Documentation
¶
Overview ¶
Package lifecycle implements workspace lifecycle automation: the Reaper loop finds RUNNING agent runs idle past their policy's AutoStopAfterSec threshold and stops them, emitting a "run.autostop" audit event for each.
Idleness is wall-clock age of agent_runs.updated_at, reset only by a write through the store (state transitions, sandbox_ref updates, TouchRun). The proxy's decision-ingest touch is coalesced to one UPDATE per TouchDebounce window, so thresholdFor adds TouchDebounce as slack. Activity that never leaves the sandbox (CPU, file writes) does NOT reset the clock, so a busy run can still be reaped; operators needing an unbounded session use the never-reap escape hatch (AutoStopAfterSec <= 0).
AutoStopAfterSec (policy auto_stop_after_sec): >0 idle timeout in seconds; 0 DISABLED/never reaped (default, matches docs/POLICIES.md); <0 also never reaped but kept distinct so an operator can express intent loudly.
Index ¶
Constants ¶
const TouchDebounce = 30 * time.Second
TouchDebounce covers the ingest touch's coalesced-UPDATE lag, added as slack in thresholdFor so debounce can never make an active run look idle.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
// Interval is how often the reaper scans. Default: 1 minute.
Interval time.Duration
// Now overrides the wall clock. Nil means use real time.
Now func() time.Time
// TickLock, when non-nil, makes each tick single-flight across control
// planes: TRY a cluster-wide lock, return a release func, or (nil, false)
// if held elsewhere — then the tick is skipped, not queued. wardynd wires
// a Postgres try-advisory-lock; kept as a func so lifecycle has no DB
// dependency. Nil = ungated (single-process default; what tests use).
TickLock func(ctx context.Context) (release func(), ok bool)
}
Config holds optional overrides for Reaper behaviour; zero value is valid.
type Reaper ¶
type Reaper struct {
// contains filtered or unexported fields
}
Reaper is the idle-workspace garbage collector: a periodic loop that finds RUNNING workspaces idle past their policy threshold and stops them. Reaper.Run blocks until ctx is cancelled; a stopper error is logged and skipped so one broken sandbox never blocks the rest of the tick.
func (*Reaper) Run ¶
Run starts the reap loop, returning when ctx is cancelled. The first tick fires after one full Interval so startup finishes before the first scan.
func (*Reaper) Tick ¶
Tick is one reap scan, gated by Config.TickLock when wired. Exported for integration callers/tests to drive directly; production code uses Run.
Deadline is Interval+defaultStopTimeout, not just Interval: a child context.WithTimeout can only shorten its parent's deadline, so budgeting at bare Interval would silently cap every per-stop deadline short of the full defaultStopTimeout the constant promises.
type Recorder ¶
type Recorder interface {
Record(ctx context.Context, ev types.AuditEvent) error
}
Recorder matches audit.Recorder exactly so a store.Recorder can be passed directly.
type RunSummary ¶
type RunSummary struct {
ID uuid.UUID
UpdatedAt time.Time
// PolicyAutoStopAfterSec: 0 or negative means never reap, positive is the
// idle timeout in seconds. Store must JOIN to the policy table; zero is
// intentional, not a missing join.
PolicyAutoStopAfterSec int
}
RunSummary is the minimal projection a Store must return for idle detection.
type StopOutcome ¶
type StopOutcome struct {
// Applied is false when a concurrent kill/complete already moved the run
// terminal, or the idleness guard no-op'd because updated_at advanced past
// the snapshot (an active attach). Either way, no spurious run.autostop.
Applied bool
// Errors, when non-empty, are post-transition teardown/revocation failures
// ("identity_error"/"broker_error": creds may still be live;
// "teardown_error": sandbox may still be routable) that leave the run not
// fully contained. Emitted as a distinct run.revoke/failure audit event,
// mirroring handleKillRun/revokeRunCascade. Nil on a clean stop.
Errors map[string]string
}
StopOutcome is what StopRun reports back to the Reaper beyond the raw error.
type Stopper ¶
type Stopper interface {
// StopRun must be idempotent: stopping an already-stopped run returns
// ({Applied:false}, nil). notAfter is the snapshot's updated_at; the
// transition is conditional on updated_at not having advanced past it, so
// a run touched by an active attach since the snapshot is NOT stopped. A
// non-nil error means the stop failed outright and the reaper logs/skips.
StopRun(ctx context.Context, runID uuid.UUID, notAfter time.Time) (StopOutcome, error)
}
Stopper stops a single run: transitions RUNNING->STOPPED guarded on the snapshot's idleness, then calls runner.StopSandbox + the revoke cascade. Hidden behind this interface so lifecycle stays target-agnostic.
type Store ¶
type Store interface {
// ListRunningWithPolicy returns RUNNING runs with their policy's
// auto_stop_after_sec (0 if unattached/unset), plus the store's own clock
// read once for the whole scan. The reaper measures idleness against that
// clock, not its own: UpdatedAt is Postgres-stamped, so wardynd clock skew
// vs. the database eats directly into TouchDebounce's 30s margin — enough
// skew stops an actively-attached run and revokes its credentials, or
// (skewed the other way) never reaps at all. A ZERO returned time means
// the store has no clock (in-memory doubles), and the reaper falls back
// to its own.
ListRunningWithPolicy(ctx context.Context) ([]RunSummary, time.Time, error)
}
Store is the narrow persistence interface the Reaper requires; the real adapter maps agent_runs JOIN run_policies, tests supply a fake.
Source Files