Documentation
¶
Overview ¶
Package suspension defines a paused run and everything needed to continue it.
Index ¶
- Constants
- Variables
- type ClaimVersion
- type Continuation
- type PendingCall
- type ReplyDiscoveryStore
- type ResumeAdmission
- type ResumeAdmissionPage
- type ResumeAdmissionRequest
- type ResumeAdmissionStore
- type ResumeCloser
- type ResumeClosure
- type ResumeClosureAuthorizer
- type ResumeClosureFence
- type ResumeClosureRequest
- type ResumeClosureStore
- type ResumeExpiry
- type ResumeRecoveryStore
- type ResumeStart
- type RunConfig
- type Store
- type SuspendReason
- type Suspension
- type VersionedStore
Constants ¶
const MaxResumeInputBytes = 8 << 20
MaxResumeInputBytes bounds retained encoded input, including JSON escaping.
Variables ¶
var ( ErrResumeAdmissionNotFound = errors.New("cortex: resume admission not found") ErrResumeAdmissionConflict = errors.New("cortex: resume admission conflict") ErrResumeAlreadyStarted = errors.New("cortex: resume execution may have started") )
Resume admission errors distinguish missing evidence from uncertain execution.
ErrVersionClaimUnavailable means storage cannot atomically claim a pinned pause.
Functions ¶
This section is empty.
Types ¶
type ClaimVersion ¶
type ClaimVersion struct {
RunID id.AgentRunID
SuspensionID id.SuspensionID
RunRevision int64
}
ClaimVersion identifies the pause and run revision validated before resume. A later suspension of the same run must never satisfy this claim.
type Continuation ¶
type Continuation struct {
ToolResults []verification.ToolResult `json:"tool_results,omitempty"`
SchemaVersion int `json:"schema_version,omitempty"`
Prepared []byte `json:"prepared,omitempty"`
Provenance run.ExecutionProvenance `json:"provenance,omitempty"`
Usage execution.Usage `json:"usage,omitempty"`
Messages []llm.Message `json:"messages"`
SystemPrompt string `json:"system_prompt"`
StepIndex int `json:"step_index"`
TokensUsed int `json:"tokens_used"`
// NewMessagesFrom is the index in Messages where this run's own
// messages begin. Everything before it is conversation history the
// run loaded at startup and must never save again.
//
// Without it a resume has no way to tell the two apart, and its
// first SaveConversation writes the whole history back as new rows.
// That is not hypothetical: it shipped in v1.8.0, and agents went
// deaf after about six runs because the duplicated rows filled the
// fixed read window until it held nothing recent. The engine tracks
// this boundary as newMessagesFrom in both react loops; this field
// is the same number, carried across the pause.
NewMessagesFrom int `json:"new_messages_from"`
// SessionID is the session the run was loading from and saving to. A
// resume cannot re-derive it: resolveSession lazily creates a
// default session under a create race, so asking it again after a
// pause can land on a different session than the one the messages
// above came from.
SessionID id.SessionID `json:"session_id,omitempty"`
// Config is the resolved configuration the run was executing under.
// See RunConfig: a resume builds its config from this and never
// re-merges the agent's, so the run's own overrides survive the
// pause.
Config RunConfig `json:"config"`
}
Continuation is everything the loop needs to pick up where it stopped. It is stored in typed columns rather than an untyped metadata map, so a malformed continuation is a scan error at the boundary instead of a half-restored run several steps later.
SystemPrompt holds the ASSEMBLED string, not the sections it came from. That is deliberate and it matters once v1.12.0 turns prompt assembly into a pipeline over sections and overlays: a run suspended before an overlay changed, and resumed after, continues with the prompt it began with. Consistency within one run beats picking up a mid-run change to the agent's identity.
type PendingCall ¶
type PendingCall struct {
// ReplyWith binds an agent reply to the question created by this pause.
// A tool-call ID alone may be reused by a later model response.
ReplyWith string `json:"reply_with,omitempty"`
ID string `json:"id"`
Name string `json:"name"`
// Arguments is the JSON-encoded argument object, verbatim from the
// model's tool call.
Arguments string `json:"arguments"`
}
PendingCall is one tool call awaiting a result from outside the engine.
Arguments is carried here rather than left to be recovered from the continuation's last assistant message. A caller resuming an external tool has to actually execute the call, and making it cross-reference the message history by call id to find out with what would be a needless puzzle for the one consumer this type exists to serve.
type ReplyDiscoveryStore ¶
type ReplyDiscoveryStore interface {
ListAgentReplyRuns(context.Context, id.AgentRunID, int) ([]id.AgentRunID, error)
}
ReplyDiscoveryStore lists only run identities with retained agent-reply suspensions or running resume admissions in the exact context scope. Results are ordered by run ID after the exclusive cursor. Limit must be between 1 and 256. Discovery includes expired and already-admitted suspensions, plus running admissions after their pause was deleted, so hosts can reconcile interrupted execution.
type ResumeAdmission ¶
type ResumeAdmission struct {
SchemaVersion int `json:"schema_version"`
Request ResumeAdmissionRequest `json:"request"`
RequestDigest string `json:"request_digest"`
SuspensionDigest string `json:"suspension_digest"`
ClaimedRevision int64 `json:"claimed_revision"`
AdmittedAt time.Time `json:"admitted_at"`
Fresh bool `json:"-"`
}
ResumeAdmission records the atomic pause claim and its retained input. Fresh describes the acknowledgment and never grants execution permission.
type ResumeAdmissionPage ¶
type ResumeAdmissionPage struct {
Admissions []ResumeAdmission
Next string
}
ResumeAdmissionPage is exact-scope history with an opaque bounded cursor.
type ResumeAdmissionRequest ¶
type ResumeAdmissionRequest struct {
Scope cortex.Scope `json:"scope"`
Version ClaimVersion `json:"version"`
Input []byte `json:"input"`
}
ResumeAdmissionRequest binds the exact agent-reply input to a validated pause. Input is encoded bytes so storage cannot normalize its JSON representation.
type ResumeAdmissionStore ¶
type ResumeAdmissionStore interface {
AdmitResume(context.Context, ResumeAdmissionRequest) (ResumeAdmission, error)
GetResumeAdmission(context.Context, id.AgentRunID, id.SuspensionID) (ResumeAdmission, error)
ListResumeAdmissions(context.Context, string, int) (ResumeAdmissionPage, error)
StartResume(context.Context, id.AgentRunID, id.SuspensionID, string) (ResumeStart, error)
GetResumeStart(context.Context, id.AgentRunID, id.SuspensionID) (ResumeStart, error)
}
ResumeAdmissionStore separates claiming an agent-reply pause from permission to execute its continuation. Errors return no execution authority, including ambiguous commits. Host recovery must recheck current authority before Start.
type ResumeCloser ¶
type ResumeCloser interface {
Close(context.Context, ResumeClosureRequest) (ResumeClosure, error)
}
ResumeCloser terminates uncertain execution as incomplete under host guards. It neither retries work nor accepts successful results. Errors can follow a commit; retry identical contents to recover the original durable receipt.
func NewResumeCloser ¶
func NewResumeCloser(storage ResumeClosureStore, fence ResumeClosureFence, authorizer ResumeClosureAuthorizer) (ResumeCloser, error)
NewResumeCloser requires explicit host proof authentication and fencing. Each operation is bounded to thirty seconds. Host ports must honor cancellation.
type ResumeClosure ¶
type ResumeClosure struct {
Request ResumeClosureRequest `json:"request"`
RequestDigest string `json:"request_digest"`
EventID id.ID `json:"event_id"`
RunRevision int64 `json:"run_revision"`
ClosedAt time.Time `json:"closed_at"`
Fresh bool `json:"-"`
}
ResumeClosure is immutable evidence of an incomplete terminal transition. Its metadata event is committed in the same transaction as the run state.
func (ResumeClosure) Validate ¶
func (r ResumeClosure) Validate() error
Validate checks receipt integrity without granting access or new execution.
type ResumeClosureAuthorizer ¶
type ResumeClosureAuthorizer interface {
AuthorizeResumeClosure(context.Context, ResumeClosureRequest) error
}
ResumeClosureAuthorizer authenticates the actor, proof issuer/revision and proof contents bound to the exact request, then checks current run/team access. Original worker identities must come from retained dispatch evidence. Current agent configuration cannot replace them. Recheck access and original proof on historical retries; a stored receipt or a hash is not permission.
type ResumeClosureFence ¶
type ResumeClosureFence interface {
WithResumeClosure(context.Context, ResumeClosureRequest, func(ResumeClosureRequest) error) error
}
ResumeClosureFence holds current run/team grants and permanent quiescence of every original worker and tool through one synchronous callback. Include remote workers, pending callbacks and delayed external actions. Stopping one process is sufficient only when the host proves it owned all possible execution.
type ResumeClosureRequest ¶
type ResumeClosureRequest struct {
SchemaVersion int `json:"schema_version"`
Scope cortex.Scope `json:"scope"`
Version ClaimVersion `json:"version"`
AdmissionDigest string `json:"admission_digest"`
StartDigest string `json:"start_digest"`
RunRevision int64 `json:"run_revision"`
Actor string `json:"actor"`
Issuer string `json:"issuer"`
IssuerRevision string `json:"issuer_revision"`
ProofRef string `json:"proof_ref"`
ProofDigest string `json:"proof_digest"`
ObservedAt time.Time `json:"observed_at"`
ExpiresAt time.Time `json:"expires_at"`
}
ResumeClosureRequest closes an uncertain start as incomplete. The host must authenticate permanent worker and tool quiescence, including delayed actions, for this exact admission, start and current run revision. A deadline, a stopped local process or a digest alone does not establish that external work stopped. Closure never accepts output, refunds usage, clears effects or permits replay.
func (ResumeClosureRequest) Freeze ¶
func (r ResumeClosureRequest) Freeze() (ResumeClosureRequest, string, error)
Freeze checks structure and detaches the request. It does not authenticate proof.
type ResumeClosureStore ¶
type ResumeClosureStore interface {
CloseResume(context.Context, ResumeClosureRequest) (ResumeClosure, error)
GetResumeClosure(context.Context, id.AgentRunID, id.SuspensionID) (ResumeClosure, error)
}
ResumeClosureStore requires current host proof and authority before writes. Closure serializes with run writers, binds the latest admission and start, and rejects changed requests. Retry the identical request after a lost ack. Historical retries return the original receipt even after proof expiry.
type ResumeExpiry ¶
type ResumeExpiry struct {
SchemaVersion int `json:"schema_version"`
Version ClaimVersion `json:"version"`
Scope cortex.Scope `json:"scope"`
RequestDigest string `json:"request_digest"`
EventID id.ID `json:"event_id"`
RunRevision int64 `json:"run_revision"`
ExpiredAt time.Time `json:"expired_at"`
Fresh bool `json:"-"`
}
ResumeExpiry proves an admitted continuation expired before execution started. It grants no execution or retry authority and does not release reservations.
type ResumeRecoveryStore ¶
type ResumeRecoveryStore interface {
LatestResumeAdmission(context.Context, id.AgentRunID) (ResumeAdmission, error)
ExpireResumeAdmission(context.Context, id.AgentRunID, id.SuspensionID, string) (ResumeExpiry, error)
GetResumeExpiry(context.Context, id.AgentRunID, id.SuspensionID) (ResumeExpiry, error)
}
ResumeRecoveryStore retains exact-scope recovery evidence independently of the live pause. LatestResumeAdmission orders by suspension ID. Expiry and execution start must share the run lock; an existing start refuses expiry. Hosts check current authority before every operation.
type ResumeStart ¶
type ResumeStart struct {
SchemaVersion int `json:"schema_version"`
Version ClaimVersion `json:"version"`
Scope cortex.Scope `json:"scope"`
RequestDigest string `json:"request_digest"`
StartedAt time.Time `json:"started_at"`
Fresh bool `json:"-"`
Suspension *Suspension `json:"-"`
}
ResumeStart records a single execution admission. Only a fresh acknowledged start includes a suspension. Observing or retrying a start never grants it.
func (ResumeStart) Digest ¶
func (s ResumeStart) Digest() (string, error)
Digest binds a retained start without its transient execution authority.
type RunConfig ¶
type RunConfig struct {
Context *cognitive.ContextPolicy `json:"context,omitempty"`
Publication safety.Policy `json:"publication"`
Budget execution.Budget `json:"budget,omitempty"`
AllToolsRestricted bool `json:"all_tools_restricted,omitempty"`
Model string `json:"model,omitempty"`
Temperature *float64 `json:"temperature,omitempty"`
MaxSteps int `json:"max_steps,omitempty"`
MaxTokens int `json:"max_tokens,omitempty"`
ReasoningLoop string `json:"reasoning_loop,omitempty"`
Tools []string `json:"tools,omitempty"`
PersonaRef string `json:"persona_ref,omitempty"`
// ToolsRestricted says Tools is an explicit allowlist even when it
// is empty, which is what an overlay that withdrew the agent's last
// tool produces. It travels with the continuation because a resumed
// run rebuilds its config from here and from nowhere else: without
// it, an empty list would read as "every registered tool" and the
// resume would hand back exactly what the overlay took away.
//
// It is absent from rows written before this field existed, and
// decodes there as false, which is the behavior those runs had.
// The flag can therefore be LOST across an upgrade, but it can
// never be spuriously SET. That is the safe direction here: losing
// it resumes an old run under the tool rules it was suspended
// under, where inventing it would strip tools from a run nobody
// restricted.
ToolsRestricted bool `json:"tools_restricted,omitempty"`
}
RunConfig is the resolved configuration a run was executing under: the merge of engine defaults, agent config and that run's overrides, already flattened.
It is stored for the same reason SystemPrompt is stored assembled rather than as the sections it came from. A run's overrides live only in the call that started it, so a resume that rebuilt the config from the agent would silently drop them: a run deliberately started with a narrowed Tools list would come back with the agent's full tool set, which is authority widening across a pause, and a run started with a raised MaxSteps would come back on the agent's smaller budget and refuse to continue at all.
Storing the resolved values also pins what the run was actually executing, so an agent-config edit landing during the pause cannot shift a conversation already in flight.
type Store ¶
type Store interface {
// CreateSuspension writes a new suspension row for a run that just
// paused.
CreateSuspension(ctx context.Context, s *Suspension) error
// GetSuspension reads the suspension for a run without changing
// anything. Callers that intend to resume the run must use
// ClaimSuspension instead: a Get followed by a separate write is the
// race ClaimSuspension exists to close.
GetSuspension(ctx context.Context, runID id.AgentRunID) (*Suspension, error)
// ClaimSuspension is the concurrency primitive for resume.
// Implementations must perform the run's paused-to-running
// transition and the suspension read as ONE atomic operation, not a
// read followed by a separate write: that gap is exactly the race
// this method exists to close, since two concurrent resume calls
// must not both observe a paused run and proceed. The caller that
// loses the race gets cortex.ErrNotSuspended, not the suspension it
// lost the race for.
ClaimSuspension(ctx context.Context, runID id.AgentRunID) (*Suspension, error)
// DeleteSuspension removes a suspension row. Callers use this once a
// resume has fully completed, or as part of the expiry sweep once an
// expired suspension has been handled.
DeleteSuspension(ctx context.Context, runID id.AgentRunID) error
// ListExpired returns suspensions whose ExpiresAt is at or before
// now, bounded to at most limit rows, within the caller's scope.
ListExpired(ctx context.Context, now time.Time, limit int) ([]*Suspension, error)
// ListExpiredAcrossScopes is ListExpired with the scope filter
// DELIBERATELY BYPASSED: it returns expired suspensions from every
// scope in the database, and it is the one read in this interface
// that does.
//
// It exists for the engine's expiry sweeper and nothing else. A
// sweeper is process infrastructure started by Engine.Start, so
// there is no request scope on its context and no way to enumerate
// the scopes that have rows; ListExpired would sweep whichever scope
// the sweeper's context happened to carry, which in practice is
// none. Hence a second method rather than a flag or a sentinel
// scope: a cross-scope read has to be visible as one at the call
// site, and no handler can reach this by passing a zero value.
//
// Crossing scopes to FIND the work does not license doing the work
// unscoped. Callers must rebind their context to each returned
// suspension's own Scope before touching that run, exactly as Resume
// does.
ListExpiredAcrossScopes(ctx context.Context, now time.Time, limit int) ([]*Suspension, error)
// ClaimExpiredSuspension is ClaimSuspension's mirror for the expiry
// sweeper: the same atomic paused-to-running transition, conditioned
// on the suspension being PAST its deadline rather than short of it.
//
// The two predicates partition the same row, which is what makes a
// resume and a sweep exclusive rather than merely unlikely to
// collide. A resume that beats the deadline claims the run and the
// sweeper can no longer take it; a sweep that finds the deadline
// passed takes the run and no resume can claim it afterwards. Both
// answer against the same stored ExpiresAt, so there is no window
// where the two agree the row is theirs.
//
// Zero rows matched means somebody else got there first, and it
// returns cortex.ErrNotSuspended. The sweeper skips such a run
// rather than failing it: it is either gone, already resumed, or no
// longer paused.
ClaimExpiredSuspension(ctx context.Context, runID id.AgentRunID, now time.Time) (*Suspension, error)
}
Store persists suspensions and provides the concurrency primitive that resuming a run depends on.
type SuspendReason ¶
type SuspendReason string
SuspendReason says why a run stopped. Both reasons share one mechanism because they are the same problem: the loop cannot proceed until something outside it returns.
const ( // ReasonApproval means a human must grant or deny before the run // continues. ReasonApproval SuspendReason = "approval" // ReasonExternalTool means the caller, not the engine, executes the // pending tool call and reports the result back. ReasonExternalTool SuspendReason = "external_tool" // ReasonAgentReply means the run is waiting on another agent's answer. // // It is not ReasonExternalTool even though both wait on something // outside the loop, because the two say different things about who // acts next. External says the CALLER executes the call and reports // back. Agent-reply says cortex itself is waiting on a peer, and a // caller answering it would be forging a message that peer never // sent. The messaging bus resumes these, off its own correlation // ledger; nobody else may. ReasonAgentReply SuspendReason = "agent_reply" )
type Suspension ¶
type Suspension struct {
// ClaimedRun is the snapshot returned by an atomic claim, including its
// new revision. It is transient and is never saved as suspension content.
ClaimedRun *run.Run `json:"-"`
cortex.Entity
ID id.SuspensionID `json:"id"`
RunID id.AgentRunID `json:"run_id"`
// Scope is stored so Resume re-derives authorization from the scope
// the run STARTED under, rather than trusting the scope of whoever
// happens to call Resume. A resume arriving with different authority
// must not quietly continue with it.
Scope cortex.Scope `json:"scope"`
Reason SuspendReason `json:"reason"`
Pending []PendingCall `json:"pending,omitempty"`
Cont Continuation `json:"continuation"`
ExpiresAt *time.Time `json:"expires_at,omitempty"`
}
Suspension is a paused run.
type VersionedStore ¶
type VersionedStore interface {
ClaimSuspensionVersion(context.Context, ClaimVersion) (*Suspension, error)
}
VersionedStore fences the suspension identity and run revision in the same transaction as the paused-to-running transition. Failed or ambiguous commits return no suspension. Retrying cannot claim a later pause in the same run. This primitive does not retain a resume payload or grant recovery authority.