Documentation
¶
Overview ¶
Package coordinator provides the orchestration seam between model-facing tools and the subagent execution pool. It owns orchestration policy, state transitions, display-name allocation, and the LedgerRepository boundary.
Package coordinator provides the orchestration seam between model-facing tools and the subagent execution pool.
Index ¶
- Constants
- Variables
- func ContextWithPanelWaitOnlyJoin(ctx context.Context) context.Context
- func IsTaskTerminal(status string) bool
- func NewRunID() string
- func RequestFingerprint(tasks []subagents.Task, policy ledger.RunPolicy) (string, error)
- func ResultsFromSnapshots(tasks []ledger.TaskSnapshot) []subagents.Result
- type Coordinator
- type EnsureRunRequest
- type LifecycleSubscriber
- type MessageSummary
- type ParkedQuestion
- type RecoveredRun
- type ReferralSpawnMeta
- type RetryPolicy
- type RetryState
- type RunHandle
- type RunResult
- type TaskProgress
Constants ¶
const LifecycleKindTaskAskDeclined = "task_ask_declined"
LifecycleKindTaskAskDeclined is appended when an ask is declined because its target task reached terminal status without answering. Attributed to the ASKER task/attempt. Payload carries {ask_id, reason}; run_messages surfaces it as an "ask_declined" entry (plan 53.04 observability).
const LifecycleKindTaskMessage = "task_message"
Lifecycle kind for agent-to-agent message announcements (plan 53.01). Payload is ID + synopsis only (never bodies) - see agentmsg.LifecyclePayload.
const MessageKindAskDeclined agentmsg.Kind = "ask_declined"
MessageKindAskDeclined is the run_messages kind surfaced for a task_ask_declined lifecycle event. The ask itself never persisted as a message on this path — it is a decline announcement attributed to the asker.
Variables ¶
var DefaultRetryPolicy = RetryPolicy{ MaxRetries: 3, BaseBackoff: 1 * time.Second, MaxBackoff: 30 * time.Second, BackoffFactor: 2.0, JitterFraction: 0.25, }
DefaultRetryPolicy is a sensible default: 3 retries, 1s base, 30s cap, exponential (2x), 25% jitter.
var ErrIdempotencyConflict = errors.New("idempotency key already used for a different request")
var ErrIdempotencyKeyContended = errors.New("idempotency key is being created/recovered by another process; retry")
ErrIdempotencyKeyContended is returned by recoverByIdempotencyKey (and surfaced by Spawn after a bounded retry) when the idempotency key is mid-creation or mid-recovery by another process: either a live creator is between its durable CreateRun and its first durable CreateTask, or another process is reclaiming the abandoned run right now. The caller must back off and retry until the winner's run is durably visible, so the retry converges to dedup onto that run instead of racing a second execution of the keyed work.
var ( // ErrRunHeldByAnotherExecutor is returned by ResumeInterruptedRun and // Spawn when the run is already claimed by another executor process. ErrRunHeldByAnotherExecutor = errors.New("run is held by another executor") )
var ErrWaitOnlyJoinLost = errors.New("panel wait-only join became locally runnable")
ErrWaitOnlyJoinLost means a remote child became locally runnable before a wait-only join could attach. The caller must obtain an actor permit first.
var NoRetry = RetryPolicy{}
NoRetry is a zero-value policy that disables retry entirely.
Functions ¶
func ContextWithPanelWaitOnlyJoin ¶
ContextWithPanelWaitOnlyJoin prevents a wait-only caller from taking over a child that is no longer held by another executor.
func IsTaskTerminal ¶
IsTaskTerminal reports whether the task's ledger status is terminal.
func NewRunID ¶
func NewRunID() string
NewRunID returns an unguessable run identifier. Unguessability is load-bearing (INV-AG-9): run IDs must not be enumerable. crypto/rand.Read never returns an error and always fills its buffer, crashing the program if the operating system's source fails, so there is no error path - and no weaker fallback would be acceptable if there were.
func RequestFingerprint ¶
RequestFingerprint returns the canonical coordinator work fingerprint. Callers use it to verify a persisted, non-authority work specification.
func ResultsFromSnapshots ¶
func ResultsFromSnapshots(tasks []ledger.TaskSnapshot) []subagents.Result
ResultsFromSnapshots converts recorded task snapshots into results.
Exported so a caller can salvage a run whose Join was cut short by the caller's own context. The run's work is recorded in the ledger, so its results stay recoverable even though the handle never resolved - without this, a caller whose budget expired reported a bare error and dropped every task that had finished.
Types ¶
type Coordinator ¶
type Coordinator interface {
Spawn(context.Context, []subagents.Task, string) (*RunHandle, error)
// SpawnNew is Spawn plus an isNew signal: false when the idempotency-key
// lookup returned an existing run some other caller started, so a
// caller can tell whether it is safe to treat itself as the run's sole
// owner (e.g. before canceling it on its own unrelated context dying).
SpawnNew(context.Context, []subagents.Task, string) (*RunHandle, bool, error)
EnsureRun(context.Context, EnsureRunRequest) (*RunHandle, error)
EnsureSingleTaskRun(context.Context, EnsureRunRequest) (*RunHandle, error)
EnsureTerminalSingleTaskRun(context.Context, EnsureRunRequest, ledger.TaskStatus) (*RunHandle, error)
// JoinAsRecovered returns a recovered, wait-only handle for an already
// admitted run, or ledger.ErrNotFound if none is admitted yet. It never
// claims to run the child, dispatches its handler, or resumes it as a
// local actor: Cancel on the returned handle always takes the fail-closed
// recovered path, which refuses when a task's persisted status looks
// nonterminal with no verifiable live owner.
JoinAsRecovered(context.Context, EnsureRunRequest) (*RunHandle, error)
Inspect(context.Context, *RunHandle) (ledger.RunSnapshot, error)
Join(context.Context, *RunHandle) (*RunResult, error)
Cancel(context.Context, *RunHandle) error
SetTimeSource(func() time.Time)
WithRetryPolicy(RetryPolicy) Coordinator
ResumeInterruptedRun(context.Context, string) (*RunHandle, error)
ListInterruptedRuns(context.Context) ([]RecoveredRun, error)
SubscribeLifecycle(LifecycleSubscriber) func()
// PostTaskMessage persists a typed agent message and announces a
// task_message lifecycle event (ID + synopsis only). Plan 53.01 seam.
PostTaskMessage(ctx context.Context, runID, taskID string, msg agentmsg.Message) error
// ParkQuestion / DeliverAnswer / Transition* support plan 53.02 questions.
// maxWait is the asker's effective max wait; the park expires at
// max(parkTTL, maxWait+parkSlack) so long waits are never evicted early.
ParkQuestion(runID, taskID, messageID string, maxWait ...time.Duration) (answerCh <-chan string, unpark func(), err error)
// DeliverAnswer unblocks a park when inReplyTo matches the parked message id
// (empty inReplyTo matches any live park for the task).
DeliverAnswer(runID, taskID, inReplyTo, body string) bool
TransitionToAwaitingInput(ctx context.Context, runID, taskID string) error
TransitionFromAwaitingInput(ctx context.Context, runID, taskID, newStatus string) error
ConsumeMessageQuota(runID, taskID string, max int) error
// RefundMessageQuota decrements the per-task upstream message count after a
// failed persist so a failed message never permanently burns a budget slot
// (messageQuota is otherwise increment-only). Floored at zero: it only ever
// undoes a prior ConsumeMessageQuota.
RefundMessageQuota(runID, taskID string)
CountPendingQuestions(runID, taskID string) int
// ParkedQuestions returns the live parked questions for a run
// (TaskID/MessageID/ExpiresAt), read under the question registry lock.
// Expired parks are treated as absent via the existing eviction.
ParkedQuestions(runID string) []ParkedQuestion
ListRunMessages(ctx context.Context, runID, taskID string) ([]MessageSummary, error)
LoadMessageBody(ctx context.Context, contentRef string) (agentmsg.Message, error)
// SendToTask enqueues a parent→child message (steer/answer) after ledger persist.
SendToTask(ctx context.Context, h *RunHandle, taskID string, msg agentmsg.Message) (delivered bool, err error)
// WithMessagingLimits applies body/mailbox budgets from [subagents.messaging].
WithMessagingLimits(maxBodyBytes, mailboxCapacity int) Coordinator
// Ask registry (plan 53.04).
RegisterAsk(runID, askerTaskID, askerRole, askID string, ancestors []string)
TryRegisterAsk(runID, askerTaskID, askerRole, askID string, ancestors []string, maxAsks int) bool
AsksUsedByTask(runID, taskID string) int
ReferralSpawnsUsed(runID string) int
IncReferralSpawn(runID string)
TryIncReferralSpawn(runID string, max int) bool
DecReferralSpawn(runID string)
AskLookup(askID string) (askerTaskID string, ok bool)
AskChainInfo(parentAskID, toRole string) (depth int, cycle bool, ancestors []string)
CompleteAskAnswer(askID string) error
ClaimAskAnswer(askID string) (askerTaskID string, err error)
// BeginAskAnswer claims an open registry ask for parent/peer one-shot.
// claimed=false,err=nil means not a registry ask (question path).
BeginAskAnswer(askID string) (askerTaskID string, claimed bool, err error)
IsAskAnswered(askID string) bool
CloseAsk(askID string)
// SealAskAnswer closes open/claimed ask; true only if this call sealed.
SealAskAnswer(askID string) bool
UnclaimAskAnswer(askID, askerTaskID string)
// FindLiveTaskByRole returns a running/awaiting task whose AgentName matches role.
FindLiveTaskByRole(ctx context.Context, runID, role string) (taskID string, ok bool, err error)
// HandleForRun returns the in-memory handle for an active run, if any.
HandleForRun(runID string) *RunHandle
// MailboxSend delivers an already-persisted message to a task mailbox.
MailboxSend(h *RunHandle, taskID string, msg agentmsg.Message) (delivered bool, err error)
// SpawnReferralFromAsk starts a same-run referral task for a non-blocking ask.
// Optional meta supplies agent digest/provider/model for production agents.
SpawnReferralFromAsk(ctx context.Context, runID, toRole string, ask agentmsg.Message, meta ...ReferralSpawnMeta) (taskID string, err error)
// SpawnReferral starts a same-run task by role/name with the given input.
// askID, when non-empty, is bound before the referral goroutine starts.
SpawnReferral(ctx context.Context, runID string, task subagents.Task, askID string) (taskID string, err error)
}
func New ¶
func New(repo ledger.LedgerRepository, pool *subagents.Pool) Coordinator
type EnsureRunRequest ¶
type EnsureRunRequest struct {
RunID string
Tasks []subagents.Task
IdempotencyKey string
ForceResume bool
// NonInteractiveParent marks the run's parent as a non-interactive
// controller that can never answer child questions. Parked questions for
// tasks in such a run are declined immediately at park time so the child
// proceeds instead of burning its full wait budget. Generic mechanism: the
// coordinator never assumes who the parent is, only that it cannot answer.
NonInteractiveParent bool
Policy ledger.RunPolicy
}
EnsureRunRequest identifies one host-admitted run and its exact work.
type LifecycleSubscriber ¶
type LifecycleSubscriber func(event ledger.LifecycleEvent)
type MessageSummary ¶
type MessageSummary struct {
MessageID string `json:"message_id"`
Kind agentmsg.Kind `json:"kind"`
Synopsis string `json:"synopsis"`
ContentRef string `json:"content_ref,omitempty"`
TaskID string `json:"task_id,omitempty"`
Sequence uint64 `json:"sequence,omitempty"`
}
MessageSummary is the model-visible synopsis entry for a task_message event.
type ParkedQuestion ¶
type ParkedQuestion struct {
TaskID string `json:"task_id"`
MessageID string `json:"message_id"`
ExpiresAt time.Time `json:"expires_at"`
}
ParkedQuestion is one live parked question surfaced by run inspection.
type RecoveredRun ¶
type RecoveredRun struct {
RunID string `json:"run_id"`
DisplayName string `json:"display_name"`
Status string `json:"status"`
WasInterrupted bool `json:"was_interrupted"`
// CreatedAt lets a listing show a run's age. Recover classifies a run
// abandoned days ago identically to one interrupted moments ago, so age is
// the only thing that distinguishes news from noise.
CreatedAt time.Time `json:"created_at"`
// HeldByAnotherExecutor is true when the run has an execution claim held
// by a different repository instance (i.e. another mivia process). The
// dashboard shows this separately so users do not try to resume it.
HeldByAnotherExecutor bool `json:"held_by_another_executor"`
}
RecoveredRun is a summary of a recovered orchestration run.
type ReferralSpawnMeta ¶
ReferralSpawnMeta carries optional agent-routing fields for production agents (digest/provider/model). Zero values leave Task defaults empty.
type RetryPolicy ¶
type RetryPolicy struct {
// MaxRetries is the maximum number of retry attempts per task.
// 0 means no retry (failed/timed_out tasks are terminal).
MaxRetries int `toml:"max_retries" json:"max_retries"`
// BaseBackoff is the initial backoff duration before the first retry.
// Each subsequent retry multiplies by BackoffFactor, capped at MaxBackoff.
BaseBackoff time.Duration `toml:"base_backoff" json:"base_backoff"`
// MaxBackoff caps the per-retry backoff duration.
MaxBackoff time.Duration `toml:"max_backoff" json:"max_backoff"`
// BackoffFactor is the multiplier applied to backoff after each attempt.
// Default 2.0 (exponential). A value of 0 is treated as 2.0.
BackoffFactor float64 `toml:"backoff_factor" json:"backoff_factor"`
// JitterFraction adds randomisation: each backoff is multiplied by
// 1 ± JitterFraction/2. E.g. 0.25 means ±12.5%. 0 disables jitter.
JitterFraction float64 `toml:"jitter_fraction" json:"jitter_fraction"`
}
RetryPolicy controls the automatic retry behaviour for failed or timed-out tasks within a DAG run. Zero values disable retry.
func (RetryPolicy) EffectiveBackoff ¶
func (p RetryPolicy) EffectiveBackoff(attempt int) time.Duration
EffectiveBackoff returns the wall-clock delay before the nth retry attempt (zero-based: attempt 0 = first retry after initial failure). It applies exponential backoff and optional jitter.
func (RetryPolicy) IsZero ¶
func (p RetryPolicy) IsZero() bool
IsZero returns true when the policy is zero-valued (no retry).
type RetryState ¶
type RetryState struct {
TaskID string
Attempts int // number of retries already performed
Policy RetryPolicy
// contains filtered or unexported fields
}
RetryState tracks retry progress for a single task within a run. Must be used from a single goroutine (no locking).
func NewRetryState ¶
func NewRetryState(taskID string, policy RetryPolicy) *RetryState
NewRetryState creates a RetryState for a task with the given policy.
func (*RetryState) CanRetry ¶
func (rs *RetryState) CanRetry() bool
CanRetry returns true if more retry attempts are available.
func (*RetryState) Done ¶
func (rs *RetryState) Done() <-chan struct{}
Done returns a channel that closes when retries are exhausted.
func (*RetryState) Exhausted ¶
func (rs *RetryState) Exhausted()
Exhausted marks the retry budget as exhausted (terminal).
func (*RetryState) NextBackoff ¶
func (rs *RetryState) NextBackoff() time.Duration
NextBackoff returns the delay before the upcoming retry and advances the internal attempt counter.
type RunHandle ¶
type RunHandle struct {
// contains filtered or unexported fields
}
RunHandle is a handle to an active orchestration run.
func (*RunHandle) LocalActor ¶
LocalActor reports whether this process owns execution of the run.
func (*RunHandle) MarkTaskMailboxTerminal ¶
MarkTaskMailboxTerminal is called when a task reaches a terminal status so further sends fail cleanly without close-on-terminal panics. Any asks that were delivered to this task's mailbox are declined to their parked askers (they will never be answered now that the task is terminal), gated on the ledger task status so a retry that is pending or queued is never declined.
func (*RunHandle) TaskProgress ¶ added in v0.1.1
func (h *RunHandle) TaskProgress() map[string]TaskProgress
TaskProgress returns the live per-task tool-call liveness view for parent-facing inspection (inspect_agents): tool-call counts and last-activity stamps that survive the raw trace buffer's caps, so a chatty task never reads stale. Nil-receiver safe; nil when nothing has run.
type RunResult ¶
type RunResult struct {
Snapshot ledger.RunSnapshot
Results []subagents.Result
Err error
}
type TaskProgress ¶ added in v0.1.1
type TaskProgress struct {
ToolCalls int // count of tool-call START events, dropped or not
LastTool string // most recent tool name (start or end)
LastActivity time.Time // most recent sink event, any kind
}
TaskProgress is the cap-proof per-task liveness view behind RunHandle.TaskProgress: counters updated on EVERY sink event regardless of the raw buffer's caps, so a chatty task never reads stale the way the capped raw steps would. Zero value = no tool activity observed yet - on a long-running task that zero is itself the wedge signal (dispatched, never reached its first tool call).
Source Files
¶
- ask_lookup.go
- ask_registry.go
- cancel.go
- claim_lease.go
- coordinator.go
- dag.go
- dag_retry.go
- dag_settled.go
- ensure.go
- handle_lifecycle.go
- join_recovered.go
- list_messages.go
- mailbox.go
- panel_wait_only.go
- post_message.go
- questions.go
- record_results.go
- recovery.go
- recovery_output.go
- recovery_reclaim.go
- recovery_tasks.go
- referral_spawn.go
- retry.go
- retry_gate.go
- run_policy.go
- send_to_task.go
- spawn.go
- task_context.go
- task_done.go
- tool_call_buffer.go
- types.go
- validation.go