Documentation
¶
Overview ¶
Package attempts runs one task up to N times (best-of-N), each attempt as a child tinyagent run in its own Git worktree, grades every finished attempt with user-configured check commands, and picks a verified winner: the lowest-numbered attempt whose run ended Submitted or Completed and whose checks all exited 0.
Attempts run sequentially and, by default, stop at the first pass (RunAll runs every attempt). tinyagent never commits, merges or pushes an attempt. Every worktree is kept unless Keep asks to remove non-winners once an attempt wins.
Index ¶
- Constants
- func ExecAgent(exe string) func(context.Context, []string, io.Writer, io.Writer) error
- func LinkEnv(base []string, l Link) []string
- func NewID() string
- func RemoveWorktree(repoRoot, path, branch string) error
- func ReportPath(out string) string
- func TrajectoryPath(out string, i int) string
- type Attempt
- type Check
- type CheckResult
- type CheckRunner
- type Link
- type Options
- type Report
Constants ¶
const ( DefaultCheckTimeout = 600 MaxCheckTimeout = 3600 )
DefaultCheckTimeout and MaxCheckTimeout bound one check, in seconds.
const ( StopPassed = "passed" StopExhausted = "exhausted" StopBudget = "budget exhausted" StopInterrupt = "interrupted" StopError = "error" // StopCompleted: RunAll ran every attempt and at least one passed. StopCompleted = "completed" )
Stop reasons recorded in the report.
const ( KeepAll = "all" KeepFailed = "failed" KeepNone = "none" )
Keep modes: which non-winning worktrees survive once an attempt wins.
const ( EventAttemptStarted = "attempt.started" EventAttemptCheck = "attempt.check" EventAttemptResult = "attempt.result" EventAttemptsResult = "attempts.result" )
Event types written to Options.Events (--json with --attempts).
const ( EnvAttemptsID = "TINYAGENT_ATTEMPTS_ID" EnvAttempt = "TINYAGENT_ATTEMPT" EnvAttempts = "TINYAGENT_ATTEMPTS" )
Environment variables a parent --attempts run sets on each child so the child's log DB row and OpenTelemetry spans can be tied back to the group.
const MaxAttempts = 4
MaxAttempts is the largest --attempts value.
const MaxCheckOutput = 8 << 10
MaxCheckOutput caps the check output kept in the report (the tail is kept).
const SchemaVersion = 1
SchemaVersion of the attempts report.
Variables ¶
This section is empty.
Functions ¶
func ExecAgent ¶
ExecAgent returns a RunAgent that runs exe as a child process. On context cancellation the child gets an interrupt so it can end Interrupted and save its trajectory.
func LinkEnv ¶
LinkEnv returns base with l's variables set, replacing any inherited attempt variables (and TRACEPARENT/TRACESTATE when l carries one).
func RemoveWorktree ¶
RemoveWorktree deletes the worktree at path and its branch from the repository at repoRoot, discarding the worktree's uncommitted changes.
func ReportPath ¶
ReportPath returns the attempts report path for the -o path out.
func TrajectoryPath ¶
TrajectoryPath returns attempt i's trajectory path for the -o path out.
Types ¶
type Attempt ¶
type Attempt struct {
Attempt int `json:"attempt"`
Model string `json:"model"`
Status string `json:"status"`
Passed bool `json:"passed"`
Steps int `json:"steps"`
InputTokens int `json:"input_tokens"`
OutputTokens int `json:"output_tokens"`
CostUSD float64 `json:"cost_usd"`
WallTimeMS int64 `json:"wall_time_ms"`
Worktree string `json:"worktree"`
Branch string `json:"branch"`
FilesChanged int `json:"files_changed"`
Trajectory string `json:"trajectory,omitempty"`
Checks []CheckResult `json:"checks"`
Error string `json:"error,omitempty"`
// Removed is true when Keep cleanup deleted the worktree and branch.
Removed bool `json:"removed,omitempty"`
// contains filtered or unexported fields
}
Attempt is one attempt's entry in the report.
type Check ¶
Check is one verification command, run with bash -c in the attempt's working directory.
type CheckResult ¶
type CheckResult struct {
Command string `json:"command"`
ExitCode int `json:"exit_code"`
DurationMS int64 `json:"duration_ms"`
TimedOut bool `json:"timed_out"`
Skipped bool `json:"skipped"`
Output string `json:"output"`
}
CheckResult is one check's outcome for one attempt.
type CheckRunner ¶
CheckRunner builds an unstarted `bash -c command` in root. Cancelling ctx must kill the command and everything it spawned.
type Link ¶
type Link struct {
ID string // attempts group id, shared by every attempt
Index int // 1-based attempt number
Count int // attempts requested
TraceParent string // W3C traceparent for the child's root span, or ""
}
Link identifies one attempt within an attempts group.
func LinkFromContext ¶
LinkFromContext returns the attempt link runAttempt put on ctx, if any.
type Options ¶
type Options struct {
// Models holds one model spec per attempt; its length is N.
Models []string
Checks []Check
// BaseName names the worktrees: attempt i uses <BaseName>-a<i>.
BaseName string
// Output is the -o path. Attempt i's trajectory goes to <stem>.a<i><ext>
// and the report to <stem>.attempts<ext>. Empty means trajectories go to
// a temporary directory and no report file is written.
Output string
CostLimit float64 // invocation-wide USD budget; 0 = none
TimeLimit time.Duration // invocation-wide wall-clock budget; 0 = none
// ExtraArgs are forwarded to every child run (the task among them).
ExtraArgs []string
Stderr io.Writer
// Stdout receives the winner's stdout (its submission) at the end.
Stdout io.Writer
// Redact masks secrets in check output; nil leaves it unchanged.
Redact func(string) string
// RunAgent runs one child tinyagent with args. Its stdout is buffered
// per attempt; its stderr is passed through.
RunAgent func(ctx context.Context, args []string, stdout, stderr io.Writer) error
// Now is the clock; nil means time.Now.
Now func() time.Time
// RunAll runs every attempt instead of stopping at the first pass. The
// lowest-numbered passing attempt still wins.
RunAll bool
// Keep is KeepAll (or empty), KeepFailed or KeepNone. With a winner,
// KeepFailed removes passing non-winners and KeepNone removes every
// non-winner through Remove. Nothing is removed without a winner.
Keep string
// Remove deletes one attempt's worktree and branch. Nil disables
// cleanup.
Remove func(a Attempt) error
// Events, when non-nil, receives JSONL events (--json): each child
// run's stdout events tagged with "attempt", plus attempt.started,
// attempt.check, attempt.result and one final attempts.result. The
// winner's stdout is then not replayed to Stdout.
Events io.Writer
// AttemptsID, when set, links the children to this group: each child
// gets it (and its attempt number) in its environment, and it is
// recorded in the report and events.
AttemptsID string
// TraceParent, when set with AttemptsID, is passed to each child as
// TRACEPARENT so every attempt's spans share the parent's trace.
TraceParent string
// CheckEnv, when non-nil, prepares where one attempt's checks run
// (the run's sandbox, container or command_env) for the attempt's
// working directory, and returns a release function. An error fails
// every check of that attempt (fail closed); nil runs checks on the
// host with tinyagent's environment.
CheckEnv func(ctx context.Context, root string) (CheckRunner, func(), error)
}
Options configures one best-of-N invocation.
type Report ¶
type Report struct {
SchemaVersion int `json:"schema_version"`
AttemptsID string `json:"attempts_id,omitempty"`
Requested int `json:"attempts_requested"`
Winner *int `json:"winner"`
WinnerBranch string `json:"winner_branch,omitempty"`
StopReason string `json:"stop_reason"`
StopOnPass bool `json:"stop_on_pass"`
Keep string `json:"keep"`
Attempts []Attempt `json:"attempts"`
TotalCostUSD float64 `json:"total_cost_usd"`
TotalWallMS int64 `json:"total_wall_time_ms"`
}
Report is the attempts report (<stem>.attempts.json).