Help us shape the future of Go! Take the 2026 Go Developer Survey

attempts

package
v0.10.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 30, 2026 License: MIT Imports: 16 Imported by: 0

Documentation

Overview

Package attempts runs one task up to N times (best-of-N), each attempt as a child tinyagent run in its own Git worktree, grades every finished attempt with user-configured check commands, and picks a verified winner: the lowest-numbered attempt whose run ended Submitted or Completed and whose checks all exited 0.

Attempts run sequentially and, by default, stop at the first pass (RunAll runs every attempt). tinyagent never commits, merges or pushes an attempt. Every worktree is kept unless Keep asks to remove non-winners once an attempt wins.

Index

Constants

View Source
const (
	DefaultCheckTimeout = 600
	MaxCheckTimeout     = 3600
)

DefaultCheckTimeout and MaxCheckTimeout bound one check, in seconds.

View Source
const (
	StopPassed    = "passed"
	StopExhausted = "exhausted"
	StopBudget    = "budget exhausted"
	StopInterrupt = "interrupted"
	StopError     = "error"
	// StopCompleted: RunAll ran every attempt and at least one passed.
	StopCompleted = "completed"
)

Stop reasons recorded in the report.

View Source
const (
	KeepAll    = "all"
	KeepFailed = "failed"
	KeepNone   = "none"
)

Keep modes: which non-winning worktrees survive once an attempt wins.

View Source
const (
	EventAttemptStarted = "attempt.started"
	EventAttemptCheck   = "attempt.check"
	EventAttemptResult  = "attempt.result"
	EventAttemptsResult = "attempts.result"
)

Event types written to Options.Events (--json with --attempts).

View Source
const (
	EnvAttemptsID = "TINYAGENT_ATTEMPTS_ID"
	EnvAttempt    = "TINYAGENT_ATTEMPT"
	EnvAttempts   = "TINYAGENT_ATTEMPTS"
)

Environment variables a parent --attempts run sets on each child so the child's log DB row and OpenTelemetry spans can be tied back to the group.

View Source
const MaxAttempts = 4

MaxAttempts is the largest --attempts value.

View Source
const MaxCheckOutput = 8 << 10

MaxCheckOutput caps the check output kept in the report (the tail is kept).

View Source
const SchemaVersion = 1

SchemaVersion of the attempts report.

Variables

This section is empty.

Functions

func ExecAgent

func ExecAgent(exe string) func(context.Context, []string, io.Writer, io.Writer) error

ExecAgent returns a RunAgent that runs exe as a child process. On context cancellation the child gets an interrupt so it can end Interrupted and save its trajectory.

func LinkEnv

func LinkEnv(base []string, l Link) []string

LinkEnv returns base with l's variables set, replacing any inherited attempt variables (and TRACEPARENT/TRACESTATE when l carries one).

func NewID

func NewID() string

NewID returns a random attempts group id (32 hex characters).

func RemoveWorktree

func RemoveWorktree(repoRoot, path, branch string) error

RemoveWorktree deletes the worktree at path and its branch from the repository at repoRoot, discarding the worktree's uncommitted changes.

func ReportPath

func ReportPath(out string) string

ReportPath returns the attempts report path for the -o path out.

func TrajectoryPath

func TrajectoryPath(out string, i int) string

TrajectoryPath returns attempt i's trajectory path for the -o path out.

Types

type Attempt

type Attempt struct {
	Attempt      int           `json:"attempt"`
	Model        string        `json:"model"`
	Status       string        `json:"status"`
	Passed       bool          `json:"passed"`
	Steps        int           `json:"steps"`
	InputTokens  int           `json:"input_tokens"`
	OutputTokens int           `json:"output_tokens"`
	CostUSD      float64       `json:"cost_usd"`
	WallTimeMS   int64         `json:"wall_time_ms"`
	Worktree     string        `json:"worktree"`
	Branch       string        `json:"branch"`
	FilesChanged int           `json:"files_changed"`
	Trajectory   string        `json:"trajectory,omitempty"`
	Checks       []CheckResult `json:"checks"`
	Error        string        `json:"error,omitempty"`
	// Removed is true when Keep cleanup deleted the worktree and branch.
	Removed bool `json:"removed,omitempty"`
	// contains filtered or unexported fields
}

Attempt is one attempt's entry in the report.

type Check

type Check struct {
	Command        string `json:"command"`
	TimeoutSeconds int    `json:"timeout_seconds"`
}

Check is one verification command, run with bash -c in the attempt's working directory.

type CheckResult

type CheckResult struct {
	Command    string `json:"command"`
	ExitCode   int    `json:"exit_code"`
	DurationMS int64  `json:"duration_ms"`
	TimedOut   bool   `json:"timed_out"`
	Skipped    bool   `json:"skipped"`
	Output     string `json:"output"`
}

CheckResult is one check's outcome for one attempt.

type CheckRunner

type CheckRunner interface {
	Command(ctx context.Context, root, command string) *exec.Cmd
}

CheckRunner builds an unstarted `bash -c command` in root. Cancelling ctx must kill the command and everything it spawned.

type Link struct {
	ID          string // attempts group id, shared by every attempt
	Index       int    // 1-based attempt number
	Count       int    // attempts requested
	TraceParent string // W3C traceparent for the child's root span, or ""
}

Link identifies one attempt within an attempts group.

func LinkFromContext

func LinkFromContext(ctx context.Context) (Link, bool)

LinkFromContext returns the attempt link runAttempt put on ctx, if any.

func LinkFromEnv

func LinkFromEnv(getenv func(string) string) (Link, bool)

LinkFromEnv reads the link a parent --attempts run set; ok is false when this process is not an attempt.

type Options

type Options struct {
	// Models holds one model spec per attempt; its length is N.
	Models []string
	Checks []Check
	// BaseName names the worktrees: attempt i uses <BaseName>-a<i>.
	BaseName string
	// Output is the -o path. Attempt i's trajectory goes to <stem>.a<i><ext>
	// and the report to <stem>.attempts<ext>. Empty means trajectories go to
	// a temporary directory and no report file is written.
	Output    string
	CostLimit float64 // invocation-wide USD budget; 0 = none
	// TokenLimit is the invocation-wide input+output token budget; 0 = none.
	TokenLimit int
	TimeLimit  time.Duration // invocation-wide wall-clock budget; 0 = none
	// ExtraArgs are forwarded to every child run (the task among them).
	ExtraArgs []string
	Stderr    io.Writer
	// Stdout receives the winner's stdout (its submission) at the end.
	Stdout io.Writer
	// Redact masks secrets in check output; nil leaves it unchanged.
	Redact func(string) string
	// RunAgent runs one child tinyagent with args. Its stdout is buffered
	// per attempt; its stderr is passed through.
	RunAgent func(ctx context.Context, args []string, stdout, stderr io.Writer) error
	// Now is the clock; nil means time.Now.
	Now func() time.Time
	// RunAll runs every attempt instead of stopping at the first pass. The
	// lowest-numbered passing attempt still wins.
	RunAll bool
	// Keep is KeepAll (or empty), KeepFailed or KeepNone. With a winner,
	// KeepFailed removes passing non-winners and KeepNone removes every
	// non-winner through Remove. Nothing is removed without a winner.
	Keep string
	// Remove deletes one attempt's worktree and branch. Nil disables
	// cleanup.
	Remove func(a Attempt) error
	// Events, when non-nil, receives JSONL events (--json): each child
	// run's stdout events tagged with "attempt", plus attempt.started,
	// attempt.check, attempt.result and one final attempts.result. The
	// winner's stdout is then not replayed to Stdout.
	Events io.Writer
	// AttemptsID, when set, links the children to this group: each child
	// gets it (and its attempt number) in its environment, and it is
	// recorded in the report and events.
	AttemptsID string
	// TraceParent, when set with AttemptsID, is passed to each child as
	// TRACEPARENT so every attempt's spans share the parent's trace.
	TraceParent string
	// CheckEnv, when non-nil, prepares where one attempt's checks run
	// (the run's sandbox, container or command_env) for the attempt's
	// working directory, and returns a release function. An error fails
	// every check of that attempt (fail closed); nil runs checks on the
	// host with tinyagent's environment.
	CheckEnv func(ctx context.Context, root string) (CheckRunner, func(), error)
}

Options configures one best-of-N invocation.

type Report

type Report struct {
	SchemaVersion int       `json:"schema_version"`
	AttemptsID    string    `json:"attempts_id,omitempty"`
	Requested     int       `json:"attempts_requested"`
	Winner        *int      `json:"winner"`
	WinnerBranch  string    `json:"winner_branch,omitempty"`
	StopReason    string    `json:"stop_reason"`
	StopOnPass    bool      `json:"stop_on_pass"`
	Keep          string    `json:"keep"`
	Attempts      []Attempt `json:"attempts"`
	TotalCostUSD  float64   `json:"total_cost_usd"`
	TotalWallMS   int64     `json:"total_wall_time_ms"`
}

Report is the attempts report (<stem>.attempts.json).

func Run

func Run(ctx context.Context, o Options) (Report, int)

Run executes the attempts and returns the report and the exit code: 0 when an attempt passed, 1 otherwise.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL