externagent

package
v0.1.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 15, 2026 License: Apache-2.0 Imports: 34 Imported by: 0

Documentation

Overview

Package externagent runs another agent's command-line harness as a backend, so a run can be driven by an external agent CLI (installed on the host, authenticated on its own subscription) instead of a single model call. The external tool owns its own inner loop: handed one turn, it runs a whole agentic episode (many inner model calls, its own tool calls and retries) before it yields. That is the opposite of a model port, which is one request in and one response out, so an external agent is a loop strategy, not a model.

The external tool must never be the effector. Its native execution surface is locked down (a read-only sandbox, native writes and shell denied), and the agent's own tools are offered to it through a loopback Model Context Protocol bridge the run hosts. Every tool call the external agent makes comes back through that bridge and is admitted, contained, braked, and recorded at the same dispatch waist as a native loop. So swapping in another harness never widens what a run may do or escapes a halt; the waist is outside the loop and applies to every action, whichever harness took it.

What an external harness cannot make observable is recorded as a gap rather than hidden: its inner model calls, the context it compacts, and its direct channel to its own provider are outside the run's tracing. Each projected action carries a provenance Tier saying how strongly the record can vouch for it, so a run driven by an external agent never claims the integrity of a native run.

One Adapter describes one external CLI; the codex adapter is the first. The runner, the bridge, and the governance are shared, so a second CLI is a new Adapter, not a new subsystem.

Index

Constants

View Source
const ProbeToolName = "conformance_check"

ProbeToolName is the bridged tool a session's reachability probe calls. It is named for what it does rather than for the run that serves it, so the external harness sees no fingerprint of who is governing it.

Variables

View Source
var ErrProgramNotFound = errors.New("externagent: cli not found on PATH")

ErrProgramNotFound reports that an external agent CLI is not installed, as opposed to installed but unreachable from inside the confinement. The two need different fixes (install it, versus grant the confined child access to it), so they are different errors and the onboarding message for each says something the user can act on.

Functions

func ContinueEpisode added in v0.1.3

func ContinueEpisode(status goal.Status, text string) (goal.Status, error)

ContinueEpisode reopens a settled external goal for another turn: it clears the episode's completion so the next reconcile runs a new one, sets the turn to put to the harness, and keeps the conversation id so the CLI continues the session it already holds rather than starting cold. It mirrors what a native loop does by appending a user message to its transcript; here the transcript is the CLI's, and this is the handle to it.

The goal's objective is left alone: it records what the run set out to do, and the turn is not a new objective.

func Instructions added in v0.1.3

func Instructions(probes []Probe) []string

Instructions renders the probes' instructions for the episode's preamble.

Types

type Adapter

type Adapter interface {
	// Name is the CLI's stable identifier (for example "codex"), used in the model
	// spec that selects it and recorded on the run.
	Name() string
	// Detect probes whether the CLI is installed, logged in, and new enough to be
	// constrained. It runs the CLI's own version and auth probes and never starts an
	// episode.
	Detect(ctx context.Context) (Readiness, error)
	// Command builds the subprocess invocation for one episode: the argv that locks
	// the CLI's native execution down (read-only, native effects denied) and points
	// its MCP client at the bridge, plus how the turn and the final message are
	// carried. It does not run anything.
	Command(ep Episode) (Invocation, error)
	// Parse turns one line of the CLI's stdout into zero or more typed events. A line
	// it does not recognize yields an attested progress event rather than an error, so
	// an unfamiliar line is recorded, not dropped.
	Parse(line []byte) ([]Event, error)
}

Adapter describes how to drive one external agent CLI as a subprocess. The codex adapter is the first; the same port fits any CLI that runs an episode as a child process and reports it as a stream of lines. An implementation must be safe for concurrent use.

type Bridge

type Bridge struct {
	// Name is the MCP server name the CLI registers the bridge under. Tool names the
	// CLI reports may be namespaced by it, which the driver maps back to real tool
	// identities for capability matching.
	Name string
	// URL is the streamable-HTTP endpoint the CLI connects to, on the loopback
	// interface. It is the address the child uses, which is not always the address the
	// bridge listens on: where the child runs in its own network namespace, this is an
	// in-namespace address the sandbox forwards to the listening one (see ForwardTo).
	URL string
	// ForwardTo is the host-loopback address the sandbox forwards the child's URL to when
	// the child cannot reach the host loopback directly (a separate network namespace). It
	// is empty when the child reaches URL directly (a shared network stack), and is set by
	// the runner from what the spawner reports, so the spawner knows to stand up the forward.
	ForwardTo string
	// Token is the bearer token the CLI must present, so another local process cannot
	// drive the bridge. The adapter passes it to the CLI through an environment
	// variable rather than an argument, keeping it out of the process table.
	Token string
	// TokenEnv is the environment variable the CLI reads the bearer token from.
	TokenEnv string
}

Bridge is the loopback endpoint an episode points the external CLI at, so the CLI's tool calls come back through the dispatch waist. It is a streamable-HTTP MCP server the run hosts on the local host for the life of the episode.

type Claude added in v0.1.3

type Claude struct {
	// contains filtered or unexported fields
}

Claude drives the claude CLI (Claude Code) as an external agent backend over its headless print mode, which streams an episode as JSON lines. Unlike codex, claude has no native filesystem-sandbox flag; its native execution surface is locked down by tool denial instead. The effector tools (shell, edit, write, web) are denied and the permission mode never prompts and never auto-approves, so the only tools the model can act through are the bridged ones offered over the loopback MCP bridge. This is defense in depth: Flynn's own sandbox spawner is the boundary that actually confines the child, and it holds regardless of whether the CLI honors the flags.

func NewClaude added in v0.1.3

func NewClaude(bin string, spawner Spawner) *Claude

NewClaude builds the claude adapter. bin overrides the executable to run (empty uses "claude" resolved on PATH); spawner runs the detection probes through the sandbox boundary and may be nil when the adapter is used only for Command and Parse.

func (*Claude) Command added in v0.1.3

func (c *Claude) Command(ep Episode) (Invocation, error)

Command builds the claude headless invocation for one episode. It runs in print mode with the stream-json event format, points claude's MCP client at the loopback bridge over HTTP with the bearer token carried in the environment (not on the command line), and locks the native surface down by tool denial: the effector tools are denied and the permission mode never prompts and never auto-approves, so effects must reach the workspace through the bridge. The turn is written on stdin; a standing instruction is prepended as a lower-authority preamble, since claude's own harness prompt outranks anything injected.

func (*Claude) Detect added in v0.1.3

func (c *Claude) Detect(ctx context.Context) (Readiness, error)

Detect probes that claude is installed, logged in on a subscription, and new enough to be constrained to the bridge. It runs claude's own version, help, and auth-status probes and never starts an episode. A missing binary or a build without the lockdown knobs (headless print, the stream-json event format, the HTTP MCP client, and the tool and permission controls) is a hard refusal, so the driver stops rather than running claude with unattested effects; a healthy but logged-out CLI is a recoverable onboarding prompt.

func (*Claude) Name added in v0.1.3

func (*Claude) Name() string

Name identifies the adapter in the model spec and on the run.

func (*Claude) Parse added in v0.1.3

func (c *Claude) Parse(line []byte) ([]Event, error)

Parse projects one claude stream-json line to typed events. The init and progress events become attested progress; an assistant message projects each of its blocks (text as attested text, a bridged tool call as a bridge call, a native tool call as a native command); a tool result is progress, since the bridged call it answers is enforced and recorded at the dispatch waist; the terminal result event becomes the episode's final text plus a done event carrying the total usage, or an error event when it reports a failure. A line it cannot decode becomes an attested progress event carrying the raw line, so nothing is dropped and noise does not end the episode.

type Codex

type Codex struct {
	// contains filtered or unexported fields
}

Codex drives the codex CLI as an external agent backend over its non-interactive exec mode, which streams an episode as JSON lines and writes the final message to a file. It is constrained to a read-only native sandbox with native approvals denied, so the model cannot write or run a command directly; its effects reach the workspace only through the loopback MCP bridge, where they are governed.

func NewCodex

func NewCodex(bin string, spawner Spawner) *Codex

NewCodex builds the codex adapter. bin overrides the executable to run (empty uses "codex" resolved on PATH); spawner runs the detection probes through the sandbox boundary and may be nil when the adapter is used only for Command and Parse.

func (*Codex) Command

func (c *Codex) Command(ep Episode) (Invocation, error)

Command builds the codex exec invocation for one episode. It pins the read-only sandbox and denies native approvals so codex cannot write or run a command itself, points codex's MCP client at the bridge over streamable HTTP with the bearer token carried in the environment (not on the command line), and reads the final message from a file rather than the event stream. The turn is written on stdin; a standing instruction is prepended as a lower-authority preamble, since codex's own harness prompt outranks anything injected.

func (*Codex) Detect

func (c *Codex) Detect(ctx context.Context) (Readiness, error)

Detect probes that codex is installed, logged in, and new enough to be constrained to the bridge. It runs codex's own version, auth, and help probes and never starts an episode. A missing binary or a build without the lockdown knobs (the read-only sandbox, the JSON event stream, the streamable-HTTP MCP client) is a hard refusal, so the driver stops rather than running codex with unattested effects; a healthy but logged-out CLI is a recoverable onboarding prompt.

func (*Codex) Name

func (*Codex) Name() string

Name identifies the adapter in the model spec and on the run.

func (*Codex) Parse

func (c *Codex) Parse(line []byte) ([]Event, error)

Parse projects one codex exec --json line to typed events. Recognized boundaries (thread and turn start) become attested progress; a reported error or a failed turn becomes an error event, terminal when the message names a permanent condition; a completed turn becomes a done event carrying any usage; an assistant message item becomes attested text. A line it cannot decode becomes an attested progress event carrying the raw line, so nothing is dropped and noise does not end the episode.

type ConformanceReport added in v0.1.3

type ConformanceReport struct {
	Results  []ProbeResult
	Steering Steering
}

ConformanceReport is the outcome of every probe run against an episode, plus the steering counts observed while it ran.

func (ConformanceReport) Failed added in v0.1.3

func (r ConformanceReport) Failed() []ProbeResult

Failed lists the probes the harness did not comply with.

func (ConformanceReport) Refused added in v0.1.3

func (r ConformanceReport) Refused() bool

Refused reports whether a required probe failed, meaning the session must not proceed on the assumption that the harness honors the contract.

func (ConformanceReport) Summary added in v0.1.3

func (r ConformanceReport) Summary() string

Summary renders the report for the live trace and the record: which probes failed and how the harness chose its tools.

type Driver

type Driver struct {
	// contains filtered or unexported fields
}

Driver adapts an external agent CLI to the driver.Driver port, so an external harness is a selectable run loop like any other: an additive registry entry, no edit to the default loop. Its loop shape is one CLI episode per goal step. It composes with governance rather than replacing it: the same grant, containment, brake, and event spine that bound a native loop bound every tool call the external harness makes, because those sit at the dispatch waist outside the loop.

Construct it with the adapter for the CLI and the spawner that runs the CLI under the sandbox's confinement. The workspace is where the CLI reads and where its bridged effects land.

func NewDriver

func NewDriver(adapter Adapter, spawner Spawner, workdir string) *Driver

NewDriver builds the driver for one external CLI. spawner runs the CLI confined; workdir is the directory the episode operates in (the CLI reads it and its bridged tools act in it).

func (*Driver) Build

Build assembles the episode loop from the Spec. It captures the governance ingredients (tools, grant, sandbox gate, brake, event sink, reporter) and defers the wiring to each step, so a step's bridge is scoped and attributed to the goal it runs under. The Spec's Model (an llm.Model) is intentionally unused: an external harness drives its own model, selected by the goal's model string, not a model port.

func (*Driver) Close added in v0.1.3

func (d *Driver) Close() error

Close releases what the run's episodes shared: the harness's credential-and-state home, where the CLI kept the conversation this run's turns continued. It is called when the run ends. A spawner that holds nothing (a test fake) closes to nothing.

func (*Driver) Drift added in v0.1.3

func (d *Driver) Drift() map[string]int

Drift returns how many episodes failed each conformance probe, by probe name. An empty map means the harness honored the session contract on every episode.

func (*Driver) Name

func (d *Driver) Name() string

Name is the adapter's identifier, the name this loop is selected by and recorded under on the run.

func (*Driver) SetRecorder added in v0.1.3

func (d *Driver) SetRecorder(r Recorder)

SetRecorder binds the sink the harness's attested events are recorded to. The host calls it once, after the run's stream exists and before the first episode runs; the driver is constructed earlier (detection happens before a run is assembled), which is why this is a setter rather than a constructor argument.

func (*Driver) Steering added in v0.1.3

func (d *Driver) Steering() Steering

Steering returns the run's tool-choice counts across every episode: how often the harness used the bridged tools it was told to use, and how often it reached for its own instead. It is the number the tool descriptions and the preamble are tuned against.

func (*Driver) Tiers added in v0.1.3

func (d *Driver) Tiers() map[Tier]int

Tiers returns a copy of the provenance-tier tally of every event the run's episodes projected: how many actions the record vouches for at each tier. The host reads it after the run to declare the tier mix on the sealed record. Bridged effects are not counted here; they are recorded at the dispatch waist as they happen.

func (*Driver) Unrecorded added in v0.1.3

func (d *Driver) Unrecorded() (int, error)

Unrecorded reports how many of the harness's attested events could not be written to the record, and the last failure. Zero and nil mean the record carries the harness's whole account.

type Episode

type Episode struct {
	Input   string
	Workdir string
	Model   string
	System  string
	Bridge  Bridge
	// Probes are conformance instructions folded into the turn the harness reads. Their
	// compliance is checked against the episode's own event stream once it finishes.
	Probes []string
	// Session, when set, is a conversation the CLI already holds (a Session an earlier
	// episode reported), which this episode continues instead of opening a fresh one. It
	// is how an interactive session gives the harness the context of its own earlier
	// turns: the CLI kept that conversation, and only the CLI can. Empty starts a new
	// conversation. An adapter whose CLI cannot continue a conversation ignores it, so
	// the turn still runs, with only this turn's input.
	Session string
}

Episode is one turn handed to an external CLI: the user input to act on, where to act, which model to drive, the standing instruction to layer in, and the bridge to route effects through. The system instruction lands as a lower-authority layer because the CLI's own harness prompt outranks anything injected, so a behavioral contract is a request to an external harness, not a guarantee.

type Event

type Event struct {
	Kind     EventKind
	Text     string
	Usage    Usage
	Err      string
	Terminal bool // for EventError: the failure is terminal, not worth retrying
	Tier     Tier
	Raw      json.RawMessage

	// Tool is the tool a bridge call named (EventBridgeCall), and Server the bridge it
	// named it on. A call to another MCP server is not ours and is not counted as bridged.
	Tool   string
	Server string
	// Args is the arguments a bridge call carried, verbatim, so a conformance probe can
	// look for the nonce it asked the harness to echo.
	Args json.RawMessage
	// Command is the command line the CLI reported running natively (EventNativeCommand).
	Command string
	// Status is the CLI's terminal status for the call or command it reported: completed,
	// failed, declined, or in_progress. A started call and its completion are two events,
	// so counting one status avoids double counting.
	Status string
	// Session is the conversation id the CLI announced for this episode, when it
	// announces one. Both bundled harnesses open their stream with it (claude's init
	// event, codex's thread.started), and both accept it back to continue that same
	// conversation. Carrying it is what lets a multi-turn session hand the CLI its own
	// context instead of replaying a transcript the CLI never wrote.
	Session string
}

Event is one typed projection of an episode's output line. The runner forwards it to the reporter and, with its Tier, to the record. Raw preserves the original CLI line for the attested record, so the CLI's own account is kept verbatim alongside the typed projection.

type EventKind

type EventKind int

EventKind is the sort of thing an episode's output line projects to.

const (
	// EventProgress is an intermediate signal from the CLI with no effect of its own
	// (a thread or turn boundary, a status line). It is attested.
	EventProgress EventKind = iota
	// EventText is assistant-visible text the CLI produced. It is attested: the run
	// did not generate it and cannot attest the context that produced it.
	EventText
	// EventUsage carries token accounting the CLI reported for the episode.
	EventUsage
	// EventError is a failure the CLI reported (a provider error, a turn failure). Its
	// Class distinguishes a terminal failure from a transient one for retry.
	EventError
	// EventDone marks the episode's own completion (the turn finished), carrying any
	// final usage. It is distinct from EventError, which ends an episode abnormally.
	EventDone
	// EventBridgeCall is the CLI's report of a tool call it made on the loopback bridge.
	// The call itself is enforced at the dispatch waist, which records it independently;
	// this event is the CLI's own account of the same call, used to measure how the
	// harness chose its tools.
	EventBridgeCall
	// EventNativeCommand is the CLI's report that it ran a command or edited a file with
	// its own built-in tools rather than the bridged ones. Its effects are contained (the
	// CLI runs read-only with native approvals denied), but the run did not observe them,
	// so a run producing these is being steered poorly, not breached.
	EventNativeCommand
)

func (EventKind) String added in v0.1.3

func (k EventKind) String() string

String names the event kind for the record. The values are a wire contract the record's readers match on, so they must not change.

type Invocation

type Invocation struct {
	Path            string
	Args            []string
	Env             []string
	Stdin           string
	LastMessageFile string
}

Invocation is the subprocess to run for one episode: the program to exec, its arguments, environment additions (KEY=VALUE, merged onto a minimal base by the runner), and the input to write on stdin when the CLI reads the turn from stdin rather than an argument. LastMessageFile, when set, is the path the CLI writes its final assistant message to, which the runner reads as the episode's result rather than reconstructing it from the event stream.

type Probe added in v0.1.3

type Probe struct {
	// Name identifies the probe on the record and in the drift report.
	Name string
	// Instruction is the text handed to the harness. It must ask for something observable
	// in the event stream, since a probe that cannot fail proves nothing.
	Instruction string
	// Required marks a probe whose failure makes the session unsafe to trust rather than
	// merely degraded. A required probe that fails refuses the run.
	Required bool
	// contains filtered or unexported fields
}

Probe is one checkable instruction. Instruction is folded into the episode's opening preamble; the probe then watches the episode's projected events as they stream and reports whether the harness complied.

A probe is evaluated as a match over the stream rather than a pass over a retained slice, so an episode's events are never held in memory: a bridged tool result echoed in the CLI's output can be megabytes, and a long episode produces many of them.

func SessionProbes added in v0.1.3

func SessionProbes(tool *ProbeTool, continuing bool) []Probe

SessionProbes are the probes an episode runs under: the harness must reach the bridge (settled on the tool's own record of being called), and it must not reach for its own tools (advisory, read off the event stream).

continuing marks an episode that continues a conversation the harness already holds (a later turn of an interactive session). Reachability is a property of this run's harness and bridge, and the episode that opened the conversation already proved it, so on a later turn the probe is evidence rather than a gate: a harness that skips it is recorded as drift instead of having the turn refused. Requiring it every turn would refuse any turn that needs no tools at all, which in a conversation is most of them, and a model that has already performed the ritual once in this same conversation will reasonably decline to repeat it. Nothing about containment rests on the repeat: a bridged call is still admitted and recorded at the dispatch waist, and a native one is still contained and counted against the harness's steering.

type ProbeResult added in v0.1.3

type ProbeResult struct {
	Name     string
	Passed   bool
	Required bool
}

ProbeResult is one probe's outcome.

type ProbeTool added in v0.1.3

type ProbeTool struct {
	// contains filtered or unexported fields
}

ProbeTool is an effect-free bridged tool whose only purpose is to prove that the external harness can reach the run's bridge and will use it when told to. It grants nothing: it touches no filesystem, runs no command, and reaches no network. Adding it to a run's grant therefore widens the harness's authority by exactly nothing, which is what makes it safe to admit unconditionally for the probe.

Its value is that a call must cross the dispatch waist to arrive here. A harness that writes "I called conformance_check" into its output stream without dispatching does not move this tool, so the probe it settles cannot be passed by narration.

func NewProbeTool added in v0.1.3

func NewProbeTool(nonce string) *ProbeTool

NewProbeTool builds the probe tool for one episode. nonce must be unique per episode, so a harness cannot satisfy the probe by replaying an argument it saw earlier.

func (*ProbeTool) Called added in v0.1.3

func (p *ProbeTool) Called() bool

Called reports whether the harness dispatched a call carrying the episode's nonce.

func (*ProbeTool) Def added in v0.1.3

func (*ProbeTool) Def() llm.Tool

Def describes the tool to the harness. The description states the contract the probe is testing, since the description is the only channel that reaches the harness's own tool-selection reasoning.

func (*ProbeTool) Invoke added in v0.1.3

func (p *ProbeTool) Invoke(_ context.Context, input json.RawMessage) (string, error)

Invoke records that the harness reached the bridge with the right nonce. A call with a wrong or missing nonce is answered but not counted: it proves the channel is open, not that the harness followed the instruction it was given, and the probe is testing the latter.

It is safe for concurrent use: the bridge serves the external subprocess, which may dispatch more than one call at a time.

func (*ProbeTool) Name added in v0.1.3

func (*ProbeTool) Name() string

Name is the tool name the harness calls and the action the waist admits.

type Process

type Process interface {
	Stdout() io.Reader
	Wait() error
}

Process is a running episode subprocess: its stdout stream and a wait handle. Cancellation is the context's job (Start binds the process to it), so there is no separate kill; Wait returns once the process ends, including after a context-driven kill.

type Program added in v0.1.3

type Program struct {
	// Path is the absolute path of the native executable to launch.
	Path string
	// ReadableDirs are the directories the confined child is granted read and execute
	// access on so it can load the executable and the helper binaries shipped beside it.
	ReadableDirs []string
}

Program is a resolved external agent CLI: the executable to launch and the directories a confined child must be able to read in order to run it.

The path is always absolute and always the real executable, never a launcher script. A confined child gets read access to nothing outside its workspace by default, so a script that shells out to an interpreter would need that interpreter's directory granted too, and an interpreter installed under a system-owned directory cannot be granted at all by an unprivileged process. Resolving to the native executable removes the interpreter from the picture, so the only directory that has to be reachable is the one the CLI ships in, which lives under the user's own install tree.

func LocateClaude added in v0.1.3

func LocateClaude(bin string) (Program, error)

LocateClaude resolves the claude CLI (Claude Code) to the native executable to launch and the directories a confined child needs to read to run it. bin names the CLI (empty means "claude" on PATH) or gives an absolute path to it.

Claude Code ships a single self-contained native binary per platform, fronted by an npm launcher (a .cmd shim on Windows, an sh shim or symlinked entry point elsewhere) that execs it. Launching the launcher would drag its shell or interpreter into the confinement, so this resolves straight to the native binary the launcher stands for. When the CLI is installed some other way that already puts a native binary on PATH, the resolved path is used as is.

A CLI that is not installed yields ErrProgramNotFound. A launcher whose native binary cannot be found yields a distinct error naming the path that was looked for, because that is a broken or unfamiliar installation rather than a missing one.

func LocateCodex added in v0.1.3

func LocateCodex(bin string) (Program, error)

LocateCodex resolves the codex CLI to the native executable to launch and the directories a confined child needs to read to run it. bin names the CLI (empty means "codex" on PATH) or gives an absolute path to it.

The npm distribution installs a launcher (a .cmd shim on Windows, a symlinked .js entry point elsewhere) that finds and executes a vendored native binary for the host's platform. Launching the launcher would drag its interpreter into the confinement, so this resolves straight to the vendored binary. When the CLI is installed some other way (a package manager that puts a native binary on PATH) the resolved path is already native and is used as is.

A CLI that is not installed yields ErrProgramNotFound. A launcher whose vendored binary cannot be found yields a distinct error naming the path that was looked for, because that is a broken or unfamiliar installation rather than a missing one.

type Readiness

type Readiness struct {
	// Available is true when the CLI is installed and answered a version probe.
	Available bool
	// LoggedIn is true when the CLI holds usable credentials (a subscription session
	// or an API key), so an episode would not stall on an auth prompt.
	LoggedIn bool
	// Version is the CLI's reported version, recorded on the run and used to refuse a
	// build too old to constrain.
	Version string
	// Reason is a one-line, actionable message when the CLI is not ready to run (for
	// example, an instruction to log in). It is empty when Ready is true.
	Reason string
	// Refuse marks a hard refusal the driver must not start on (a too-old CLI, a
	// missing lockdown knob) as distinct from a recoverable onboarding prompt (not yet
	// logged in). A refusal is terminal; an onboarding prompt is not.
	Refuse bool
}

Readiness reports whether an external agent CLI can start an episode, and if not, why. The driver consults it before every session: a CLI that is not installed or not logged in yields an actionable Reason, and a CLI that cannot be constrained to route its effects through the bridge (too old to offer the sandbox, approval, or MCP knobs the lockdown needs) sets Refuse, so the driver stops rather than running an external harness with unattested effects.

func (Readiness) Ready

func (r Readiness) Ready() bool

Ready reports whether an episode can start: the CLI is installed, logged in, and not under a hard refusal.

type Recorder added in v0.1.3

type Recorder interface {
	// Record persists one attested event. An error is reported by the caller and does not
	// fail the episode: losing the harness's account of a line is a gap in the record, not
	// a reason to abandon a run whose effects are still enforced and recorded.
	Record(ctx context.Context, ev Event) error
}

Recorder persists the harness's own account of its episode: each event the episode projected, with its tier and the CLI's original line. It is a narrow port because the only thing this package knows about the record is that one exists; the host binds an implementation that writes to the run's event stream.

The dispatch waist records the effects the run enforced. This records what the harness said it was doing, so a reader can hold the two accounts side by side and see where they part. A nil Recorder records nothing, leaving the run's stream as it was.

type Result

type Result struct {
	Text     string
	Usage    Usage
	Failed   bool
	Err      string
	Terminal bool
	Tiers    map[Tier]int
	// Conformance is how the harness answered the episode's probes and how it chose its
	// tools. It is the evidence that the run's instructions took, rather than the
	// assumption that they did.
	Conformance ConformanceReport
	// Session is the conversation id the CLI announced, empty when it announced none.
	// Handing it back on a later Episode continues that conversation, which is how a
	// multi-turn session keeps the harness's own context across turns.
	Session string
}

Result is the outcome of one episode: the final assistant message, the token usage the CLI reported, whether it failed and why, and the provenance-tier tally of the projected events. Reasoning is not in the tally because it is unobservable with an external harness in the loop; the record names that gap separately.

type Runner

type Runner struct {
	// contains filtered or unexported fields
}

Runner drives one external-agent episode end to end: it hosts the loopback MCP bridge over a local streamable-HTTP port, spawns the CLI constrained to route its effects through that bridge, projects the CLI's output stream to typed events, and tears everything down. The bridge is served on the governed context passed to Run, so every tool call the CLI makes is admitted, contained, braked, and recorded at the dispatch waist; cancelling that context (a halt or shutdown) both refuses further tool calls and kills the subprocess.

func NewRunner

func NewRunner(adapter Adapter, server *mcp.Server, spawner Spawner, report func(Event)) *Runner

NewRunner builds a runner that drives adapter, hosting server as the bridge, spawning the episode subprocess through spawner (the sandbox-confined process host), and forwarding each projected event to report (nil drops them).

func (*Runner) Run

func (r *Runner) Run(ctx context.Context, ep Episode) (Result, error)

Run hosts the bridge, runs one episode of ep, and returns its Result. ctx must carry the run's governance bindings (the grant and the brake) so bridged calls are governed; cancelling ctx halts the episode and kills the subprocess. Run returns a non-nil error only on a failure to start or host the episode (a bad invocation, a bridge that will not bind); an episode that runs and the CLI reports as failed is a completed Run with Failed set on the Result.

func (*Runner) WithProbes added in v0.1.3

func (r *Runner) WithProbes(probes []Probe) *Runner

WithProbes sets the conformance probes the runner watches the episode against. Their instructions must also be folded into the episode (see Episode.Probes); a probe whose instruction was never sent measures nothing but the model's guesswork.

type SandboxConfig

type SandboxConfig struct {
	// AllowedHosts is the destination-name allowlist for an episode's egress: the
	// external provider's API and auth endpoints, the only names the confined child may
	// reach out to. It is a name gate on the egress proxy ("deny all egress except these
	// providers"): a name not on the list is denied, and a listed name that resolves to a
	// private or rebinding address is denied too (the address gate still applies). An
	// entry beginning with a dot (".example.com") matches any subdomain. The loopback MCP
	// bridge is not listed here: it is not reached through the egress proxy at all. Where the
	// child shares the host's network stack it dials the bridge's host loopback directly;
	// where the child is confined to its own network namespace it dials an in-namespace
	// address the sandbox forwards to the bridge (see the loopback forward in Start). Either
	// way the bridge stays reachable while the internet does not. An empty list permits no
	// egress at all beyond the bridge (deny-all), enough for an offline detection run but not
	// a live episode. These are supplied by the adapter, so no provider name is baked into
	// this generic host.
	AllowedHosts []string
	// AuthDir is the external CLI's credential and config home (its OAuth token lives
	// there), which sits outside the episode workspace. The confined child is granted
	// read (and traverse) on it for the life of the episode and the grant is revoked on
	// teardown, so the credential stays in its home directory and is never copied into the
	// workspace, the vault, or the record. Empty grants no extra read. On Linux and macOS
	// a read-only host already permits the read, so this takes effect only where the
	// confinement denies reads by default (a Windows AppContainer).
	AuthDir string
	// AuthEnv is the environment variable the external CLI reads to find its credential
	// and config home (CODEX_HOME for codex). The confined child inherits none of the
	// host's environment, so it has no HOME or USERPROFILE to derive that home from and
	// would look for its credentials somewhere that does not exist, reporting itself
	// logged out on a host where it is logged in. Granting AuthDir makes the directory
	// readable; naming it here is what makes the child look in it. Empty passes no
	// variable.
	AuthEnv string
	// AuthSeedPaths names individual source files, by absolute path, that together make up
	// the CLI's credential and config home. It is the multi-source counterpart of AuthDir
	// plus AuthSeedFiles: where those copy named files out of one directory, this gathers
	// files that live in different directories (or under a home the confined child has no
	// way to derive) into one directory holding exactly their base names and nothing else.
	// A CLI whose credential and config are split across two locations (Claude Code keeps
	// its config in the home directory and its OAuth token in a subdirectory) is given one
	// directory it can be pointed at, for both detection and an episode. When set it takes
	// precedence over AuthSeedFiles; AuthEnv still names the variable that points the child
	// at the assembled directory, and a source file that does not exist is skipped so a
	// partially configured CLI assembles what it has and detection reports the rest as
	// not-ready. Like the AuthSeedFiles copy, the assembled directory is a per-episode home
	// the run writes and deletes, so a token the harness refreshes lives only in the copy
	// and the host credential is never made writable to it.
	AuthSeedPaths []string
	// AuthSeedFiles names the files inside AuthDir that an episode's credential home must
	// contain (for codex: the OAuth token and the CLI's own config). Naming them switches
	// an episode from pointing the child straight at AuthDir to giving it a per-episode
	// copy of just these files, in a writable directory outside the workspace that the run
	// creates and deletes.
	//
	// An episode needs this because the CLI writes its own home as it runs (a session
	// rollout, a log, a PATH shim) and every confinement tier denies writes outside the
	// workspace, so pointing it at a read-only AuthDir fails partway through the episode.
	// The two obvious fixes are both worse. Granting write on the host's AuthDir hands an
	// untrusted harness the credential file it authenticates with, to corrupt or replace,
	// and leaves whatever it wrote behind after the run. Putting the home inside the
	// workspace copies the OAuth token into the tree the record captures. The per-episode
	// copy has neither problem: the child can write all it likes, it writes only a copy
	// that dies with the episode, and the copy sits outside the recorded workspace.
	//
	// The cost is that a token the CLI refreshes mid-episode is refreshed only in the copy,
	// so the host's credential stays as it was; the refresh token it was issued from
	// remains valid, so the next episode still authenticates. Empty keeps the read-only
	// AuthDir behaviour (correct for a detection probe, which writes nothing).
	AuthSeedFiles []string
	// ProgramDirs are the directories holding the external CLI's own executable and the
	// helper binaries shipped beside it. The confined child is granted read (and execute)
	// on them for the life of the launch and the grant is revoked on teardown. Without it
	// the child cannot load the very program it is meant to run: confinement is
	// default-deny for reads where the platform enforces it, and the CLI lives outside the
	// episode workspace. They come from resolving the CLI to its native executable (see
	// LocateCodex), so a launcher script's interpreter, which may sit under a system-owned
	// directory an unprivileged process cannot grant at all, never enters the confinement.
	// Empty grants no extra read, which is correct where the confinement leaves the host
	// filesystem readable.
	ProgramDirs []string
	// HostReadable selects the confinement tier that confines the child's writes to the
	// episode workspace but leaves the host filesystem readable to it (see
	// sandbox.WithHostReadable). It is set for a CLI whose runtime cannot start under a
	// deny-by-default read posture: the codex CLI is a Rust program, and every Rust program
	// that canonicalizes a path fails inside a Windows AppContainer, whose token cannot
	// perform the final step that maps a file handle back to a DOS path. The weakening is
	// real (the harness can read the host user's files) and bounded (it can still write
	// nothing outside the workspace, and its egress is still gated), so it is named per
	// backend rather than defaulted, and the tier that ran is recorded on the episode. On
	// Linux and macOS the kernel tier already leaves the host readable, so this changes
	// nothing there.
	HostReadable bool
	// MinContainment is the floor the host must actually enforce or an episode is refused
	// rather than run less contained (refuse-rather-than-downgrade: an untrusted harness
	// never silently drops to a weaker boundary). The zero value is treated as
	// sandbox.ContainmentKernel, the boundary for semi-trusted, model-authored code over a
	// shared kernel; a caller can raise it (a microVM tier) but not lower it below the
	// kernel floor by leaving it zero.
	MinContainment sandbox.Containment
	// ProbeTimeout caps how long a detection probe may run before it is killed, so a
	// hung CLI cannot stall detection. Zero applies no cap beyond the caller's context.
	ProbeTimeout time.Duration
}

SandboxConfig is the confinement envelope a SandboxSpawner runs an external CLI under. The external harness is untrusted: its own sandbox is bypassed and is not the boundary, so this profile is the only one. A zero value refuses a live episode (no provider egress, kernel containment required), which is the safe default.

type SandboxSpawner

type SandboxSpawner struct {
	// contains filtered or unexported fields
}

SandboxSpawner is the production Spawner: it runs an external agent CLI as an untrusted subprocess inside the sandbox's containment envelope, the single security boundary around a harness whose own code the run does not control. It backs the Spawner port with the sandbox's streaming and capture launch primitives (no direct os/exec, which the repo confines to the sandbox package), composing a read-only host, the syscall filter, a deny-all-except-provider egress gate, and a read grant for the CLI's own auth home into one launch. Detection runs through Probe; an episode runs through Start, whose process is bound to the context so a halt kills the CLI.

func NewSandboxSpawner

func NewSandboxSpawner(cfg SandboxConfig) *SandboxSpawner

NewSandboxSpawner builds the production Spawner for the given confinement envelope. A MinContainment left at the zero value is raised to sandbox.ContainmentKernel, so the default is the kernel-confined floor rather than an unconfined process jail.

func (*SandboxSpawner) Close added in v0.1.3

func (s *SandboxSpawner) Close() error

Close removes the run's credential-and-state home, with whatever the CLI wrote into it (its own conversations). It is called when the run that owns the spawner ends; the credentials it held were already deleted with each episode.

func (*SandboxSpawner) ForwardBridge added in v0.1.3

func (s *SandboxSpawner) ForwardBridge(hostURL string) (childURL, forwardTo string)

ForwardBridge reports how the confined child reaches a bridge on the host loopback. The child runs behind the sandbox's network confinement, which on Linux is a separate network namespace whose loopback is not the host's, so the bridge is forwarded in: the child is given an in-namespace address and the sandbox forwards it to the host one. Where the child shares the host's stack the sandbox reports the host URL unchanged and no forward. The runner calls this before building the episode command, so the child is configured with an address it can actually reach.

func (*SandboxSpawner) Probe

func (s *SandboxSpawner) Probe(ctx context.Context, path string, args ...string) (string, error)

Probe runs path with args to completion under best-effort confinement and returns its combined output, for detection (a version or auth-status probe). Detection must work wherever the CLI is installed, so the probe uses the always-on baseline confinement that degrades to the process-jail floor rather than refusing on a host that cannot set up the kernel tier; the episode path (Start) is the one that refuses. The CLI's auth home is granted read so an auth-status probe can see whether it is logged in. A non-zero exit is returned as an error with the output preserved, so detection reads it as "not present" or "not ready" while the adapter can still parse the reason.

func (*SandboxSpawner) Start

func (s *SandboxSpawner) Start(ctx context.Context, ep Episode, inv Invocation) (Process, error)

Start launches one episode's subprocess inside the containment envelope and returns its live stdout and a wait handle. The child runs with a read-only host and the syscall filter (so its bypassed native writes and dangerous syscalls are refused by the OS, not by trusting the CLI), a deny-all-except-provider egress gate (so its only way out is the allowlisted provider, and its direct provider channel stays unobserved-but-contained), and a read grant for its auth home. Before it launches, the host's actual containment is checked against the configured floor and the launch is refused if the host cannot meet it, so an untrusted harness never runs less contained than required. The process is bound to ctx: cancelling it (a halt or shutdown) kills the CLI, and the per-episode sandbox (its egress proxy and read grant) is released when Wait returns.

type Spawner

type Spawner interface {
	// Probe runs path with args to completion and returns its combined output. A
	// non-nil error means the command could not be run or exited non-zero, which
	// detection reads as "not present" or "not ready".
	Probe(ctx context.Context, path string, args ...string) (string, error)
	// Start launches one episode's subprocess and returns its live stdout and a wait
	// handle. The process is bound to ctx: a cancellation (a halt or shutdown) kills it.
	Start(ctx context.Context, ep Episode, inv Invocation) (Process, error)
}

Spawner runs the external CLI's processes. Process spawning is confined to the sandbox boundary in this project, so this package never spawns directly: the caller provides a Spawner backed by that boundary (a semi-trusted external-agent containment profile), and the adapter and runner drive it. Probe runs a short command to completion for detection; Start launches an episode subprocess bound to the context, so cancelling the context kills it.

type Steering added in v0.1.3

type Steering struct {
	// BridgeCalls is how many tool calls the harness made on the run's own bridge, where
	// the dispatch waist admitted, contained, braked, and recorded each one.
	BridgeCalls int
	// ForeignCalls is how many tool calls the harness made on some other MCP server. The
	// run neither serves nor governs those, so they are counted apart from its own.
	ForeignCalls int
	// NativeCommands is how many commands or file edits the harness ran with its own
	// built-in tools rather than the bridged ones.
	NativeCommands int
	// NativeDeclined is how many of those native attempts the CLI's own sandbox refused
	// (a declined command, a failed patch). Each is a turn the harness spent learning it
	// cannot act natively, which better steering would not have spent.
	NativeDeclined int
}

Steering counts how an episode's harness chose its tools. The external CLI carries its own built-in shell and patch tools and its own harness prompt, which outranks anything the run injects, so the instruction to route effects through the bridged tools is a request rather than a guarantee. These counts turn the request's outcome into a number: a run whose harness keeps reaching for its native shell is being steered badly, and the tool descriptions or the preamble need tuning.

A native command is not a containment breach. The CLI runs under a read-only sandbox with its approval path denied, so a native write cannot land. It is an observability loss: a native read succeeds and the run never sees what was read.

func (Steering) NativeRate added in v0.1.3

func (s Steering) NativeRate() float64

NativeRate is the share of the harness's tool attempts that reached for its own tools instead of the bridged ones, in [0,1]. It is the headline steering number: zero means the preamble and tool descriptions steered it perfectly. An episode that attempted no tools at all reports zero rather than dividing by zero, since there was nothing to steer.

func (Steering) Total added in v0.1.3

func (s Steering) Total() int

Total is every tool attempt the harness reported, bridged or native.

type Tier

type Tier int

Tier is the provenance tier of a recorded action: how strongly the sealed record can vouch for it. A run driven by an external harness mixes tiers, and verify reports the mix so an external-harness run never claims the integrity of a native one.

const (
	// TierEnforced is an action that ran through the dispatch waist: a bridged tool
	// call, admitted against the grant and contained, exactly like a native action.
	TierEnforced Tier = iota
	// TierAttested is an action the external CLI reported that the run did not
	// independently enforce, for example the CLI's own progress and context events.
	// The record carries the CLI's claim, marked as its claim.
	TierAttested
	// TierUnobserved is a declared gap: work the external harness does that is
	// structurally outside the run's tracing (its inner model calls, its direct egress
	// to its own provider). The record names the gap rather than pretending to cover it.
	TierUnobserved
)

func (Tier) String

func (t Tier) String() string

String names the tier for the record and verify output.

type Usage

type Usage struct {
	InputTokens  int
	OutputTokens int
}

Usage is the token accounting an episode reports. It mirrors the shape a native run records, so an external episode's cost lands on the same accounting.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL