chatwright

Overview ¶

arena.go wires chatwright.dev/runtime/arena into the CLI as `chatwright arena run` and `chatwright arena report`. It is deliberately thin per this repository's own AGENTS.md ("the CLI is deliberately thin ... engine or wire logic never lives here"): every metric, every retry- breakdown count and the whole matrix-execution loop live in the arena package; this file only parses an arena.yaml config into arena.Matrix, calls arena.Run/arena.WriteReport, and persists what the arena package itself never writes to disk (see arena's own package doc comment) — bundles, the markdown report, and a machine-readable results.json.

completion.go implements `chatwright completion bash|zsh|fish` — item 9 of the UX brief ("Shell completion (bash/zsh/fish) if it can be done without a heavy dependency"). It genuinely can: this repository's own command surface is small and static (a fixed set of top-level commands, a fixed set of flags per subcommand), so the three scripts below are plain hand-written shell text with no code-generation framework behind them (no cobra, no urfave/cli) — the same "small internal helper is right, a framework is not" line the brief draws for terminal capability detection applies here too. Keeping the three vocab lists (topLevelCommands, runFlags, arenaSubcommands, serverSubcommands) as the single source of truth for all three shells is this file's own discipline against drift between them; there is no schema forcing it, only TestCompletionScriptsCoverEveryCommand checking it.

example.go embeds this repository's own copy of the standard self-contained-scenario-document worked example — GreetBot's language- onboarding fixture, copied from chatwright.dev/runtime's own scenario/testdata/ (see run_test.go's own greetbotFixturePath) — so `chatwright run example` has something real to execute the moment a user installs this binary: no files of their own, no network call, no API key. The document's "bot" is exampleBot:greetbot (a bot compiled into chatwright.dev/runtime, no URL) and its one cast member is a cassette-replay provider (a canned recording, not a live model), so nothing about running it ever leaves the process.

ONE DELIBERATE DIVERGENCE FROM THE UPSTREAM COPY: this copy's own top-level "ceiling" object is NOT present in chatwright.dev/runtime's scenario/testdata/greetbot-language-onboarding.json and must not be removed by a future re-sync. Without it, `chatwright run example` — this CLI's own bundled, first-thing-a-stranger-tries demo — printed its own "no-run-ceiling" validation warning as the very first line a new user ever saw (scenario/validate.go's validateCeiling); shipping a warning about your own demo is a bad first impression, and the fix belongs here (a document-authoring choice), not in the runtime's validation rule. Adding it is safe: Ceiling is a Run-level execution control (run.RunCeiling), never part of what feeds an actor's Prompt, so it does not affect the cassette's replay-key hashing (see the next paragraph) — TestRunExampleEndToEnd still passes unchanged. Its values (maxSteps/maxDurationSeconds) deliberately match the document's one Part own goal.Budgets exactly: this document has a single Part, so the run-level ceiling and that Part's own task budget are the same bound restated, not a materially different aggregate limit.

KEEPING THE EMBEDDED CASSETTE IN SYNC: the cassette's entries are keyed by a hash of the whole actor Prompt, and Prompt.History embeds run-bundle wire types — so ANY change to those types (a renamed field, an added one) silently invalidates every key in this copy, and `chatwright run example` starts failing with a replay cache miss. It is not a copy that only drifts when someone edits it. When bumping chatwright.dev/runtime, re-copy scenario/testdata/cassettes/greetbot-language-onboarding.json from that module (it regenerates its own via a build-tag-guarded recorder), re-apply this file's own "ceiling" divergence to the re-copied document if the upstream fixture still lacks one, and re-run the tests here — TestRunExampleEndToEnd catches a cassette mismatch loudly, which is how the sdk Verdict->Freshness rename was caught on 2026-07-25.

chatwright.dev/runtime/scenario has no seam for supplying a Document or a cassette as in-memory bytes — scenario.ScenarioProvider.Load and, deeper, scenario.Build's own cassette loading (actor.LoadCassette) are both hard-wired to a filesystem path (see their own doc comments in that package: "Loading a cassette file is I/O this package only ever performs here, in Build"). So the embedded bytes are, whichever of the two materialize functions below is used, always written back out as real files before scenario.FileScenarioProvider ever sees them — a temporary, self-cleaning directory for a plain `chatwright run example`, or a directory the caller chose for `chatwright run example --write`. This is not a workaround bent around the runtime's API; it is that API, used exactly as every other document already is.

interrupt.go gives `chatwright run` a graceful Ctrl-C (item 8 of the UX brief: "Ctrl-C should not lose the run"). Before this file, interrupting a multi-minute ai-goal run left nothing behind — no bundle, no partial result, just a killed process — because nothing in this repository ever derived a cancellable context or listened for os.Interrupt.

What SIGINT actually reaches: runengine.Run.Execute's own ai-goal loop (chatwright.dev/runtime/actor.Loop.RunTask) passes ctx through to Provider.Propose on every iteration, and a Propose call that returns a context-cancellation error is recorded as a LoopEvent carrying that error in ProposeError — exactly the same shape this repository's own actorFailureFromParts already reads for a cassette cache miss (see run.go). Run.Execute itself then reports that Part as PartFailed and returns the Result normally, with a nil top-level error — cancellation never becomes a Go error runRun has to specially unwrap; it already arrives as an ordinary actor-unavailable-shaped outcome. This file only adds the *signal*; runOutcome.actorFailed's own machinery does the rest. A deterministic Part is not interruptible this way: runDeterministic takes no context at all, so Ctrl-C during a purely deterministic run has no earlier interception point than the process's own default signal disposition — a real gap, out of this CLI's reach without a chatwright.dev/runtime change (see this repository's own AGENTS.md: "the CLI is deliberately thin").

Command chatwright is the local command-line entry point for the Chatwright conversation execution platform. It is deliberately thin: the heavy lifting lives in chatwright.dev/runtime (platform emulation + the testing runtime) and chatwright.dev/sdk (the run-bundle wire model); this binary only fronts them from a terminal.

progress.go renders chatwright.dev/runtime/run.Run.OnProgress events as live terminal output for `chatwright run` — this repository's own AGENTS.md "the CLI is deliberately thin" applies to what this file computes (it reads a runengine.ProgressSnapshot and formats a line; it invents no new runtime state) but not to whether it exists at all: before this file, Run.OnProgress had zero references anywhere in this repository, so a multi-minute ai-goal run printed nothing until it finished, even though the runtime already emits everything needed for live output at every part boundary and every actor-loop iteration.

A hard limit on what this can show, worth stating plainly because the obvious brief ("show what it observed and proposed") oversells the API: runengine.ProgressSnapshot (and the actor.ProgressSnapshot it forwards for an ai-goal Part) is a point-in-time GAUGE — phase, position, budget burn, retry counts by outcome kind — never a transcript. It carries no message text, no button label, no rationale for that iteration; that detail (actor.LoopEvent) only exists after a Part finishes, in sdk.AIGoalSection.Events. So this renderer cannot literally say "asked for English" or "clicked the button labelled X" live; the closest it can get is actedThisStep, which diffs two consecutive RetryCounts snapshots to name the ActionOutcomeKind that just changed — "executed", "no effect", "invalid, skipped", and so on. That is a real, derived fact (whether the loop's last attempt actually did something), not the content of what it did.

run.go wires chatwright.dev/runtime/scenario into the CLI as `chatwright run`. It is deliberately thin per this repository's own AGENTS.md ("the CLI is deliberately thin ... engine or wire logic never lives here"): loading, validating, resolving and executing a self-contained scenario document (https://chatwright.dev/formats/scenario-document/v1) all live in chatwright.dev/runtime/scenario; this file only parses flags, calls into that package, assembles the resulting sdk.Bundle (the one piece scenario.Build deliberately leaves to its caller — a bundle is a bundle-only, wire-typed concept with no runtime counterpart, exactly the same division run.AssembleBundleRun already draws for the arena subcommand) and writes it to disk. Everything about how the run is *presented* — colour, the live progress line, the JSON shape, the summary block — lives in run_output.go and progress.go; this file owns orchestration only.

run_output.go is `chatwright run`'s own presentation layer: the scannable human summary block (replacing the old flat "part status=…, outcome=…" prose line), the --json machine-readable shape, and a readable rendering of a rejected scenario document's validation issues. runRun (run.go) owns orchestration and flag parsing; this file owns turning what it computed into what a terminal or a script actually reads.

server.go wires chatwright.dev/cli/internal/server into the CLI as `chatwright server serve|start|stop|restart`. It is deliberately thin per this repository's own AGENTS.md ("the CLI is deliberately thin ... engine or wire logic never lives here"): every HTTP handler, the reverse proxy, the metrics ring buffer, the datastate evaluation seam, and the PID-file/ daemon primitives live in internal/server; this file only parses flags (falling back to environment variables, then fixed defaults), and calls into that package.