cli

Chatwright CLI

The command-line entry point for Chatwright — deterministic and AI-driven testing for conversational applications.

Module chatwright.dev/cli, binary chatwright. The CLI is deliberately thin: platform emulation and the testing runtime live in chatwright.dev/runtime, and the run-bundle wire model in chatwright.dev/sdk; this binary fronts them from a terminal.

Install

Canonical (macOS/Linux):

curl -fsSL https://chatwright.dev/install.sh | sh

Windows (PowerShell):

irm https://chatwright.dev/install.ps1 | iex

Homebrew (macOS):

brew install --cask chatwright/tap/chatwright

Go-native:

go install chatwright.dev/cli/cmd/chatwright@latest

Usage

chatwright <command>

Commands:
  platforms     List built-in messaging platform emulators
  run           Execute a self-contained scenario document (chatwright run --help)
  arena         Run and report on the actor-model arena (chatwright arena help)
  server        Run the server companion daemon (chatwright server help)
  completion    Generate a bash/zsh/fish completion script (chatwright completion help)
  self-update   Update the installed binary in place (chatwright self-update --help);
                also available as "chatwright update"
  install       List and install fleet CLIs relevant to chatwright (chatwright install --help)
  upgrade       Upgrade installed fleet CLIs, including chatwright itself (chatwright upgrade --help)
  skills        Install Chatwright Agent Skills into supported harnesses
  version       Print the CLI, runtime and sdk versions
  help          Show this help

Try it now — no files, no network, no API key:
  chatwright run example

chatwright version reports the CLI's own version plus the resolved sdk/runtime module versions it was built against, and the supported run-bundle format id. chatwright version --json prints the same identity as one JSON object (name, version, commit, date, date_source, plus runtime/sdk) and nothing else, side-effect-free — the fleet-wide probe flag every other catalog CLI's install/upgrade uses to identify an installed chatwright copy.

chatwright run

Runs a self-contained scenario document and writes the resulting run bundle — live progress on stderr while it runs, a scannable summary (or --json) once it's done:

chatwright run example                 # the built-in worked example — try this first
chatwright run my-scenario.json --out ./runs
chatwright run my-scenario.json --json --quiet   # CI-friendly: one JSON object, nothing else
chatwright run my-scenario.json --verbose        # every actor turn, not just task boundaries

Colour and the live progress line both respect a real terminal, NO_COLOR and the CLICOLOR/CLICOLOR_FORCE conventions, and degrade to plain, newline-terminated lines once piped or redirected. See chatwright run --help for the full flag reference, the --json shape, and this command's exit codes (0 verified/judged, 1 not verified, 2 usage error, 3 actor unavailable, 130 interrupted).

chatwright self-update

Updates the installed chatwright binary in place, or reports whether a newer release is available — also available as chatwright update:

chatwright self-update --check                # report availability only; never modifies
chatwright self-update --check --format json  # machine-readable verdict
chatwright self-update --yes                  # skip the confirmation prompt (for scripts/CI/agents)
chatwright self-update --dry-run              # print the exact asset URL a real run would fetch
chatwright self-update --version v0.4.0       # install a specific release (manual installs only)
chatwright update                             # alias for self-update

Every safety decision — whether this install may be replaced at all, checksum verification before extraction, the atomic swap — comes from github.com/strongo/cli-helpers/selfupdate; see spec/features/self-update for what is chatwright's own configuration versus the shared library's behavior. A Homebrew-installed binary is redirected to brew upgrade --cask chatwright and is never overwritten directly; a manual install (the install script, or go install) is what actually gets replaced. Without --yes and without an interactive terminal attached, self-update refuses rather than blocking on input. See chatwright self-update --help for the full flag reference and this command's exit codes (0 success — including a completed --check whatever its verdict; 1 a runtime failure no flag fixes; 2 a usage error, including a confirmation that was needed but neither --yes nor a terminal was available).

chatwright install

Lists the fleet CLIs relevant to chatwright (currently specscore) with their live installed status, and installs named ones consistently with how chatwright itself was installed:

chatwright install                    # list relevant fleet CLIs, with live status
chatwright install specscore           # show details, confirm once, install it
chatwright install specscore --yes     # skip the confirmation prompt
chatwright install specscore --dry-run # report the plan without installing anything

Every catalog entry, status probe, destination policy and install method comes from github.com/strongo/cli-helpers/cliinstall; see spec/features/install for what is chatwright's own command wiring versus the shared library's behavior. Exit codes follow the same 0/1/2 convention as chatwright self-update, and an unknown target name is refused before any confirmation, network request, or write.

chatwright upgrade

The fleet-wide counterpart to self-update: reports and upgrades every installed catalog CLI, including chatwright itself:

chatwright upgrade                # report every installed catalog CLI plus chatwright; changes nothing
chatwright upgrade --all          # upgrade every installed catalog CLI plus chatwright
chatwright upgrade --all --check  # report upgrade availability only; changes nothing
chatwright upgrade specscore      # show details/plan for specscore, confirm once, upgrade it

chatwright self-update is exactly chatwright upgrade chatwright: both build from the same release identity, so they never disagree. upgrade carries no update alias — that alias stays on self-update only. Same exit-code convention and unknown-target refusal as install above.

chatwright skills sync

Installs the Chatwright Agent Skills embedded in the exact CLI build:

chatwright skills sync
chatwright skills sync --harness codex
chatwright skills sync --dir ./agent-skills --format=json
chatwright skills sync --newer-compatible

Ordinary sync is offline and selects the immutable plugin revision bundled with the installed Chatwright CLI. --newer-compatible is the explicit opt-in for a newer compatible published bundle. --json is a shortcut for --format=json.

Shell completion

chatwright completion bash > /usr/local/etc/bash_completion.d/chatwright
chatwright completion zsh  > "${fpath[1]}/_chatwright"
chatwright completion fish > ~/.config/fish/completions/chatwright.fish

Actor-model arena

Compares actor models (Ollama, LM Studio, any OpenAI-compatible endpoint) on the same Chatwright scenario — see chatwright.dev/runtime/arena and spec/ideas/actor-model-arena.md in the standard repository:

chatwright arena run --config arena.yaml --out ./arena-run
chatwright arena report --dir ./arena-run   # recompute report.md later, no re-run

arena run writes bundles/ (one replayable run-bundle per cell), report.md (the comparison table) and results.json (machine-readable) into --out. See arena.example.yaml for a documented starting config.

The Chatwright repositories

Licence

Apache-2.0 — see LICENSE and NOTICE.

Spec-first

Chatwright is developed spec-first with SpecScore — product specs live in the standard repository; this repository's own specs live under spec/.