dev/

directory
v3.0.0-dev.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 20, 2026 License: Apache-2.0

README

dev/

Build- and test-tooling. Same scripts power both local development and GitHub Actions CI, so a green local run is the same green run as remote.

Quickstart

# Run every CI check locally (fast-fail order).
dev/tools/ci

# Run all checks even after a failure (collect every problem at once).
dev/tools/ci --keep-going

# Auto-fix formatting (gofmt + goimports).
dev/tools/fix-go-format

Missing tools (golangci-lint, goimports, govulncheck) auto-install into $GOBIN (or $(go env GOPATH)/bin) on first use. No setup needed beyond a Go toolchain.

Layout

dev/
├── tools/                 # entry points users run locally
│   ├── ci                 # aggregator — runs every check below
│   ├── vet                # go vet ./...
│   ├── build              # go build ./...
│   ├── test-unit          # go test -race -coverprofile
│   ├── lint-go            # golangci-lint (auto-installs v2.12.1)
│   ├── verify-go-format   # gofmt -s + goimports check (read-only)
│   ├── fix-go-format      # gofmt -s -w + goimports -w (auto-fix)
│   ├── verify-mod-tidy    # `go mod tidy` clean check
│   ├── verify-vuln        # govulncheck ./...
│   ├── verify-go-toolchain # every named Go version agrees with go.mod
│   ├── verify-coretui-guards # capability guards vs the pinned core-tui
│   ├── examples-smoke     # the credential-free examples still run
│   ├── add-license-headers # bulk-applier for SPDX + copyright headers
│   ├── common.sh          # shared bash helpers (ensure_tool, run_step)
│   └── .golangci.yml      # linter config
└── ci/
    └── presubmits/        # thin delegators called by .github/workflows/ci.yml
        ├── vet            # → dev/tools/vet
        ├── build          # → dev/tools/build
        ├── test-unit      # → dev/tools/test-unit
        ├── lint-go        # → dev/tools/lint-go
        ├── verify-go-format
        ├── verify-mod-tidy
        ├── verify-go-toolchain
        ├── verify-coretui-guards
        ├── examples-smoke
        └── verify-vuln

Adding a check

  1. Drop a new script under dev/tools/<name> (executable, set -euo pipefail, sources common.sh).
  2. Add it to the STEPS array in dev/tools/ci.
  3. Add a one-line delegator under dev/ci/presubmits/<name> that execs the tool script.
  4. Reference the presubmit from .github/workflows/ci.yml.

That's it — the delegator pattern means the GitHub workflow never has to know what the check actually does.

A check with real logic puts that logic in a Go program under dev/ rather than in the bash — dev/lookout-pin-check and dev/coretui-guard-check are the two — and the dev/tools/ script becomes a go run wrapper over it. (lookout-pin-check predates the convention and is still invoked straight from its own scheduled workflow; verify-coretui-guards is the wrapped shape.) Go programs there are part of the main module, so go build ./..., go vet and go test cover them like anything else.

examples-smoke is the one exception to "the Go program lives under dev/": its manifest is examples/internal/smokeset, and Go's internal rule refuses an import of it from dev/, so the runner sits at examples/internal/smokerun instead. The wrapper is in the usual place and behaves like every other check.

The examples smoke gate

dev/tools/examples-smoke (tool: examples/internal/smokerun) builds and runs every example program marked runnable and fails on any that does not exit 0. No arguments and no credentials: every runnable example defaults to the scripted mock provider and opts into a real one behind a flag the runner never passes, and provider keys are dropped from the child environment so the answer doesn't depend on whose machine it runs on. The whole set takes about five seconds.

Why it exists (#852): go build ./... proves the examples compile, and nothing proved they still run. examples/parallel-spawn had been exiting 1 for some time — every spawn refused after ad-hoc spawns became opt-in — while its README advertised "Exits 0"; it was found by hand during unrelated work (#476), not by a check.

The runnable set is a list, not a glob, because the examples split three ways (runnable, needs a key or an argument or binds a listener, covered by its own tests). The list lives in examples/internal/smokeset with a test that requires every examples/*/main.go to appear in one column or the other, so a new example is either covered or excluded with a written reason — never silently uncovered.

dev/tools/examples-smoke           # the gate
dev/tools/examples-smoke --print   # the disposition table

What it does not prove: that an example demonstrates what its README claims. Only parallel-spawn and compose-multi-session assert on their own results today; the rest exit non-zero only when an operation errors. The runner also fails if a run leaves the worktree dirty, since an example writing into the checkout is a defect the exit code would not catch.

The core-tui capability gate

dev/tools/verify-coretui-guards (tool: dev/coretui-guard-check) enumerates the exported interfaces of github.com/go-steer/core-tui/tui at the version go.mod resolves to and requires each of the repo's two TUI hosts to account for every one of them. A capability is accounted for when it is either:

  • guarded — a var _ coretui.X = (*T)(nil) line in the host's guard file (cmd/core-agent/coretui_guards.go for the local --tui host, internal/coretuiremote/guards.go for attach mode); or
  • declined — a //coretui:declined X directive in that file's trailing comment, immediately followed by prose naming coretui.X and saying why this host does not implement it.

Both halves of a decline are required. The directive is what the gate counts; the bullet is the decision, and the gate refuses a directive whose following prose names a different interface, so a rename cannot leave one pointing at the other. The guard files are also the only place either may appear — a guard next to its method is a hard error that points back here.

Why it exists (#812): the guards make core-tui adding a method to an interface we name a build failure. Nothing made core-tui adding a whole new interface anything at all — core-tui feature-detects by type assertion with no error and no log, so an unimplemented capability is simply absent.

dev/tools/verify-coretui-guards           # the gate
dev/tools/verify-coretui-guards --print   # the interface x adapter matrix

Run --print when picking up a core-tui bump; it is the audit table, and UNACCOUNTED in a column is what the gate fails on. What it does not catch: a decline whose reason has quietly stopped being true while the interface still exists (the #811 case, where a capability became implementable). Proving that needs a type-check of the host, not a comparison of two lists; the gate catches the after-effect — a decline left in place once the guard lands — as a contradiction.

The harness config-pin gate

dev/tools/verify-harness-config-pinned (tool: dev/harness-config-check) reads every script under dev/smoke, dev/uat, dev/tools and dev/ci, finds the places that execute the core-agent binary, and requires each one to pin its config with -c.

Why it exists (#1116): config.Find walks up from the process cwd and returns the first .agents/ it meets, so a .agents/ at the repo root — committed with a recipe, or just left behind by a local run — is inherited by every unpinned invocation anywhere under the checkout. Nothing fails. The scripts keep passing, against an agent nobody chose: a different model, a different permission mode, a tools.disable list that removes the tool under test. dev/smoke/05-headless-gate.sh tests bash gating, and a root config disabling bash would have it testing nothing.

--agents-dir is not a pin. loadConfig() reads config.json by discovery from the cwd before resolveAgentsDir() applies the flag, so --agents-dir moves the skills, MCP servers and sessions and leaves the model, permissions and budgets behind — the split splitTreeWarning() in cmd/core-agent/agents_dir.go warns about. Only -c closes it.

dev/tools/verify-harness-config-pinned           # the gate
dev/tools/verify-harness-config-pinned --print   # every site and its pin

Two shapes are exempt, structurally rather than by grant: attach and ls are dispatched before flag.Parse, read no config, and reject -c outright. Everything else is a violation until pinned.

The scanner's soundness runs the opposite way to most checks here: it fails when it finds something, so a miss is fatal and a false positive is merely loud, and every ambiguity resolves towards "treat it as code". That is why it descends into quoted strings and $(…) rather than blanking them — dev/uat/attach/run.sh builds real invocations as double-quoted strings and dispatches them through tmux, and a lexer that blanks quoted spans cannot see them. Deciding a site is pinned takes the opposite bias, because a -c credited in error is also a silent pass: a -c the quote scan places inside an argument does not count (-p 'explain the -c flag' is prose), and the value has to look like a config path. --print is the third leg: a violation scanner cannot report the site it has stopped recognising, so the site list is worth reading whenever a harness script renames its binary variable.

What it does not claim:

  • That a pinned run is hermetic. -c fixes the config and the content root. Instruction discovery still reaches ~/.core-agent/AGENTS.md and ~/.agents/AGENTS.md, so a personal instruction file in $HOME still steers a pinned smoke run.
  • That every way of executing a binary is recognised. Command position, assignment prefixes, exec/env/sudo, brace groups, the timeout/nice/stdbuf family and go run ./cmd/core-agent are. eval, xargs, find -exec, set -- … ; "$@", variable indirection and a binary held in a variable the tool does not know are not — the last of those is why binaryVars and --print exist rather than a glob.
  • That the pinned path is right. It checks that a script says which config it wants, not that the file is the one the test needs.

CI on PRs

Open a PR against main from a short-lived feature branch (e.g. feat/m3-subagents, fix/mcp-leak). CI runs on the PR; merging is gated on the four required status checks.

For this to actually gate merges, the repo's branch protection on main must require these checks (settings → branches → main):

  • test
  • lint
  • go mod tidy is clean
  • govulncheck

Docs-only PRs (**/*.md) are handled by the companion ci-docs.yml workflow, which emits the same four check names trivially-green so branch protection is satisfied without running the full Go pipeline.

Scheduled automation

A weekly job keeps an externally-owned value from drifting silently, and a new one should copy its shape:

Workflow Schedule Tool Answers
lookout-pin-check.yml Tuesdays 06:23 UTC dev/lookout-pin-check is the recipe's pin of ghcr.io/go-steer/lookout still upstream's current release?

The built-in price table used to be the other one (pricing-regen.yml, dev/regen-builtin-pricing). It now lives in go-steer/core-models, which runs the same weekly regen for both core-agent and mast; core-agent takes the result by bumping its core-models pin. dev/tools/verify-pricing-fresh runs core-models' generator in --check mode against the pinned table, and the release guards call it.

The shared conventions, each of which exists because of a specific failure:

  • --check reports on stdout, not through the exit code. Exactly drift=true or drift=false, and exit 0 either way. A non-zero exit always means the tool broke. The tool is invoked via go run, which collapses every non-zero child status to 1 — so an exit-code convention would make a network hiccup indistinguishable from real drift, and the job would open a pull request on nothing.
  • Default mode writes the change. The auto-PR carries a real diff, so it is reviewable and so path-filtered workflows (the recipe's kind e2e, for one) actually run on it.
  • Presubmits run on the rewritten tree before the PR opens. A generator that produces something lint or tests reject should fail the workflow, not open a bad pull request.
  • The App token is minted unconditionally. Most weeks there is no drift; gating the mint would leave the credentials unexercised until the one run that needs them.

Run it locally the same way CI does, and the pricing check the same way the release guards do:

go run ./dev/lookout-pin-check --check
dev/tools/verify-pricing-fresh

dev/lookout-pin-check additionally takes --releases=<file.json> to replay a captured release list with no network at all, which is how its own tests and any offline reproduction work, and --resolved=<file.json> to write out what a live run resolved. The workflow uses the pair to resolve upstream exactly once: the --check step records its answer and the rewrite step replays it, so a release cut between the two steps cannot leave the tree written to a tag the drift verdict never saw.

License headers

Every source file carries the full Apache 2.0 header at the top, attributed to Google LLC:

// Copyright 2026 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

(#-prefixed for shell, YAML, and Python.) The goheader linter inside dev/tools/lint-go enforces this on every .go file — CI fails if a new Go source is missing it. For shell, YAML, and Python files, run dev/tools/add-license-headers after creating new ones; the script is idempotent and normalizes any existing header (including the older SPDX-shorthand variant) to the current canonical form.

Pinned tool versions

Tool Version Source
golangci-lint v2.12.1 dev/tools/lint-go (GOLANGCI_LINT_VERSION env var)
goimports latest dev/tools/fix-go-format, dev/tools/verify-go-format
govulncheck latest dev/tools/verify-vuln

Bump deliberately — new linter releases can introduce findings that block CI. When you bump golangci-lint, run dev/tools/lint-go locally first to fix anything new before pushing.

Directories

Path Synopsis
Command coretui-guard-check holds the capability-guard lists in cmd/core-agent/coretui_guards.go and internal/coretuiremote/guards.go exhaustive against the core-tui this repo actually pins.
Command coretui-guard-check holds the capability-guard lists in cmd/core-agent/coretui_guards.go and internal/coretuiremote/guards.go exhaustive against the core-tui this repo actually pins.
Command harness-config-check fails when a harness script runs the core-agent binary without pinning its config with -c.
Command harness-config-check fails when a harness script runs the core-agent binary without pinning its config with -c.
Keeps this repo's pin of the upstream lookout watcher image current.
Keeps this repo's pin of the upstream lookout watcher image current.
Diagnostic: measure how often the model emits multiple tool calls per assistant turn.
Diagnostic: measure how often the model emits multiple tool calls per assistant turn.
persona-probe measures whether deleting the persona line ("You are a helpful assistant.
persona-probe measures whether deleting the persona line ("You are a helpful assistant.
smoke
cmd/approvereval command
approvereval runs auto mode's approver against the labelled corpus in dev/evals/approver with a real model (#1175 phase 5).
approvereval runs auto mode's approver against the labelled corpus in dev/evals/approver with a real model (#1175 phase 5).
cmd/evalrun command
evalrun runs one eval case at both tiers and reports (#652).
evalrun runs one eval case at both tiers and reports (#652).
cmd/inspect-grounding command
Command inspect-grounding dumps every agent_eventlog row whose author starts with "gemini/" from a SQLite eventlog database.
Command inspect-grounding dumps every agent_eventlog row whose author starts with "gemini/" from a SQLite eventlog database.
Package trajectory reads what an agent DID, in order, out of a recorded run — and reports on the shape of it.
Package trajectory reads what an agent DID, in order, out of a recorded run — and reports on the shape of it.
cmd/trajectory command
Command trajectory prints what an agent did, in order, out of a recorded GKE-drill run.
Command trajectory prints what an agent did, in order, out of a recorded GKE-drill run.
uat
scheduled-monitor command
UAT driver for the scheduled-monitoring feature (docs/scheduled-monitoring-design.md).
UAT driver for the scheduled-monitoring feature (docs/scheduled-monitoring-design.md).
selfdev-soak/dispatcher command
Command dispatcher is the self-development soak's minimal dispatcher: the A7 subset (steps 0–3 and 7) of "The dispatcher" in docs/selfdev-soak-design.md.
Command dispatcher is the self-development soak's minimal dispatcher: the A7 subset (steps 0–3 and 7) of "The dispatcher" in docs/selfdev-soak-design.md.
Command webhook-sink is a local alert target that writes down every delivery it accepts, so a UAT rig can grade that a notification was delivered rather than that the daemon said it sent one.
Command webhook-sink is a local alert target that writes down every delivery it accepts, so a UAT rig can grade that a notification was delivered rather than that the daemon said it sent one.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL