smokeset

package
v3.1.0-dev.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 3, 2026 License: Apache-2.0 Imports: 6 Imported by: 0

Documentation

Overview

Package smokeset records which of the example programs under examples/ can be run unattended, and — for every one that cannot — the reason it is excluded.

The problem it exists for (#852): `go build ./...` proves the examples compile and nothing proved they still *do what they say*, which is the only reason an example exists. examples/parallel-spawn had been exiting 1 for some time — every one of its spawns refused with "ad-hoc subagents are disabled" after ad-hoc spawns became opt-in — while its README went on advertising "Exits 0". It was found by hand during unrelated work (#476), not by a check.

The set has to be a list rather than a glob, because the examples split three ways: runnable unattended, deliberately not runnable (needs a provider key, binds a listener, wants an argument), and covered by its own tests. A list alone would rot the same way, so the companion test requires every examples/*/main.go to be accounted for here — a new example is either on the runnable list or excluded with a reason, and there is no third state in which it is silently uncovered.

What the gate does and does not prove

It proves a program runs to completion and exits 0 with no credentials and no arguments. That is strictly weaker than "the example demonstrates what its README claims": only parallel-spawn, compose-multi-session and embed/slim assert on their own results today, and the rest exit non-zero only when an operation returns an error. A stronger gate would need each example to check its own output, which is worth doing per example and is not what this package claims to have done.

The other thing it cannot catch is a Skip whose reason has stopped being true — an example that grows a default prompt, or drops its credential requirement, stays excluded until somebody re-reads the row. Proving that needs running the thing, which is the argument this package exists to have; the reason is written down so the re-reading is possible at all.

Index

Constants

View Source
const DefaultTimeout = 90 * time.Second

DefaultTimeout bounds a single example's run. The whole runnable set finishes in about four seconds on a warm build cache; the bound is generous because what it is really guarding against is an example that starts blocking forever, and a gate that flakes on a slow runner is a gate people disable.

Variables

View Source
var Entries = []Entry{
	{Name: "attach-daemon", Disposition: Skip,
		Reason: "binds an HTTP listener and serves until interrupted; it has no terminating path to exit 0 from"},
	{Name: "autonomous", Disposition: Run},
	{Name: "autonomous-handle", Disposition: Run},
	{Name: "autonomous-resume", Disposition: Run},
	{Name: "background-monitor", Disposition: Run},
	{Name: "basic", Disposition: Skip,
		Reason: "demonstrates the default provider path, so it needs a real provider key by design"},
	{Name: "bouncer-preflight", Disposition: Skip,
		Reason: "needs GOOGLE_API_KEY, and its logic is already covered by its own go tests in the same directory"},
	{Name: "compose-multi-session", Disposition: Run},
	{Name: "deploy/ax/entrypoint", Disposition: Skip,
		Reason: "the AX task-runner wrapper, not a library example: it takes the agent command after `--` and exits 2 without one, and when booted with --wait-for-egress blocks on AX's egress proxy; its own unit tests cover the exit-code, done-marker and egress-wait paths"},
	{Name: "embed/recipe", Disposition: Run},
	{Name: "embed/slim", Disposition: Run},
	{Name: "parallel-spawn", Disposition: Run},
	{Name: "replay", Disposition: Run},
	{Name: "scheduled-monitor", Disposition: Run},
	{Name: "streaming", Disposition: Skip,
		Reason: "takes the prompt as an argument and exits 2 with a usage message without one; it also needs a provider key"},
	{Name: "with-subagent", Disposition: Run},
	{Name: "with-tools", Disposition: Skip,
		Reason: "demonstrates the default provider path against real tools, so it needs a real provider key by design"},
}

Entries is the recorded disposition of every example program in the tree, sorted by name. Adding a main.go anywhere under examples/ (outside internal/ and testdata/) without adding a line here fails TestEveryExampleProgramIsAccountedFor; the name is the directory's slash-separated path relative to examples/.

Functions

func Discover

func Discover(examplesDir string) ([]string, error)

Discover lists the example programs in examplesDir: every directory under it, at any depth, holding a main.go. Names are slash-separated paths relative to examplesDir (`basic`, `embed/slim`, `deploy/ax/entrypoint`), which is also the form `go build ./examples/<name>` takes.

main.go is the rule rather than "any Go file" or "any directory" because it is what makes a directory a runnable program. Several directories under examples/ are recipes or docs — config, manifests and a README with no Go in them at all — and they are covered by examples/internal/recipecheck and by the recipe e2e workflows, not by this gate. A rule that swept them in would need a permanent row of "not a Go program" exclusions saying nothing.

The walk is depth-unbounded on purpose. The first version looked one level down, then grew an `embed/` special case, and in between `deploy/ax/entrypoint/main.go` landed two levels down and the accounting test never saw it (#1417). A program's depth is not evidence about whether CI should run it; the entry's Disposition is.

Skipped subtrees: `internal` (this package and its siblings), `testdata` (fixtures, which Go itself ignores), `node_modules`, and dot-directories (`.agents`, `.git`). A main.go inside those is not an example program.

Types

type Disposition

type Disposition string

Disposition says what CI does with an example program.

const (
	// Run means the smoke gate builds and runs it with no arguments
	// and no credentials, and requires exit 0.
	Run Disposition = "run"

	// Skip means the gate deliberately leaves it alone. Every Skip
	// entry carries a Reason, because an unexplained exclusion is
	// indistinguishable from an oversight — which is the failure this
	// package exists to prevent, one level up.
	Skip Disposition = "skip"
)

type Entry

type Entry struct {
	// Name is the directory under examples/.
	Name string

	// Disposition is Run or Skip.
	Disposition Disposition

	// Reason is required for Skip and must be empty for Run: a
	// runnable example needs no justification, and prose sitting on a
	// Run entry would read as a caveat nobody has to honour.
	Reason string

	// Timeout overrides DefaultTimeout for this example. Zero means
	// DefaultTimeout.
	Timeout time.Duration
}

Entry is one example program's disposition.

func Runnable

func Runnable() []Entry

Runnable returns the Run entries, in Entries order.

func (Entry) Bound

func (e Entry) Bound() time.Duration

Bound returns the timeout to apply to e.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL