Documentation
¶
Overview ¶
Package testutil provides shared test helpers for Cogo, most notably FakeModel — a deterministic implementation of google.golang.org/adk/v2/model.LLM that lets us drive end-to-end agent tests without burning real tokens.
Index ¶
- Variables
- func ApproverContext(ctx context.Context) context.Context
- func ApproverToolContext() adkagent.Context
- func ModelFacingBanViolations(text string) []string
- func ModelFacingText(tl tool.Tool) (texts []string, scannedSchema bool)
- func PruneWalkDir(root, path, name string) bool
- func RepoRoot(startFile string) (string, error)
- func UndeclaredArgRefs(tl tool.Tool) (refs []string, checked bool)
- type ApproverProbe
- type FakeModel
- type ScriptedResponse
Constants ¶
This section is empty.
Variables ¶
var ModelFacingBans = []struct { Phrase string Why string }{ {"generously", "a frequency instruction the persona cannot see or countermand (#905)"}, {"the codebase", "assumes a repository; many deployments have none"}, {"shipping a feature", "assumes a code workload"}, {"code review", "assumes a code workload"}, {"code search", "assumes a code workload; say what the tool does — search file contents"}, {"code investigation", "assumes a code workload; say what the tool does — read files, search them, list directories"}, {"debugging session", "assumes a code workload"}, {"source file", "assumes a code workload; a file is a file"}, {"completion summary", "names a genre, not a content obligation"}, {"status update", "names a genre, not a content obligation"}, {"one-paragraph", "prescribes a document shape the caller's AGENTS.md should pick"}, {"one-sentence detail", "names a genre and a length, not a content obligation"}, }
Tool descriptions and arg schemas are system-prompt-weight text that nothing reviews: AGENTS.md gets careful scrutiny, a `Description:` string in a Go file gets read as code. They outrank the persona at the point where the model decides what to call, and a recipe author can neither see nor override them (#909, #910).
core-agent began as an interactive coding agent and now also runs as a headless, long-lived daemon consuming machine signals, where "after shipping a feature" and "the codebase" describe nothing that happens. mark_task_done is the confirmed instance where that frame broke a live deployment: its "use this generously at natural task boundaries" beat a persona that explicitly forbade the behaviour, and its "one-paragraph completion summary" ARG SCHEMA — not the description — shaped what the model actually wrote (#905).
Two rules, in the order they matter:
A string-typed argument on a model-facing tool is a WRITING PROMPT. It must name what the text has to CONTAIN (findings, evidence, the proposed change), never what GENRE of document it is. A genre name carries a rhetorical mode with it — "summary" implies retrospection, "report" implies an audience, "status" implies a state machine — and the model honours that mode over the persona, because satisfying the schema is a precondition for the call succeeding at all.
Branch when the branch changes what is TRUE about this build (whenTool, gate.HasTool, sciontoolOnPath); delete when it only changes what is TYPICAL. There is no interactive/headless bit to switch on — the session that motivated #909 was a headless daemon with an operator attached over the TUI, i.e. both at once — so a mode-varying description would be computed from a fact that can change after it is baked. One neutral string per tool.
This list is the only thing that stops re-drift, because the review gate reads a changed description as a changed line of code. Precedent: markTaskDoneRepeatStatus is already a constant asserted by a test because the content IS the contract.
It lives here rather than beside any one catalog because the five packages that register model-facing tools — pkg/tools, pkg/tools/agentic, pkg/agent/background, pkg/agent/autonomous and pkg/agent — must not drift from each other. Two return tools disagreeing about whether a status line is acceptable is exactly the defect the audit found.
#909 shipped saying four, and pkg/agent was the missing one (#919) — which meant mark_task_done, the tool whose description started this whole thread, was the single tool with no re-drift guard. If you add a package that registers a tool a model can call, it gets a description_neutrality_test.go and this sentence gets a new name in it. The count is load-bearing: it is the only record of what "every registered tool" is being claimed over.
Matching is on lowercased text, so entries here must be lowercase.
Functions ¶
func ApproverContext ¶
ApproverContext returns ctx carrying an approver context with a non-empty task, which is what a turn context gives the gate.
func ApproverToolContext ¶
ApproverToolContext is ApproverContext as an adkagent.Context, for calling a tool's handler directly. Only the context.Context methods are backed; anything else panics on the nil embedded interface, which is the right failure for a handler that needs more.
func ModelFacingBanViolations ¶
ModelFacingBanViolations returns one message per banned phrase found in text. Empty means clean.
func ModelFacingText ¶
ModelFacingText returns every string a tool puts in front of a model: its description plus its whole arg schema. The second return is false when the tool does not expose a declaration, i.e. its arg schema went unscanned and the caller should say so rather than pass quietly.
The schema goes in as marshalled JSON rather than a typed walk. functiontool populates ParametersJsonSchema (an `any` holding a jsonschema document), not the typed genai.Schema, so there is no struct to walk — and scanning the serialized form is what keeps this robust to a nested object or an array-of-objects arg growing a description later, which a hand-written walk would silently miss.
func PruneWalkDir ¶
PruneWalkDir reports whether a directory met during a walk of this repo's source should be skipped entirely. path is the directory's full path, name its base; root is the walk root and is never pruned, however it happens to be named.
Every dot-directory is pruned, which is a rule about working state rather than a list of known offenders. `.git` is the obvious one; `.claude` is the one that actually broke a suite, because this repo's git worktrees live there and each is a **full second checkout**, so a walk that descends into it reports another branch's source as though it were this tree's. The 2026-09-04 analysis saw that as two test failures on a machine with 26 worktrees present, against a tree that passes `go test ./...` from a clean shallow clone (#964).
Pruning the class rather than the names is the point: `.claude` was added by name after it broke something, which left the next tool to put a checkout under a dot-directory to break it again. A suite that fails on the developer's untracked working state teaches people to ignore the suite, and that is a worse outcome than any single false negative it might now miss — a walk of this repo's own source has no business in a dot-directory in the first place, since nothing the module builds or ships lives in one.
Callers keep their own additional exclusions (an unswept top-level tree, a self-exemption) where those are about what the walk *means* rather than about what counts as this repo's source.
func RepoRoot ¶
RepoRoot walks up from startFile to the directory holding go.mod. Callers pass their own compiled-in path from runtime.Caller, so the answer is derived from the source tree rather than from the working directory, which `go test` sets to the package directory.
func UndeclaredArgRefs ¶
UndeclaredArgRefs returns the argument names a tool's own description tells the model to populate that the tool does not actually declare.
This exists because #909's first draft rewrote the autonomous loop's default report_done description to say "put your actual findings in the result argument" — and that branch builds a coretools.NewLifecycleTool, whose arguments are {state, detail}. ADK validates function args with additionalProperties:false, so a model that obeyed would emit a hard validation error and the run would never receive its done signal. The two return tools in this repo take different argument names, and a sweep that only reads prose cannot tell that apart from a wording change.
The second return is false when the tool exposes no declaration or no arg schema, i.e. nothing was checked.
Types ¶
type ApproverProbe ¶
type ApproverProbe struct {
// contains filtered or unexported fields
}
ApproverProbe is a permissions.Approver that records every request and denies it, so a tool under test reaches the approver and then does nothing: no command runs, no file is written, no request is sent. It exists to prove a call site hands the gate the call's full arguments (#1175 phase 2).
func AutoGate ¶
func AutoGate(t testing.TB, eligible ...string) (*permissions.Gate, *ApproverProbe)
AutoGate returns a ModeAuto gate with no prompter whose approver is the returned probe and may decide calls matching eligible ("tool:pattern", as in permissions.allow). Pair it with ApproverToolContext: without an approver context every call escalates before the approver is asked.
func AutoGateInScope ¶
func AutoGateInScope(t testing.TB, root string, eligible ...string) (*permissions.Gate, *ApproverProbe)
AutoGateInScope is AutoGate with root as the declared path scope (the project root), for file tools: a write outside the scope is a path-scope prompt, which never reaches the approver.
func (*ApproverProbe) Judge ¶
func (p *ApproverProbe) Judge(_ context.Context, req permissions.ApproverRequest) (permissions.Verdict, error)
Judge records req and denies it.
func (*ApproverProbe) Requests ¶
func (p *ApproverProbe) Requests() []permissions.ApproverRequest
Requests returns what the approver was asked, in order.
func (*ApproverProbe) RequireArgs ¶
func (p *ApproverProbe) RequireArgs(t testing.TB, toolName string, want ...string)
RequireArgs fails t unless the approver was asked exactly once, about toolName, with Args containing every one of want. A want that only the full arguments carry — write_file's content, a prompt past byte 200 — is what distinguishes a call site that passes its arguments from one that passes the summary alone.
type FakeModel ¶
type FakeModel struct {
ModelName string
Script []ScriptedResponse
// contains filtered or unexported fields
}
FakeModel implements model.LLM with scripted output for tests.
Each call to GenerateContent advances the script by one entry. If the script is exhausted, an empty TurnComplete response is returned.
func (*FakeModel) Calls ¶
Calls returns the number of GenerateContent invocations seen so far. Useful in tests that want to assert on call count.
func (*FakeModel) GenerateContent ¶
func (f *FakeModel) GenerateContent(_ context.Context, _ *model.LLMRequest, stream bool) iter.Seq2[*model.LLMResponse, error]
GenerateContent yields the next scripted response. When stream is true, each TextChunk is emitted as its own Partial event; when false, only the final consolidated event is emitted.
type ScriptedResponse ¶
ScriptedResponse describes one turn of FakeModel output.
TextChunks are emitted as individual streaming Partial events (one per chunk) when the caller asks for a stream. After the chunks, FakeModel emits a single TurnComplete=true event whose Content is the concatenation of all chunks. In non-streaming mode (stream=false) only the final TurnComplete event is emitted.
InputTokens / OutputTokens populate the final event's UsageMetadata when set, so usage-tracking tests don't need a real model.