Documentation
¶
Overview ¶
This file is the version half of the package (#680): which daemon release a recipe's config actually needs.
The tool-reachability check in recipecheck.go answers "can the agent reach what the content names" against the CURRENT source tree. That is the right question for a developer and the wrong one for an operator, who runs a pinned image. pkg/config does not set DisallowUnknownFields, so a 2.8.0 binary handed this repo's gke-troubleshoot config boots cleanly, silently drops `alerts` and `tools.wait_and_verify`, registers neither tool, and then hands the model a skill that instructs it to call both. Structurally that is #644 one layer down: the config states a property, the deployed runtime does not have it, and nothing says so.
The three pieces here:
- GatedFeatures maps a config path to the first release that understands it.
- RequiredVersion reads a recipe's config and returns the highest such release it depends on.
- ReleasedVersions reads CHANGELOG.md for the set of versions that exist at all, so a pin can be checked for existence and not just for order. kube-platform-agent pinned "2.9.0", which has never been released — an ordering check alone waves that through.
imagepin.go compares the answer against what the recipe's overlays actually pin.
Package recipecheck cross-checks a config-only recipe's skill content against the tool surface its own config actually produces (#645).
The gke-troubleshoot-agent recipe shipped green while its playbook told the model to run `kubectl rollout undo` — with `bash` in the recipe's own `tools.disable` list and no MCP tool exposing that verb (#644). The existing recipe tests validated *structure* (does the config parse, do the skills load) and structure was fine. Nothing checked executability.
This package is the missing check. It answers one question per finding: "the content names this — can the agent reach it?" Reachability is not re-derived here; it is read off the real registry by running the same tools.Default() → Disable(…) → tools.Build(…) sequence cmd/core-agent runs at boot, so a change to registration conditions can't drift from what the checker believes.
What is and isn't decidable offline ¶
An MCP server's tool list only exists once you can dial the server, and recipe tests run with no credentials and no cluster. So the checker does not claim to know whether `gke_get_k8s_resource` exists. It checks the things that ARE decidable without a network:
A built-in named in content but absent from the built registry — because it's in `tools.disable`, or because a registration precondition isn't met (`fetch_url` needs a URL allowlist, `alert` needs targets). This is the #644 failure exactly.
A shell command named as something to run while `bash` is not registered. `kubectl` is a tool reference too; its reachability is `bash`'s. "Named as something to run" is load-bearing: the name has to sit in an *executable position* — inside a shell code fence, on a `$ ` transcript line, or in an inline code span that invokes it — a span reading "kubectl apply -f x.yaml": the command, then an argv. A skill that correctly disclaims the shell — "there is **no** `kubectl` and **no** `gcloud`" — names it with no argv and is a mention, not a step (#766). See executablePosition; the argv requirement, not the fence, is what carries this rule.
A double-underscore MCP name (`gke__get_pod`). pkg/mcp/namespace.go joins with ONE underscore, so these match nothing — and an unmatched name is not a config error, so it fails silently at call time (#648).
A name in an unambiguous *tool position* — `wait_and_verify`'s `tool:` argument, or a `tools.wait_and_verify.poll_allow` entry — that is neither a registered built-in nor namespaced onto a server the recipe declares. In a tool position there's no ambiguity with config keys, so this can be checked hard.
A `wait_and_verify` target in content that is missing from `poll_allow`, which the runtime refuses at call time.
A populated `poll_allow` in a recipe whose `wait_and_verify` is not registered at all — a list of assertions about a tool that isn't there.
Scopes ¶
A recipe is not one tool surface. The parent loads its own skills/ plus one tree per `content_roots` entry, and since #619 each declarative subagent with a `root` loads a SEPARATE tree — its own AGENTS.md, skills/, and mcp.json — that the parent never sees. Content under a subagent root is only ever handed to that subagent, so it is checked against that subagent's effective catalog: the parent's registry narrowed by the `subagents[].tools` allowlist, and the root's OWN mcp.json narrowed by `subagents[].mcp`. A skill under `cluster/` that names a tool the parent has and the `cluster` subagent does not is unreachable in the only place it is ever loaded, and checking it against the parent would wave it through (#766).
A subagent root does not itself compose `content_roots`: cmd/core-agent's loadSubagentRoot reads <root>/AGENTS.md, <root>/skills/ and <root>/mcp.json and never loads a config.json from the root, so there is no content_roots list to follow there.
Deliberately NOT checked: bare snake_case tokens in prose. `poll_allow`, `require_plan_artifact` and `imagePullSecrets` are shaped exactly like tool names, and a checker that guessed would be turned off within a week. The tool-position rule buys the same coverage without the guessing.
Also deliberately NOT scanned: AGENTS.md. A hardened persona states its limits by naming them — gke-troubleshoot-agent's says "`bash` is disabled […] no `kubectl`, no `gcloud`, no `curl`" and lists the four disabled write tools — so scanning it yields 8 findings on the recipe #644 exists to have fixed. Negation is the dominant idiom there and this checker cannot read it, so it stays out. The cost is real and worth stating: a promise the persona makes and the config cannot keep is not caught here.
Two knowing divergences ¶
Rule A gets NO executable-position gate, so a skill that writes "there is no `bash` here" still costs a finding, and only rule D forgives the disclaimer. That asymmetry is deliberate. The position rule's teeth are its argv requirement (see executablePosition), and a built-in has no argv: `record_plan` IS the invocation form, so a span rule applied to rule A would forgive every real hit too — including the built-in half of #644, which is the class this package exists for. Rule D can afford the gate because a CLI carries arguments; rule A cannot. Skill content phrases built-in limits without naming the tool ("there is no shell to fall back to"), which is how the shipped content is written.
subagents[].skills is NOT modeled. That field name-scopes which of the root's skills the subagent may load, so a skill file left in the tree but absent from the list is scanned here and never loaded at runtime — this package over-reports it. Modeling it means parsing SKILL.md frontmatter for the declared name, since the list keys on names and not paths; until a recipe uses it, over-reporting is the fail-loud direction and a dead file in a content root is worth a finding anyway.
The deploy-time counterpart ¶
Everything above asks whether the recipe's content is executable against the tool surface its config produces — on the daemon this repo builds today. minversion.go and imagepin.go ask the other half: whether the image the recipe's manifests actually ship can produce that surface at all (#680). It is the same bug one layer down. pkg/config has no DisallowUnknownFields, so a daemon older than a config feature does not fail on it; it boots clean, drops the block, and hands the model a skill naming tools that were never registered — which is exactly what this package's checks are blind to, because they run against HEAD's registry and not against the pinned tag's. See CheckDeployPins.
Index ¶
Constants ¶
const FoldTrailerPrefix = "_Pre-release history: cut incrementally as "
FoldTrailerPrefix is the literal dev/release/cut-ga-tag.sh writes when it folds pre-release sections into a GA entry. See ReleasedVersions.
TestFoldTrailerMatchesReleaseScript asserts this string still appears in cut-ga-tag.sh, so a reword there fails here instead of silently costing this check its memory of every dev tag.
Variables ¶
var DefaultShellCommands = []string{
"kubectl", "gcloud", "helm", "docker", "terraform", "curl", "istioctl",
}
DefaultShellCommands is the set of CLIs a Kubernetes/cloud playbook reaches for by reflex. Each one is a promise the agent cannot keep unless `bash` is in its catalog.
var GatedFeatures = []GatedFeature{
{
Path: "alerts.targets",
Min: "2.9.0-dev.1",
Why: "the `alert` tool registers only when a target is configured (#607). " +
"An older daemon drops the whole `alerts` block, never registers the tool, " +
"and skill content that calls alert() is naming a tool that is not in the catalog",
},
{
Path: "tools.wait_and_verify",
Min: "2.9.0-dev.1",
Why: "the `wait_and_verify` tool and its poll_allow assertion list (#672). " +
"An older daemon has no such tool, so a skill that treats `verified: true` as the " +
"only grounds for RESOLVED can never reach RESOLVED",
},
{
Path: "tools.call_peer",
Min: "2.9.0-dev.1",
Why: "the `call_peer` tool registers off this block; an older daemon drops it and never registers the tool",
},
{
Path: "subagents",
Min: "2.9.0-dev.1",
Why: "declarative subagents and the `spawn_agent` tool they register (#602). " +
"An older daemon drops the roster and runs as a single agent, so every delegation " +
"instruction in the content is dead",
},
{
Path: "subagents[].root",
Min: "2.9.0-dev.1",
Why: "a per-subagent content root — its own AGENTS.md, skills/ and mcp.json (#619). " +
"An older daemon drops it and the subagent boots with no instructions and no skills",
},
{
Path: "subagents[].budgets",
Min: "2.9.0-dev.4",
Why: "a per-delegation turn/cost/wall-clock cap honored on both doors the subagent " +
"is reachable through (#713). An older daemon drops the block and runs the " +
"subagent uncapped while the config reads as though it were bounded",
},
{
Path: "model.builtin_tools",
Min: "2.9.0-dev.4",
Why: "the provider's server-side built-ins — web search, URL fetching, code execution (#876). " +
"An older daemon drops the block and runs whatever the provider defaults to, which for " +
"Gemini means search and URL fetching stay ON; a recipe that declared `web_search: false` " +
"reads as though the agent cannot reach the public internet when it still can",
},
{
Path: "subagents[].model.builtin_tools",
Min: "2.9.0-dev.4",
Why: "the same block on a subagent's own model (#876). Same silent-reversal shape, and the " +
"one that matters most: a scoped read-only subagent is exactly where an unnoticed " +
"search tool turns a bounded investigation into an open-ended one",
},
{
Path: "content_roots",
Min: "2.9.0-dev.1",
Why: "extra instruction/skill trees loaded from outside the agents dir (#610). " +
"An older daemon loads only the agents dir, so most of the recipe's content never reaches the model",
},
{
Path: "permissions.plan_mode",
Min: "2.9.0-dev.1",
Why: "plan_mode selects whether `record_plan` is registered at all; an older daemon ignores the setting",
},
{
Path: "checkpoint.mode",
Min: "2.9.0-dev.5",
Why: "which parties may declare a task boundary (#905). `operator` withholds the " +
"`mark_task_done` tool while keeping /done and the heuristic; an older daemon drops " +
"the block and registers the tool anyway, so a recipe that took the model's trigger " +
"away still has a model that can end its own task",
},
{
Path: "safety.watchdog",
Min: "2.9.0-dev.1",
Why: "the runaway-loop watchdog's enforce/warn selection (#623); an older daemon runs without the backstop the config asked for",
},
{
Path: "safety.bash_search_gate",
Min: "2.9.0-dev.1",
Why: "the bash search gate; an older daemon ignores it and leaves the gate off",
},
{
Path: "agent.auto_continue",
Min: "2.8.0",
Why: "auto-continue of a restart-interrupted turn (#559); an older daemon ignores the block, including an explicit opt-out",
},
{
Path: "attach.multi_session",
Min: "2.4.0",
Why: "the multi-session attach substrate; an older daemon serves a single session and ignores the auth table",
},
{
Path: "agent.max_turn_cost_usd",
Min: "2.4.0",
Why: "the per-turn spend cap; an older daemon runs uncapped",
},
{
Path: "agent.max_session_cost_usd",
Min: "2.4.0",
Why: "the per-session spend cap; an older daemon runs uncapped",
},
}
GatedFeatures is the table. It is deliberately NOT the whole config surface.
Strictly, every field is version-gated: an older daemon drops any key it does not know. What makes a drop worth a red build is what the config was ASSERTING — this table covers the paths that (a) register a tool, (b) change the agent topology or the content the agent loads, or (c) assert a safety property. Those are the drops that leave the model holding a promise the runtime cannot keep, which is the #644 shape. `agent.max_steps` silently reverting to its default is a degradation an operator can see in the logs; `alerts` silently vanishing is a pager that never fires.
Entries at or below the current major's first release (2.0.0) are omitted: nothing can pin below v2 and still be this daemon.
Adding an entry: find the first release containing the field with
sha=$(git log --reverse --format=%H -S'json:"<field>' -- pkg/config | head -1) git tag --contains "$sha" --sort=version:refname | head -1
Forgetting to add one is caught from the other side — TestConfigSurfaceIsAccountedFor fails the build when pkg/config grows a path this package has never seen, which forces the question.
Functions ¶
func ConfigSurface ¶
func ConfigSurface() []string
ConfigSurface returns every JSON path in config.Config, sorted.
Grammar: struct fields join with ".", a slice whose elements are structs contributes "[]", a map whose values are structs contributes "{}". A container of scalars is a leaf, since there is nothing underneath it to name. So the alert webhook env var is "alerts.targets[].url_env", a hook's command is "hooks{}[].command", and the disable list is just "tools.disable".
This exists to make the GatedFeatures table's blind spot loud. A hand-maintained table's real failure mode is not a wrong row, it is a missing one — someone ships the next `alerts`-shaped block and never thinks about deployed daemons. Fingerprinting the surface turns that omission into a failing test on the PR that adds the field.
Types ¶
type Finding ¶
type Finding struct {
File string // path relative to the config root
Line int // 1-indexed
Name string // the tool or command named
Reason string // why it is unreachable
Waived bool // matched a Policy.WaiveFileGlobs pattern
}
Finding is one named-but-unreachable tool reference.
func Check ¶
Check returns every unreachable tool reference in r's skill content. A nil error with an empty slice means the recipe is executable as written, as far as anything decidable without a live MCP server goes.
"r's skill content" spans every scope the recipe loads — its own skills/, each content_roots tree, and each rooted subagent's tree — with each scope judged against the catalog it is actually loaded with. See the Scopes section of the package doc.
type GatedFeature ¶
type GatedFeature struct {
// Path is a config JSON path in the grammar ConfigSurface writes:
// struct fields joined with ".", "[]" for a slice of structs, "{}"
// for a map of structs. It must resolve against the current
// config.Config — RequiredVersion errors if it does not, so a
// renamed field turns into a red build instead of a dead table row.
Path string
// Min is the first RELEASE that understands Path, spelled as the
// GHCR tag (no leading "v"). It is the first release, not the first
// GA: 2.9.0-dev.1 ships the v2.9 config surface and a recipe pinned
// to it is correct, so requiring "2.9.0" here would be a lie in the
// strict direction.
Min string
// Why states what an older daemon does instead, in operator terms.
Why string
}
GatedFeature is one config path a daemon older than Min does not know about. Because pkg/config does not set DisallowUnknownFields, "does not know about" means "drops without a word".
type Policy ¶
type Policy struct {
// ShellCommands are argv[0]-style names whose presence in skill
// content means "run this in a shell". Reachable only when `bash` is
// registered. Empty uses DefaultShellCommands.
ShellCommands []string
// WaiveFileGlobs are filepath.Match patterns, matched against the
// path relative to Dir, whose findings are downgraded to a logged
// note. Use for vendored content the recipe deliberately does not
// modify. WaiveReason must be set alongside it.
//
// Waived findings are still counted and logged: "we ship 40 unreachable
// tool references in a vendored snapshot" is a fact a reviewer should
// see, not one the test should swallow.
WaiveFileGlobs []string
// WaiveReason explains why WaiveFileGlobs is justified. Required when
// WaiveFileGlobs is non-empty; the checker errors without it, so a
// waiver can never be added silently.
WaiveReason string
// WaiveMinFindings is a per-glob floor, keyed by an entry of
// WaiveFileGlobs: Check errors when that tree yields FEWER findings
// than the floor.
//
// A waiver is an assertion about a body of content ("these 18 vendored
// skills are Hermes-shaped and we accept that"), and an assertion whose
// subject can silently disappear is worse than no assertion. That is
// literally #766: kube-platform-agent moved six skills under a subagent
// root, the checker stopped seeing them, no test failed, and the waiver
// text went on claiming to cover both trees for months. A floor turns
// the next such move into a red build instead of a quieter number.
//
// Set it below the current count, not at it — the point is to catch a
// tree going dark, not to pin vendored content byte-for-byte.
WaiveMinFindings map[string]int
}
Policy tunes a check for one recipe.
type Recipe ¶
type Recipe struct {
// Name identifies the recipe in failure messages. For a config root
// at examples/foo/deploy/base/config it is "foo (deploy/base/config)".
Name string
// Dir is the config root itself — the path you would pass to
// config.Load / mcp.Load / skills.Load.
Dir string
}
Recipe is one resolved config root: the directory holding config.json, optionally mcp.json, and optionally a skills/ tree.
type Requirement ¶
type Requirement struct {
// Min is the highest GatedFeature.Min among the features in use.
Min Version
// Reasons are the features in use, in table order. Empty means the
// config uses nothing gated.
Reasons []Reason
}
Requirement is the floor a recipe's config puts under the daemon image it is deployed with.
func RequiredVersion ¶
func RequiredVersion(cfg *config.Config) (Requirement, error)
RequiredVersion returns the minimum daemon release cfg needs.
"In use" is read off the config value by reflection rather than by a hand-written predicate per feature, so the table stays pure data and cannot disagree with the schema about what a field is called.
func (Requirement) Empty ¶
func (r Requirement) Empty() bool
Empty reports whether the config asks for nothing beyond the baseline.
func (Requirement) Unmet ¶
func (r Requirement) Unmet(v Version) []Reason
Unmet returns the features a daemon at version v would drop. A too-old pin should be reported against these and not against every gated feature in the config: a 2.8.0 image honours attach.multi_session perfectly well, and listing it as a casualty teaches the reader to skim the list.
type Version ¶
type Version struct {
// contains filtered or unexported fields
}
Version is one of this repo's release tags, validated to a full MAJOR.MINOR.PATCH[-prerelease] shape and ordered by golang.org/x/mod/semver.
The ordering is delegated rather than hand-rolled. golang.org/x/mod is already in this module's graph (golang.org/x/text depends on it), so naming it directly costs one go.mod line at the version the build already selected, two go.sum lines, and no new modules — measurably cheaper than owning a semver §11 comparator whose prerelease rules (numeric identifiers compare numerically, numeric sorts below alphanumeric, a longer identifier list wins a tie) are easy to get subtly wrong and hard to notice when you do.
The shape check stays local because semver.IsValid is deliberately lax about it: IsValid("v2.9") is true, and a `newTag: "2.9"` that quietly parsed as a version would be exactly the kind of quiet this check exists to remove.
func ParseVersion ¶
ParseVersion parses a release tag. A floating tag ("main", "main-1a2b3c4", "latest") is not a version and returns an error — callers are expected to treat that as a finding, not as a pass.
func ReleasedVersions ¶
ReleasedVersions returns every version CHANGELOG.md records as released, newest first.
Why the changelog and not git tags ¶
The obvious oracle for "does this tag exist" is `git tag --list`, and this repo already has a presubmit that uses it. But that presubmit runs in a job whose checkout sets `fetch-depth: 0` specifically to make tags visible; the `test` job that runs `go test ./...` does not, and neither does a contributor's shallow clone, a `git archive` tarball, or a module extracted from the proxy. An oracle that answers "no versions exist" in those environments either red-builds the whole examples tree for an environmental reason or — far worse — gets a `len(tags) == 0` skip bolted onto it, at which point the check is a no-op exactly where nobody is watching. The changelog is a file in the tree; it reads the same everywhere.
What the changelog actually answers ¶
Strictly it answers "did someone write release notes", not "was an image published". Those coincide here by construction: dev/release/cut-dev-tag.sh and cut-ga-tag.sh promote [Unreleased] into a versioned section as part of the release commit, and release-images.yml publishes off the tag pushed with it. The gaps run in both directions and both are narrow:
- Permissive: the section lands in the release commit slightly BEFORE the tag is pushed and the image is built, so for a few minutes a version reads as released with no image behind it; an abandoned cut would leave the same trace permanently.
- Restrictive: the fold, below.
Neither is a reason to prefer no check. The failure this catches is a pin to "2.9.0" — a version that was never cut at all.
The fold ¶
cut-ga-tag.sh folds every pre-release section since the last GA into the new GA entry and DELETES those sections, which is correct for a human reader and would otherwise be a landmine here: the moment v2.9.0 GA is cut, "## [2.9.0-dev.1]" stops existing and every overlay pinned to it starts failing this check — on the release commit, in the required `test` job, on main. What saves it is that the fold writes the tags it removed into a machine-readable trailer, and this parses that trailer back. TestReleasedVersionsSurvivesTheGAFold is the regression.
Sections written before that trailer convention existed (2.6.0 and earlier) have no such record, so their dev tags are not recoverable. That is acceptable: nothing in this repo can usefully pin a v2.6-era pre-release, and the failure is the loud direction.
type VersionFinding ¶
type VersionFinding struct {
// Overlay is the deploy artifact at fault, relative to the examples
// dir: the directory an operator would `kubectl apply -k`, the
// Dockerfile whose FROM names the daemon, or the script that deploys
// it.
Overlay string
// Pin is the reference as written, or "" when there is none.
Pin string
// Reason says what is wrong and what to do about it.
Reason string
}
VersionFinding is one deploy artifact whose daemon pin cannot be shown to satisfy the recipe's own config.
func CheckDeployPins ¶
func CheckDeployPins(examplesDir string, r Recipe, released []Version) ([]VersionFinding, error)
CheckDeployPins reports every way r's deploy artifacts disagree with r's config about which daemon release they are for.
The three rules ¶
- A pin that names a version must name a version that EXISTS. This is the rule kube-platform-agent broke by pinning "2.9.0", a tag this repo has never cut. An ordering check alone waves that through and the Pod fails ImagePullBackOff.
- A pin must be ORDERABLE. A floating tag ("main", "main-<sha>", "latest") is not a version, cannot be compared, and moves under the operator between one `kubectl apply` and the next; a bare digest carries no version either, so it has to say which release it came from.
- When the config uses a version-gated feature, the pin must be at least the release that introduced it.
Rules 1 and 2 are unconditional. An earlier revision gated them on the recipe having a version-gated config, on the theory that a recipe asserting nothing has nothing for the pin to contradict. That was wrong twice over. Empirically it left four of the six discovered recipes with an empty floor and therefore no check at all — reverting cloud-run-deploy's FROM to the floating `:main` this issue exists to fix produced a clean run. And structurally the premise does not hold: an unorderable pin is not "unconstrained", it is a pin whose contents this check cannot see, which is the same blindness under a different name. Rule 3 is inherently conditional; it is a no-op when the floor is empty, which needs no special case.
The artifacts ¶
Four deploy shapes are covered, because the repo ships all four and the bug is identical in each: a kustomize overlay's `images:` transformer, a container `image:` in a manifest the overlay composes, a recipe Dockerfile's `FROM`, and a literal daemon reference in a deploy script. internal/imagepin documents each and its limits.
func (VersionFinding) String ¶
func (f VersionFinding) String() string
Source Files