Documentation
¶
Overview ¶
Package recipecheck cross-checks a config-only recipe's skill content against the tool surface its own config actually produces (#645).
The gke-troubleshoot-agent recipe shipped green while its playbook told the model to run `kubectl rollout undo` — with `bash` in the recipe's own `tools.disable` list and no MCP tool exposing that verb (#644). The existing recipe tests validated *structure* (does the config parse, do the skills load) and structure was fine. Nothing checked executability.
This package is the missing check. It answers one question per finding: "the content names this — can the agent reach it?" Reachability is not re-derived here; it is read off the real registry by running the same tools.Default() → Disable(…) → tools.Build(…) sequence cmd/core-agent runs at boot, so a change to registration conditions can't drift from what the checker believes.
What is and isn't decidable offline ¶
An MCP server's tool list only exists once you can dial the server, and recipe tests run with no credentials and no cluster. So the checker does not claim to know whether `gke_get_k8s_resource` exists. It checks the things that ARE decidable without a network:
A built-in named in content but absent from the built registry — because it's in `tools.disable`, or because a registration precondition isn't met (`fetch_url` needs a URL allowlist, `alert` needs targets). This is the #644 failure exactly.
A shell command named as something to run while `bash` is not registered. `kubectl` is a tool reference too; its reachability is `bash`'s.
A double-underscore MCP name (`gke__get_pod`). pkg/mcp/namespace.go joins with ONE underscore, so these match nothing — and an unmatched name is not a config error, so it fails silently at call time (#648).
A name in an unambiguous *tool position* — `wait_and_verify`'s `tool:` argument, or a `tools.wait_and_verify.poll_allow` entry — that is neither a registered built-in nor namespaced onto a server the recipe declares. In a tool position there's no ambiguity with config keys, so this can be checked hard.
A `wait_and_verify` target in content that is missing from `poll_allow`, which the runtime refuses at call time.
A populated `poll_allow` in a recipe whose `wait_and_verify` is not registered at all — a list of assertions about a tool that isn't there.
Deliberately NOT checked: bare snake_case tokens in prose. `poll_allow`, `require_plan_artifact` and `imagePullSecrets` are shaped exactly like tool names, and a checker that guessed would be turned off within a week. The tool-position rule buys the same coverage without the guessing.
Also deliberately NOT scanned: AGENTS.md. A hardened persona states its limits by naming them — gke-troubleshoot-agent's says "`bash` is disabled […] no `kubectl`, no `gcloud`, no `curl`" and lists the four disabled write tools — so scanning it yields 8 findings on the recipe #644 exists to have fixed. Negation is the dominant idiom there and this checker cannot read it, so it stays out. The cost is real and worth stating: a promise the persona makes and the config cannot keep is not caught here. Skill content avoids the same trap by phrasing limits without naming a CLI ("there is no shell to fall back to"), which is how the shipped content is written; a reference that spells out "do not use kubectl" will trip rule D, and should be reworded rather than waived.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var DefaultShellCommands = []string{
"kubectl", "gcloud", "helm", "docker", "terraform", "curl", "istioctl",
}
DefaultShellCommands is the set of CLIs a Kubernetes/cloud playbook reaches for by reflex. Each one is a promise the agent cannot keep unless `bash` is in its catalog.
Functions ¶
This section is empty.
Types ¶
type Finding ¶
type Finding struct {
File string // path relative to the config root
Line int // 1-indexed
Name string // the tool or command named
Reason string // why it is unreachable
Waived bool // matched a Policy.WaiveFileGlobs pattern
}
Finding is one named-but-unreachable tool reference.
func Check ¶
Check returns every unreachable tool reference in r's skill content. A nil error with an empty slice means the recipe is executable as written, as far as anything decidable without a live MCP server goes.
type Policy ¶
type Policy struct {
// ShellCommands are argv[0]-style names whose presence in skill
// content means "run this in a shell". Reachable only when `bash` is
// registered. Empty uses DefaultShellCommands.
ShellCommands []string
// WaiveFileGlobs are filepath.Match patterns, matched against the
// path relative to Dir, whose findings are downgraded to a logged
// note. Use for vendored content the recipe deliberately does not
// modify. WaiveReason must be set alongside it.
//
// Waived findings are still counted and logged: "we ship 40 unreachable
// tool references in a vendored snapshot" is a fact a reviewer should
// see, not one the test should swallow.
WaiveFileGlobs []string
// WaiveReason explains why WaiveFileGlobs is justified. Required when
// WaiveFileGlobs is non-empty; the checker errors without it, so a
// waiver can never be added silently.
WaiveReason string
}
Policy tunes a check for one recipe.
type Recipe ¶
type Recipe struct {
// Name identifies the recipe in failure messages. For a config root
// at examples/foo/deploy/base/config it is "foo (deploy/base/config)".
Name string
// Dir is the config root itself — the path you would pass to
// config.Load / mcp.Load / skills.Load.
Dir string
}
Recipe is one resolved config root: the directory holding config.json, optionally mcp.json, and optionally a skills/ tree.
Source Files