Documentation
¶
Overview ¶
Package registry — always-load annotation + deterministic sort for tools/list responses.
Two MCP-spec-compliance hooks live here:
SortToolsByName — emits each tool in name-sorted order at the MCP boundary. Post-SEP-2133 the spec calls out deterministic order so prompt-cache hits and client-side caching stay stable across runs. mcp-go's handleListTools already sorts in the base case, but session-tool merges, filters, and other AfterListTools hooks can disturb the order; pinning the sort here as the LAST hook guarantees the wire output stays deterministic regardless.
AnnotateAlwaysLoad — injects `_meta["anthropic/alwaysLoad"]: true` on the eight hot tools (Bash / Read / Edit / Glob / Grep / WebFetch / WebSearch / ToolSearch). Anthropic's "Code execution with MCP" engineering recipe lets Claude Code respect this hint to keep specific tools eager while deferring the long tail. Tools without AlwaysLoad=true are left untouched — the resulting JSON simply lacks the annotation, preserving backward compatibility for clients that don't read the namespace.
Both functions mutate the result in place; they're wired into the AfterListTools hook chain in internal/server/server.go.
Package registry — OpenTelemetry W3C Trace Context propagation over MCP `_meta`, per SEP-414.
MCP request `_meta` and response `_meta` carry W3C Trace Context (https://www.w3.org/TR/trace-context/) so a client and a server can stitch a single tool call into one distributed trace. The two fields are:
- `traceparent` — `00-<32-hex-trace>-<16-hex-span>-<2-hex-flags>`
- `tracestate` — vendor-specific key-value list, optional
Direction of flow:
- Client serialises the active span as `_meta.traceparent` (and optionally `_meta.tracestate`) on the CallToolRequest.
- Server middleware extracts those keys and parents the handler's ctx via the W3C TraceContext propagator. With the observability subsystem off, the remote span context is still attached but no live span is recorded — Inject on the way out simply re-emits the same trace_id/span_id so the client's reply correlates against its own active span. With observability on, agents.NewSupervisor / handler-side `obs.StartSpan` open a child span on top, and the response meta carries that child span's id so the client sees the whole subtree.
- Server middleware Inject()s the (possibly child-of-remote) span context back onto the response's `_meta` envelope.
Fallback: when the request carries no `_meta.traceparent`, the middleware honours the `TRACEPARENT` (and `TRACESTATE`) process env vars. This keeps the v0.22 env-driven shim working for hosts that don't speak SEP-414 yet — they just set the env on the spawned MCP child and clawtool stitches them in.
Backwards compatibility: requests with NO trace context (no `_meta.traceparent` and no env var) are passed through unchanged; the response carries no `traceparent` either, so older clients that weren't reading these keys see exactly the pre-SEP-414 wire shape.
Package registry — typed manifest of every clawtool MCP tool. Codex's #1 ROI architectural recommendation (BIAM task a3ef5af9): collapse server.go's hand-maintained list of RegisterX calls + CoreToolDocs's parallel description list + the slash-command + skill routing-map cross-references into ONE typed source of truth.
Step 1 (this commit): ship the package + types + an empty Manifest. server.go is unchanged. Subsequent commits migrate tool registration through the registry, one cohesive group at a time, with the surface_drift_test guarding each step.
Why type-driven, not config-driven: a TOML manifest would need a runtime registry of register funcs anyway. Putting the register-fn pointer ON the typed ToolSpec keeps the type system honest — a misspelled tool name fails to compile, not at boot.
Why a separate package, not a method on core: core/ already owns ~30 RegisterX functions. Importing core to build the manifest, then having core import registry to look up specs, would be a cycle. registry stays a leaf — core (and any future tool source) imports it; server.go calls registry.Apply.
Package registry — TypeScript stub export for code-mode hosts.
Anthropic's "Code execution with MCP" recipe (and Cloudflare's earlier "Code Mode" pattern) presents the MCP tool catalog as a TypeScript file tree the agent imports from. Quoted reduction from that recipe: 150 K → 2 K tokens (98.7%) on heavy tool-call loops. The agent writes code instead of round-tripping each `tools/call`.
`clawtool tools export-typescript --output <dir>` walks the manifest and emits one `.ts` file per registered tool, plus a barrel `index.ts`. The MVP shape is minimal: tool name, description (docstring), and a typed function signature whose input + output are `any` for now. Full JSON-Schema → TypeScript translation lands in a follow-up cut once we decide how to represent oneOf / $ref / nested objects without bringing in a full schema-codegen dependency.
The point of the MVP: operators using a code-mode host (Codex 0.125+ rollout-tracing now records "code-mode edges"; Anthropic blog endorses the pattern) can already adopt clawtool's tool catalog as a TypeScript module today, with the agent reading the docstring to learn what each tool does. Type fidelity arrives incrementally.
Package registry — usage-hint enrichment for tools/list responses.
MCP's tools/list result carries each tool's Description, but Description answers WHAT the tool does. Calling agents (Claude Code, Codex, OpenCode) frequently need a HOW pointer too: "when do I pick this over the similar one", "what's a common mistake", "one concrete example". We curate that as a separate `UsageHint` field on the ToolSpec and surface it via the MCP-spec-defined `_meta` extension envelope:
{
"name": "Glob",
"description": "List files matching a glob pattern...",
"_meta": {
"clawtool": { "usage_hint": "Use when you need a file LIST..." }
}
}
`_meta` is the canonical extension surface in the MCP spec — strict clients ignore unknown sub-keys gracefully, so adding the `clawtool` namespace under it is non-breaking. (We considered hanging the hint off `annotations.clawtool` per ADR-008's extension-namespace pattern, but mcp-go's typed `ToolAnnotation` struct has no extension map and re-shaping its MarshalJSON would force a vendored fork. `_meta.AdditionalFields` is the supported path.)
Wiring: server.go installs an `AddAfterListTools` hook that calls `EnrichListToolsResult` with the manifest's UsageHints map. The hook mutates the result in place before mcp-go serializes it.
Index ¶
- Constants
- func AnnotateAlwaysLoad(result *mcp.ListToolsResult, alwaysLoad map[string]struct{})
- func EchoTraceContextOnResult(ctx context.Context, res *mcp.CallToolResult)
- func EnrichListToolsResult(result *mcp.ListToolsResult, hints map[string]string)
- func ExtractTraceContextFromRequest(ctx context.Context, req mcp.CallToolRequest) context.Context
- func IsValidCategory(c Category) bool
- func SortToolsByName(result *mcp.ListToolsResult)
- func TraceContextMiddleware() server.ToolHandlerMiddleware
- type Category
- type Manifest
- func (m *Manifest) AlwaysLoadSet() map[string]struct{}
- func (m *Manifest) Append(spec ToolSpec)
- func (m *Manifest) Apply(s *server.MCPServer, rt Runtime, pred func(toolName string) bool)
- func (m *Manifest) ExportTypeScript(outDir string) ([]string, error)
- func (m *Manifest) LiveDescriptions(rt Runtime) map[string]string
- func (m *Manifest) Names() []string
- func (m *Manifest) SearchDocs(pred func(toolName string) bool) []search.Doc
- func (m *Manifest) SortedNames() []string
- func (m *Manifest) Specs() []ToolSpec
- func (m *Manifest) SyncDescriptionsFromRegistration(rt Runtime)
- func (m *Manifest) UsageHints() map[string]string
- type RegisterFn
- type Runtime
- type ToolSpec
Constants ¶
const ( TraceparentMetaKey = "traceparent" TracestateMetaKey = "tracestate" // TraceparentEnv / TracestateEnv name the process-env // fallback inputs. Uppercased per the v0.22 convention // (matches `TRACEPARENT` documented in the wiki). TraceparentEnv = "TRACEPARENT" TracestateEnv = "TRACESTATE" )
W3C Trace Context header names. Per the spec these are lowercase and used as both the HTTP header keys and the MCP `_meta` keys (SEP-414 reuses them verbatim so a host can pipe HTTP-extracted values straight through without renaming).
Variables ¶
This section is empty.
Functions ¶
func AnnotateAlwaysLoad ¶ added in v0.22.115
func AnnotateAlwaysLoad(result *mcp.ListToolsResult, alwaysLoad map[string]struct{})
AnnotateAlwaysLoad walks `result.Tools` and, for any tool whose name is in `alwaysLoad`, attaches `_meta["anthropic/alwaysLoad"] = true`. Tools not in the set are left untouched.
Pre-existing `_meta` content is preserved: if a tool was already shipped with `_meta.foo = bar` (or `_meta.clawtool.usage_hint = "..."` from the usage-hint hook), this function adds `_meta["anthropic/alwaysLoad"]` alongside without clobbering.
Safe to call with nil/empty inputs: a nil set or nil result is a no-op.
func EchoTraceContextOnResult ¶ added in v0.22.117
func EchoTraceContextOnResult(ctx context.Context, res *mcp.CallToolResult)
EchoTraceContextOnResult writes the active span context from `ctx` onto `res.Meta.AdditionalFields` as `traceparent` (and `tracestate` when non-empty) so the client can stitch the call into its own trace.
Safe to call when ctx has no recording span: a nil res is a no-op, and a ctx with no valid SpanContext leaves res.Meta untouched (no empty `_meta` envelope is created — preserves backwards compat for old-shape responses).
Existing `_meta` content is preserved: this function only writes the two W3C keys, never clobbering peer namespaces (e.g. `clawtool/usage_hint` from EnrichListToolsResult).
func EnrichListToolsResult ¶ added in v0.22.58
func EnrichListToolsResult(result *mcp.ListToolsResult, hints map[string]string)
EnrichListToolsResult walks `result.Tools` and, for any tool whose name appears in `hints`, attaches the hint string at `_meta.clawtool.usage_hint`. Tools without a hint are left untouched — the resulting JSON simply lacks the annotation, preserving backward compatibility for callers that don't read the namespace.
Pre-existing `_meta` content is preserved: if a tool was already shipped with `_meta.foo = bar`, this function adds `_meta.clawtool.usage_hint` alongside without clobbering.
Safe to call with nil/empty inputs: a nil hints map or nil result is a no-op.
func ExtractTraceContextFromRequest ¶ added in v0.22.117
ExtractTraceContextFromRequest reads `_meta.traceparent` / `_meta.tracestate` off `req` (with TRACEPARENT/TRACESTATE env as the fallback) and parents `ctx` with the resulting remote SpanContext via the W3C TraceContext propagator. Returns the (possibly unchanged) ctx; safe to call when the request has no trace context — in that case ctx is returned as-is.
Exposed alongside the middleware so handlers that bypass the MCP-go middleware path (e.g. direct in-process dispatch in tests) can reuse the same extraction logic.
func IsValidCategory ¶
IsValidCategory is the load-time guard. A typo in a ToolSpec's Category field crashes the manifest builder rather than slipping into the wild as a tool that no group lists.
func SortToolsByName ¶ added in v0.22.115
func SortToolsByName(result *mcp.ListToolsResult)
SortToolsByName re-orders `result.Tools` ascending by Name. Safe to call with a nil result. Stable sort so ties (which shouldn't occur — names are unique within a manifest) preserve input order.
func TraceContextMiddleware ¶ added in v0.22.117
func TraceContextMiddleware() server.ToolHandlerMiddleware
TraceContextMiddleware returns a tool-handler middleware that (1) extracts W3C Trace Context from the incoming request's `_meta` (or, as a fallback, from the TRACEPARENT/TRACESTATE process env), parents the handler's ctx with the remote span context, and (2) injects the resulting span context back onto the response's `_meta` so the client can stitch the call into its own trace.
Wire it on server bootstrap via `server.WithToolHandlerMiddleware(registry.TraceContextMiddleware())`.
The middleware is a no-op for requests that carry no trace context AND no env fallback — the response's `_meta` is left untouched, preserving the pre-SEP-414 wire shape for older clients.
Types ¶
type Category ¶
type Category string
Category enumerates the canonical groupings. New categories require code review — adding one without thinking through the existing seven leads to single-tool buckets that no UI can surface.
const ( CategoryShell Category = "shell" // Bash, BashOutput, BashKill, Verify CategoryFile Category = "file" // Read, Edit, Write, Glob, Grep CategoryWeb Category = "web" // WebFetch, WebSearch, BrowserFetch, BrowserScrape, Portal* CategoryDispatch Category = "dispatch" // SendMessage, AgentList, Task*, TaskNotify CategoryAuthoring Category = "authoring" // McpNew/Run/Build/Install/List, SkillNew, AgentNew CategorySetup Category = "setup" // Recipe*, Bridge*, Sandbox* CategoryDiscovery Category = "discovery" // ToolSearch, SemanticSearch CategoryCheckpoint Category = "checkpoint" // Commit, RulesCheck (future: Snapshot, Restore) )
type Manifest ¶
type Manifest struct {
// contains filtered or unexported fields
}
Manifest is the ordered collection of ToolSpec entries. Order matters for two reasons:
- server.go's RegisterX call order today is preserved during incremental migration so behaviour change is observable per-tool.
- tools/list output groups by Category but ties break on manifest order; deterministic output simplifies test fixtures.
func (*Manifest) AlwaysLoadSet ¶ added in v0.22.115
AlwaysLoadSet returns the set of tool names whose ToolSpec has AlwaysLoad=true. Used by the MCP server's tools/list post-processor to inject `_meta["anthropic/alwaysLoad"]: true` onto each entry. Anthropic's Claude Code respects the hint to keep specific tools eager while deferring the long-tail catalog. The map is set-shaped so a TestHotToolsAlwaysLoad invariant can assert exactly the canonical eight (Bash, Read, Edit, Glob, Grep, WebFetch, WebSearch, ToolSearch) carry the flag.
func (*Manifest) Append ¶
Append registers one ToolSpec. Duplicate names panic — the manifest is built at boot, before any user request, so a duplicate is a programmer error worth crashing on.
func (*Manifest) Apply ¶
Apply walks the manifest and calls each spec's Register fn, gated by the caller-supplied predicate. Mirrors server.go's hand-maintained `if cfg.IsEnabled(name) { core.RegisterX(s) }` chain — once the migration completes, server.go calls `manifest.Apply(s, runtime, cfg.IsEnabled)` and that chain disappears entirely.
Specs with a nil Register fn are skipped silently. This is intentional during incremental migration: a spec added to the manifest for documentation purposes (so SearchDocs picks it up) without yet being wired to the new register flow stays harmless until its turn comes.
func (*Manifest) ExportTypeScript ¶ added in v0.22.31
ExportTypeScript writes one .ts file per ToolSpec into outDir, plus an index.ts that re-exports every tool. Returns the list of files created (relative to outDir) so the CLI can echo them back to the operator.
outDir is created when missing. Existing files in outDir are overwritten silently — the export is meant to be idempotent and repeatable on every manifest change.
func (*Manifest) LiveDescriptions ¶ added in v0.22.110
LiveDescriptions is the exported probe for drift-detection tests. Builds the same throwaway-server map SyncDescriptionsFromRegistration uses internally so a unit test can compare manifest specs against the canonical `mcp.WithDescription(...)` source.
func (*Manifest) Names ¶
Names returns every spec name in insertion order. Useful for diff-against-something tests.
func (*Manifest) SearchDocs ¶
SearchDocs flattens the manifest into search.Doc entries for the bleve indexer. Always-on tools always appear; gateable tools are filtered by the caller-supplied gate predicate (typically `cfg.IsEnabled(name).Enabled`). When pred is nil every spec is included.
func (*Manifest) SortedNames ¶
SortedNames returns the manifest's tool names alphabetically. Tests that need deterministic output independent of insertion order use this; runtime code prefers Names() to preserve the gate / display ordering.
func (*Manifest) Specs ¶
Specs returns the manifest contents in insertion order. Caller MUST NOT mutate the slice.
func (*Manifest) SyncDescriptionsFromRegistration ¶ added in v0.22.110
SyncDescriptionsFromRegistration mutates each ToolSpec.Description in place to match what its Register fn ACTUALLY emits via `mcp.WithDescription(...)`. The mechanism: register every spec onto a throwaway *server.MCPServer, then walk `s.ListTools()` and copy each registered Tool's Description back onto the matching spec.
Why this exists: BuildManifest historically hardcoded a Description string per spec; the SAME description was also hand-written in each tool's `RegisterX` body via `mcp.WithDescription(...)`. Two parallel sources drifted (caught post-v0.22.108: the description rewrite touched only one side, so the bleve BM25 ToolSearch index — which reads from the manifest — kept ranking on stale prose while `tools/list` — which reads from the registered Tool — already saw the fresh copy). Calling this function at the end of BuildManifest collapses both sources to one and any future rewrite of either side stays in sync automatically.
Specs whose Register fn is nil (companion specs in multi-tool bundles like RecipeStatus / RecipeApply) keep their hardcoded Description — those bundles are registered through their first spec's Register, so the live description still surfaces under the bundle name; the unit-test invariant (`TestManifestDescriptionsMatchRegistered`) explicitly walks every registered tool, so drift on a companion spec still trips CI.
rt is the same Runtime callers pass to Apply; for description sync it's safe for Index / Secrets to be nil because every Register fn captures those values inside the call HANDLER (not at registration time) — see internal/tools/core/toolsearch.go + websearch.go for the canonical examples.
func (*Manifest) UsageHints ¶ added in v0.22.58
UsageHints returns a {tool name → curated usage hint} map for every spec whose UsageHint is non-empty. Used by the MCP server's tools/list post-processor to inject per-tool guidance under `_meta.clawtool.usage_hint`. Specs without a hint don't appear in the map — callers can range freely without nil-check noise per tool.
type RegisterFn ¶
RegisterFn is the shape every typed register callback adopts. Mirrors mcp-go's AddTool but receives Runtime so register-time dependencies stay explicit — no package-level singletons leak into tool implementations.
type Runtime ¶
type Runtime struct {
// Index is the bleve search index ToolSearch closes over.
// Step 4 wires ToolSearch through the manifest, so this
// field becomes load-bearing rather than aspirational.
Index *search.Index
// Secrets is the secrets store WebSearch reads its API key
// from at registration time. Typed as *secrets.Store at the
// importer's site (server.go / core); registry stays a leaf
// by holding it as `any` and letting the per-tool register
// fn type-assert. The trade-off (slightly worse type safety
// at registration) is preferable to having registry depend
// on internal/secrets — keeps the import graph linear.
Secrets any
}
Runtime carries the cross-cutting dependencies a register fn might need. Passed by value (struct of pointers / interfaces) so the manifest stays composable and tests can stub fields independently. Add fields as new tools demand them; never remove without a deprecation cycle.
type ToolSpec ¶
type ToolSpec struct {
// Name is the canonical MCP tool name. PascalCase per ADR-006.
// MUST be unique within a Manifest; duplicates are a load-time
// error.
Name string
// Description is the one-paragraph human form. Same string the
// tool surfaces via tools/list AND ToolSearch.
Description string
// Keywords feed the bleve BM25 index. Lowercase, single words,
// 3-12 entries is the sweet spot.
Keywords []string
// Category groups tools for introspection / grouping in
// tools/list and the README. See package-level Category*
// constants for the canonical set.
Category Category
// Gate names the config.IsEnabled key for this tool. Empty =
// always-on (BridgeAdd / Verify / SemanticSearch / etc.).
// "Bash" gate also covers BashOutput + BashKill (companions).
Gate string
// Register is the MCP wiring callback. Receives the server +
// per-tool runtime dependencies (search index, secrets store,
// sources manager) via the Runtime struct. Empty when the
// tool is documented in the manifest but registered through
// a legacy direct path — useful during incremental migration.
Register RegisterFn
// UsageHint is curated guidance for calling agents — one to
// three sentences answering "when do I pick this tool over a
// similar one", "what's a common mistake", and (optionally)
// "one concrete example". Distinct from Description: the
// description is the WHAT (one-paragraph action summary),
// the hint is the HOW (decision-time pointer). Empty value =
// no hint surfaced; serializer skips the annotation.
//
// Surface: clawtool's tools/list response carries the hint
// under each tool's `_meta.clawtool.usage_hint` — `_meta` is
// the MCP-spec-defined extension envelope, so strict clients
// ignore it gracefully and tolerant clients (Claude Code,
// Codex 0.125+) can surface it as an inline guidance line.
UsageHint string
// AlwaysLoad marks a tool as eager-load on every Claude Code
// session. Anthropic's "Code execution with MCP" recipe lets
// hosts honor `_meta["anthropic/alwaysLoad"]: true` to keep
// hot tools materialised while deferring the long-tail
// catalog. Default false — only the eight canonical hot
// tools (Bash, Read, Edit, Glob, Grep, WebFetch, WebSearch,
// ToolSearch) flip this on so the manifest stays opt-in
// tight; a runaway flag flip would defeat the deferral
// optimisation. Surfaced at `_meta["anthropic/alwaysLoad"]`
// on each tool's `tools/list` entry.
AlwaysLoad bool
}
ToolSpec is the typed manifest entry for one MCP tool. Every shipped tool is described by exactly one ToolSpec. The fields match the four planes of the shipping contract (docs/feature-shipping-contract.md):
- Name + Description + Keywords → search index + ToolSearch
- Category → introspection + grouping
- Gate → config.IsEnabled subset
- Register → the actual MCP wiring
Slash command + skill row don't live on the spec because they're *file*-shaped (commands/clawtool-X.md, skills/clawtool/SKILL.md routing rows). The surface drift test (internal/server/surface_drift_test.go) cross-references the manifest against those files at test time.