Documentation
¶
Overview ¶
Package trace is the port through which every call an agent makes to Cartograph's MCP server is recorded (docs/adr/0028), for governance and for finding where agents work well and where they vary: which tools are used most, which are refused, where work is redone. A call is recorded by its shape (the tool, the record it names, the keys it sent, its sizes, its outcome and how long it took), never by what it carried, so no document an agent ports and nothing a person wrote reaches a trace. An adapter keeps the calls; "off", the default, keeps none.
Index ¶
Constants ¶
const ( ValueAdding = "value-adding" // a write that was kept Necessary = "necessary" // a read the work needs Waste = "waste" // a defect, or a read repeated as it was )
Value classes of a call, as a value stream counts them.
const ( OK = "ok" Refused = "refused" // the server said no and why: a defect the agent can fix Failed = "failed" // the server could not answer )
Outcomes of a call.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Call ¶
type Call struct {
At time.Time `json:"time"`
// Method is always tools/call; Operation always execute_tool.
Method string `json:"mcp.method.name"`
Operation string `json:"gen_ai.operation.name"`
Tool string `json:"gen_ai.tool.name"`
RequestID string `json:"jsonrpc.request.id,omitempty"`
Protocol string `json:"mcp.protocol.version,omitempty"`
// Session groups the calls of one piece of an agent's work: the
// client's MCP session where it has one, else its agent and person.
Session string `json:"mcp.session.id"`
// Seconds is the server's time on the call.
Seconds float64 `json:"mcp.server.operation.duration"`
// ErrorType is tool_error for a call answered with isError, empty on
// success.
ErrorType string `json:"error.type,omitempty"`
// ErrorMessage is what a refusal said, with every quoted value
// masked and cut short: why a call was refused, never what was sent.
ErrorMessage string `json:"error.message,omitempty"`
Agent string `json:"cartograph.agent,omitempty"`
Person string `json:"cartograph.person,omitempty"`
ChangeSet string `json:"cartograph.change_set,omitempty"`
// Record is the Kind/id the call names, when it names one.
Record string `json:"cartograph.record,omitempty"`
// Keys are the argument's top-level keys, sorted; never their values.
Keys []string `json:"cartograph.argument.keys,omitempty"`
InputBytes int `json:"cartograph.argument.bytes"`
OutputBytes int `json:"cartograph.result.bytes"`
Outcome string `json:"cartograph.outcome"`
// Defect classifies a refusal or failure, low in cardinality: schema,
// structure, open-checks, not-found, conflict, input, access, other.
Defect string `json:"cartograph.defect,omitempty"`
// Problems counts what a refusal, or an answer, says is wrong, and
// Paths are the fields it names, list items as "-": where the defect
// is, never what was sent there.
Problems int `json:"cartograph.problems,omitempty"`
Paths []string `json:"cartograph.problem.paths,omitempty"`
}
Call is one tool call. Its fields carry the names OpenTelemetry's semantic conventions give an MCP server span (mcp.*, gen_ai.*, error.type), so a collector reading the lines needs no mapping; Cartograph's own are under cartograph.*. Arguments and results, opt-in and sensitive under those conventions, are never recorded: only the argument keys.
type Recorder ¶
type Recorder interface {
Record(c Call)
}
Recorder keeps calls. Record never fails the call it records.
type Report ¶
type Report struct {
Calls int `json:"calls"`
Defects int `json:"defects"`
DPMO float64 `json:"dpmo"`
Sigma float64 `json:"sigma"`
// FirstPassYield is the share of calls that were not refused and were
// not a retry of one that was.
FirstPassYield float64 `json:"firstPassYield"`
// RolledThroughputYield is the chance a session goes through every
// write step it takes right the first time: the product of each
// write tool's first-pass yield.
RolledThroughputYield float64 `json:"rolledThroughputYield"`
// Rework counts calls that repeat a tool on the same record after a
// defect there; MeanAttempts is how many tries such a step took.
Rework int `json:"rework"`
MeanAttempts float64 `json:"meanAttempts"`
Value map[string]int `json:"valueStream"`
// ValueAddedRatio is the share of calls that added value.
ValueAddedRatio float64 `json:"valueAddedRatio"`
// CallsPerSessionCV is how much sessions vary in length: the standard
// deviation of calls per session over its mean.
CallsPerSessionCV float64 `json:"callsPerSessionCV"`
Tools []ToolStats `json:"tools"`
Sessions []SessionStats `json:"sessions"`
// Transitions are the most taken steps from one tool to the next.
Transitions []Transition `json:"transitions"`
}
Report is the analysis of a set of calls.
type SessionStats ¶
type SessionStats struct {
Session string `json:"session"`
Calls int `json:"calls"`
Defects int `json:"defects"`
Rework int `json:"rework"`
Proposed bool `json:"proposed"`
Writes int `json:"writes"`
}
SessionStats is one session's line.
type ToolStats ¶
type ToolStats struct {
Tool string `json:"tool"`
Calls int `json:"calls"`
Defects int `json:"defects"`
DPMO float64 `json:"dpmo"`
Sigma float64 `json:"sigma"`
P50Millis int64 `json:"p50Millis"`
P95Millis int64 `json:"p95Millis"`
// FirstPassYield is the share of this tool's steps (a tool on a
// record in a session) that were right the first time.
FirstPassYield float64 `json:"firstPassYield"`
MeanBytes int `json:"meanInputBytes"`
ByDefect map[string]int `json:"byDefect,omitempty"`
// ByPath counts the fields this tool\'s refusals named.
ByPath map[string]int `json:"byPath,omitempty"`
}
ToolStats is one tool's line in the report.
type Transition ¶
Transition is a step from one tool to the next within a session.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package jsonl keeps traced calls as JSON lines appended to a file, one call a line, for any tool that reads JSON lines (and for cartograph traces) to analyse.
|
Package jsonl keeps traced calls as JSON lines appended to a file, one call a line, for any tool that reads JSON lines (and for cartograph traces) to analyse. |