trace

package
v2.10.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 9, 2026 License: Apache-2.0 Imports: 4 Imported by: 0

Documentation

Overview

Package trace is the port through which every call an agent makes to Cartograph's MCP server is recorded (docs/adr/0028), for governance and for finding where agents work well and where they vary: which tools are used most, which are refused, where work is redone. A call is recorded by its shape (the tool, the record it names, the keys it sent, its sizes, its outcome and how long it took), never by what it carried, so no document an agent ports and nothing a person wrote reaches a trace. An adapter keeps the calls; "off", the default, keeps none.

Index

Constants

View Source
const (
	ValueAdding = "value-adding" // a write that was kept
	Necessary   = "necessary"    // a read the work needs
	Waste       = "waste"        // a defect, or a read repeated as it was
)

Value classes of a call, as a value stream counts them.

View Source
const (
	OK      = "ok"
	Refused = "refused" // the server said no and why: a defect the agent can fix
	Failed  = "failed"  // the server could not answer
)

Outcomes of a call.

Variables

This section is empty.

Functions

This section is empty.

Types

type Call

type Call struct {
	At time.Time `json:"time"`
	// Method is always tools/call; Operation always execute_tool.
	Method    string `json:"mcp.method.name"`
	Operation string `json:"gen_ai.operation.name"`
	Tool      string `json:"gen_ai.tool.name"`
	RequestID string `json:"jsonrpc.request.id,omitempty"`
	Protocol  string `json:"mcp.protocol.version,omitempty"`
	// Session groups the calls of one piece of an agent's work: the
	// client's MCP session where it has one, else its agent and person.
	Session string `json:"mcp.session.id"`
	// Seconds is the server's time on the call.
	Seconds float64 `json:"mcp.server.operation.duration"`
	// ErrorType is tool_error for a call answered with isError, empty on
	// success.
	ErrorType string `json:"error.type,omitempty"`
	// ErrorMessage is what a refusal said, with every quoted value
	// masked and cut short: why a call was refused, never what was sent.
	ErrorMessage string `json:"error.message,omitempty"`
	Agent        string `json:"cartograph.agent,omitempty"`
	Person       string `json:"cartograph.person,omitempty"`
	ChangeSet    string `json:"cartograph.change_set,omitempty"`
	// Record is the Kind/id the call names, when it names one.
	Record string `json:"cartograph.record,omitempty"`
	// Keys are the argument's top-level keys, sorted; never their values.
	Keys        []string `json:"cartograph.argument.keys,omitempty"`
	InputBytes  int      `json:"cartograph.argument.bytes"`
	OutputBytes int      `json:"cartograph.result.bytes"`
	Outcome     string   `json:"cartograph.outcome"`
	// Defect classifies a refusal or failure, low in cardinality: schema,
	// structure, open-checks, not-found, conflict, input, access, other.
	Defect string `json:"cartograph.defect,omitempty"`
	// Problems counts what a refusal, or an answer, says is wrong, and
	// Paths are the fields it names, list items as "-": where the defect
	// is, never what was sent there.
	Problems int      `json:"cartograph.problems,omitempty"`
	Paths    []string `json:"cartograph.problem.paths,omitempty"`
}

Call is one tool call. Its fields carry the names OpenTelemetry's semantic conventions give an MCP server span (mcp.*, gen_ai.*, error.type), so a collector reading the lines needs no mapping; Cartograph's own are under cartograph.*. Arguments and results, opt-in and sensitive under those conventions, are never recorded: only the argument keys.

type Off

type Off struct{}

Off records nothing.

func (Off) Record

func (Off) Record(Call)

Record does nothing.

type Recorder

type Recorder interface {
	Record(c Call)
}

Recorder keeps calls. Record never fails the call it records.

type Report

type Report struct {
	Calls   int     `json:"calls"`
	Defects int     `json:"defects"`
	DPMO    float64 `json:"dpmo"`
	Sigma   float64 `json:"sigma"`
	// FirstPassYield is the share of calls that were not refused and were
	// not a retry of one that was.
	FirstPassYield float64 `json:"firstPassYield"`
	// RolledThroughputYield is the chance a session goes through every
	// write step it takes right the first time: the product of each
	// write tool's first-pass yield.
	RolledThroughputYield float64 `json:"rolledThroughputYield"`
	// Rework counts calls that repeat a tool on the same record after a
	// defect there; MeanAttempts is how many tries such a step took.
	Rework       int            `json:"rework"`
	MeanAttempts float64        `json:"meanAttempts"`
	Value        map[string]int `json:"valueStream"`
	// ValueAddedRatio is the share of calls that added value.
	ValueAddedRatio float64 `json:"valueAddedRatio"`
	// CallsPerSessionCV is how much sessions vary in length: the standard
	// deviation of calls per session over its mean.
	CallsPerSessionCV float64        `json:"callsPerSessionCV"`
	Tools             []ToolStats    `json:"tools"`
	Sessions          []SessionStats `json:"sessions"`
	// Transitions are the most taken steps from one tool to the next.
	Transitions []Transition `json:"transitions"`
}

Report is the analysis of a set of calls.

func Analyse

func Analyse(calls []Call) Report

Analyse reads a set of calls, in the order they were made.

type SessionStats

type SessionStats struct {
	Session  string `json:"session"`
	Calls    int    `json:"calls"`
	Defects  int    `json:"defects"`
	Rework   int    `json:"rework"`
	Proposed bool   `json:"proposed"`
	Writes   int    `json:"writes"`
}

SessionStats is one session's line.

type ToolStats

type ToolStats struct {
	Tool      string  `json:"tool"`
	Calls     int     `json:"calls"`
	Defects   int     `json:"defects"`
	DPMO      float64 `json:"dpmo"`
	Sigma     float64 `json:"sigma"`
	P50Millis int64   `json:"p50Millis"`
	P95Millis int64   `json:"p95Millis"`
	// FirstPassYield is the share of this tool's steps (a tool on a
	// record in a session) that were right the first time.
	FirstPassYield float64        `json:"firstPassYield"`
	MeanBytes      int            `json:"meanInputBytes"`
	ByDefect       map[string]int `json:"byDefect,omitempty"`
	// ByPath counts the fields this tool\'s refusals named.
	ByPath map[string]int `json:"byPath,omitempty"`
}

ToolStats is one tool's line in the report.

type Transition

type Transition struct {
	From  string `json:"from"`
	To    string `json:"to"`
	Count int    `json:"count"`
}

Transition is a step from one tool to the next within a session.

Directories

Path Synopsis
Package jsonl keeps traced calls as JSON lines appended to a file, one call a line, for any tool that reads JSON lines (and for cartograph traces) to analyse.
Package jsonl keeps traced calls as JSON lines appended to a file, one call a line, for any tool that reads JSON lines (and for cartograph traces) to analyse.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL