metrics

package
v0.1.0-rc.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 10, 2026 License: Apache-2.0 Imports: 6 Imported by: 0

Documentation

Overview

Package metrics provides token usage metrics collection and aggregation.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Accumulator 

type Accumulator struct {
	// contains filtered or unexported fields
}

Accumulator collects token usage metrics with thread-safe operations. Session totals use atomic counters. Historical data is stored in a ring buffer of pre-aggregated 1-minute time buckets.

func NewAccumulator 

func NewAccumulator(maxDataPoints int) *Accumulator

NewAccumulator creates a metrics accumulator with the given ring buffer capacity. Each slot holds one minute of aggregated data, so 10000 slots ≈ ~7 days.

func (*Accumulator) Clear 

func (a *Accumulator) Clear()

Clear resets all metrics — session totals, per-server totals, and history.

func (*Accumulator) PromptUsageSnapshot 

func (a *Accumulator) PromptUsageSnapshot() map[string]ToolStat

PromptUsageSnapshot returns a deep copy of the per-skill prompts/get call counters. Empty (nil) when no prompt has been served yet. Reuses the ToolStat value shape so the persistence and API layers share one type.

func (*Accumulator) Query 

func (a *Accumulator) Query(duration time.Duration) TimeSeriesResponse

Query returns historical time-series data for the given duration. For ranges > 6h, data points are downsampled to hourly buckets.

func (*Accumulator) Record 

func (a *Accumulator) Record(serverName string, inputTokens, outputTokens int)

Record adds a token usage observation from a tool call. Equivalent to RecordReplica with replicaID=-1 (i.e. do not attribute to a replica).

func (*Accumulator) RecordFormatSavings 

func (a *Accumulator) RecordFormatSavings(serverName string, originalTokens, formattedTokens int)

RecordFormatSavings records token counts before and after format conversion. Normal token usage tracking is handled separately by the ToolCallObserver; this method only tracks the format savings delta.

func (*Accumulator) RecordPromptGet 

func (a *Accumulator) RecordPromptGet(name string)

RecordPromptGet increments the call counter for a single skill (prompt) served via prompts/get and stamps the last-called timestamp. Powers the Skills Library "Never used" facet. An empty name is a no-op so callers without attribution can invoke unconditionally.

Kept parallel to RecordToolCall rather than reusing it: routing prompt serving through the tool-usage map would surface synthetic entries in Tools Audit Mode.

func (*Accumulator) RecordReplica 

func (a *Accumulator) RecordReplica(serverName string, replicaID, inputTokens, outputTokens int)

RecordReplica adds a token usage observation attributed to a specific replica. Per-server aggregates are updated in all cases. Pass replicaID < 0 to skip the per-replica update (used for servers that are not part of a replica set).

func (*Accumulator) RecordReplicaWithClient 

func (a *Accumulator) RecordReplicaWithClient(serverName string, replicaID int, clientID string, inputTokens, outputTokens int)

RecordReplicaWithClient is the client-aware variant of RecordReplica. It updates the per-client token counters in addition to session, per-server, and per-replica aggregates. An empty clientID skips the per-client update, matching the replicaID < 0 convention so callers without attribution can continue to use the same code path.

func (*Accumulator) RecordToolCall 

func (a *Accumulator) RecordToolCall(serverName, toolName string)

RecordToolCall increments per-(server, tool) call counters and stamps the last-called timestamp. Used by pkg/optimize's unused_tool heuristic.

An empty serverName or toolName is a no-op so callers without per-tool attribution (legacy ToolCallObserver path) can invoke unconditionally.

func (*Accumulator) RecordToolCallUsage 

func (a *Accumulator) RecordToolCallUsage(serverName, toolName string, inputTokens, outputTokens int)

RecordToolCallUsage is RecordToolCall plus the call's token counts: one bucket lookup increments the call counter, stamps the last-called timestamp, and adds input/output tokens, so the observer's hot path touches the tool-usage map once per call.

An empty serverName or toolName is a no-op so callers without per-tool attribution (legacy ToolCallObserver path) can invoke unconditionally.

func (*Accumulator) ReplaySnapshot 

func (a *Accumulator) ReplaySnapshot(serverName string, ts time.Time, inputTokens, outputTokens int64)

ReplaySnapshot adds a historical observation to the time-series ring buffers (aggregate + per-server) without touching cumulative counters. Used by telemetry.MetricsFlusher.SeedFromFile to rehydrate per-minute bucket history from each persisted Diff line — the chart shows pre-restart activity continuously alongside live data instead of resetting to a single post-restart point.

Cumulative counters are restored separately via Restore. Calling both with the same source data reproduces the on-disk state.

ts is bucketed to the minute via the same key the live Record path uses, so chronological replay produces one bucket per flush minute and live observations after replay continue advancing the same ring naturally.

func (*Accumulator) Restore 

func (a *Accumulator) Restore(perServer map[string]TokenCounts)

Restore replaces per-server token totals with the supplied map and recomputes session totals as the sum across all servers (matching the invariant Record/RecordReplica maintains). Used on daemon startup to repopulate cumulative counters from a persisted metrics.jsonl file.

Existing per-server counters are overwritten for any server present in the map; servers absent from the map retain their current state. Replicas and format-savings counters are not restored — those carry no on-disk equivalent in the snapshot format. Time-series ring buckets are populated separately via ReplaySnapshot.

func (*Accumulator) RestorePromptUsage 

func (a *Accumulator) RestorePromptUsage(perSkill map[string]ToolStat)

RestorePromptUsage seeds per-skill prompts/get counters from a persisted snapshot so usage history survives a gateway restart. Mirrors RestoreToolUsage: max-wins per counter (an existing in-memory value is kept when it already exceeds the restored one), entries with no recorded calls are skipped, and an empty map is a no-op.

func (*Accumulator) RestoreToolUsage 

func (a *Accumulator) RestoreToolUsage(perServer map[string]map[string]ToolStat)

RestoreToolUsage seeds per-(server, tool) call counters from a persisted snapshot so Audit Mode's usage history survives a gateway restart. Called on startup by telemetry.MetricsFlusher.SeedFromFile before the gateway serves traffic, so the counters it re-creates are the same *toolUsage buckets RecordToolCall increments afterward — live calls continue from the restored count rather than starting at zero.

Tool-call attribution flows through the same observer for direct and code-mode calls (Gateway.CallTool → HandleToolsCall → Observer → RecordToolCall), so a restored snapshot reflects both equally.

Restore is max-wins per counter — calls and tokens alike: an existing in-memory value is kept when it already exceeds the restored one (defensive against a seed racing late initialization; the counters are monotonic between resets, so max-wins never double-counts). Entries with no recorded calls are skipped so the snapshot stays sparse. An empty map is a no-op.

func (*Accumulator) Snapshot 

func (a *Accumulator) Snapshot() TokenUsage

Snapshot returns the current token usage summary.

func (*Accumulator) StartedAt 

func (a *Accumulator) StartedAt() time.Time

StartedAt returns the wall-clock time the accumulator was created. Clear does not reset this value — the start-of-observation window stays anchored to the gateway lifetime, which is what pkg/optimize uses to gate "<24h of data" findings.

func (*Accumulator) ToolUsageSnapshot 

func (a *Accumulator) ToolUsageSnapshot() map[string]map[string]ToolStat

ToolUsageSnapshot returns a deep copy of the per-(server, tool) call counters. Empty when no per-tool calls have been recorded (typical for gateways still on the legacy ToolCallObserver path).

type DataPoint 

type DataPoint struct {
	Timestamp    time.Time `json:"timestamp"`
	InputTokens  int64     `json:"input_tokens"`
	OutputTokens int64     `json:"output_tokens"`
	TotalTokens  int64     `json:"total_tokens"`
}

DataPoint is a single time-series data point with token counts.

type FormatSavings 

type FormatSavings struct {
	OriginalTokens  int64   `json:"original_tokens"`
	FormattedTokens int64   `json:"formatted_tokens"`
	SavedTokens     int64   `json:"saved_tokens"`
	SavingsPercent  float64 `json:"savings_percent"`
}

FormatSavings tracks token savings from output formatting.

type Observer 

type Observer struct {
	// contains filtered or unexported fields
}

Observer implements mcp.ToolCallObserver and mcp.ClientObserver by counting tokens and recording them into an Accumulator.

func NewObserver 

func NewObserver(counter token.Counter, accumulator *Accumulator) *Observer

NewObserver creates a ToolCallObserver that counts tokens and records metrics.

func (*Observer) ObservePromptGet 

func (o *Observer) ObservePromptGet(obs mcp.PromptGetObservation)

ObservePromptGet records that a registry skill was served via prompts/get, incrementing its cumulative count and last-used timestamp in the parallel prompt-usage namespace. The token path does not apply: prompts are static content, not tool calls.

func (*Observer) ObserveToolCall 

func (o *Observer) ObserveToolCall(serverName string, replicaID int, arguments map[string]any, result *mcp.ToolCallResult)

ObserveToolCall counts input/output tokens and records them.

func (*Observer) ObserveToolCallWithClient 

func (o *Observer) ObserveToolCallWithClient(_ context.Context, obs mcp.ToolCallObservation) mcp.ToolCallSummary

ObserveToolCallWithClient is the ClientObserver entry point. It records the same tokens as ObserveToolCall, additionally attributes them to the supplied client, and returns a summary the gateway uses to populate OTel GenAI semantic span attributes without re-counting tokens.

type TimeSeriesResponse 

type TimeSeriesResponse struct {
	Range     string                 `json:"range"`
	Interval  string                 `json:"interval"`
	Points    []DataPoint            `json:"data_points"`
	PerServer map[string][]DataPoint `json:"per_server"`
}

TimeSeriesResponse is returned by the historical metrics endpoint.

type TokenCounts 

type TokenCounts struct {
	InputTokens  int64 `json:"input_tokens"`
	OutputTokens int64 `json:"output_tokens"`
	TotalTokens  int64 `json:"total_tokens"`
}

TokenCounts holds input/output/total token counts.

type TokenUsage 

type TokenUsage struct {
	Session    TokenCounts                    `json:"session"`
	PerServer  map[string]TokenCounts         `json:"per_server"`
	PerReplica map[string]map[int]TokenCounts `json:"per_replica,omitempty"`
	// PerClient groups token usage by the originating MCP client (for example
	// "claude-code", "cursor"). The field is omitempty so consumers built
	// before per-client attribution shipped continue to see the same JSON
	// shape. Future per-user / per-team dimensions land as sibling fields
	// (per_user, per_team) under this same shape rather than reshaping
	// per_client.
	PerClient     map[string]TokenCounts `json:"per_client,omitempty"`
	FormatSavings FormatSavings          `json:"format_savings"`
}

TokenUsage is the top-level token usage snapshot returned by the API.

type ToolStat 

type ToolStat struct {
	Calls        int64     `json:"calls"`
	LastCalledAt time.Time `json:"last_called_at,omitempty"`
	InputTokens  int64     `json:"input_tokens,omitempty"`
	OutputTokens int64     `json:"output_tokens,omitempty"`
}

ToolStat is the snapshot shape for per-(server, tool) call tracking. Used by pkg/optimize to detect tools that have not seen any calls inside a freshness window and to attribute observed spend per tool. Calls is the cumulative count since the accumulator was created or last cleared; LastCalledAt is the wall-clock time the most recent call was recorded, or the zero value when no calls have been recorded. InputTokens/OutputTokens are the cumulative token counts of the tool's own calls. The token fields are omitempty so persisted lines written before per-tool attribution stay byte-identical.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.