Documentation
¶
Overview ¶
Package usage tracks token + cost accounting for the agent loop.
Every model call returns a UsageMetadata block with input and output token counts; a Tracker accumulates these across a session. Pricing numbers come from a built-in table that callers may override per model via .agents/config.json (model.pricing).
Index ¶
- func ContextWindowSizeFor(model string) int
- func KnownModelsCount() int
- func RebuildTrackerFromEvents(ctx context.Context, t *Tracker, events iter.Seq2[*session.Event, error], ...) error
- func SetCatalog(c *pricing.Catalog)
- type DigestSavingsRecord
- type DigestSavingsTotals
- type Pricing
- type Totals
- type Tracker
- func (t *Tracker) All() []Turn
- func (t *Tracker) Append(model string, inputTokens, outputTokens int, p Pricing) Turn
- func (t *Tracker) AppendDigestSavings(rec DigestSavingsRecord)
- func (t *Tracker) AppendUsage(model string, u TurnUsage, p Pricing) Turn
- func (t *Tracker) ContextWindowSize() int
- func (t *Tracker) ContextWindowUsed() int
- func (t *Tracker) DigestSavings() DigestSavingsTotals
- func (t *Tracker) Duration() time.Duration
- func (t *Tracker) Last() (Turn, bool)
- func (t *Tracker) SetOnAppend(f func())
- func (t *Tracker) Totals() Totals
- func (t *Tracker) TotalsByModel() map[string]Totals
- type Turn
- type TurnTap
- type TurnUsage
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ContextWindowSizeFor ¶
contextWindowSizeFor returns the configured max input window for model. Hardcoded table; bump when new models land. Unknown models return 0. Substring match — model IDs come in many flavors ("gemini-3.1-pro-preview-customtools", "claude-sonnet-4-6-1m", …) and we want the limit to land regardless of suffix.
Exported as a package-level function so callers that have a model name in hand (without going through the Tracker) can resolve it directly. The Tracker methods above are the common path.
func KnownModelsCount ¶
func KnownModelsCount() int
KnownModelsCount returns the total number of models across every layer of the installed pricing catalog (cfg override + project file + user manual + user external + builtin). Returns 0 when no catalog is installed. Used by the attach /pricing endpoint's snapshot so operators can see how many models the daemon knows about at a glance — the previous default of hard-coded 0 was actively misleading during the v2.7.0-dev.3 demo drive.
func RebuildTrackerFromEvents ¶
func RebuildTrackerFromEvents( ctx context.Context, t *Tracker, events iter.Seq2[*session.Event, error], defaultModel string, pricingFor func(model string) Pricing, ) error
RebuildTrackerFromEvents replays a persisted-event stream into t, reconstructing the per-turn totals via the same TurnTap.Observe + TurnTap.Commit pattern the live turn loop uses. Called on the session-resume path (cmd/core-agent/multi_session.go's reproduceAgent when origin=="resumed") so the newly-minted per-session tracker carries the historical totals instead of starting at zero.
Motivation: PR #275 correctly isolated per-session usage.Trackers to stop cross-session contamination, but every session-resume path (SessionResumer / registry eviction miss / daemon restart) then began at zero — the eventlog was intact, but the tracker was fresh. The visible bug: /stats and the TUI's status-bar aggregate showed "0 in / 0 out / $0.00" for sessions with real historical work. Per-turn footers kept working because they replay from live SSE events, not from the tracker.
Best-effort semantics:
- Events missing UsageMetadata: skipped by TurnTap.Observe (safe).
- Events missing ModelVersion: fall back to defaultModel. In practice the session's primary model is stable across its lifetime, so this is fine.
- pricingFor returning zero: tracker records tokens but $0.00 cost for that model. Downstream cost totals reflect this; operators re-appending after a pricing refresh land the correct cost on the next real turn.
- Context cancellation: early return with ctx.Err().
- Iterator errors: early return with the error (caller decides whether to fail-open or fail-loud).
Caller MUST invoke this BEFORE wiring tracker.SetOnAppend (which happens later inside agent.New's option evaluation), otherwise each rebuild AppendUsage would fire the OnAppend callback and broadcast N synthetic usage-update SSE events. The reproduceAgent call site respects this ordering by construction.
func SetCatalog ¶
SetCatalog installs the catalog PriceFor consults. Safe to call from any goroutine; lookups in flight see either the old or new catalog atomically, never a torn read.
Types ¶
type DigestSavingsRecord ¶
type DigestSavingsRecord struct {
Path string
ParentTokensSaved int // max(0, OriginalTokensEst - DigestTokensEst)
SubagentModel string
SubagentInputTokens int
SubagentOutputTokens int
SubagentCostUSD float64
}
DigestSavingsRecord is one per-call sample of the MCP digest wrap's effect on the parent's context. Aggregated into DigestSavingsTotals via Tracker.AppendDigestSavings; callers construct one per Process result the wrap hands back.
Path mirrors digest.Method (structural_json / llm_fallback / passthrough). Passthrough records still flow through — a call the router decided to pass through verbatim IS a data point (told the operator "the wrap layer thought this was small enough to skip").
type DigestSavingsTotals ¶
type DigestSavingsTotals struct {
StructuralCalls int
StructuralTokensSaved int
AgenticCalls int
AgenticTokensSaved int // parent-side tokens saved BEFORE subagent offset
AgenticSubagentInTokens int
AgenticSubagentOutTokens int
AgenticSubagentCostUSD float64
PassthroughCalls int
}
DigestSavingsTotals is the cumulative session view rendered by /context and (when wired) OTel session-close attributes. Structural and agentic-path counts are broken out because their cost math differs (agentic pays a subagent bill, structural doesn't).
type Pricing ¶
type Pricing struct {
InputPerMTok float64
CachedInputPerMTok float64
OutputPerMTok float64
// UpdatedAt is when the rate was last verified against its
// source. Threads through from internal/pricing.Rates so /pricing
// can surface staleness. Zero when unknown.
UpdatedAt time.Time
}
Pricing is the per-million-token rate for one model. Fields are USD per million tokens (the same unit upstream providers publish public list rates in). CachedInputPerMTok is the reduced rate applied to prompt-cache-hit input tokens — Gemini charges 25% of the base input rate for both implicit and explicit caches. A zero Pricing carries no useful pricing — callers should distinguish "rate unknown" from "free" (e.g. echo models). See pricing.Rates / pricing.Catalog for the layered resolution behind PriceFor.
func PriceFor ¶
PriceFor returns the Pricing for modelID. Resolution chain (first exact match wins; longest-prefix fallback at the end):
- cfg.Model.Pricing[modelID] — operator override
- .agents/pricing.json models[modelID] — project file
- ~/.core-agent/pricing.json — user file (manual + external)
- compiled-in builtin — fallback
- longest-prefix match across (1)..(4) — suffix variants
- Pricing{} — rate unknown
cfg is consulted via the catalog (if installed via SetCatalog) or via an on-the-fly lookup when no catalog is installed. The no-catalog path covers tests + library use that doesn't go through cmd/core-agent's startup.
func PriceForWithSource ¶
PriceForWithSource is PriceFor + the catalog layer name that served the rate (pricing.SourceCfgOverride / SourceProjectFile / SourceUserManual / SourceUserExternal / SourceBuiltin). Empty source when no rate was found. Used by /pricing so operators can spot when a rate came from a stale builtin instead of the freshly-refreshed LiteLLM external catalog they were expecting.
The cfg override path (used only when no globalCatalog is installed) reports source SourceCfgOverride when the model resolves through it.
func (Pricing) CostUSD ¶
CostUSD returns the dollar cost of (input, output) tokens at p. Treats every input token as uncached — see CostUSDWithCache for the cached-vs-uncached split.
func (Pricing) CostUSDWithCache ¶
CostUSDWithCache returns the dollar cost with cache-hit tokens billed at CachedInputPerMTok. When CachedInputPerMTok is zero (rate unknown) cached tokens fall back to InputPerMTok so the estimate never silently drops to zero cost for cached input.
type Totals ¶
type Totals struct {
Turns int
InputTokens int
CachedInputTokens int
OutputTokens int
ThoughtsTokens int
ToolUseTokens int
CostUSD float64
}
Totals aggregates a slice of Turns. Cached / thoughts / tool-use mirror the Turn fields so callers projecting Totals into wire formats can render every dimension without walking All().
type Tracker ¶
type Tracker struct {
// contains filtered or unexported fields
}
Tracker accumulates per-turn usage for one session.
Thread-safe: the agent goroutine (or run loop) calls Append; readers access via Last/Totals/All.
func NewTracker ¶
func NewTracker() *Tracker
NewTracker returns a tracker with its session-start time set to now.
func (*Tracker) Append ¶
Append records one turn's usage with input/output only. Cost is computed via the supplied Pricing; pass a zero Pricing to skip cost tracking. If SetOnAppend has been called with a non-nil callback, the callback fires after the new turn is durable in the tracker and the lock has been released.
Callers that have a full per-turn breakdown (cache hits, thoughts, tool-use) should use AppendUsage instead so the extra dimensions flow through to Totals + wire formats.
func (*Tracker) AppendDigestSavings ¶
func (t *Tracker) AppendDigestSavings(rec DigestSavingsRecord)
AppendDigestSavings accumulates one MCP digest-wrap result into the session's cumulative counters. Negative ParentTokensSaved is clamped to zero — a "digest" longer than the original happens occasionally on the passthrough path when the wrap adds a truncation marker, and we don't want that to subtract from savings totals.
func (*Tracker) AppendUsage ¶
AppendUsage records one turn's usage with the full per-field breakdown. CachedInputTokens > InputTokens is clamped to InputTokens (defensive against occasional provider quirks where the cached counter over-reports; the input/uncached math must stay non-negative downstream). Cost applies CostUSDWithCache when any cache hits are present so the cached-vs-uncached rate split is reflected in the stored Turn.
func (*Tracker) ContextWindowSize ¶
ContextWindowSize returns the model's max input window from a hardcoded table, keyed on the most recent turn's model name. Returns 0 for unknown models (or when no turn has landed yet) — consumers should treat 0 as "unknown; suppress any per-context UI segment and skip threshold-based behaviors like compaction." See contextWindowSizeFor for the lookup table.
Lifted from cmd/core-agent/coretui_enabled.go where it was first implemented as part of the core-tui adapter tier-3+ work (commit be8dae5). Agent-level code (compaction trigger, micro-subagents) needs the same accessor, so it lives on the substrate type rather than the adapter bridge.
func (*Tracker) ContextWindowUsed ¶
ContextWindowUsed approximates the current context fill as the most recent turn's input-token count. Each turn re-sends the full conversation, so the input count is the rolling context size. Returns 0 before any turn has landed (matches "unknown" semantics — consumers should suppress the segment).
func (*Tracker) DigestSavings ¶
func (t *Tracker) DigestSavings() DigestSavingsTotals
DigestSavings returns the session-cumulative snapshot of the digest-wrap's effect. Safe to call from any goroutine.
func (*Tracker) SetOnAppend ¶
func (t *Tracker) SetOnAppend(f func())
SetOnAppend registers a callback that fires after every Append call. The callback runs after the lock is released, so it can safely call Totals(), TotalsByModel(), or any other Tracker accessor without risking a re-entrant deadlock.
Used by the attach layer to push usage-update events on the SSE stream as turn cost lands — each Append represents a turn whose cumulative impact should reach connected operators.
Pass nil to unregister. Safe to set multiple times (last wins); callers wiring this from the broadcaster do so on first subscriber and clear it on last detach.
func (*Tracker) TotalsByModel ¶
TotalsByModel groups the session's turns by model name and returns the per-model totals. Useful for surfaces that want to break down "$X.YY total" into "$A.BB parent model + $C.DD subtask model" so the cost-efficiency win of routing subtasks to a cheaper model is directly visible. Empty map when no turns recorded.
type Turn ¶
type Turn struct {
Model string
InputTokens int
CachedInputTokens int
OutputTokens int
ThoughtsTokens int
ToolUseTokens int
CostUSD float64
At time.Time
}
Turn captures one model call's resource use. Times are wall clock so summary lines can include session duration without a monotonic ref.
InputTokens is the total effective prompt size — for Gemini this matches PromptTokenCount, which already includes any cache-hit tokens (google.golang.org/genai types.go: "the total effective prompt size meaning this includes the number of tokens in the cached content"). CachedInputTokens is therefore a subset of InputTokens, not an addition to it. Uncached = InputTokens - CachedInputTokens.
type TurnTap ¶
type TurnTap struct {
// contains filtered or unexported fields
}
TurnTap accumulates per-model-turn usage from a stream of *session.Event, applying Gemini's "cumulative UsageMetadata per chunk, final on TurnComplete" convention: overwrite last-seen per event, commit exactly once on TurnComplete, reset between turns.
Motivation: Gemini's UsageMetadata is cumulative across streaming chunks within a single model turn — earlier chunks carry running totals, the final chunk carries the per-turn total. Naïve Append- on-every-event both inflates the tracker's turn count (one Append per chunk) and double-counts tokens (summing cumulative running totals). This bug bit us in the core-tui adapter (fixed in #156, surfaced in the field as "totals exactly 2x the last turn") and #157 extracted the pattern so future adapters get it right by default.
Zero-value ready. Not safe for concurrent use — one TurnTap per event iterator.
Typical usage (bookkeeping only):
var tap usage.TurnTap
for ev, err := range agent.Run(ctx, prompt) {
tap.Observe(ev)
if u, ok := tap.Commit(ev); ok {
tracker.AppendUsage(model, u, pricing)
}
// ... other per-event work
}
TUI-style usage (live per-event running total AND commit):
tap.Observe(ev)
if peek := tap.Peek(); peek.InputTokens > 0 {
stampLiveCounter(peek) // reflects running total mid-turn
}
if u, ok := tap.Commit(ev); ok {
turn := tracker.AppendUsage(model, u, pricing)
stampFinalCost(turn.CostUSD)
}
func (*TurnTap) Commit ¶
Commit returns (per-turn totals, true) exactly when ev is a TurnComplete carrying non-zero accumulated usage, after resetting internal state so the next turn's chunks accumulate cleanly. Returns (zero, false) otherwise. Call after Observe.
type TurnUsage ¶
type TurnUsage struct {
InputTokens int
CachedInputTokens int
OutputTokens int
ThoughtsTokens int
ToolUseTokens int
}
TurnUsage is the per-call token breakdown a provider adapter hands to Tracker.AppendUsage. Provider-independent: adapters normalize their per-response metadata into this shape (see TurnUsageFromGenaiMetadata for the Gemini/Vertex path).
func TurnUsageFromGenaiMetadata ¶
func TurnUsageFromGenaiMetadata(u *genai.GenerateContentResponseUsageMetadata) TurnUsage
TurnUsageFromGenaiMetadata projects one genai UsageMetadata block into the provider-independent TurnUsage shape. All Gemini/Vertex tap sites use this to get identical field extraction — call it once per event with UsageMetadata != nil, overwriting the "last seen" turn snapshot (matching the existing lastIn/lastOut overwrite pattern).
PromptTokenCount is the total effective prompt size and already includes cache-hit tokens (see the Turn docstring in tracker.go). Returns a zero TurnUsage for a nil input.
Source Files