usage

package
v2.9.0-dev.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 3, 2026 License: Apache-2.0 Imports: 14 Imported by: 0

Documentation

Overview

Package usage tracks token + cost accounting for the agent loop.

Every model call returns a UsageMetadata block with input and output token counts; a Tracker accumulates these across a session. Pricing numbers come from a built-in table that callers may override per model via .agents/config.json (model.pricing).

Index

Constants

View Source 
const (
	// #nosec G101 -- OTel GenAI semconv metric name, not a credential.
	MetricGenAITokenUsage = "gen_ai.client.token.usage"
	MetricSessionTurns    = "core_agent.session.turns"
	MetricSessionCost     = "core_agent.session.cost_usd"
	MetricSessionDuration = "core_agent.session.duration"
	// MetricDigestSubagentCost lives here rather than pkg/digest
	// because the accumulator is Tracker.DigestSavings() — the digest
	// package's own TelemetrySnapshot carries no cost field.
	MetricDigestSubagentCost = "core_agent.digest.subagent.cost_usd"
)

Metric names — GenAI semconv where a stable name exists, otherwise core_agent.* for our own surface. See docs/metrics-design.md for the full rationale on why we adopt GenAI semconv (cloud vendor dashboards render token / cost panels automatically).

View Source 
const (
	AttrGenAIModel = "gen_ai.request.model"
	// #nosec G101 -- OTel GenAI semconv attribute key, not a credential.
	AttrGenAITokenType = "gen_ai.token.type"
	AttrSessionID      = "session.id"
	AttrAppName        = "app.name"
	AttrUserID         = "user.id"
	// AttrPriced tags the cost series with whether every turn for the
	// model had a known catalog rate. priced=false means CostUSD is a
	// lower bound because at least one turn was unpriced (rate unknown,
	// billed as $0) — dashboards can filter it out of "exact spend"
	// panels rather than under-reporting. See #368.
	AttrPriced = "priced"
)

Attribute keys. GenAI semconv keys use the exact upstream spelling so consumers with dashboards keyed on gen_ai.* work out of the box. Session identity uses session.id (semconv-adjacent — the GenAI SIG has proposed but not finalized a gen_ai.session.id).

View Source 
const (
	TokenTypeInput  = "input"
	TokenTypeOutput = "output"
	TokenTypeCached = "cached"
	// TokenTypeCacheWrite is input tokens that WROTE a cache entry,
	// billed at a premium rather than the cached discount (#263). Its
	// own series because it is disjoint from both `input` (which here
	// carries the whole prompt) and `cached`, and because the spend it
	// represents is what a dashboard tracking cache ROI has to net
	// against the savings on `cached`.
	TokenTypeCacheWrite = "cache_write"
	TokenTypeThought    = "thoughts"
	TokenTypeToolUse    = "tool_use"
)

Token-type attribute values. Match the Turn breakdown fields.

View Source 
const CacheCreation1hTokensMetadataKey = "cache_creation_1h_input_tokens"

CacheCreation1hTokensMetadataKey is the CustomMetadata key under which a provider reports how much of CacheCreationTokensMetadataKey was written at a 1-hour breakpoint TTL rather than the default 5-minute one — a SUBSET of that count, billed at 2x base input instead of 1.25x (#770).

Absent for every provider that offers one cache TTL or none, and absent from recordings made before #770; both read as zero, which prices the whole write bucket at the 5-minute rate exactly as before.

Written by pkg/models/anthropic (from usage.cache_creation.ephemeral_1h_input_tokens); read by TurnUsageFromMetadata. Same numeric-shape caveat as CacheCreationTokensMetadataKey.

View Source 
const CacheCreationTokensMetadataKey = "cache_creation_input_tokens"

CacheCreationTokensMetadataKey is the LLMResponse.CustomMetadata key under which a provider reports the turn's cache-WRITE token count — tokens billed at a premium for establishing a cache entry rather than at the base input rate.

Why a CustomMetadata sidecar and not a UsageMetadata field: genai's GenerateContentResponseUsageMetadata models Gemini's two input buckets (total prompt + cache reads) and has nowhere to put a third. CustomMetadata is the one per-event map that survives ADK's persist round-trip, which is what lets usage.Rebuild reconstruct correct cost from a reloaded eventlog — pkg/eventlog piggy-backs the same mechanism for FinishReason, for the same reason.

Written by pkg/models/anthropic (from cache_creation_input_tokens); read by TurnUsageFromMetadata. The value is an int64 when freshly stamped and a float64 or json.Number after a JSON round-trip, so readers must accept all three — see cacheCreationTokens.

Variables

This section is empty.

Functions

func ContextWindowSizeFor 

func ContextWindowSizeFor(model string) int

ContextWindowSizeFor returns the max input window for model, or 0 when it isn't known.

Two tiers, in order:

  1. pricing.BuiltinContextWindow — LiteLLM's max_input_tokens for every model we ship a rate for, generated by dev/regen-builtin-pricing. Exact match on the lowercased id.
  2. The substring table below, for ids LiteLLM never publishes: long-context "-1m" suffixes, Vertex publication names, and anything an operator pins that upstream hasn't catalogued.

The generated tier exists because the substring table was wrong and nobody noticed: it claimed gemini-2.5-pro held 2,000,000 input tokens — Gemini 1.5 Pro's number — against a real 1,048,576 cap. Mid-tier compaction fires at 0.65 of the window, so it was scheduled for ~1.3M tokens on a session that the provider would have hard- failed first. Generated numbers can't rot that way.

Exported as a package-level function so callers that have a model name in hand (without going through the Tracker) can resolve it directly. The Tracker methods above are the common path.

func KnownModelsCount 

func KnownModelsCount() int

KnownModelsCount returns the total number of models across every layer of the installed pricing catalog (cfg override + project file + user manual + user external + builtin). Returns 0 when no catalog is installed. Used by the attach /pricing endpoint's snapshot so operators can see how many models the daemon knows about at a glance — the previous default of hard-coded 0 was actively misleading during the v2.7.0-dev.3 demo drive.

func RebuildTrackerFromEvents 

func RebuildTrackerFromEvents(
	ctx context.Context,
	t *Tracker,
	events iter.Seq2[*session.Event, error],
	defaultModel string,
	pricingFor func(model string) Pricing,
) error

RebuildTrackerFromEvents replays a persisted-event stream into t, reconstructing the per-turn totals via the same TurnTap.Observe + TurnTap.Commit pattern the live turn loop uses. Called on the session-resume path (cmd/core-agent/multi_session.go's reproduceAgent when origin=="resumed") so the newly-minted per-session tracker carries the historical totals instead of starting at zero.

Motivation: PR #275 correctly isolated per-session usage.Trackers to stop cross-session contamination, but every session-resume path (SessionResumer / registry eviction miss / daemon restart) then began at zero — the eventlog was intact, but the tracker was fresh. The visible bug: /stats and the TUI's status-bar aggregate showed "0 in / 0 out / $0.00" for sessions with real historical work. Per-turn footers kept working because they replay from live SSE events, not from the tracker.

Best-effort semantics:

  • Events missing UsageMetadata: skipped by TurnTap.Observe (safe).
  • Events missing ModelVersion: fall back to defaultModel. In practice the session's primary model is stable across its lifetime, so this is fine.
  • pricingFor returning zero: tracker records tokens but $0.00 cost for that model. Downstream cost totals reflect this; operators re-appending after a pricing refresh land the correct cost on the next real turn.
  • Context cancellation: early return with ctx.Err().
  • Iterator errors: early return with the error (caller decides whether to fail-open or fail-loud).

Caller MUST invoke this BEFORE wiring tracker.SetOnAppend (which happens later inside agent.New's option evaluation), otherwise each rebuild AppendUsage would fire the OnAppend callback and broadcast N synthetic usage-update SSE events. The compose.ReproduceAgent call site respects this ordering by construction.

func RegisterMetrics added in v2.8.0 

func RegisterMetrics(mp metric.MeterProvider, tp TrackerProvider, opts ...RegisterOption) (metric.Registration, error)

RegisterMetrics wires the async usage observers against mp. Callers pass the process-global MeterProvider (obtained via otel.GetMeterProvider()) so metric points flow into whichever reader(s) telemetry.SetupMetrics installed. tp is called on every export interval — implementations must be cheap and thread-safe.

Returns the registered Registration so the caller can Unregister on shutdown if desired; typical usage discards it (the MeterProvider shutdown cleans up).

func SetCatalog 

func SetCatalog(c *pricing.Catalog)

SetCatalog installs the catalog PriceFor consults. Safe to call from any goroutine; lookups in flight see either the old or new catalog atomically, never a torn read.

Types

type DigestSavingsRecord 

type DigestSavingsRecord struct {
	Path                 string
	ParentTokensSaved    int // max(0, OriginalTokensEst - DigestTokensEst)
	SubagentModel        string
	SubagentInputTokens  int
	SubagentOutputTokens int
	SubagentCostUSD      float64

	// The two cache buckets inside SubagentInputTokens, so a caller
	// charging this record to a session can rebuild the TurnUsage the
	// subagent actually spent instead of a flat uncached one (#771).
	SubagentCachedInputTokens        int
	SubagentCacheCreationInputTokens int

	// SubagentCacheCreation1hInputTokens is the 1-hour-TTL share of
	// the write bucket — a SUBSET of the field above, priced at 2x
	// base input rather than 1.25x (#770).
	SubagentCacheCreation1hInputTokens int

	// SubagentThoughtsTokens is the reasoning bucket, ADDITIVE to
	// SubagentOutputTokens and billed at the output rate (#927). A
	// sidecar from a producer predating that field carries zero, which
	// prices as it always did.
	SubagentThoughtsTokens int
}

DigestSavingsRecord is one per-call sample of the MCP digest wrap's effect on the parent's context. Aggregated into DigestSavingsTotals via Tracker.AppendDigestSavings; callers construct one per Process result the wrap hands back.

Path mirrors digest.Method (structural_json / llm_fallback / passthrough). Passthrough records still flow through — a call the router decided to pass through verbatim IS a data point (told the operator "the wrap layer thought this was small enough to skip").

func (DigestSavingsRecord) SubagentTurn added in v2.9.0 

func (r DigestSavingsRecord) SubagentTurn() TurnUsage

SubagentTurn rebuilds the TurnUsage the digest subagent spent, for callers that need to price or append it. Clamped, so a sidecar carrying contradictory buckets can only shrink the premium-rated write bucket rather than invent a negative uncached remainder.

type DigestSavingsTotals 

type DigestSavingsTotals struct {
	StructuralCalls          int
	StructuralTokensSaved    int
	AgenticCalls             int
	AgenticTokensSaved       int // parent-side tokens saved BEFORE subagent offset
	AgenticSubagentInTokens  int
	AgenticSubagentOutTokens int
	AgenticSubagentCostUSD   float64
	PassthroughCalls         int
}

DigestSavingsTotals is the cumulative session view rendered by /context and (when wired) OTel session-close attributes. Structural and agentic-path counts are broken out because their cost math differs (agentic pays a subagent bill, structural doesn't).

type Pricing 

type Pricing struct {
	InputPerMTok                float64
	CachedInputPerMTok          float64
	CacheCreationInputPerMTok   float64
	CacheCreation1hInputPerMTok float64
	OutputPerMTok               float64
	// UpdatedAt is when the rate was last verified against its
	// source. Threads through from pkg/pricing.Rates so /pricing
	// can surface staleness. Zero when unknown.
	UpdatedAt time.Time
	// Unpriced is true when no catalog layer had a rate for the model
	// (pricing.Catalog.Lookup returned found=false). It disambiguates
	// "rate unknown" (Unpriced=true, cost should render "$—") from a
	// genuinely free model (Unpriced=false, all rates zero). Without
	// this flag a $0 cost is indistinguishable from an unknown-price
	// model in Totals, per-model breakdowns, and the
	// core_agent.session.cost_usd metric — see #368.
	Unpriced bool
}

Pricing is the per-million-token rate for one model. Fields are USD per million tokens (the same unit upstream providers publish public list rates in). CachedInputPerMTok is the reduced rate applied to prompt-cache-hit input tokens — Gemini charges 25% of the base input rate for both implicit and explicit caches. CacheCreationInputPerMTok is the PREMIUM rate applied to input tokens that write a cache entry (Anthropic's cache_creation_input_tokens, 1.25x base input); zero for providers that don't bill writes separately. Like pricing.Rates.CacheCreationInputPerMTok it holds the 5-minute-TTL rate; CacheCreation1hInputPerMTok holds the 1-hour one (2x base input), and falls back to the 5-minute rate when zero. A zero Pricing carries no useful pricing — callers should distinguish "rate unknown" from "free" (e.g. echo models). See pricing.Rates / pricing.Catalog for the layered resolution behind PriceFor.

func PriceFor 

func PriceFor(modelID string, cfg *config.Config) Pricing

PriceFor returns the Pricing for modelID. Resolution chain (first exact match wins; longest-prefix fallback at the end):

  1. cfg.Model.Pricing[modelID] — operator override
  2. .agents/pricing.json models[modelID] — project file
  3. ~/.core-agent/pricing.json — user file (manual + external)
  4. compiled-in builtin — fallback
  5. longest-prefix match across (1)..(4) — suffix variants
  6. Pricing{} — rate unknown

cfg is consulted via the catalog (if installed via SetCatalog) or via an on-the-fly lookup when no catalog is installed. The no-catalog path covers tests + library use that doesn't go through cmd/core-agent's startup.

func PriceForRefreshed added in v2.9.0 

func PriceForRefreshed(modelID string, fallback Pricing) Pricing

PriceForRefreshed re-resolves modelID against the process-wide catalog, falling back to the caller's captured rate when no catalog is installed.

It exists for billing sites that hold a Pricing *value* resolved once and then bill many turns against it (#930). `POST /pricing/refresh` and `/pricing/set` rebuild the catalog and install it with SetCatalog, which swaps what the value was derived FROM and cannot reach the copy — so `GET /pricing` reported the new rate while the ledger, WriteSummary, and the --max-session-cost-usd ceiling all kept charging the old one. Reporting right and billing wrong is the worst of the two: an operator who refreshes because they believe the rates are stale gets told they are now correct.

The catalog wins over the fallback whenever one is installed, and that is the point rather than a caveat: cmd/core-agent installs one at boot, and from then on the catalog IS the definition of the rate — including the operator's cfg.Model.Pricing override, which SetCatalog folds in as the CfgOverride layer (see PriceFor, which likewise ignores its cfg argument once a catalog is present).

The fallback covers library and test use that never calls SetCatalog. Those callers keep the value they passed in, so an embedder supplying an explicit rate is not quietly overruled by a builtin table they never opted into. It is a no-catalog fallback and not a lookup-miss one: under an installed catalog an unknown model resolves unpriced rather than to the captured value, matching what GET /pricing has always reported for it (cmd/core-agent's single-session provider re-resolves with no fallback at all) and keeping the card and the ledger on one answer.

func PriceForWithSource 

func PriceForWithSource(modelID string, cfg *config.Config) (Pricing, string)

PriceForWithSource is PriceFor + the catalog layer name that served the rate (pricing.SourceCfgOverride / SourceProjectFile / SourceUserManual / SourceUserExternal / SourceBuiltin). Empty source when no rate was found. Used by /pricing so operators can spot when a rate came from a stale builtin instead of the freshly-refreshed LiteLLM external catalog they were expecting.

The cfg override path (used only when no globalCatalog is installed) reports source SourceCfgOverride when the model resolves through it.

func (Pricing) CostUSD 

func (p Pricing) CostUSD(inputTokens, outputTokens int) float64

CostUSD returns the dollar cost of (input, output) tokens at p. Treats every input token as uncached — see CostUSDWithCache for the cached-vs-uncached split.

func (Pricing) CostUSDForTurn added in v2.9.0 

func (p Pricing) CostUSDForTurn(u TurnUsage) float64

CostUSDForTurn prices one turn's full token breakdown, applying TurnUsage.Clamped first so the three input buckets are disjoint and non-negative. This is the one place turn cost is defined: Tracker's AppendUsage and the tracker-less fallbacks in pkg/agent all route through it so a cache-warming turn can't be priced two different ways depending on which call site saw it.

ThoughtsTokens is billed at the output rate, on top of OutputTokens. Gemini reports thoughts as a bucket ADDITIVE to candidates rather than a subset of it — live metadata reads promptTokenCount 12455 + candidatesTokenCount 85 + thoughtsTokenCount 570 = totalTokenCount 13110 — and Google charges for them as output. Tracked since the first tap but never priced, which is not a rounding error on a thinking model: a measured agentic turn spent 6,449 thought tokens against 1,180 candidate tokens, so 85% of the billable output was invisible to the ledger, to /usage, and to the --max-session-cost-usd ceiling that reads it.

Adding the bucket unconditionally is provider-safe rather than an Anthropic double-count: Anthropic bills thinking inside Usage.OutputTokens, and pkg/models/anthropic's usageMetadata maps that field to CandidatesTokenCount and never populates ThoughtsTokenCount — so on that path this term is zero and the turn prices exactly as it did before. Only providers that report thoughts out of band (Gemini/Vertex, via ThoughtsTokenCount) move.

A negative count cannot subtract from the bill, because Clamped floors the three top-line buckets. That belongs there rather than here: pricing was never the only reader of these numbers, and a thoughts term guarded locally would have kept the negative out of the invoice while leaving it in Totals and in the monotonic OTel counter.

func (Pricing) CostUSDWithCache 

func (p Pricing) CostUSDWithCache(uncachedInputTokens, cachedInputTokens, outputTokens int) float64

CostUSDWithCache returns the dollar cost with cache-hit tokens billed at CachedInputPerMTok. When CachedInputPerMTok is zero (rate unknown) cached tokens fall back to InputPerMTok so the estimate never silently drops to zero cost for cached input.

Providers that also report cache-WRITE tokens should call CostUSDWithCacheWrites; this signature has no bucket for them, so callers fold them into uncached and understate the bill (#263).

func (Pricing) CostUSDWithCacheTTLs added in v2.9.0 

func (p Pricing) CostUSDWithCacheTTLs(uncachedInputTokens, cacheReadTokens, cacheWriteTokens, cacheWrite1hTokens, outputTokens int) float64

CostUSDWithCacheTTLs is CostUSDWithCacheWrites with the write bucket split by breakpoint TTL. cacheWrite1hTokens is a SUBSET of cacheWriteTokens — Anthropic's `usage.cache_creation.ephemeral_1h_input_tokens` — billed at CacheCreation1hInputPerMTok, with the remainder at the 5-minute rate.

An unknown 1-hour rate falls back to the 5-minute rate rather than to base input: the nearer neighbour is the better estimate for a catalog row that is missing a field. Mirrors pricing.Rates.CostUSDWithCacheTTLs.

func (Pricing) CostUSDWithCacheWrites added in v2.9.0 

func (p Pricing) CostUSDWithCacheWrites(uncachedInputTokens, cacheReadTokens, cacheWriteTokens, outputTokens int) float64

CostUSDWithCacheWrites is CostUSDWithCache plus the cache-write bucket, billed at CacheCreationInputPerMTok.

The three input buckets must be disjoint: pass uncached = total prompt - cache reads - cache writes. Both cache rates fall back to InputPerMTok when unknown, so a model missing from the catalog degrades to the old understated number rather than billing cache traffic as free.

Every write token is billed at the 5-minute TTL rate; callers that know the TTL split should use CostUSDWithCacheTTLs (#770).

func (Pricing) IsZero 

func (p Pricing) IsZero() bool

IsZero reports whether the rates carry no useful pricing. CachedInputPerMTok isn't part of the check: a row that carries only a cache rate but no base input/output rates is still "unpriced".

type RegisterOption added in v2.8.0 

type RegisterOption func(*registerOptions)

RegisterOption tunes RegisterMetrics.

func WithoutSessionLabels added in v2.8.0 

func WithoutSessionLabels() RegisterOption

WithoutSessionLabels drops the per-session identity attributes (session.id, app.name, user.id) from every usage metric and aggregates values across sessions before observing. For fleet operators where per-session labels would blow up series cardinality. Two consequences of aggregation:

  • core_agent.session.duration is not emitted at all — a wall-clock duration aggregated across sessions is meaningless.
  • `priced` on the cost series is the AND across sessions: false if ANY session had an unpriced turn for that model.

Aggregation (rather than merely stripping attributes) is load- bearing: two Observe calls with identical attribute sets in one callback are last-wins in the OTel SDK, so stripping alone would silently report only one session's totals.

type SingleTracker added in v2.8.0 

type SingleTracker struct {
	Tracker   *Tracker
	SessionID string
	AppName   string
	UserID    string
}

SingleTracker adapts a single (tracker, sessionID) pair to TrackerProvider. Used by daemons running without attach-mode where there is exactly one session and no registry to iterate.

func (SingleTracker) Trackers added in v2.8.0 

func (s SingleTracker) Trackers() []TrackedSession

Trackers implements TrackerProvider. Returns a single-element slice (or empty if Tracker is nil).

type Totals 

type Totals struct {
	Turns                    int
	InputTokens              int
	CachedInputTokens        int
	CacheCreationInputTokens int
	OutputTokens             int
	ThoughtsTokens           int
	ToolUseTokens            int
	CostUSD                  float64
	// UnpricedTurns counts turns whose model had no catalog rate, so
	// their contribution to CostUSD was 0 for lack of a price rather
	// than because the model is free. When > 0, CostUSD is a lower
	// bound and consumers should flag the total as incomplete (e.g.
	// "$X.YY+" or a "$—" marker) rather than presenting it as exact.
	// See #368.
	UnpricedTurns int
}

Totals aggregates a slice of Turns. Cached / thoughts / tool-use mirror the Turn fields so callers projecting Totals into wire formats can render every dimension without walking All().

func (Totals) UncachedInputTokens added in v2.9.0 

func (t Totals) UncachedInputTokens() int

UncachedInputTokens is the session's fresh-input remainder — the aggregate counterpart to Turn.UncachedInputTokens.

type TrackedSession added in v2.8.0 

type TrackedSession struct {
	Tracker   *Tracker
	SessionID string
	AppName   string
	UserID    string
}

TrackedSession pairs a Tracker with the session-identity fields we stamp as metric attributes. Populated by the daemon at observer registration time; a session's identity does not change over its lifetime so callers can snapshot once.

AppName and UserID are optional. Empty values are dropped from the attribute set — the resulting series carry SessionID only, which is the load-bearing identity for the "per-session drill-down" story.

type Tracker 

type Tracker struct {
	// contains filtered or unexported fields
}

Tracker accumulates per-turn usage for one session.

Thread-safe: the agent goroutine (or run loop) calls Append; readers access via Last/Totals/All.

func NewTracker 

func NewTracker() *Tracker

NewTracker returns a tracker with its session-start time set to now.

func (*Tracker) All 

func (t *Tracker) All() []Turn

All returns a copy of every recorded turn.

func (*Tracker) Append 

func (t *Tracker) Append(model string, inputTokens, outputTokens int, p Pricing) Turn

Append records one turn's usage with input/output only. Cost is computed via the supplied Pricing; pass a zero Pricing to skip cost tracking. If SetOnAppend has been called with a non-nil callback, the callback fires after the new turn is durable in the tracker and the lock has been released.

Callers that have a full per-turn breakdown (cache hits, thoughts, tool-use) should use AppendUsage instead so the extra dimensions flow through to Totals + wire formats.

func (*Tracker) AppendDigestSavings 

func (t *Tracker) AppendDigestSavings(rec DigestSavingsRecord)

AppendDigestSavings accumulates one MCP digest-wrap result into the session's cumulative counters. Negative ParentTokensSaved is clamped to zero — a "digest" longer than the original happens occasionally on the passthrough path when the wrap adds a truncation marker, and we don't want that to subtract from savings totals.

func (*Tracker) AppendUsage 

func (t *Tracker) AppendUsage(model string, u TurnUsage, p Pricing) Turn

AppendUsage records one turn's usage with the full per-field breakdown. Cost goes through CostUSDForTurn so all three input buckets — uncached, cache-read, cache-write — are billed at their own rates in the stored Turn, and thinking tokens at the output rate.

The cache buckets are clamped into InputTokens — see TurnUsage.Clamped for why and in what order.

func (*Tracker) ContextWindowSize 

func (t *Tracker) ContextWindowSize() int

ContextWindowSize returns the model's max input window from a hardcoded table, keyed on the most recent turn's model name. Returns 0 for unknown models (or when no turn has landed yet) — consumers should treat 0 as "unknown; suppress any per-context UI segment and skip threshold-based behaviors like compaction." See contextWindowSizeFor for the lookup table.

Lifted from cmd/core-agent/coretui_enabled.go where it was first implemented as part of the core-tui adapter tier-3+ work (commit be8dae5). Agent-level code (compaction trigger, micro-subagents) needs the same accessor, so it lives on the substrate type rather than the adapter bridge.

func (*Tracker) ContextWindowUsed 

func (t *Tracker) ContextWindowUsed() int

ContextWindowUsed approximates the current context fill as the most recent turn's input-token count. Each turn re-sends the full conversation, so the input count is the rolling context size. Returns 0 before any turn has landed (matches "unknown" semantics — consumers should suppress the segment).

func (*Tracker) DigestSavings 

func (t *Tracker) DigestSavings() DigestSavingsTotals

DigestSavings returns the session-cumulative snapshot of the digest-wrap's effect. Safe to call from any goroutine.

func (*Tracker) Duration 

func (t *Tracker) Duration() time.Duration

Duration reports wall-clock time since NewTracker was called.

func (*Tracker) Last 

func (t *Tracker) Last() (Turn, bool)

Last returns the most recently appended turn, or zero if none yet.

func (*Tracker) SetOnAppend 

func (t *Tracker) SetOnAppend(f func())

SetOnAppend registers a callback that fires after every Append call. The callback runs after the lock is released, so it can safely call Totals(), TotalsByModel(), or any other Tracker accessor without risking a re-entrant deadlock.

Used by the attach layer to push usage-update events on the SSE stream as turn cost lands — each Append represents a turn whose cumulative impact should reach connected operators.

Pass nil to unregister. Safe to set multiple times (last wins); callers wiring this from the broadcaster do so on first subscriber and clear it on last detach.

func (*Tracker) Totals 

func (t *Tracker) Totals() Totals

Totals returns the cumulative usage across all turns.

func (*Tracker) TotalsByModel 

func (t *Tracker) TotalsByModel() map[string]Totals

TotalsByModel groups the session's turns by model name and returns the per-model totals. Useful for surfaces that want to break down "$X.YY total" into "$A.BB parent model + $C.DD subtask model" so the cost-efficiency win of routing subtasks to a cheaper model is directly visible. Empty map when no turns recorded.

type TrackerProvider added in v2.8.0 

type TrackerProvider interface {
	Trackers() []TrackedSession
}

TrackerProvider enumerates live sessions for the metrics observer to sample on each export interval. Implementations are responsible for thread-safe access to the underlying session registry.

The interface is deliberately narrow so callers can supply either an attach-mode SessionRegistry adapter (multi-session daemons) or a SingleTracker wrapper (non-attach daemons with a single primary tracker) without pulling pkg/attach into this package's imports.

type Turn 

type Turn struct {
	Model                    string
	InputTokens              int
	CachedInputTokens        int
	CacheCreationInputTokens int
	OutputTokens             int
	ThoughtsTokens           int
	ToolUseTokens            int
	CostUSD                  float64
	At                       time.Time
	// Unpriced is true when the model had no rate in the pricing
	// catalog, so CostUSD is 0 because the price is unknown rather
	// than because the model is free. Threads through from the
	// Pricing passed to AppendUsage; consumers rendering cost should
	// show "$—" for unpriced turns instead of "$0.00". See #368.
	Unpriced bool
}

Turn captures one model call's resource use. Times are wall clock so summary lines can include session duration without a monotonic ref.

InputTokens is the total effective prompt size — for Gemini this matches PromptTokenCount, which already includes any cache-hit tokens (google.golang.org/genai types.go: "the total effective prompt size meaning this includes the number of tokens in the cached content"). CachedInputTokens and CacheCreationInputTokens are therefore both subsets of InputTokens, not additions to it, and they never overlap with each other. Uncached = InputTokens - CachedInputTokens - CacheCreationInputTokens.

CacheCreationInputTokens is the write bucket: tokens this turn spent establishing a cache entry, billed at a premium (Anthropic charges 1.25x base input on the 5-minute TTL, 2x on the 1-hour TTL). Zero for providers that don't bill writes separately — Gemini's explicit caches charge per-hour storage, not per written token.

func (Turn) UncachedInputTokens added in v2.9.0 

func (t Turn) UncachedInputTokens() int

UncachedInputTokens is the turn's fresh-input remainder: the prompt minus cache reads minus cache writes. Never negative. Wire projections use it instead of open-coding the subtraction, which is how the cache-write bucket got double-counted into "uncached" before #263.

type TurnTap 

type TurnTap struct {
	// contains filtered or unexported fields
}

TurnTap accumulates per-model-turn usage from a stream of *session.Event, applying Gemini's "cumulative UsageMetadata per chunk, final on TurnComplete" convention: overwrite last-seen per event, commit exactly once on TurnComplete, reset between turns.

Motivation: Gemini's UsageMetadata is cumulative across streaming chunks within a single model turn — earlier chunks carry running totals, the final chunk carries the per-turn total. Naïve Append- on-every-event both inflates the tracker's turn count (one Append per chunk) and double-counts tokens (summing cumulative running totals). This bug bit us in the core-tui adapter (fixed in #156, surfaced in the field as "totals exactly 2x the last turn") and #157 extracted the pattern so future adapters get it right by default.

Zero-value ready. Not safe for concurrent use — one TurnTap per event iterator.

Typical usage (bookkeeping only): 

var tap usage.TurnTap
for ev, err := range agent.Run(ctx, prompt) {
    tap.Observe(ev)
    if u, ok := tap.Commit(ev); ok {
        tracker.AppendUsage(model, u, pricing)
    }
    // ... other per-event work
}

TUI-style usage (live per-event running total AND commit): 

tap.Observe(ev)
if peek := tap.Peek(); peek.InputTokens > 0 {
    stampLiveCounter(peek)  // reflects running total mid-turn
}
if u, ok := tap.Commit(ev); ok {
    turn := tracker.AppendUsage(model, u, pricing)
    stampFinalCost(turn.CostUSD)
}

func (*TurnTap) Commit 

func (t *TurnTap) Commit(ev *session.Event) (TurnUsage, bool)

Commit returns (per-turn totals, true) exactly when ev is a TurnComplete carrying non-zero accumulated usage, after resetting internal state so the next turn's chunks accumulate cleanly. Returns (zero, false) otherwise. Call after Observe.

func (*TurnTap) Observe 

func (t *TurnTap) Observe(ev *session.Event)

Observe updates the last-seen usage from ev.UsageMetadata, plus the cache-write sidecar on ev.CustomMetadata. Ignores events without UsageMetadata (and nil events); safe to call for every event in the iterator.

func (*TurnTap) Peek 

func (t *TurnTap) Peek() TurnUsage

Peek returns the current last-seen usage without touching state. Reflects the running cumulative total mid-turn and the final total at the instant of TurnComplete (before Commit resets it).

type TurnUsage 

type TurnUsage struct {
	InputTokens              int
	CachedInputTokens        int
	CacheCreationInputTokens int
	// CacheCreation1hInputTokens is the share of
	// CacheCreationInputTokens written at a 1-hour breakpoint TTL
	// rather than the default 5-minute one — a SUBSET of the write
	// bucket, not a fourth input bucket. Anthropic bills it at 2x base
	// input against the 5-minute TTL's 1.25x and reports the split on
	// the response, so a mixed-TTL request has a right answer to bill
	// rather than a guess (#770).
	//
	// Zero for every provider that offers one TTL or none, which
	// prices exactly as it did before.
	CacheCreation1hInputTokens int
	OutputTokens               int
	// ThoughtsTokens is the turn's reasoning tokens, reported by
	// providers that meter them separately (Gemini's
	// thoughtsTokenCount). ADDITIVE to OutputTokens, not a subset of
	// it — the provider's own total is prompt + candidates + thoughts —
	// and billed at the output rate, which CostUSDForTurn does. Zero
	// for providers that fold thinking into their output count, which
	// is why adding it there can't double-charge them.
	ThoughtsTokens int
	ToolUseTokens  int
}

TurnUsage is the per-call token breakdown a provider adapter hands to Tracker.AppendUsage. Provider-independent: adapters normalize their per-response metadata into this shape (see TurnUsageFromGenaiMetadata for the Gemini/Vertex path).

func TurnUsageFromGenaiMetadata 

func TurnUsageFromGenaiMetadata(u *genai.GenerateContentResponseUsageMetadata) TurnUsage

TurnUsageFromGenaiMetadata projects one genai UsageMetadata block into the provider-independent TurnUsage shape.

PromptTokenCount is the total effective prompt size and already includes cache-hit tokens (see the Turn docstring in tracker.go). Returns a zero TurnUsage for a nil input.

This signature has no access to the cache-write sidecar, so turns that wrote cache entries come back with CacheCreationInputTokens == 0 and their cost is understated by the write premium. Callers holding an *session.Event or *model.LLMResponse should use TurnUsageFromMetadata and pass CustomMetadata through.

func TurnUsageFromMetadata added in v2.9.0 

func TurnUsageFromMetadata(u *genai.GenerateContentResponseUsageMetadata, custom map[string]any) TurnUsage

TurnUsageFromMetadata projects one turn's genai UsageMetadata plus the provider's CustomMetadata sidecar into TurnUsage. All tap sites use this so field extraction — including the cache-write bucket providers can only report out-of-band — stays identical: call it once per event with UsageMetadata != nil, overwriting the "last seen" turn snapshot (matching the existing lastIn/lastOut overwrite pattern).

custom may be nil; the cache-write buckets then read zero, which is correct for every provider that doesn't bill writes separately.

func (TurnUsage) Clamped added in v2.9.0 

func (u TurnUsage) Clamped() TurnUsage

Clamped returns u with the two cache buckets forced inside InputTokens, so the uncached remainder can never go negative. Defensive against provider quirks where a cache counter over-reports; reads are clamped first and writes get whatever room is left, so a contradictory pair can only shrink the premium-rated write bucket — an under-estimate, never a phantom charge.

CacheCreation1hInputTokens is clamped last, into the write bucket it is a subset of, for the same reason and in the same direction: the 1-hour share is the dearer of the two TTLs, so an over-report shrinks to what the write bucket can hold rather than billing tokens the turn never wrote.

The three top-line counts — InputTokens, OutputTokens, ThoughtsTokens — are floored at zero first, so a provider (or a JSON sidecar) reporting a negative can neither subtract from the bill nor push a monotonic OTel counter backwards. That mattered less when nothing but cost read these; ThoughtsTokens is now billed at the output rate (#927) and every one of the three is summed into Totals and observed on gen_ai.client.token.usage, which is an Int64ObservableCounter and rejects a negative observation outright. Flooring input before the cache buckets is deliberate: the cache clamps below are all relative to InputTokens, so a negative prompt count has to become zero before they can mean anything.

Applied by Tracker.AppendUsage and by Pricing.CostUSDForTurn, so tracker-backed and tracker-less call sites agree on what a turn cost.

func (TurnUsage) UncachedInputTokens added in v2.9.0 

func (u TurnUsage) UncachedInputTokens() int

UncachedInputTokens is the fresh-input remainder: the prompt minus what was served from cache and minus what was written to cache. Never negative (the buckets are clamped first).

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.