usage

package
v3.1.0-rc.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 5, 2026 License: Apache-2.0 Imports: 16 Imported by: 0

Documentation

Overview

Package usage tracks token + cost accounting for the agent loop.

Every model call returns a UsageMetadata block with input and output token counts; a Tracker accumulates these across a session. Pricing numbers come from a built-in table that callers may override per model via .agents/config.json (model.pricing).

Index

Constants

View Source
const (
	// #nosec G101 -- OTel GenAI semconv metric name, not a credential.
	MetricGenAITokenUsage = "gen_ai.client.token.usage"
	MetricSessionTurns    = "core_agent.session.turns"
	MetricSessionCost     = "core_agent.session.cost_usd"
	MetricSessionDuration = "core_agent.session.duration"
	// MetricDigestSubagentCost lives here rather than pkg/digest
	// because the accumulator is Tracker.DigestSavings() — the digest
	// package's own TelemetrySnapshot carries no cost field.
	MetricDigestSubagentCost = "core_agent.digest.subagent.cost_usd"
)

Metric names — GenAI semconv where a stable name exists, otherwise core_agent.* for our own surface. See docs/metrics-design.md for the full rationale on why we adopt GenAI semconv (cloud vendor dashboards render token / cost panels automatically).

View Source
const (
	AttrGenAIModel = "gen_ai.request.model"
	// #nosec G101 -- OTel GenAI semconv attribute key, not a credential.
	AttrGenAITokenType = "gen_ai.token.type"
	AttrSessionID      = "session.id"
	AttrAppName        = "app.name"
	AttrUserID         = "user.id"
	// AttrPriced tags the cost series with whether every turn for the
	// model had a known catalog rate. priced=false means CostUSD is a
	// lower bound because at least one turn was unpriced (rate unknown,
	// billed as $0) — dashboards can filter it out of "exact spend"
	// panels rather than under-reporting. See #368.
	AttrPriced = "priced"
)

Attribute keys. GenAI semconv keys use the exact upstream spelling so consumers with dashboards keyed on gen_ai.* work out of the box. Session identity uses session.id (semconv-adjacent — the GenAI SIG has proposed but not finalized a gen_ai.session.id).

View Source
const (
	TokenTypeInput  = "input"
	TokenTypeOutput = "output"
	TokenTypeCached = "cached"
	// TokenTypeCacheWrite is input tokens that WROTE a cache entry,
	// billed at a premium rather than the cached discount (#263). Its
	// own series because it is disjoint from both `input` (which here
	// carries the whole prompt) and `cached`, and because the spend it
	// represents is what a dashboard tracking cache ROI has to net
	// against the savings on `cached`.
	TokenTypeCacheWrite = "cache_write"
	TokenTypeThought    = "thoughts"
	TokenTypeToolUse    = "tool_use"
)

Token-type attribute values. Match the Turn breakdown fields.

View Source
const (
	// SkipParentUsageMetadataKey marks a subtask event whose spend is
	// attributed to a different session via a tool-response savings
	// sidecar (e.g. the primary agent's MCP digest LLM fallback, #717,
	// #1359) so a session-tree rebuild skips the subtask's own
	// UsageMetadata and does not double-count or mis-bill it.
	SkipParentUsageMetadataKey = "skip_parent_usage"

	// ModelVersionMetadataKey is the CustomMetadata key under which
	// eventlog and BranchInjectingService persist LLMResponse.ModelVersion
	// across ADK database serialization (#1359).
	ModelVersionMetadataKey = "model_version"

	// BillingBackendMetadataKey is the CustomMetadata key under which
	// BranchInjectingService persists a subagent's billing backend
	// across ADK database serialization (#1359).
	BillingBackendMetadataKey = "billing_backend"
)
View Source
const BytesPerTokenEstimate = 3

BytesPerTokenEstimate converts appended bytes to tokens for the unmeasured part of ContextWindowUsed. Three, not the usual four.

The usual English-prose rule of thumb is ~4 bytes per token, and using it here would be wrong in the unsafe direction twice over. First, what is being estimated is overwhelmingly not prose: it is JSON, YAML and log output, which tokenize far worse than prose because punctuation, indentation and long unbroken identifiers each cost a token. Second, the asymmetry — underestimating means the threshold does not fire and the defect this exists to fix survives, while overestimating means compaction fires somewhat early, which costs a summarizer call and is immediately visible. Three is a deliberate over-count for the payload shapes that actually blow the window, and it is only ever applied to the unmeasured tail: the moment the next model call lands, a measured number replaces it.

View Source
const CacheCreation1hTokensMetadataKey = "cache_creation_1h_input_tokens"

CacheCreation1hTokensMetadataKey is the CustomMetadata key under which a provider reports how much of CacheCreationTokensMetadataKey was written at a 1-hour breakpoint TTL rather than the default 5-minute one — a SUBSET of that count, billed at 2x base input instead of 1.25x (#770).

Absent for every provider that offers one cache TTL or none, and absent from recordings made before #770; both read as zero, which prices the whole write bucket at the 5-minute rate exactly as before.

Written by pkg/models/anthropic (from usage.cache_creation.ephemeral_1h_input_tokens); read by TurnUsageFromMetadata. Same numeric-shape caveat as CacheCreationTokensMetadataKey.

View Source
const CacheCreationTokensMetadataKey = "cache_creation_input_tokens"

CacheCreationTokensMetadataKey is the LLMResponse.CustomMetadata key under which a provider reports the turn's cache-WRITE token count — tokens billed at a premium for establishing a cache entry rather than at the base input rate.

Why a CustomMetadata sidecar and not a UsageMetadata field: genai's GenerateContentResponseUsageMetadata models Gemini's two input buckets (total prompt + cache reads) and has nowhere to put a third. CustomMetadata is the one per-event map that survives ADK's persist round-trip, which is what lets usage.Rebuild reconstruct correct cost from a reloaded eventlog — pkg/eventlog piggy-backs the same mechanism for FinishReason, for the same reason.

Written by pkg/models/anthropic (from cache_creation_input_tokens); read by TurnUsageFromMetadata. The value is an int64 when freshly stamped and a float64 or json.Number after a JSON round-trip, so readers must accept all three — see cacheCreationTokens.

View Source
const SideUsageKey = "side_usage"

SideUsageKey marks an event's CustomMetadata as one side call's usage (AppendSideUsage), so RebuildTrackerFromEvents can replay it after a restart. Without it the cost ceilings would forget that spend on a pod roll, which they must not (#643).

The usage rides in CustomMetadata and not in the event's UsageMetadata on purpose: every reader of UsageMetadata — the turn tap, the attach and core-tui projections — takes it for a conversation turn.

Variables

This section is empty.

Functions

func BackendFor

func BackendFor(cfg *config.Config) string

BackendFor returns the backend that bills cfg's model: a provider profile's backend name (profile.Profile.BackendName, extends applied), or the core provider's own name, which is already the core-models backend name. An empty model.provider (auto-detection) falls back to the backend installed with SetBackend, and a nil cfg likewise.

The backend matters for rows the bare table can't express: OpenAI's and xAI's models (priced only as "openai/…" and "xai/…"), Vertex AI partner models, and Claude Sonnet 4.5's long-context rate on Vertex. For every other model the qualified and bare rows agree.

func ContextWindowSizeFor

func ContextWindowSizeFor(model string) int

ContextWindowSizeFor returns the max input window for model, or 0 when it isn't known.

Three tiers, in order:

  1. The window a provider profile declares for the model (SetDeclaredContextWindows). That is the window the operator's server actually serves, so it outranks the model's published one: a vLLM run with --max-model-len 65536 rejects past 64K whatever the model card says.
  2. pricing.BuiltinContextWindow — LiteLLM's max_input_tokens for every model we ship a rate for, generated by dev/regen-builtin-pricing. Exact match on the lowercased id.
  3. The substring table below, for ids LiteLLM never publishes: long-context "-1m" suffixes, Vertex publication names, and anything an operator pins that upstream hasn't catalogued.

The generated tier exists because the substring table was wrong and nobody noticed: it claimed gemini-2.5-pro held 2,000,000 input tokens — Gemini 1.5 Pro's number — against a real 1,048,576 cap. Mid-tier compaction fires at 0.65 of the window, so it was scheduled for ~1.3M tokens on a session that the provider would have hard- failed first. Generated numbers can't rot that way.

Exported as a package-level function so callers that have a model name in hand (without going through the Tracker) can resolve it directly. The Tracker methods above are the common path.

func DeclaredRates

func DeclaredRates(cfg *config.Config) map[string]pricing.ModelRates

DeclaredRates is the provider profiles' declared rates, for the catalog's lowest layer (pricing.Options.Declared).

Each rate is keyed twice. "<backend>/<model>" (pricing.DeclaredFromProfiles) is the one LookupFor reaches when the caller knows the backend, so two profiles that declare different prices for one model id each get their own. The bare id (config.Config.ProfileRates, first profile wins) is kept for lookups that can't name a backend — a subagent priced with a nil cfg on a daemon whose backend is another profile's — which priced from it before backends existed and would otherwise turn unpriced, and be refused outright under a cost ceiling.

func EstimateTokensFromBytes

func EstimateTokensFromBytes(n int) int

EstimateTokensFromBytes converts a byte count to an estimated token count. Exported because the agent-side budget check reports the estimate in operator-facing text and should not re-derive the ratio.

func KnownModelsCount

func KnownModelsCount() int

KnownModelsCount returns the total number of models across every layer of the installed pricing catalog (cfg override + project file + user manual + user external + builtin). Returns 0 when no catalog is installed. Used by the attach /pricing endpoint's snapshot so operators can see how many models the daemon knows about at a glance — the previous default of hard-coded 0 was actively misleading during the v2.7.0-dev.3 demo drive.

func NewCatalogFromConfig added in v3.1.0

func NewCatalogFromConfig(cfg *config.Config, agentsDir, userHome string) (*pricing.Catalog, error)

NewCatalogFromConfig constructs a *pricing.Catalog from cfg (including cfg.Model.Pricing overrides, provider profiles' declared rates and pricing: "isolated" policy, .agents/pricing.json under agentsDir, and ~/.core-agent/pricing.json under userHome when non-empty).

func RebuildTrackerFromEvents

func RebuildTrackerFromEvents(
	ctx context.Context,
	t *Tracker,
	events iter.Seq2[*session.Event, error],
	defaultModel string,
	pricingFor func(model string) Pricing,
) error

RebuildTrackerFromEvents replays a persisted-event stream into t, reconstructing the per-turn totals via the same TurnTap.Observe + TurnTap.Commit pattern the live turn loop uses. Called on the session-resume path (pkg/compose/multi_session.go's ReproduceAgent when origin=="resumed") so the newly-minted per-session tracker carries the historical totals — including delegated and background subagent turns across the session tree (#1359) — instead of starting at zero.

Motivation: PR #275 correctly isolated per-session usage.Trackers to stop cross-session contamination, but every session-resume path (SessionResumer / registry eviction miss / daemon restart) then began at zero — the eventlog was intact, but the tracker was fresh. The visible bug: /stats and the TUI's status-bar aggregate showed "0 in / 0 out / $0.00" for sessions with real historical work. Per-turn footers kept working because they replay from live SSE events, not from the tracker.

Best-effort semantics:

  • Events missing UsageMetadata: skipped by TurnTap.Observe (safe).
  • Events with CustomMetadata["skip_parent_usage"] == true: skipped so SkipParentUsage subtasks (such as the shared MCP digest LLM fallback) are not billed to the host session on a session-tree rebuild; the calling session's tool-response "savings" sidecar is replayed instead (#1359).
  • Events missing ModelVersion: fall back to CustomMetadata["model_version"] (persisted by eventlog / BranchInjectingService across ADK database round-trips) and then to defaultModel (#1359).
  • Subtask branches ("sub.<name>"): replayed via AppendSideUsage so Tracker.Last() remains the conversation's own turn (#1191, #1359).
  • pricingFor returning zero: tracker records tokens but $0.00 cost for that model. Downstream cost totals reflect this; operators re-appending after a pricing refresh land the correct cost on the next real turn.
  • Context cancellation: early return with ctx.Err().
  • Iterator errors: early return with the error (caller decides whether to fail-open or fail-loud).

Caller MUST invoke this BEFORE wiring tracker.SetOnAppend (which happens later inside agent.New's option evaluation), otherwise each rebuild AppendUsage would fire the OnAppend callback and broadcast N synthetic usage-update SSE events. The compose.ReproduceAgent call site respects this ordering by construction.

func RegisterMetrics

func RegisterMetrics(mp metric.MeterProvider, tp TrackerProvider, opts ...RegisterOption) (metric.Registration, error)

RegisterMetrics wires the async usage observers against mp. Callers pass the process-global MeterProvider (obtained via otel.GetMeterProvider()) so metric points flow into whichever reader(s) telemetry.SetupMetrics installed. tp is called on every export interval — implementations must be cheap and thread-safe.

Returns the registered Registration so the caller can Unregister on shutdown if desired; typical usage discards it (the MeterProvider shutdown cleans up).

func SetBackend

func SetBackend(backend string)

SetBackend installs the backend PriceFor uses when its caller names none — a core-models backend name ("gemini", "vertex", "anthropic", "anthropic-vertex", "openai", "xai", "vertex-maas", or a profile's own), as BackendFor returns it.

func SetCatalog

func SetCatalog(c *pricing.Catalog)

SetCatalog installs the catalog PriceFor consults. Safe to call from any goroutine; lookups in flight see either the old or new catalog atomically, never a torn read.

func SetDeclaredContextWindows

func SetDeclaredContextWindows(m map[string]int)

SetDeclaredContextWindows installs the windows provider profiles declare (config.Config.ProfileContextWindows), the first tier of ContextWindowSizeFor. Install it before the compaction threshold is first computed. Nil or empty clears it.

func SideUsageMetadata

func SideUsageMetadata(model string, u TurnUsage) map[string]any

SideUsageMetadata is the CustomMetadata recording one side call.

Types

type DigestSavingsRecord

type DigestSavingsRecord struct {
	Path                 string
	ParentTokensSaved    int // max(0, OriginalTokensEst - DigestTokensEst)
	SubagentModel        string
	SubagentInputTokens  int
	SubagentOutputTokens int
	SubagentCostUSD      float64

	// The two cache buckets inside SubagentInputTokens, so a caller
	// charging this record to a session can rebuild the TurnUsage the
	// subagent actually spent instead of a flat uncached one (#771).
	SubagentCachedInputTokens        int
	SubagentCacheCreationInputTokens int

	// SubagentCacheCreation1hInputTokens is the 1-hour-TTL share of
	// the write bucket — a SUBSET of the field above, priced at 2x
	// base input rather than 1.25x (#770).
	SubagentCacheCreation1hInputTokens int

	// SubagentThoughtsTokens is the reasoning bucket, ADDITIVE to
	// SubagentOutputTokens and billed at the output rate (#927). A
	// sidecar from a producer predating that field carries zero, which
	// prices as it always did.
	SubagentThoughtsTokens int
}

DigestSavingsRecord is one per-call sample of the MCP digest wrap's effect on the parent's context. Aggregated into DigestSavingsTotals via Tracker.AppendDigestSavings; callers construct one per Process result the wrap hands back.

Path mirrors digest.Method (structural_json / llm_fallback / passthrough). Passthrough records still flow through — a call the router decided to pass through verbatim IS a data point (told the operator "the wrap layer thought this was small enough to skip").

func (DigestSavingsRecord) SubagentTurn

func (r DigestSavingsRecord) SubagentTurn() TurnUsage

SubagentTurn rebuilds the TurnUsage the digest subagent spent, for callers that need to price or append it. Clamped, so a sidecar carrying contradictory buckets can only shrink the premium-rated write bucket rather than invent a negative uncached remainder.

type DigestSavingsTotals

type DigestSavingsTotals struct {
	StructuralCalls          int
	StructuralTokensSaved    int
	AgenticCalls             int
	AgenticTokensSaved       int // parent-side tokens saved BEFORE subagent offset
	AgenticSubagentInTokens  int
	AgenticSubagentOutTokens int
	AgenticSubagentCostUSD   float64
	PassthroughCalls         int
}

DigestSavingsTotals is the cumulative session view rendered by /context and (when wired) OTel session-close attributes. Structural and agentic-path counts are broken out because their cost math differs (agentic pays a subagent bill, structural doesn't).

type Pricing

type Pricing struct {
	InputPerMTok                float64
	CachedInputPerMTok          float64
	CacheCreationInputPerMTok   float64
	CacheCreation1hInputPerMTok float64
	OutputPerMTok               float64
	// UpdatedAt is when the rate was last verified against its
	// source. Threads through from pkg/pricing.Rates so /pricing
	// can surface staleness. Zero when unknown.
	UpdatedAt time.Time
	// Unpriced is true when no catalog layer had a rate for the model
	// (pricing.Catalog.Lookup returned found=false). It disambiguates
	// "rate unknown" (Unpriced=true, cost should render "$—") from a
	// genuinely free model (Unpriced=false, all rates zero). Without
	// this flag a $0 cost is indistinguishable from an unknown-price
	// model in Totals, per-model breakdowns, and the
	// core_agent.session.cost_usd metric — see #368.
	Unpriced bool
	// LongContext, when set, is the model's above-threshold price
	// (Gemini's 200K, Claude Sonnet 4.5's 200K on Vertex AI, OpenAI's
	// 272K). A turn whose whole prompt is over the threshold bills every
	// token at it, output included — see pricing.Rates.At. Nil for a
	// model with one price at every size.
	LongContext *pricing.LongContext
	// Backend is the backend the rate was looked up for, as BackendFor
	// or PriceOn's caller named it: "" when none was known, which
	// prices from the bare rows. Tracker copies it onto each Turn, so a
	// later re-pricing of the turn (the /usage savings reference) asks
	// the same backend the ledger did, even for a subagent whose
	// provider isn't its parent's.
	Backend string
}

Pricing is the per-million-token rate for one model. Fields are USD per million tokens (the same unit upstream providers publish public list rates in). CachedInputPerMTok is the reduced rate applied to prompt-cache-hit input tokens — Gemini charges 25% of the base input rate for both implicit and explicit caches. CacheCreationInputPerMTok is the PREMIUM rate applied to input tokens that write a cache entry (Anthropic's cache_creation_input_tokens, 1.25x base input); zero for providers that don't bill writes separately. Like pricing.Rates.CacheCreationInputPerMTok it holds the 5-minute-TTL rate; CacheCreation1hInputPerMTok holds the 1-hour one (2x base input), and falls back to the 5-minute rate when zero. A zero Pricing carries no useful pricing — callers should distinguish "rate unknown" from "free" (e.g. echo models). See pricing.Rates / pricing.Catalog for the layered resolution behind PriceFor.

func PriceFor

func PriceFor(modelID string, cfg *config.Config) Pricing

PriceFor returns the Pricing for modelID as billed by cfg's backend (BackendFor). Resolution chain (first exact match wins; longest-prefix fallback at the end), each layer consulting "<backend>/<model>" before the bare id — see pricing.Catalog.LookupFor:

  1. cfg.Model.Pricing[modelID] — operator override
  2. .agents/pricing.json models[modelID] — project file
  3. ~/.core-agent/pricing.json — user file (manual + external)
  4. compiled-in builtin — core-models' shared table
  5. provider profiles' declared rates — models no catalog covers
  6. longest-prefix match, layer by layer — suffix variants
  7. Pricing{} — rate unknown

cfg is consulted via the catalog (if installed via SetCatalog) or via an on-the-fly lookup when no catalog is installed. The no-catalog path covers tests + library use that doesn't go through cmd/core-agent's startup.

func PriceForRefreshed

func PriceForRefreshed(modelID string, fallback Pricing) Pricing

PriceForRefreshed re-resolves modelID against the process-wide catalog, falling back to the caller's captured rate when no catalog is installed.

It exists for billing sites that hold a Pricing *value* resolved once and then bill many turns against it (#930). `POST /pricing/refresh` and `/pricing/set` rebuild the catalog and install it with SetCatalog, which swaps what the value was derived FROM and cannot reach the copy — so `GET /pricing` reported the new rate while the ledger, WriteSummary, and the --max-session-cost-usd ceiling all kept charging the old one. Reporting right and billing wrong is the worst of the two: an operator who refreshes because they believe the rates are stale gets told they are now correct.

The catalog wins over the fallback whenever one is installed, and that is the point rather than a caveat: cmd/core-agent installs one at boot, and from then on the catalog IS the definition of the rate — including the operator's cfg.Model.Pricing override, which SetCatalog folds in as the CfgOverride layer (see PriceFor, which likewise ignores its cfg argument once a catalog is present).

The fallback covers library and test use that never calls SetCatalog. Those callers keep the value they passed in, so an embedder supplying an explicit rate is not quietly overruled by a builtin table they never opted into. It is a no-catalog fallback and not a lookup-miss one: under an installed catalog an unknown model resolves unpriced rather than to the captured value, matching what GET /pricing has always reported for it (cmd/core-agent's single-session provider re-resolves with no fallback at all) and keeping the card and the ledger on one answer.

func PriceForWithSource

func PriceForWithSource(modelID string, cfg *config.Config) (Pricing, string)

PriceForWithSource is PriceFor + the catalog layer name that served the rate (pricing.SourceCfgOverride / SourceProjectFile / SourceUserManual / SourceUserExternal / SourceBuiltin). Empty source when no rate was found. Used by /pricing so operators can spot when a rate came from a stale builtin instead of the freshly-refreshed LiteLLM external catalog they were expecting.

The cfg override path (used only when no globalCatalog is installed) reports source SourceCfgOverride when the model resolves through it.

func PriceOn

func PriceOn(backend, modelID string) Pricing

PriceOn returns the Pricing for modelID as billed by backend, for a cost site that knows which provider served the call but has no cfg naming it: a subagent declared on a provider other than its parent's, whose spend PriceFor(model, nil) would otherwise price at the process backend (SetBackend). An empty backend is exactly PriceFor(modelID, nil).

With no catalog installed (tests, library use) there are no declared rates to consult, so it prices from the builtin table.

func PriceOnCatalog added in v3.1.0

func PriceOnCatalog(c *pricing.Catalog, backend, modelID string) Pricing

PriceOnCatalog returns the Pricing for modelID as billed by backend against catalog c. When c is nil, it falls back to PriceOn(backend, modelID). When backend is empty and c is non-nil, it resolves the fallback backend installed via SetBackend before querying c.

func (Pricing) CostUSD

func (p Pricing) CostUSD(inputTokens, outputTokens int) float64

CostUSD returns the dollar cost of (input, output) tokens at p. Treats every input token as uncached — see CostUSDWithCache for the cached-vs-uncached split.

func (Pricing) CostUSDForTurn

func (p Pricing) CostUSDForTurn(u TurnUsage) float64

CostUSDForTurn prices one turn's full token breakdown, applying TurnUsage.Clamped first so the three input buckets are disjoint and non-negative. This is the one place turn cost is defined: Tracker's AppendUsage and the tracker-less fallbacks in pkg/agent all route through it so a cache-warming turn can't be priced two different ways depending on which call site saw it.

ThoughtsTokens is billed at the output rate, on top of OutputTokens. Gemini reports thoughts as a bucket ADDITIVE to candidates rather than a subset of it — live metadata reads promptTokenCount 12455 + candidatesTokenCount 85 + thoughtsTokenCount 570 = totalTokenCount 13110 — and Google charges for them as output. Tracked since the first tap but never priced, which is not a rounding error on a thinking model: a measured agentic turn spent 6,449 thought tokens against 1,180 candidate tokens, so 85% of the billable output was invisible to the ledger, to /usage, and to the --max-session-cost-usd ceiling that reads it.

Adding the bucket unconditionally is provider-safe rather than an Anthropic double-count: Anthropic bills thinking inside Usage.OutputTokens, and pkg/models/anthropic's usageMetadata maps that field to CandidatesTokenCount and never populates ThoughtsTokenCount — so on that path this term is zero and the turn prices exactly as it did before. Only providers that report thoughts out of band (Gemini/Vertex, via ThoughtsTokenCount) move.

A negative count cannot subtract from the bill, because Clamped floors the three top-line buckets. That belongs there rather than here: pricing was never the only reader of these numbers, and a thoughts term guarded locally would have kept the negative out of the invoice while leaving it in Totals and in the monotonic OTel counter.

func (Pricing) CostUSDWithCache

func (p Pricing) CostUSDWithCache(uncachedInputTokens, cachedInputTokens, outputTokens int) float64

CostUSDWithCache returns the dollar cost with cache-hit tokens billed at CachedInputPerMTok. When CachedInputPerMTok is zero (rate unknown) cached tokens fall back to InputPerMTok so the estimate never silently drops to zero cost for cached input.

Providers that also report cache-WRITE tokens should call CostUSDWithCacheWrites; this signature has no bucket for them, so callers fold them into uncached and understate the bill (#263).

func (Pricing) CostUSDWithCacheTTLs

func (p Pricing) CostUSDWithCacheTTLs(uncachedInputTokens, cacheReadTokens, cacheWriteTokens, cacheWrite1hTokens, outputTokens int) float64

CostUSDWithCacheTTLs is CostUSDWithCacheWrites with the write bucket split by breakpoint TTL. cacheWrite1hTokens is a SUBSET of cacheWriteTokens — Anthropic's `usage.cache_creation.ephemeral_1h_input_tokens` — billed at CacheCreation1hInputPerMTok, with the remainder at the 5-minute rate.

An unknown 1-hour rate falls back to the 5-minute rate rather than to base input: the nearer neighbour is the better estimate for a catalog row that is missing a field. A prompt (all three input buckets) over the model's LongContext threshold bills every bucket and the output at the long-context rates. The arithmetic is pricing.Rates.CostUSDWithCacheTTLs, shared with mast, so the two products can't price one turn two ways.

func (Pricing) CostUSDWithCacheWrites

func (p Pricing) CostUSDWithCacheWrites(uncachedInputTokens, cacheReadTokens, cacheWriteTokens, outputTokens int) float64

CostUSDWithCacheWrites is CostUSDWithCache plus the cache-write bucket, billed at CacheCreationInputPerMTok.

The three input buckets must be disjoint: pass uncached = total prompt - cache reads - cache writes. Both cache rates fall back to InputPerMTok when unknown, so a model missing from the catalog degrades to the old understated number rather than billing cache traffic as free.

Every write token is billed at the 5-minute TTL rate; callers that know the TTL split should use CostUSDWithCacheTTLs (#770).

func (Pricing) IsZero

func (p Pricing) IsZero() bool

IsZero reports whether the rates carry no useful pricing. CachedInputPerMTok isn't part of the check: a row that carries only a cache rate but no base input/output rates is still "unpriced".

type RegisterOption

type RegisterOption func(*registerOptions)

RegisterOption tunes RegisterMetrics.

func WithoutSessionLabels

func WithoutSessionLabels() RegisterOption

WithoutSessionLabels drops the per-session identity attributes (session.id, app.name, user.id) from every usage metric and aggregates values across sessions before observing. For fleet operators where per-session labels would blow up series cardinality. Two consequences of aggregation:

  • core_agent.session.duration is not emitted at all — a wall-clock duration aggregated across sessions is meaningless.
  • `priced` on the cost series is the AND across sessions: false if ANY session had an unpriced turn for that model.

Aggregation (rather than merely stripping attributes) is load- bearing: two Observe calls with identical attribute sets in one callback are last-wins in the OTel SDK, so stripping alone would silently report only one session's totals.

type SingleTracker

type SingleTracker struct {
	Tracker   *Tracker
	SessionID string
	AppName   string
	UserID    string
}

SingleTracker adapts a single (tracker, sessionID) pair to TrackerProvider. Used by daemons running without attach-mode where there is exactly one session and no registry to iterate.

func (SingleTracker) Trackers

func (s SingleTracker) Trackers() []TrackedSession

Trackers implements TrackerProvider. Returns a single-element slice (or empty if Tracker is nil).

type Totals

type Totals struct {
	Turns                    int
	InputTokens              int
	CachedInputTokens        int
	CacheCreationInputTokens int
	OutputTokens             int
	ThoughtsTokens           int
	ToolUseTokens            int
	CostUSD                  float64
	// UnpricedTurns counts turns whose model had no catalog rate, so
	// their contribution to CostUSD was 0 for lack of a price rather
	// than because the model is free. When > 0, CostUSD is a lower
	// bound and consumers should flag the total as incomplete (e.g.
	// "$X.YY+" or a "$—" marker) rather than presenting it as exact.
	// See #368.
	UnpricedTurns int
}

Totals aggregates a slice of Turns. Cached / thoughts / tool-use mirror the Turn fields so callers projecting Totals into wire formats can render every dimension without walking All().

func (Totals) UncachedInputTokens

func (t Totals) UncachedInputTokens() int

UncachedInputTokens is the session's fresh-input remainder — the aggregate counterpart to Turn.UncachedInputTokens.

type TrackedSession

type TrackedSession struct {
	Tracker   *Tracker
	SessionID string
	AppName   string
	UserID    string
}

TrackedSession pairs a Tracker with the session-identity fields we stamp as metric attributes. Populated by the daemon at observer registration time; a session's identity does not change over its lifetime so callers can snapshot once.

AppName and UserID are optional. Empty values are dropped from the attribute set — the resulting series carry SessionID only, which is the load-bearing identity for the "per-session drill-down" story.

type Tracker

type Tracker struct {
	// contains filtered or unexported fields
}

Tracker accumulates per-turn usage for one session.

Thread-safe: the agent goroutine (or run loop) calls Append; readers access via Last/Totals/All.

func NewTracker

func NewTracker() *Tracker

NewTracker returns a tracker with its session-start time set to now.

func (*Tracker) AddPendingContextBytes

func (t *Tracker) AddPendingContextBytes(n int)

AddPendingContextBytes records that n bytes of content have been appended to the conversation since the last measured input count. Cleared by the next AppendUsage, whose InputTokens already counts them. Negative and zero values are ignored.

func (*Tracker) All

func (t *Tracker) All() []Turn

All returns a copy of every recorded turn.

func (*Tracker) Append

func (t *Tracker) Append(model string, inputTokens, outputTokens int, p Pricing) Turn

Append records one turn's usage with input/output only. Cost is computed via the supplied Pricing; pass a zero Pricing to skip cost tracking. If SetOnAppend has been called with a non-nil callback, the callback fires after the new turn is durable in the tracker and the lock has been released.

Callers that have a full per-turn breakdown (cache hits, thoughts, tool-use) should use AppendUsage instead so the extra dimensions flow through to Totals + wire formats.

func (*Tracker) AppendDigestSavings

func (t *Tracker) AppendDigestSavings(rec DigestSavingsRecord)

AppendDigestSavings accumulates one MCP digest-wrap result into the session's cumulative counters. Negative ParentTokensSaved is clamped to zero — a "digest" longer than the original happens occasionally on the passthrough path when the wrap adds a truncation marker, and we don't want that to subtract from savings totals.

func (*Tracker) AppendSideUsage

func (t *Tracker) AppendSideUsage(model string, u TurnUsage, p Pricing) Turn

AppendSideUsage records a model call that is not a measurement of the conversation: its cost counts toward Totals, TotalsByModel and so the cost ceilings, but Last skips it, the #975 context estimate is left alone, and the SetOnAppend callback does not fire (it would re-ship the previous turn as the last one). The next AppendUsage brings the totals on the wire up to date. The SetOnTurn spend observer does fire: a side call is spend, and the ledger must see it.

func (*Tracker) AppendUsage

func (t *Tracker) AppendUsage(model string, u TurnUsage, p Pricing) Turn

AppendUsage records one turn's usage with the full per-field breakdown. Cost goes through CostUSDForTurn so all three input buckets — uncached, cache-read, cache-write — are billed at their own rates in the stored Turn, and thinking tokens at the output rate.

The cache buckets are clamped into InputTokens — see TurnUsage.Clamped for why and in what order.

func (*Tracker) ContextWindowSize

func (t *Tracker) ContextWindowSize() int

ContextWindowSize returns the model's max input window from a hardcoded table, keyed on the most recent turn's model name. Returns 0 for unknown models (or when no turn has landed yet) — consumers should treat 0 as "unknown; suppress any per-context UI segment and skip threshold-based behaviors like compaction." See contextWindowSizeFor for the lookup table.

Lifted from cmd/core-agent/coretui_enabled.go where it was first implemented as part of the core-tui adapter tier-3+ work (commit be8dae5). Agent-level code (compaction trigger, micro-subagents) needs the same accessor, so it lives on the substrate type rather than the adapter bridge.

func (*Tracker) ContextWindowUsed

func (t *Tracker) ContextWindowUsed() int

ContextWindowUsed approximates the current context fill as the most recent model call's input-token count PLUS an estimate of everything appended to history since that call (#975). Each call re-sends the full conversation, so the input count is the rolling context size — but only as of the last call, and the gap matters.

The gap is the whole bug. The sequence inside one agentic turn is: model call N completes and commits its usage, a tool runs and returns a payload, model call N+1 is built with that payload included. Nothing evaluates context between the second and third steps, so a single large tool result — a full pod log, a wide `kubectl get -o json`, an MCP response over a big cluster — takes the next request over the window while the utilization figure compaction and the UI both read still reflects the comfortable state before it arrived. On the GKE path large read results are the normal case, not an edge case.

Returns 0 before any turn has landed (matches "unknown" semantics — consumers should suppress the segment). Bytes appended before the first turn are deliberately not reported as usage on their own: with no measured floor to add them to, the estimate alone would be a number with no relationship to the window.

func (*Tracker) ContextWindowUsedEstimated

func (t *Tracker) ContextWindowUsedEstimated() (used int, estimated bool)

ContextWindowUsedEstimated is ContextWindowUsed plus whether the answer includes an estimate, i.e. whether anything has been appended to history since the last measured input count.

Exported separately because the assessment's note on this is right: a utilization number that can only be stale is worse than one that admits it is estimated. A surface rendering a percentage should say which of the two it has. Callers that only need the number keep calling ContextWindowUsed.

func (*Tracker) DigestSavings

func (t *Tracker) DigestSavings() DigestSavingsTotals

DigestSavings returns the session-cumulative snapshot of the digest-wrap's effect. Safe to call from any goroutine.

func (*Tracker) Duration

func (t *Tracker) Duration() time.Duration

Duration reports wall-clock time since NewTracker was called.

func (*Tracker) Last

func (t *Tracker) Last() (Turn, bool)

Last returns the most recently appended turn that is not a side call, or zero if none yet. Its model and input count stand for the conversation (context window, compaction tier, the last-turn cost on the wire), which a side call's do not.

func (*Tracker) PendingContextBytes

func (t *Tracker) PendingContextBytes() int

PendingContextBytes reports the unmeasured byte count currently folded into ContextWindowUsed. For tests and for an operator surface that wants to show the size of the estimate rather than only its existence.

func (*Tracker) RestoreTurns added in v3.1.0

func (t *Tracker) RestoreTurns(turns ...Turn)

RestoreTurns appends turns that were already billed — read back from the durable spend ledger on session resume (#1433) — exactly as stored: CostUSD, At, Side, EventID and the rest are kept, nothing is re-priced, and neither the SetOnAppend nor the SetOnTurn callback fires. Afterwards Totals, TotalsByModel and Last read as they did on the tracker the turns came from.

func (*Tracker) SetOnAppend

func (t *Tracker) SetOnAppend(f func())

SetOnAppend registers a callback that fires after every Append call. The callback runs after the lock is released, so it can safely call Totals(), TotalsByModel(), or any other Tracker accessor without risking a re-entrant deadlock.

Used by the attach layer to push usage-update events on the SSE stream as turn cost lands — each Append represents a turn whose cumulative impact should reach connected operators.

Pass nil to unregister. Safe to set multiple times (last wins); callers wiring this from the broadcaster do so on first subscriber and clear it on last detach.

func (*Tracker) SetOnTurn added in v3.1.0

func (t *Tracker) SetOnTurn(f func(Turn))

SetOnTurn registers a spend observer: f fires with the stored Turn after every AppendUsage and AppendSideUsage, once the lock is released, so f may call any Tracker accessor. It differs from SetOnAppend in both directions: it sees side calls (which spend money without being a conversation turn), and it stays silent for RestoreTurns, which replays spend already recorded rather than new spend.

The durable spend ledger hangs off this hook (#1433): pkg/compose installs one per session that writes each Turn, at the price it was billed, to eventlog.SpendStore.

Pass nil to unregister. Last call wins.

func (*Tracker) Totals

func (t *Tracker) Totals() Totals

Totals returns the cumulative usage across all turns.

func (*Tracker) TotalsByModel

func (t *Tracker) TotalsByModel() map[string]Totals

TotalsByModel groups the session's turns by model name and returns the per-model totals. Useful for surfaces that want to break down "$X.YY total" into "$A.BB parent model + $C.DD subtask model" so the cost-efficiency win of routing subtasks to a cheaper model is directly visible. Empty map when no turns recorded.

type TrackerProvider

type TrackerProvider interface {
	Trackers() []TrackedSession
}

TrackerProvider enumerates live sessions for the metrics observer to sample on each export interval. Implementations are responsible for thread-safe access to the underlying session registry.

The interface is deliberately narrow so callers can supply either an attach-mode SessionRegistry adapter (multi-session daemons) or a SingleTracker wrapper (non-attach daemons with a single primary tracker) without pulling pkg/attach into this package's imports.

type Turn

type Turn struct {
	Model                    string
	InputTokens              int
	CachedInputTokens        int
	CacheCreationInputTokens int
	OutputTokens             int
	ThoughtsTokens           int
	ToolUseTokens            int
	CostUSD                  float64
	At                       time.Time
	// Unpriced is true when the model had no rate in the pricing
	// catalog, so CostUSD is 0 because the price is unknown rather
	// than because the model is free. Threads through from the
	// Pricing passed to AppendUsage; consumers rendering cost should
	// show "$—" for unpriced turns instead of "$0.00". See #368.
	Unpriced bool
	// Side is true for a call recorded with AppendSideUsage: one that
	// spent money on the session's behalf without sending the
	// conversation: the auto-mode approver (#1175), agentic_* subtask
	// turns and digest-summarizer calls (#1191). It counts in Totals
	// and TotalsByModel, and Last skips it.
	Side bool
	// Backend is the backend the turn was priced on (Pricing.Backend):
	// "" when none was known. Re-pricing the turn must ask the same
	// backend, or a subagent's turn on another provider re-prices at
	// the parent's rate.
	Backend string
	// EventID identifies the ADK event this turn was committed on
	// (TurnUsage.EventID, stamped by TurnTap.Commit and tool-savings
	// observers). It is the idempotency key of the durable spend ledger
	// (#1433): a row per event, so the same turn billed twice lands
	// once. Empty for a call with no event at call time (the approver,
	// internal compaction/side-question calls), which the ledger stores
	// with a NULL event_id and does not deduplicate.
	EventID string
}

Turn captures one model call's resource use. Times are wall clock so summary lines can include session duration without a monotonic ref.

InputTokens is the total effective prompt size — for Gemini this matches PromptTokenCount, which already includes any cache-hit tokens (google.golang.org/genai types.go: "the total effective prompt size meaning this includes the number of tokens in the cached content"). CachedInputTokens and CacheCreationInputTokens are therefore both subsets of InputTokens, not additions to it, and they never overlap with each other. Uncached = InputTokens - CachedInputTokens - CacheCreationInputTokens.

CacheCreationInputTokens is the write bucket: tokens this turn spent establishing a cache entry, billed at a premium (Anthropic charges 1.25x base input on the 5-minute TTL, 2x on the 1-hour TTL). Zero for providers that don't bill writes separately — Gemini's explicit caches charge per-hour storage, not per written token.

func (Turn) UncachedInputTokens

func (t Turn) UncachedInputTokens() int

UncachedInputTokens is the turn's fresh-input remainder: the prompt minus cache reads minus cache writes. Never negative. Wire projections use it instead of open-coding the subtraction, which is how the cache-write bucket got double-counted into "uncached" before #263.

type TurnTap

type TurnTap struct {
	// contains filtered or unexported fields
}

TurnTap accumulates per-model-turn usage from a stream of *session.Event, applying Gemini's "cumulative UsageMetadata per chunk, final on TurnComplete" convention: overwrite last-seen per event, commit exactly once on TurnComplete, reset between turns.

Motivation: Gemini's UsageMetadata is cumulative across streaming chunks within a single model turn — earlier chunks carry running totals, the final chunk carries the per-turn total. Naïve Append- on-every-event both inflates the tracker's turn count (one Append per chunk) and double-counts tokens (summing cumulative running totals). This bug bit us in the core-tui adapter (fixed in #156, surfaced in the field as "totals exactly 2x the last turn") and #157 extracted the pattern so future adapters get it right by default.

Zero-value ready. Not safe for concurrent use — one TurnTap per event iterator.

Typical usage (bookkeeping only):

var tap usage.TurnTap
for ev, err := range agent.Run(ctx, prompt) {
    tap.Observe(ev)
    if u, ok := tap.Commit(ev); ok {
        tracker.AppendUsage(model, u, pricing)
    }
    // ... other per-event work
}

TUI-style usage (live per-event running total AND commit):

tap.Observe(ev)
if peek := tap.Peek(); peek.InputTokens > 0 {
    stampLiveCounter(peek)  // reflects running total mid-turn
}
if u, ok := tap.Commit(ev); ok {
    turn := tracker.AppendUsage(model, u, pricing)
    stampFinalCost(turn.CostUSD)
}

func (*TurnTap) Commit

func (t *TurnTap) Commit(ev *session.Event) (TurnUsage, bool)

Commit returns (per-turn totals, true) exactly when ev is a TurnComplete carrying non-zero accumulated usage, after resetting internal state so the next turn's chunks accumulate cleanly. Returns (zero, false) otherwise. Call after Observe.

The returned TurnUsage.EventID is the committing event's ID when it has one, else the ID of the last usage-bearing chunk observed — the per-turn identity the durable spend ledger keys on (#1433).

func (*TurnTap) Observe

func (t *TurnTap) Observe(ev *session.Event)

Observe updates the last-seen usage from ev.UsageMetadata, plus the cache-write sidecar on ev.CustomMetadata. Ignores events without UsageMetadata (and nil events); safe to call for every event in the iterator.

func (*TurnTap) Peek

func (t *TurnTap) Peek() TurnUsage

Peek returns the current last-seen usage without touching state. Reflects the running cumulative total mid-turn and the final total at the instant of TurnComplete (before Commit resets it).

type TurnUsage

type TurnUsage struct {
	InputTokens              int
	CachedInputTokens        int
	CacheCreationInputTokens int
	// CacheCreation1hInputTokens is the share of
	// CacheCreationInputTokens written at a 1-hour breakpoint TTL
	// rather than the default 5-minute one — a SUBSET of the write
	// bucket, not a fourth input bucket. Anthropic bills it at 2x base
	// input against the 5-minute TTL's 1.25x and reports the split on
	// the response, so a mixed-TTL request has a right answer to bill
	// rather than a guess (#770).
	//
	// Zero for every provider that offers one TTL or none, which
	// prices exactly as it did before.
	CacheCreation1hInputTokens int
	OutputTokens               int
	// ThoughtsTokens is the turn's reasoning tokens, reported by
	// providers that meter them separately (Gemini's
	// thoughtsTokenCount). ADDITIVE to OutputTokens, not a subset of
	// it — the provider's own total is prompt + candidates + thoughts —
	// and billed at the output rate, which CostUSDForTurn does. Zero
	// for providers that fold thinking into their output count, which
	// is why adding it there can't double-charge them.
	ThoughtsTokens int
	ToolUseTokens  int
	// EventID is the identity of the ADK event the turn was committed
	// on, not a token count. TurnTap.Commit stamps it so every
	// tap-driven call site threads it to Tracker.AppendUsage without
	// a signature change, and the Turn carries it on to the durable
	// spend ledger as its idempotency key (#1433). Not priced, not
	// clamped, and ignored by every arithmetic method on this type.
	EventID string
}

TurnUsage is the per-call token breakdown a provider adapter hands to Tracker.AppendUsage. Provider-independent: adapters normalize their per-response metadata into this shape (see TurnUsageFromGenaiMetadata for the Gemini/Vertex path).

func TurnUsageFromGenaiMetadata

func TurnUsageFromGenaiMetadata(u *genai.GenerateContentResponseUsageMetadata) TurnUsage

TurnUsageFromGenaiMetadata projects one genai UsageMetadata block into the provider-independent TurnUsage shape.

PromptTokenCount is the total effective prompt size and already includes cache-hit tokens (see the Turn docstring in tracker.go). Returns a zero TurnUsage for a nil input.

This signature has no access to the cache-write sidecar, so turns that wrote cache entries come back with CacheCreationInputTokens == 0 and their cost is understated by the write premium. Callers holding an *session.Event or *model.LLMResponse should use TurnUsageFromMetadata and pass CustomMetadata through.

func TurnUsageFromMetadata

func TurnUsageFromMetadata(u *genai.GenerateContentResponseUsageMetadata, custom map[string]any) TurnUsage

TurnUsageFromMetadata projects one turn's genai UsageMetadata plus the provider's CustomMetadata sidecar into TurnUsage. All tap sites use this so field extraction — including the cache-write bucket providers can only report out-of-band — stays identical: call it once per event with UsageMetadata != nil, overwriting the "last seen" turn snapshot (matching the existing lastIn/lastOut overwrite pattern).

custom may be nil; the cache-write buckets then read zero, which is correct for every provider that doesn't bill writes separately.

func (TurnUsage) Clamped

func (u TurnUsage) Clamped() TurnUsage

Clamped returns u with the two cache buckets forced inside InputTokens, so the uncached remainder can never go negative. Defensive against provider quirks where a cache counter over-reports; reads are clamped first and writes get whatever room is left, so a contradictory pair can only shrink the premium-rated write bucket — an under-estimate, never a phantom charge.

CacheCreation1hInputTokens is clamped last, into the write bucket it is a subset of, for the same reason and in the same direction: the 1-hour share is the dearer of the two TTLs, so an over-report shrinks to what the write bucket can hold rather than billing tokens the turn never wrote.

The three top-line counts — InputTokens, OutputTokens, ThoughtsTokens — are floored at zero first, so a provider (or a JSON sidecar) reporting a negative can neither subtract from the bill nor push a monotonic OTel counter backwards. That mattered less when nothing but cost read these; ThoughtsTokens is now billed at the output rate (#927) and every one of the three is summed into Totals and observed on gen_ai.client.token.usage, which is an Int64ObservableCounter and rejects a negative observation outright. Flooring input before the cache buckets is deliberate: the cache clamps below are all relative to InputTokens, so a negative prompt count has to become zero before they can mean anything.

Applied by Tracker.AppendUsage and by Pricing.CostUSDForTurn, so tracker-backed and tracker-less call sites agree on what a turn cost.

func (TurnUsage) UncachedInputTokens

func (u TurnUsage) UncachedInputTokens() int

UncachedInputTokens is the fresh-input remainder: the prompt minus what was served from cache and minus what was written to cache. Never negative (the buckets are clamped first).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL