usage

package
v2.9.0-dev.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 17, 2026 License: Apache-2.0 Imports: 14 Imported by: 0

Documentation

Overview

Package usage tracks token + cost accounting for the agent loop.

Every model call returns a UsageMetadata block with input and output token counts; a Tracker accumulates these across a session. Pricing numbers come from a built-in table that callers may override per model via .agents/config.json (model.pricing).

Index

Constants

View Source 
const (
	// #nosec G101 -- OTel GenAI semconv metric name, not a credential.
	MetricGenAITokenUsage = "gen_ai.client.token.usage"
	MetricSessionTurns    = "core_agent.session.turns"
	MetricSessionCost     = "core_agent.session.cost_usd"
	MetricSessionDuration = "core_agent.session.duration"
	// MetricDigestSubagentCost lives here rather than pkg/digest
	// because the accumulator is Tracker.DigestSavings() — the digest
	// package's own TelemetrySnapshot carries no cost field.
	MetricDigestSubagentCost = "core_agent.digest.subagent.cost_usd"
)

Metric names — GenAI semconv where a stable name exists, otherwise core_agent.* for our own surface. See docs/metrics-design.md for the full rationale on why we adopt GenAI semconv (cloud vendor dashboards render token / cost panels automatically).

View Source 
const (
	AttrGenAIModel = "gen_ai.request.model"
	// #nosec G101 -- OTel GenAI semconv attribute key, not a credential.
	AttrGenAITokenType = "gen_ai.token.type"
	AttrSessionID      = "session.id"
	AttrAppName        = "app.name"
	AttrUserID         = "user.id"
	// AttrPriced tags the cost series with whether every turn for the
	// model had a known catalog rate. priced=false means CostUSD is a
	// lower bound because at least one turn was unpriced (rate unknown,
	// billed as $0) — dashboards can filter it out of "exact spend"
	// panels rather than under-reporting. See #368.
	AttrPriced = "priced"
)

Attribute keys. GenAI semconv keys use the exact upstream spelling so consumers with dashboards keyed on gen_ai.* work out of the box. Session identity uses session.id (semconv-adjacent — the GenAI SIG has proposed but not finalized a gen_ai.session.id).

View Source 
const (
	TokenTypeInput  = "input"
	TokenTypeOutput = "output"
	TokenTypeCached = "cached"
	// TokenTypeCacheWrite is input tokens that WROTE a cache entry,
	// billed at a premium rather than the cached discount (#263). Its
	// own series because it is disjoint from both `input` (which here
	// carries the whole prompt) and `cached`, and because the spend it
	// represents is what a dashboard tracking cache ROI has to net
	// against the savings on `cached`.
	TokenTypeCacheWrite = "cache_write"
	TokenTypeThought    = "thoughts"
	TokenTypeToolUse    = "tool_use"
)

Token-type attribute values. Match the Turn breakdown fields.

View Source 
const CacheCreationTokensMetadataKey = "cache_creation_input_tokens"

CacheCreationTokensMetadataKey is the LLMResponse.CustomMetadata key under which a provider reports the turn's cache-WRITE token count — tokens billed at a premium for establishing a cache entry rather than at the base input rate.

Why a CustomMetadata sidecar and not a UsageMetadata field: genai's GenerateContentResponseUsageMetadata models Gemini's two input buckets (total prompt + cache reads) and has nowhere to put a third. CustomMetadata is the one per-event map that survives ADK's persist round-trip, which is what lets usage.Rebuild reconstruct correct cost from a reloaded eventlog — pkg/eventlog piggy-backs the same mechanism for FinishReason, for the same reason.

Written by pkg/models/anthropic (from cache_creation_input_tokens); read by TurnUsageFromMetadata. The value is an int64 when freshly stamped and a float64 or json.Number after a JSON round-trip, so readers must accept all three — see cacheCreationTokens.

Variables

This section is empty.

Functions

func ContextWindowSizeFor 

func ContextWindowSizeFor(model string) int

ContextWindowSizeFor returns the max input window for model, or 0 when it isn't known.

Two tiers, in order:

  1. pricing.BuiltinContextWindow — LiteLLM's max_input_tokens for every model we ship a rate for, generated by dev/regen-builtin-pricing. Exact match on the lowercased id.
  2. The substring table below, for ids LiteLLM never publishes: long-context "-1m" suffixes, Vertex publication names, and anything an operator pins that upstream hasn't catalogued.

The generated tier exists because the substring table was wrong and nobody noticed: it claimed gemini-2.5-pro held 2,000,000 input tokens — Gemini 1.5 Pro's number — against a real 1,048,576 cap. Mid-tier compaction fires at 0.65 of the window, so it was scheduled for ~1.3M tokens on a session that the provider would have hard- failed first. Generated numbers can't rot that way.

Exported as a package-level function so callers that have a model name in hand (without going through the Tracker) can resolve it directly. The Tracker methods above are the common path.

func KnownModelsCount 

func KnownModelsCount() int

KnownModelsCount returns the total number of models across every layer of the installed pricing catalog (cfg override + project file + user manual + user external + builtin). Returns 0 when no catalog is installed. Used by the attach /pricing endpoint's snapshot so operators can see how many models the daemon knows about at a glance — the previous default of hard-coded 0 was actively misleading during the v2.7.0-dev.3 demo drive.

func RebuildTrackerFromEvents 

func RebuildTrackerFromEvents(
	ctx context.Context,
	t *Tracker,
	events iter.Seq2[*session.Event, error],
	defaultModel string,
	pricingFor func(model string) Pricing,
) error

RebuildTrackerFromEvents replays a persisted-event stream into t, reconstructing the per-turn totals via the same TurnTap.Observe + TurnTap.Commit pattern the live turn loop uses. Called on the session-resume path (cmd/core-agent/multi_session.go's reproduceAgent when origin=="resumed") so the newly-minted per-session tracker carries the historical totals instead of starting at zero.

Motivation: PR #275 correctly isolated per-session usage.Trackers to stop cross-session contamination, but every session-resume path (SessionResumer / registry eviction miss / daemon restart) then began at zero — the eventlog was intact, but the tracker was fresh. The visible bug: /stats and the TUI's status-bar aggregate showed "0 in / 0 out / $0.00" for sessions with real historical work. Per-turn footers kept working because they replay from live SSE events, not from the tracker.

Best-effort semantics:

  • Events missing UsageMetadata: skipped by TurnTap.Observe (safe).
  • Events missing ModelVersion: fall back to defaultModel. In practice the session's primary model is stable across its lifetime, so this is fine.
  • pricingFor returning zero: tracker records tokens but $0.00 cost for that model. Downstream cost totals reflect this; operators re-appending after a pricing refresh land the correct cost on the next real turn.
  • Context cancellation: early return with ctx.Err().
  • Iterator errors: early return with the error (caller decides whether to fail-open or fail-loud).

Caller MUST invoke this BEFORE wiring tracker.SetOnAppend (which happens later inside agent.New's option evaluation), otherwise each rebuild AppendUsage would fire the OnAppend callback and broadcast N synthetic usage-update SSE events. The compose.ReproduceAgent call site respects this ordering by construction.

func RegisterMetrics added in v2.8.0 

func RegisterMetrics(mp metric.MeterProvider, tp TrackerProvider, opts ...RegisterOption) (metric.Registration, error)

RegisterMetrics wires the async usage observers against mp. Callers pass the process-global MeterProvider (obtained via otel.GetMeterProvider()) so metric points flow into whichever reader(s) telemetry.SetupMetrics installed. tp is called on every export interval — implementations must be cheap and thread-safe.

Returns the registered Registration so the caller can Unregister on shutdown if desired; typical usage discards it (the MeterProvider shutdown cleans up).

func SetCatalog 

func SetCatalog(c *pricing.Catalog)

SetCatalog installs the catalog PriceFor consults. Safe to call from any goroutine; lookups in flight see either the old or new catalog atomically, never a torn read.

Types

type DigestSavingsRecord 

type DigestSavingsRecord struct {
	Path                 string
	ParentTokensSaved    int // max(0, OriginalTokensEst - DigestTokensEst)
	SubagentModel        string
	SubagentInputTokens  int
	SubagentOutputTokens int
	SubagentCostUSD      float64
}

DigestSavingsRecord is one per-call sample of the MCP digest wrap's effect on the parent's context. Aggregated into DigestSavingsTotals via Tracker.AppendDigestSavings; callers construct one per Process result the wrap hands back.

Path mirrors digest.Method (structural_json / llm_fallback / passthrough). Passthrough records still flow through — a call the router decided to pass through verbatim IS a data point (told the operator "the wrap layer thought this was small enough to skip").

type DigestSavingsTotals 

type DigestSavingsTotals struct {
	StructuralCalls          int
	StructuralTokensSaved    int
	AgenticCalls             int
	AgenticTokensSaved       int // parent-side tokens saved BEFORE subagent offset
	AgenticSubagentInTokens  int
	AgenticSubagentOutTokens int
	AgenticSubagentCostUSD   float64
	PassthroughCalls         int
}

DigestSavingsTotals is the cumulative session view rendered by /context and (when wired) OTel session-close attributes. Structural and agentic-path counts are broken out because their cost math differs (agentic pays a subagent bill, structural doesn't).

type Pricing 

type Pricing struct {
	InputPerMTok              float64
	CachedInputPerMTok        float64
	CacheCreationInputPerMTok float64
	OutputPerMTok             float64
	// UpdatedAt is when the rate was last verified against its
	// source. Threads through from pkg/pricing.Rates so /pricing
	// can surface staleness. Zero when unknown.
	UpdatedAt time.Time
	// Unpriced is true when no catalog layer had a rate for the model
	// (pricing.Catalog.Lookup returned found=false). It disambiguates
	// "rate unknown" (Unpriced=true, cost should render "$—") from a
	// genuinely free model (Unpriced=false, all rates zero). Without
	// this flag a $0 cost is indistinguishable from an unknown-price
	// model in Totals, per-model breakdowns, and the
	// core_agent.session.cost_usd metric — see #368.
	Unpriced bool
}

Pricing is the per-million-token rate for one model. Fields are USD per million tokens (the same unit upstream providers publish public list rates in). CachedInputPerMTok is the reduced rate applied to prompt-cache-hit input tokens — Gemini charges 25% of the base input rate for both implicit and explicit caches. CacheCreationInputPerMTok is the PREMIUM rate applied to input tokens that write a cache entry (Anthropic's cache_creation_input_tokens, 1.25x base input); zero for providers that don't bill writes separately. Like pricing.Rates.CacheCreationInputPerMTok it holds the 5-minute-TTL rate only — see that field's doc before adding a 1-hour TTL anywhere. A zero Pricing carries no useful pricing — callers should distinguish "rate unknown" from "free" (e.g. echo models). See pricing.Rates / pricing.Catalog for the layered resolution behind PriceFor.

func PriceFor 

func PriceFor(modelID string, cfg *config.Config) Pricing

PriceFor returns the Pricing for modelID. Resolution chain (first exact match wins; longest-prefix fallback at the end):

  1. cfg.Model.Pricing[modelID] — operator override
  2. .agents/pricing.json models[modelID] — project file
  3. ~/.core-agent/pricing.json — user file (manual + external)
  4. compiled-in builtin — fallback
  5. longest-prefix match across (1)..(4) — suffix variants
  6. Pricing{} — rate unknown

cfg is consulted via the catalog (if installed via SetCatalog) or via an on-the-fly lookup when no catalog is installed. The no-catalog path covers tests + library use that doesn't go through cmd/core-agent's startup.

func PriceForWithSource 

func PriceForWithSource(modelID string, cfg *config.Config) (Pricing, string)

PriceForWithSource is PriceFor + the catalog layer name that served the rate (pricing.SourceCfgOverride / SourceProjectFile / SourceUserManual / SourceUserExternal / SourceBuiltin). Empty source when no rate was found. Used by /pricing so operators can spot when a rate came from a stale builtin instead of the freshly-refreshed LiteLLM external catalog they were expecting.

The cfg override path (used only when no globalCatalog is installed) reports source SourceCfgOverride when the model resolves through it.

func (Pricing) CostUSD 

func (p Pricing) CostUSD(inputTokens, outputTokens int) float64

CostUSD returns the dollar cost of (input, output) tokens at p. Treats every input token as uncached — see CostUSDWithCache for the cached-vs-uncached split.

func (Pricing) CostUSDForTurn added in v2.9.0 

func (p Pricing) CostUSDForTurn(u TurnUsage) float64

CostUSDForTurn prices one turn's full token breakdown, applying TurnUsage.Clamped first so the three input buckets are disjoint and non-negative. This is the one place turn cost is defined: Tracker's AppendUsage and the tracker-less fallbacks in pkg/agent all route through it so a cache-warming turn can't be priced two different ways depending on which call site saw it.

func (Pricing) CostUSDWithCache 

func (p Pricing) CostUSDWithCache(uncachedInputTokens, cachedInputTokens, outputTokens int) float64

CostUSDWithCache returns the dollar cost with cache-hit tokens billed at CachedInputPerMTok. When CachedInputPerMTok is zero (rate unknown) cached tokens fall back to InputPerMTok so the estimate never silently drops to zero cost for cached input.

Providers that also report cache-WRITE tokens should call CostUSDWithCacheWrites; this signature has no bucket for them, so callers fold them into uncached and understate the bill (#263).

func (Pricing) CostUSDWithCacheWrites added in v2.9.0 

func (p Pricing) CostUSDWithCacheWrites(uncachedInputTokens, cacheReadTokens, cacheWriteTokens, outputTokens int) float64

CostUSDWithCacheWrites is CostUSDWithCache plus the cache-write bucket, billed at CacheCreationInputPerMTok.

The three input buckets must be disjoint: pass uncached = total prompt - cache reads - cache writes. Both cache rates fall back to InputPerMTok when unknown, so a model missing from the catalog degrades to the old understated number rather than billing cache traffic as free.

func (Pricing) IsZero 

func (p Pricing) IsZero() bool

IsZero reports whether the rates carry no useful pricing. CachedInputPerMTok isn't part of the check: a row that carries only a cache rate but no base input/output rates is still "unpriced".

type RegisterOption added in v2.8.0 

type RegisterOption func(*registerOptions)

RegisterOption tunes RegisterMetrics.

func WithoutSessionLabels added in v2.8.0 

func WithoutSessionLabels() RegisterOption

WithoutSessionLabels drops the per-session identity attributes (session.id, app.name, user.id) from every usage metric and aggregates values across sessions before observing. For fleet operators where per-session labels would blow up series cardinality. Two consequences of aggregation:

  • core_agent.session.duration is not emitted at all — a wall-clock duration aggregated across sessions is meaningless.
  • `priced` on the cost series is the AND across sessions: false if ANY session had an unpriced turn for that model.

Aggregation (rather than merely stripping attributes) is load- bearing: two Observe calls with identical attribute sets in one callback are last-wins in the OTel SDK, so stripping alone would silently report only one session's totals.

type SingleTracker added in v2.8.0 

type SingleTracker struct {
	Tracker   *Tracker
	SessionID string
	AppName   string
	UserID    string
}

SingleTracker adapts a single (tracker, sessionID) pair to TrackerProvider. Used by daemons running without attach-mode where there is exactly one session and no registry to iterate.

func (SingleTracker) Trackers added in v2.8.0 

func (s SingleTracker) Trackers() []TrackedSession

Trackers implements TrackerProvider. Returns a single-element slice (or empty if Tracker is nil).

type Totals 

type Totals struct {
	Turns                    int
	InputTokens              int
	CachedInputTokens        int
	CacheCreationInputTokens int
	OutputTokens             int
	ThoughtsTokens           int
	ToolUseTokens            int
	CostUSD                  float64
	// UnpricedTurns counts turns whose model had no catalog rate, so
	// their contribution to CostUSD was 0 for lack of a price rather
	// than because the model is free. When > 0, CostUSD is a lower
	// bound and consumers should flag the total as incomplete (e.g.
	// "$X.YY+" or a "$—" marker) rather than presenting it as exact.
	// See #368.
	UnpricedTurns int
}

Totals aggregates a slice of Turns. Cached / thoughts / tool-use mirror the Turn fields so callers projecting Totals into wire formats can render every dimension without walking All().

func (Totals) UncachedInputTokens added in v2.9.0 

func (t Totals) UncachedInputTokens() int

UncachedInputTokens is the session's fresh-input remainder — the aggregate counterpart to Turn.UncachedInputTokens.

type TrackedSession added in v2.8.0 

type TrackedSession struct {
	Tracker   *Tracker
	SessionID string
	AppName   string
	UserID    string
}

TrackedSession pairs a Tracker with the session-identity fields we stamp as metric attributes. Populated by the daemon at observer registration time; a session's identity does not change over its lifetime so callers can snapshot once.

AppName and UserID are optional. Empty values are dropped from the attribute set — the resulting series carry SessionID only, which is the load-bearing identity for the "per-session drill-down" story.

type Tracker 

type Tracker struct {
	// contains filtered or unexported fields
}

Tracker accumulates per-turn usage for one session.

Thread-safe: the agent goroutine (or run loop) calls Append; readers access via Last/Totals/All.

func NewTracker 

func NewTracker() *Tracker

NewTracker returns a tracker with its session-start time set to now.

func (*Tracker) All 

func (t *Tracker) All() []Turn

All returns a copy of every recorded turn.

func (*Tracker) Append 

func (t *Tracker) Append(model string, inputTokens, outputTokens int, p Pricing) Turn

Append records one turn's usage with input/output only. Cost is computed via the supplied Pricing; pass a zero Pricing to skip cost tracking. If SetOnAppend has been called with a non-nil callback, the callback fires after the new turn is durable in the tracker and the lock has been released.

Callers that have a full per-turn breakdown (cache hits, thoughts, tool-use) should use AppendUsage instead so the extra dimensions flow through to Totals + wire formats.

func (*Tracker) AppendDigestSavings 

func (t *Tracker) AppendDigestSavings(rec DigestSavingsRecord)

AppendDigestSavings accumulates one MCP digest-wrap result into the session's cumulative counters. Negative ParentTokensSaved is clamped to zero — a "digest" longer than the original happens occasionally on the passthrough path when the wrap adds a truncation marker, and we don't want that to subtract from savings totals.

func (*Tracker) AppendUsage 

func (t *Tracker) AppendUsage(model string, u TurnUsage, p Pricing) Turn

AppendUsage records one turn's usage with the full per-field breakdown. Cost applies CostUSDWithCacheWrites so all three input buckets — uncached, cache-read, cache-write — are billed at their own rates in the stored Turn.

The cache buckets are clamped into InputTokens — see TurnUsage.Clamped for why and in what order.

func (*Tracker) ContextWindowSize 

func (t *Tracker) ContextWindowSize() int

ContextWindowSize returns the model's max input window from a hardcoded table, keyed on the most recent turn's model name. Returns 0 for unknown models (or when no turn has landed yet) — consumers should treat 0 as "unknown; suppress any per-context UI segment and skip threshold-based behaviors like compaction." See contextWindowSizeFor for the lookup table.

Lifted from cmd/core-agent/coretui_enabled.go where it was first implemented as part of the core-tui adapter tier-3+ work (commit be8dae5). Agent-level code (compaction trigger, micro-subagents) needs the same accessor, so it lives on the substrate type rather than the adapter bridge.

func (*Tracker) ContextWindowUsed 

func (t *Tracker) ContextWindowUsed() int

ContextWindowUsed approximates the current context fill as the most recent turn's input-token count. Each turn re-sends the full conversation, so the input count is the rolling context size. Returns 0 before any turn has landed (matches "unknown" semantics — consumers should suppress the segment).

func (*Tracker) DigestSavings 

func (t *Tracker) DigestSavings() DigestSavingsTotals

DigestSavings returns the session-cumulative snapshot of the digest-wrap's effect. Safe to call from any goroutine.

func (*Tracker) Duration 

func (t *Tracker) Duration() time.Duration

Duration reports wall-clock time since NewTracker was called.

func (*Tracker) Last 

func (t *Tracker) Last() (Turn, bool)

Last returns the most recently appended turn, or zero if none yet.

func (*Tracker) SetOnAppend 

func (t *Tracker) SetOnAppend(f func())

SetOnAppend registers a callback that fires after every Append call. The callback runs after the lock is released, so it can safely call Totals(), TotalsByModel(), or any other Tracker accessor without risking a re-entrant deadlock.

Used by the attach layer to push usage-update events on the SSE stream as turn cost lands — each Append represents a turn whose cumulative impact should reach connected operators.

Pass nil to unregister. Safe to set multiple times (last wins); callers wiring this from the broadcaster do so on first subscriber and clear it on last detach.

func (*Tracker) Totals 

func (t *Tracker) Totals() Totals

Totals returns the cumulative usage across all turns.

func (*Tracker) TotalsByModel 

func (t *Tracker) TotalsByModel() map[string]Totals

TotalsByModel groups the session's turns by model name and returns the per-model totals. Useful for surfaces that want to break down "$X.YY total" into "$A.BB parent model + $C.DD subtask model" so the cost-efficiency win of routing subtasks to a cheaper model is directly visible. Empty map when no turns recorded.

type TrackerProvider added in v2.8.0 

type TrackerProvider interface {
	Trackers() []TrackedSession
}

TrackerProvider enumerates live sessions for the metrics observer to sample on each export interval. Implementations are responsible for thread-safe access to the underlying session registry.

The interface is deliberately narrow so callers can supply either an attach-mode SessionRegistry adapter (multi-session daemons) or a SingleTracker wrapper (non-attach daemons with a single primary tracker) without pulling pkg/attach into this package's imports.

type Turn 

type Turn struct {
	Model                    string
	InputTokens              int
	CachedInputTokens        int
	CacheCreationInputTokens int
	OutputTokens             int
	ThoughtsTokens           int
	ToolUseTokens            int
	CostUSD                  float64
	At                       time.Time
	// Unpriced is true when the model had no rate in the pricing
	// catalog, so CostUSD is 0 because the price is unknown rather
	// than because the model is free. Threads through from the
	// Pricing passed to AppendUsage; consumers rendering cost should
	// show "$—" for unpriced turns instead of "$0.00". See #368.
	Unpriced bool
}

Turn captures one model call's resource use. Times are wall clock so summary lines can include session duration without a monotonic ref.

InputTokens is the total effective prompt size — for Gemini this matches PromptTokenCount, which already includes any cache-hit tokens (google.golang.org/genai types.go: "the total effective prompt size meaning this includes the number of tokens in the cached content"). CachedInputTokens and CacheCreationInputTokens are therefore both subsets of InputTokens, not additions to it, and they never overlap with each other. Uncached = InputTokens - CachedInputTokens - CacheCreationInputTokens.

CacheCreationInputTokens is the write bucket: tokens this turn spent establishing a cache entry, billed at a premium (Anthropic charges 1.25x base input on the 5-minute TTL, 2x on the 1-hour TTL). Zero for providers that don't bill writes separately — Gemini's explicit caches charge per-hour storage, not per written token.

func (Turn) UncachedInputTokens added in v2.9.0 

func (t Turn) UncachedInputTokens() int

UncachedInputTokens is the turn's fresh-input remainder: the prompt minus cache reads minus cache writes. Never negative. Wire projections use it instead of open-coding the subtraction, which is how the cache-write bucket got double-counted into "uncached" before #263.

type TurnTap 

type TurnTap struct {
	// contains filtered or unexported fields
}

TurnTap accumulates per-model-turn usage from a stream of *session.Event, applying Gemini's "cumulative UsageMetadata per chunk, final on TurnComplete" convention: overwrite last-seen per event, commit exactly once on TurnComplete, reset between turns.

Motivation: Gemini's UsageMetadata is cumulative across streaming chunks within a single model turn — earlier chunks carry running totals, the final chunk carries the per-turn total. Naïve Append- on-every-event both inflates the tracker's turn count (one Append per chunk) and double-counts tokens (summing cumulative running totals). This bug bit us in the core-tui adapter (fixed in #156, surfaced in the field as "totals exactly 2x the last turn") and #157 extracted the pattern so future adapters get it right by default.

Zero-value ready. Not safe for concurrent use — one TurnTap per event iterator.

Typical usage (bookkeeping only): 

var tap usage.TurnTap
for ev, err := range agent.Run(ctx, prompt) {
    tap.Observe(ev)
    if u, ok := tap.Commit(ev); ok {
        tracker.AppendUsage(model, u, pricing)
    }
    // ... other per-event work
}

TUI-style usage (live per-event running total AND commit): 

tap.Observe(ev)
if peek := tap.Peek(); peek.InputTokens > 0 {
    stampLiveCounter(peek)  // reflects running total mid-turn
}
if u, ok := tap.Commit(ev); ok {
    turn := tracker.AppendUsage(model, u, pricing)
    stampFinalCost(turn.CostUSD)
}

func (*TurnTap) Commit 

func (t *TurnTap) Commit(ev *session.Event) (TurnUsage, bool)

Commit returns (per-turn totals, true) exactly when ev is a TurnComplete carrying non-zero accumulated usage, after resetting internal state so the next turn's chunks accumulate cleanly. Returns (zero, false) otherwise. Call after Observe.

func (*TurnTap) Observe 

func (t *TurnTap) Observe(ev *session.Event)

Observe updates the last-seen usage from ev.UsageMetadata, plus the cache-write sidecar on ev.CustomMetadata. Ignores events without UsageMetadata (and nil events); safe to call for every event in the iterator.

func (*TurnTap) Peek 

func (t *TurnTap) Peek() TurnUsage

Peek returns the current last-seen usage without touching state. Reflects the running cumulative total mid-turn and the final total at the instant of TurnComplete (before Commit resets it).

type TurnUsage 

type TurnUsage struct {
	InputTokens              int
	CachedInputTokens        int
	CacheCreationInputTokens int
	OutputTokens             int
	ThoughtsTokens           int
	ToolUseTokens            int
}

TurnUsage is the per-call token breakdown a provider adapter hands to Tracker.AppendUsage. Provider-independent: adapters normalize their per-response metadata into this shape (see TurnUsageFromGenaiMetadata for the Gemini/Vertex path).

func TurnUsageFromGenaiMetadata 

func TurnUsageFromGenaiMetadata(u *genai.GenerateContentResponseUsageMetadata) TurnUsage

TurnUsageFromGenaiMetadata projects one genai UsageMetadata block into the provider-independent TurnUsage shape.

PromptTokenCount is the total effective prompt size and already includes cache-hit tokens (see the Turn docstring in tracker.go). Returns a zero TurnUsage for a nil input.

This signature has no access to the cache-write sidecar, so turns that wrote cache entries come back with CacheCreationInputTokens == 0 and their cost is understated by the write premium. Callers holding an *session.Event or *model.LLMResponse should use TurnUsageFromMetadata and pass CustomMetadata through.

func TurnUsageFromMetadata added in v2.9.0 

func TurnUsageFromMetadata(u *genai.GenerateContentResponseUsageMetadata, custom map[string]any) TurnUsage

TurnUsageFromMetadata projects one turn's genai UsageMetadata plus the provider's CustomMetadata sidecar into TurnUsage. All tap sites use this so field extraction — including the cache-write bucket providers can only report out-of-band — stays identical: call it once per event with UsageMetadata != nil, overwriting the "last seen" turn snapshot (matching the existing lastIn/lastOut overwrite pattern).

custom may be nil; the cache-write bucket then reads zero, which is correct for every provider that doesn't bill writes separately.

func (TurnUsage) Clamped added in v2.9.0 

func (u TurnUsage) Clamped() TurnUsage

Clamped returns u with the two cache buckets forced inside InputTokens, so the uncached remainder can never go negative. Defensive against provider quirks where a cache counter over-reports; reads are clamped first and writes get whatever room is left, so a contradictory pair can only shrink the premium-rated write bucket — an under-estimate, never a phantom charge.

Applied by Tracker.AppendUsage and by Pricing.CostUSDForTurn, so tracker-backed and tracker-less call sites agree on what a turn cost.

func (TurnUsage) UncachedInputTokens added in v2.9.0 

func (u TurnUsage) UncachedInputTokens() int

UncachedInputTokens is the fresh-input remainder: the prompt minus what was served from cache and minus what was written to cache. Never negative (the buckets are clamped first).

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.