usage

package
v0.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 30, 2026 License: Apache-2.0 Imports: 3 Imported by: 0

Documentation

Overview

Package usage carries the provider-usage record that genai's UsageMetadata has no room for.

genai.GenerateContentResponseUsageMetadata is Gemini's shape. Every provider mast speaks to is projected onto it, and the projection is lossy in both directions: Anthropic's cache_creation_input_tokens has nowhere to go (core-models' Anthropic adapter folds it into the prompt total, where on its own it would bill at 1x instead of 1.25x, or 2x for a 1-hour write), and a counter a provider simply did not report is indistinguishable from one it reported as zero.

Detail is the sidecar that fixes both. It carries cache reads, cache writes and the 1-hour share of the writes as separate counts, so the meter bills each at its own rate. An adapter attaches one beside the genai record, under budget.DetailKey in model.LLMResponse.CustomMetadata; pkg/budget's meter reads the buckets it prices and ignores the rest. See docs/model-support-design.md §4.3, and #352 for why this landed before pkg/budget froze rather than with the rest of the multi-provider work.

Scope: this package is the M0 subset — what the two shipped adapters can state today. The profile-shaped fields §4.3 also names (backend, region, and the self-hosted KV block) belong to M1–M4 and are #312's territory, post-v1.0.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Attach

func Attach(resp *adkmodel.LLMResponse, d *Detail)

Attach puts d on resp under budget.DetailKey, allocating the CustomMetadata map if the response has none and leaving any other keys alone. A nil resp or a nil d is a no-op: an adapter with nothing to say attaches nothing, rather than an empty record that reads as "asked and answered".

func Int64

func Int64(n int64) *int64

Int64 returns a pointer to n, for building a Detail from counters a provider stated. Use it only where the provider really did say: a pointer to zero and a nil pointer are different claims.

Types

type Detail

type Detail struct {
	// CacheReadTokens is the prompt subset served from a cache —
	// Anthropic's cache_read_input_tokens, Gemini's
	// cachedContentTokenCount, and (M3, not yet) OpenAI's cached_tokens.
	CacheReadTokens *int64 `json:"cache_read_tokens,omitempty"`

	// CacheWriteTokens is the prompt subset that created a cache entry —
	// Anthropic's cache_creation_input_tokens, billed at a premium over
	// fresh input rather than a discount. The bucket this package exists
	// for: genai has no field for it and pkg/budget cannot price what it
	// cannot see.
	//
	// Nil for Gemini, which has no equivalent: its explicit caches bill
	// storage per hour, not per written token.
	CacheWriteTokens *int64 `json:"cache_write_tokens,omitempty"`

	// CacheWrite1hTokens is the subset of CacheWriteTokens written with a
	// 1-hour breakpoint (a bundle's prompt_cache.ttl: 1h), which bills at
	// 2x base input against the 5-minute 1.25x. Nil when the provider did
	// not split the bucket.
	CacheWrite1hTokens *int64 `json:"cache_write_1h_tokens,omitempty"`

	// ReasoningTokens is thinking billed apart from the visible output —
	// Gemini's thoughtsTokenCount. Nil for Anthropic, whose output count
	// already includes thinking, so it is a genuine "did not say" rather
	// than a zero.
	//
	// Recorded, not priced: no rate card mast knows charges reasoning
	// separately from output, and the meter keeps folding it into the
	// output bucket. When one does, the count is already here.
	ReasoningTokens *int64 `json:"reasoning_tokens,omitempty"`

	// ToolUseTokens is prompt spent on tool definitions and server-side
	// tool traffic — Gemini's toolUsePromptTokenCount. Prompt-side, and
	// already inside the prompt total, so it is a breakdown rather than
	// an addition.
	ToolUseTokens *int64 `json:"tool_use_tokens,omitempty"`

	// ServedModel is what the backend says it actually ran, which is not
	// always what was asked for. Duplicated from LLMResponse.ModelVersion
	// on purpose: that field is a pricing key the adapter may have had to
	// substitute (core-models' Anthropic adapter falls back to the
	// requested id when the echo is a Vertex resource path), and this one
	// is the unedited echo.
	ServedModel string `json:"served_model,omitempty"`

	// ProviderRequestID is the id a support ticket needs — Anthropic's
	// message id, Gemini's responseId.
	ProviderRequestID string `json:"provider_request_id,omitempty"`
}

Detail is one model call as the provider described it, normalized across providers.

Every count is a pointer, and a nil one means the provider did not say — which is not zero (model-support-design R3). The distinction is the point of the type: "this turn wrote no cache entry" and "this backend has no cache-write concept" price the same today and must not be reported the same, or a bucket that goes missing after a provider upgrade looks exactly like a bucket that was empty.

The value is in-process only. It rides on an ADK map that does get persisted with the event, so it must stay JSON-marshalable, but nothing reads it back: an event rehydrated from storage carries the sidecar as a plain map and prices from the genai fields, the same as an event that never had one.

func FromEvent

func FromEvent(ev *session.Event) *Detail

FromEvent returns the sidecar an event carries, or nil. Only the in-process form is recognized — see Detail on why a rehydrated event has none.

func (*Detail) UsageBuckets

func (d *Detail) UsageBuckets() budget.Buckets

UsageBuckets implements budget.Detailer: the subset of Detail the meter prices. A nil *Detail reports nothing said, so a caller need not branch on whether an adapter produced one.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL