Documentation
¶
Overview ¶
Package usage carries the provider-usage record that genai's UsageMetadata has no room for.
genai.GenerateContentResponseUsageMetadata is Gemini's shape. Every provider mast speaks to is projected onto it, and the projection is lossy in both directions: Anthropic's cache_creation_input_tokens has nowhere to go (core-models' Anthropic adapter folds it into the prompt total, where on its own it would bill at 1x instead of 1.25x, or 2x for a 1-hour write), and a counter a provider simply did not report is indistinguishable from one it reported as zero.
Detail is the sidecar that fixes both. It carries cache reads, cache writes and the 1-hour share of the writes as separate counts, so the meter bills each at its own rate. An adapter attaches one beside the genai record, under budget.DetailKey in model.LLMResponse.CustomMetadata; pkg/budget's meter reads the buckets it prices and ignores the rest. See docs/model-support-design.md §4.3, and #352 for why this landed before pkg/budget froze rather than with the rest of the multi-provider work.
Scope: this package is the M0 subset — what the two shipped adapters can state today. The profile-shaped fields §4.3 also names (backend, region, and the self-hosted KV block) belong to M1–M4 and are #312's territory, post-v1.0.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Attach ¶
func Attach(resp *adkmodel.LLMResponse, d *Detail)
Attach puts d on resp under budget.DetailKey, allocating the CustomMetadata map if the response has none and leaving any other keys alone. A nil resp or a nil d is a no-op: an adapter with nothing to say attaches nothing, rather than an empty record that reads as "asked and answered".
Types ¶
type Detail ¶
type Detail struct {
// CacheReadTokens is the prompt subset served from a cache —
// Anthropic's cache_read_input_tokens, Gemini's
// cachedContentTokenCount, and (M3, not yet) OpenAI's cached_tokens.
CacheReadTokens *int64 `json:"cache_read_tokens,omitempty"`
// CacheWriteTokens is the prompt subset that created a cache entry —
// Anthropic's cache_creation_input_tokens, billed at a premium over
// fresh input rather than a discount. The bucket this package exists
// for: genai has no field for it and pkg/budget cannot price what it
// cannot see.
//
// Nil for Gemini, which has no equivalent: its explicit caches bill
// storage per hour, not per written token.
CacheWriteTokens *int64 `json:"cache_write_tokens,omitempty"`
// CacheWrite1hTokens is the subset of CacheWriteTokens written with a
// 1-hour breakpoint (a bundle's prompt_cache.ttl: 1h), which bills at
// 2x base input against the 5-minute 1.25x. Nil when the provider did
// not split the bucket.
CacheWrite1hTokens *int64 `json:"cache_write_1h_tokens,omitempty"`
// ReasoningTokens is thinking billed apart from the visible output —
// Gemini's thoughtsTokenCount. Nil for Anthropic, whose output count
// already includes thinking, so it is a genuine "did not say" rather
// than a zero.
//
// Recorded, not priced: no rate card mast knows charges reasoning
// separately from output, and the meter keeps folding it into the
// output bucket. When one does, the count is already here.
ReasoningTokens *int64 `json:"reasoning_tokens,omitempty"`
// ToolUseTokens is prompt spent on tool definitions and server-side
// tool traffic — Gemini's toolUsePromptTokenCount. Prompt-side, and
// already inside the prompt total, so it is a breakdown rather than
// an addition.
ToolUseTokens *int64 `json:"tool_use_tokens,omitempty"`
// ServedModel is what the backend says it actually ran, which is not
// always what was asked for. Duplicated from LLMResponse.ModelVersion
// on purpose: that field is a pricing key the adapter may have had to
// substitute (core-models' Anthropic adapter falls back to the
// requested id when the echo is a Vertex resource path), and this one
// is the unedited echo.
ServedModel string `json:"served_model,omitempty"`
// ProviderRequestID is the id a support ticket needs — Anthropic's
// message id, Gemini's responseId.
ProviderRequestID string `json:"provider_request_id,omitempty"`
}
Detail is one model call as the provider described it, normalized across providers.
Every count is a pointer, and a nil one means the provider did not say — which is not zero (model-support-design R3). The distinction is the point of the type: "this turn wrote no cache entry" and "this backend has no cache-write concept" price the same today and must not be reported the same, or a bucket that goes missing after a provider upgrade looks exactly like a bucket that was empty.
The value is in-process only. It rides on an ADK map that does get persisted with the event, so it must stay JSON-marshalable, but nothing reads it back: an event rehydrated from storage carries the sidecar as a plain map and prices from the genai fields, the same as an event that never had one.
func FromEvent ¶
FromEvent returns the sidecar an event carries, or nil. Only the in-process form is recognized — see Detail on why a rehydrated event has none.
func (*Detail) UsageBuckets ¶
UsageBuckets implements budget.Detailer: the subset of Detail the meter prices. A nil *Detail reports nothing said, so a caller need not branch on whether an adapter produced one.