usage

package
v0.15.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 14, 2026 License: Apache-2.0 Imports: 2 Imported by: 0

Documentation

Overview

Package usage is the normalized usage record a core-models adapter attaches to every Response, beside genai's UsageMetadata.

genai's struct has no cache-write bucket and no way to say "the provider did not report this". Detail has both: every count is a pointer, nil when the provider said nothing and a pointer to zero when it said zero. The shape starts from mast's shipped internal/providers/usage.Detail (mast #352) and adds the fields a shared layer can populate (docs/design.md §6).

Index

Constants

View Source
const MetadataKey = "core_models.usage"

MetadataKey is the Response.CustomMetadata key Detail is attached under.

Variables

This section is empty.

Functions

func Attach

func Attach(resp *llm.Response, d *Detail)

Attach puts d on resp under MetadataKey, leaving other keys alone.

func Int64

func Int64(n int64) *int64

Int64 returns a pointer to n, for filling Detail's counts.

Types

type Detail

type Detail struct {
	// CacheReadTokens is the prompt subset served from a provider-side
	// cache.
	CacheReadTokens *int64 `json:"cache_read_tokens,omitempty"`
	// CacheWriteTokens is the prompt subset written to a cache, across
	// every TTL. CacheWrite1hTokens is the one-hour share of it, where a
	// provider bills that differently.
	CacheWriteTokens   *int64 `json:"cache_write_tokens,omitempty"`
	CacheWrite1hTokens *int64 `json:"cache_write_1h_tokens,omitempty"`
	// ReasoningTokens and ToolUseTokens are reported counts the provider
	// broke out of its totals.
	ReasoningTokens *int64 `json:"reasoning_tokens,omitempty"`
	ToolUseTokens   *int64 `json:"tool_use_tokens,omitempty"`

	// ServedModel is what the backend said it ran. It may not be a key
	// any price table resolves (a Vertex resource path, a deployment
	// name, a weights path); Response.ModelVersion is the priceable one.
	ServedModel string `json:"served_model,omitempty"`
	// ProviderRequestID is the id a support ticket needs.
	ProviderRequestID string `json:"provider_request_id,omitempty"`
	// Backend is the profile backend that served the call: the key a
	// price is looked up under, beside the model.
	Backend string `json:"backend,omitempty"`
	// Region is the serving region where the backend has one.
	Region string `json:"region,omitempty"`
}

Detail is what a provider reported about one call beyond genai's usage struct. A nil count means "not reported", never zero.

func FromMetadata

func FromMetadata(md map[string]any) (*Detail, bool)

FromMetadata reads a Detail out of a CustomMetadata map. It accepts the value Attach stored, and also the map[string]any that value becomes after a JSON round trip through an event log — the shape a replay sees.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL