Documentation
¶
Overview ¶
Package pricing implements the pricing table and cost formula the cost_meter middleware uses to convert LLM token usage into a USD cost estimate. The table's content arrives from the management server inside cost_meter's middleware config (synthesized from the catalog plus the operator's stored per-provider prices) — the proxy carries no embedded price list. Price updates ride the ordinary mapping push: a chain rebuild constructs a fresh table, so there is nothing to reload.
Index ¶
- func NewEntries(raw map[string]map[string]EntryJSON) (map[string]map[string]Entry, error)
- type Costs
- type Entry
- type EntryJSON
- type Table
- func (t *Table) Cost(provider, model string, inTokens, outTokens, cachedInput, cacheCreation int64) (float64, bool)
- func (t *Table) Costs(provider, model string, inTokens, outTokens, cachedInput, cacheCreation int64) (Costs, bool)
- func (t *Table) Has(provider, model string) bool
- func (t *Table) Lookup(provider, model string) (Entry, bool)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NewEntries ¶ added in v0.76.1
NewEntries validates and converts a wire-shape map (surface-or-record -> model -> rates) into the internal representation. Every rate must be a finite, non-negative USD amount; a violation is returned as an error so a corrupt config fails the chain build loudly instead of mispricing. Management validates the same constraints at its API boundary, so this is defense-in-depth. Nil input yields an empty (never-matching) map.
Types ¶
type Costs ¶ added in v0.75.1
type Costs struct {
InputUSD float64
CachedInputUSD float64
CacheCreationUSD float64
OutputUSD float64
TotalUSD float64
CacheUSD float64
}
Costs is a per-request cost split. The four per-bucket fields are the base of the breakdown — one per token bucket the provider bills separately — and the two aggregates are derived from them:
TotalUSD = InputUSD + CachedInputUSD + CacheCreationUSD + OutputUSD CacheUSD = CachedInputUSD + CacheCreationUSD
InputUSD is always the cost of the *non-cached* input bucket, for both provider shapes: on OpenAI the cached subset is carved out of inTokens and billed as CachedInputUSD, so the two never double-count. Buckets a provider doesn't bill are zero, which keeps the identities above true everywhere.
func EntryCosts ¶ added in v0.76.1
func EntryCosts(entry Entry, surface string, inTokens, outTokens, cachedInput, cacheCreation int64) Costs
EntryCosts computes the USD cost split for the given entry and token counts. The surface (the llm.provider value the request parser stamps) selects the cache formula; the entry may come from the surface-keyed defaults table or from a per-provider-record override — the math is identical either way.
Provider-shape semantics for cached / cache-creation counts:
- "openai": cachedInput is a SUBSET of inTokens. The cached portion is billed at CachedInputPer1K (or InputPer1K when no override), and the non-cached remainder of inTokens at InputPer1K. cacheCreation is ignored (OpenAI has no analogue).
- "anthropic", "bedrock": cachedInput (cache_read) and cacheCreation are ADDITIVE to inTokens. The three buckets are billed at CacheReadPer1K, CacheCreationPer1K, and InputPer1K respectively, each falling back to InputPer1K when the corresponding rate is zero.
- Other surfaces: cached and cacheCreation are ignored; cost is inTokens*InputPer1K + outTokens*OutputPer1K.
type Entry ¶
type Entry struct {
InputPer1K float64
OutputPer1K float64
CachedInputPer1K float64
CacheReadPer1K float64
CacheCreationPer1K float64
}
Entry is a single model's input and output pricing, expressed in USD per 1000 tokens.
CachedInputPer1K applies to OpenAI's cached prompt tokens, which are a subset of input_tokens — when set, the cached portion is billed at this rate and the non-cached remainder at InputPer1K. Zero means "no discount configured", and cached tokens are billed at InputPer1K.
CacheReadPer1K and CacheCreationPer1K apply to Anthropic's two prompt- cache fields, which are additive to input_tokens: cache_read is the cheaper read-from-cache rate, cache_creation is the more expensive write-to-cache rate. Zero means "no rate configured" and the corresponding token bucket is billed at InputPer1K.
type EntryJSON ¶ added in v0.76.1
type EntryJSON struct {
InputPer1K float64 `json:"input_per_1k"`
OutputPer1K float64 `json:"output_per_1k"`
CachedInputPer1K float64 `json:"cached_input_per_1k"`
CacheReadPer1K float64 `json:"cache_read_per_1k"`
CacheCreationPer1K float64 `json:"cache_creation_per_1k"`
}
EntryJSON is the wire shape of a pricing entry inside cost_meter's middleware config. Field names are the management→proxy contract; the management synthesizer marshals the same names (its pricing.Entry).
type Table ¶
type Table struct {
// contains filtered or unexported fields
}
Table is a provider-surface-to-model pricing lookup. Instances are immutable once built; a mapping update builds a whole new middleware instance (and with it a new table) rather than mutating this one.
func NewTable ¶ added in v0.76.1
NewTable builds an immutable Table from the wire-shape defaults map. See NewEntries for validation semantics.
func (*Table) Cost ¶
func (t *Table) Cost(provider, model string, inTokens, outTokens, cachedInput, cacheCreation int64) (float64, bool)
Cost returns the estimated USD cost for the given token counts. ok is false when the provider or model is not present in the table; the caller can still emit token metrics with a model=unknown label.
func (*Table) Costs ¶ added in v0.75.1
func (t *Table) Costs(provider, model string, inTokens, outTokens, cachedInput, cacheCreation int64) (Costs, bool)
Costs returns the estimated USD cost split for the given token counts. The provider surface selects the cache formula; see EntryCosts.
Source Files