Documentation
¶
Overview ¶
Package pricing implements the embedded-default + override pricing table shared by middleware that converts LLM token usage into a USD cost estimate. The table is hot-reloadable from a basename under the proxy data directory; missing override files keep the embedded defaults so cost annotation works without operator action.
Index ¶
Constants ¶
const ( // ReloadInterval is the mtime-poll cadence for the background reloader. ReloadInterval = 30 * time.Second )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Entry ¶
type Entry struct {
InputPer1K float64
OutputPer1K float64
CachedInputPer1K float64
CacheReadPer1K float64
CacheCreationPer1K float64
}
Entry is a single model's input and output pricing, expressed in USD per 1000 tokens.
CachedInputPer1K applies to OpenAI's cached prompt tokens, which are a subset of input_tokens — when set, the cached portion is billed at this rate and the non-cached remainder at InputPer1K. Zero means "no discount configured", and cached tokens are billed at InputPer1K (matches current behaviour where cached counts weren't extracted at all).
CacheReadPer1K and CacheCreationPer1K apply to Anthropic's two prompt- cache fields, which are additive to input_tokens: cache_read is the cheaper read-from-cache rate, cache_creation is the more expensive write-to-cache rate. Zero means "no rate configured" and the corresponding token bucket is billed at InputPer1K. This is more accurate than today's behaviour, where Anthropic's cache tokens are ignored and not charged at all.
type Loader ¶
type Loader struct {
// contains filtered or unexported fields
}
Loader is a confined, hot-reloadable pricing table reader. Construction must succeed against the target file; subsequent reload failures keep the previously-loaded table so callers never observe a blank price list.
func NewLoader ¶
func NewLoader(baseDir, relPath, pluginID string, failures metric.Int64Counter) (*Loader, error)
NewLoader returns a pricing loader that overlays an optional file-based table on top of the embedded defaults. Missing override file, baseDir, or relPath is not an error: the loader keeps the embedded defaults so cost metadata is still emitted for known models.
Errors:
- bad basename, traversal segment, or absolute relPath are rejected so a misconfigured target surfaces immediately.
- permission errors and YAML parse errors keep the defaults but log a warning; cost annotation does not silently break.
failures is optional; pass nil in tests that do not care about reload-failure telemetry.
func (*Loader) Get ¶
Get returns the current pricing table. The returned pointer is immutable; callers must not mutate its contents.
func (*Loader) Reload ¶
Reload runs a polling loop that checks the pricing file mtime every ReloadInterval (or the value passed to SetReloadInterval). Returns when ctx is cancelled.
func (*Loader) SetReloadInterval ¶
SetReloadInterval overrides the mtime-poll cadence used by Reload. Calls after Reload has started have no effect on the running loop. Intended for tests; production code uses the default ReloadInterval.
func (*Loader) WatchesFile ¶
WatchesFile reports whether this loader is bound to an override file on disk. False for defaults-only loaders (no operator override given). Callers use this to decide whether to spawn the mtime-poll goroutine.
type Table ¶
type Table struct {
// contains filtered or unexported fields
}
Table is a provider-to-model pricing lookup. Instances are immutable once built and are swapped atomically by Loader.
func DefaultTable ¶
func DefaultTable() *Table
DefaultTable returns the pricing table embedded in the binary. The result is parsed once and shared; callers must not mutate the returned value. Cost annotation works without any operator action because every loader starts with this table.
func (*Table) Cost ¶
func (t *Table) Cost(provider, model string, inTokens, outTokens, cachedInput, cacheCreation int64) (float64, bool)
Cost returns the estimated USD cost for the given token counts. ok is false when the provider or model is not present in the table; the caller can still emit token metrics with a model=unknown label.
Provider-shape semantics for cached / cache-creation counts:
- OpenAI: cachedInput is a SUBSET of inTokens. The cached portion is billed at CachedInputPer1K (or InputPer1K when no override), and the non-cached remainder of inTokens at InputPer1K. cacheCreation is ignored (OpenAI has no analogue).
- Anthropic: cachedInput (cache_read) and cacheCreation are ADDITIVE to inTokens. The three buckets are billed at CacheReadPer1K, CacheCreationPer1K, and InputPer1K respectively, each falling back to InputPer1K when the corresponding rate is zero.
- Other providers: cached and cacheCreation are ignored; cost is inTokens*InputPer1K + outTokens*OutputPer1K.
Source Files