Documentation
¶
Overview ¶
Package pricing implements the price table and cost math for the FinOps circuit-breaker proxy.
Money is always integer micro-USD (µUSD): 1 USD = 1_000_000 µUSD. The floating-point dollar amounts in config/pricing.json are normalized to integers exactly once at Load time; every cost computation afterwards is integer-only.
Network price-table fetcher: pulls the OpenRouter model catalog, parses each entry's prompt/completion pricing strings into ModelPrice, and returns the table plus skip diagnostics. This layer is pure I/O — it owns no state and no background loop; the Registry decides when to fetch and what to do with the result (including treating an empty table as an error, D7).
Thread-safe price registry: the single owner of the active price table, the background refresh loop, and the fallback state machine. It starts from the static config/pricing.json table (the guaranteed seed) and, when a fetcher is configured, overlays fresh catalog prices in the background.
Reader invariants (D4/D7): readers never see an empty/partial/torn table and never block on the network. cur is an immutable *Pricing swapped whole under mu; a refresh failure keeps the last-good table and never reverts to the static file once a network table has been adopted.
Index ¶
- Constants
- Variables
- func Cost(mp ModelPrice, promptTok, completionTok int) int64
- func EstimatePromptTokens(body []byte) int
- func ExtractModel(body []byte) string
- type Config
- type FetchResult
- type Fetcher
- type ModelPrice
- type PriceSource
- type Pricing
- type RefreshError
- type Registry
- func (r *Registry) AssumedCompletionTokens() int
- func (r *Registry) Bootstrap(ctx context.Context)
- func (r *Registry) EstimatePromptTokens(body []byte) int
- func (r *Registry) LastRefresh() (time.Time, error)
- func (r *Registry) PriceFor(model string) (ModelPrice, bool)
- func (r *Registry) RefreshNow(ctx context.Context) error
- func (r *Registry) Replace(tbl *Pricing)
- func (r *Registry) ReplaceLive(tbl *Pricing, lastSync time.Time)
- func (r *Registry) SavedCost(body []byte) int64
- func (r *Registry) SetOnSync(fn func(ctx context.Context, models map[string]ModelPrice) (*Pricing, error))
- func (r *Registry) Snapshot() *Pricing
- func (r *Registry) SourceKind() SourceKind
- func (r *Registry) Start(ctx context.Context)
- func (r *Registry) Stop()
- type SkippedModel
- type SourceKind
Constants ¶
const MicroUSDPerUSD = 1_000_000
MicroUSDPerUSD is the fixed-point scale: 1 USD = 1_000_000 µUSD.
Variables ¶
var ErrEmptyTable = errors.New("pricing: fetch succeeded but produced no priced models")
ErrEmptyTable marks a fetch that succeeded syntactically but produced no priced models. The registry treats it as a refresh failure (D7) rather than swapping in an empty table.
Functions ¶
func Cost ¶
func Cost(mp ModelPrice, promptTok, completionTok int) int64
Cost returns the cost in µUSD of a completed call: (promptTok*InputUSD + completionTok*OutputUSD) / 1_000_000.
func EstimatePromptTokens ¶
EstimatePromptTokens estimates token count using the default chars-per-token heuristic (4.0). Prefer (*Pricing).EstimatePromptTokens to honor the loaded prompt_estimation_chars_per_token knob.
func ExtractModel ¶
ExtractModel returns the "model" field of a chat-completions request body, "" if absent or unparseable.
Types ¶
type Config ¶
type Config struct {
PricingPath string // static baseline file (config/pricing.json)
Endpoint string // "" => static-only mode (no fetcher)
APIKey string // optional Bearer token
UserAgent string // default "finops-proxy/1.0 (openrouter-fetcher)"
RefreshInterval time.Duration // default 6h; <=0 disables background refresh
StartupTimeout time.Duration // default 3s (bootstrap budget)
HTTPTimeout time.Duration // default 30s
BackoffBase time.Duration // default 10s
BackoffMax time.Duration // default 5m
StaleAfter time.Duration // default 24h
HardFailOnEmpty bool // default true
MaxBodyBytes int64 // default 8 MiB
}
Config is the single, consolidated configuration for the pricing layer. It is stdlib-only and carries no dependencies. Zero fields get their documented defaults via DefaultConfig (or a defensive normalize inside the constructors); the two fields with meaningful zero values are Endpoint ("" ⇒ static-only) and RefreshInterval (<=0 ⇒ no background refresh).
func DefaultConfig ¶
func DefaultConfig() Config
DefaultConfig returns the §10 defaults. Endpoint defaults to the OpenRouter catalog; an operator who wants static-only mode overrides it to "".
type FetchResult ¶
type FetchResult struct {
Models map[string]ModelPrice
Default ModelPrice // informative seed default (the registry keeps base's)
Skipped []SkippedModel
Updated time.Time
}
FetchResult is the outcome of one catalog fetch: the priced models, an informative (seed) default, the models skipped for being unpriceable, and when the table was pulled.
type Fetcher ¶
type Fetcher struct {
// contains filtered or unexported fields
}
Fetcher is a stateless HTTP client for the model catalog. It owns no table and no loop; callers drive it.
func NewFetcher ¶
NewFetcher builds a Fetcher for cfg. It returns nil in static-only mode (cfg.Endpoint == ""), in which case there is nothing to fetch from.
func (*Fetcher) Fetch ¶
func (f *Fetcher) Fetch(ctx context.Context) (FetchResult, error)
Fetch is FetchModels plus skip diagnostics and an Updated timestamp.
func (*Fetcher) FetchModels ¶
FetchModels fetches the catalog and returns the priced models, dropping unpriceable entries silently. endpoint == "" falls back to f.cfg.Endpoint. An empty map is not an error here — the registry decides (D7).
type ModelPrice ¶
type ModelPrice struct {
InputUSD int64 // µUSD per 1M input (prompt) tokens
OutputUSD int64 // µUSD per 1M output (completion) tokens
}
ModelPrice is one model's price in µUSD per 1,000,000 tokens.
type PriceSource ¶
type PriceSource interface {
// PriceFor resolves one model's price (exact key, else longest configured
// prefix, else the default). ok=false means "fell back to default".
PriceFor(model string) (ModelPrice, bool)
// EstimatePromptTokens estimates the prompt token count of a request body
// using the table's chars-per-token knob.
EstimatePromptTokens(body []byte) int
// SavedCost estimates the µUSD spend avoided by blocking this request.
SavedCost(body []byte) int64
// AssumedCompletionTokens is the completion token count credited to a
// blocked request's saved-cost estimate.
AssumedCompletionTokens() int
}
PriceSource is the single contract the proxy's cost layer consumes. Both the static *Pricing and the dynamic *Registry satisfy it, so the handler never knows — or needs to know — where the table came from.
Implementations MUST be safe for concurrent reads and MUST NOT return an empty or partial table: every model resolves (exact key → longest prefix → default), and no read ever blocks on the network.
type Pricing ¶
type Pricing struct {
// PromptEstimationCharsPerToken is the chars-per-token heuristic used to
// estimate prompt tokens. Default 4.0.
PromptEstimationCharsPerToken float64
// contains filtered or unexported fields
}
Pricing is an immutable price table. Safe for concurrent reads (no mutex — the model map is never mutated after Load returns).
func Build ¶
func Build(models map[string]ModelPrice, def ModelPrice, assumedCompletionTokens int, promptEstimationCharsPerToken float64) *Pricing
Build constructs an immutable serving *Pricing from a persisted price book: the exact-key model map plus the shared default price and estimation knobs that a remote catalog or database overlay never supplies. It is the rebuild seam the enterprise module and the reference stores use to turn a persisted price book (SQLite model_prices rows, or any in-memory map) into a live serving table.
models is used as-is (not copied): the caller must not mutate it after the call, mirroring the immutability contract of *Pricing. promptEstimationCharsPerToken is clamped to the default (4.0) when non-positive.
func Load ¶
Load reads, validates and normalizes config/pricing.json. It errors on any invalid entry — a proxy that cannot price its traffic must not start silently with a wrong table.
func (*Pricing) AssumedCompletionTokens ¶
AssumedCompletionTokens returns the completion token count credited to a blocked request's saved-cost estimate. It exists so *Pricing satisfies the PriceSource contract; the registry delegates to the same knob on its current table. The backing value is stored unexported because a Go type cannot have a field and a method with the same name.
func (*Pricing) Default ¶
func (p *Pricing) Default() ModelPrice
Default returns the fallback price used when a model resolves to neither an exact key nor a configured prefix. It pairs with Build so a caller can carry a table's default price and estimation knobs across a rebuild. A nil receiver yields the zero price.
func (*Pricing) EstimatePromptTokens ¶
EstimatePromptTokens estimates token count using p's chars-per-token knob (prompt_estimation_chars_per_token; default 4.0). For JSON bodies it slightly overestimates, which is the safe direction for a saved-cost estimate.
func (*Pricing) Models ¶
func (p *Pricing) Models() map[string]ModelPrice
Models returns a shallow copy of the exact-key price map. It exists for boot seeding: main seeds the SQLite price book from the static config table (with INSERT OR IGNORE) so a DB-first Rebuild preserves the guaranteed seed. A nil receiver yields nil.
func (*Pricing) PriceFor ¶
func (p *Pricing) PriceFor(model string) (ModelPrice, bool)
PriceFor returns the price for model: exact key, else longest configured key that is a prefix of model, else the default. ok=false means "fell back to default".
type RefreshError ¶
type RefreshError struct {
Endpoint string
Status int // 0 if no HTTP response
Snippet string
Err error
}
RefreshError describes a failed table refresh: which endpoint, the HTTP status (0 when no HTTP response was received — network/DNS/timeout), a short response snippet, and the underlying error. Status lets a caller tell a permanent 401/403 from a transient 5xx or network failure.
func (*RefreshError) Error ¶
func (e *RefreshError) Error() string
func (*RefreshError) Unwrap ¶
func (e *RefreshError) Unwrap() error
Unwrap exposes the underlying error for errors.Is/As.
type Registry ¶
type Registry struct {
// contains filtered or unexported fields
}
Registry is a thread-safe PriceSource. See the package comment for the reader guarantees and the fallback ordering (live > stale-last-good > static).
func NewRegistry ¶
NewRegistry builds a registry seeded with base (the loaded static table). base must be non-nil — it is the guaranteed seed served until the first successful network fetch. fetcher may be nil for static-only mode.
func (*Registry) AssumedCompletionTokens ¶
func (*Registry) Bootstrap ¶
Bootstrap performs one synchronous, bounded startup fetch so the proxy can serve a live table immediately on boot. On failure it logs a WARN and keeps the static table — a network outage must not prevent startup (8.2). It is a no-op in static-only mode.
func (*Registry) EstimatePromptTokens ¶
func (*Registry) LastRefresh ¶
LastRefresh reports the time of the last successful refresh and the last refresh error (nil on success, or when a fetch was never attempted).
func (*Registry) RefreshNow ¶
RefreshNow performs one fetch+merge synchronously and returns the outcome. It is safe to call concurrently with the background loop — refreshMu serializes.
func (*Registry) Replace ¶
Replace swaps in tbl as the current serving table, preserving the current kind, last-sync time and failure counter. It is the manual-PUT path: the caller has already persisted the row and rebuilt a fresh *Pricing. A nil tbl is a no-op (defensive).
func (*Registry) ReplaceLive ¶
ReplaceLive swaps in tbl and marks the table live, resetting the failure counter, last error and last-sync time. It is the manual-sync path (POST /api/v1/pricing/sync). A nil tbl is a no-op (defensive).
func (*Registry) SetOnSync ¶
func (r *Registry) SetOnSync(fn func(ctx context.Context, models map[string]ModelPrice) (*Pricing, error))
SetOnSync installs the persistence hook for the background refresh loop. When set, refresh calls fn with the freshly fetched models instead of merge, and swaps in the *Pricing fn returns. The hook is the Service.OnSync seam; it is installed once at startup before Bootstrap/Start. Safe for concurrent use.
func (*Registry) Snapshot ¶
Snapshot returns the current immutable table. The registry never mutates the returned *Pricing, so a caller may hold it indefinitely.
func (*Registry) SourceKind ¶
func (r *Registry) SourceKind() SourceKind
SourceKind reports the current table source (static/live/stale).
func (*Registry) Start ¶
Start launches the background refresh loop and returns immediately (7.3). The loop does an immediate first fetch — unless Bootstrap already populated the table (lastSync non-zero), which avoids a redundant double-fetch at startup — then re-fetches on a timer with exponential backoff on failure. Start is a no-op in static-only mode or when background refresh is disabled (RefreshInterval <= 0).
func (*Registry) Stop ¶
func (r *Registry) Stop()
Stop stops the background loop and waits for any in-flight refresh to finish (bounded by HTTPTimeout). It is idempotent and safe to call multiple times. Call it after Start; the wg counter is set by Start, so Stop without Start is a no-op that just closes stopCh.
type SkippedModel ¶
type SkippedModel struct {
ID, Reason string
}
SkippedModel is one catalog entry dropped for a non-fatal reason (empty id or unpriceable pricing). It is diagnostic only — a bad entry never fails a fetch.
type SourceKind ¶
type SourceKind int
SourceKind identifies the current state of the registry's table.
const ( // SourceStatic: the static config/pricing.json table; no successful network // fetch has ever replaced it (never a successful fetch). SourceStatic SourceKind = iota // SourceLive: a network table, last refresh OK. SourceLive // SourceStale: a network table was adopted, but the last refresh FAILED. SourceStale )