pricing

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 10, 2026 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Overview

Package pricing implements the price table and cost math for the FinOps circuit-breaker proxy.

Money is always integer micro-USD (µUSD): 1 USD = 1_000_000 µUSD. The floating-point dollar amounts in config/pricing.json are normalized to integers exactly once at Load time; every cost computation afterwards is integer-only.

Network price-table fetcher: pulls the OpenRouter model catalog, parses each entry's prompt/completion pricing strings into ModelPrice, and returns the table plus skip diagnostics. This layer is pure I/O — it owns no state and no background loop; the Registry decides when to fetch and what to do with the result (including treating an empty table as an error, D7).

Thread-safe price registry: the single owner of the active price table, the background refresh loop, and the fallback state machine. It starts from the static config/pricing.json table (the guaranteed seed) and, when a fetcher is configured, overlays fresh catalog prices in the background.

Reader invariants (D4/D7): readers never see an empty/partial/torn table and never block on the network. cur is an immutable *Pricing swapped whole under mu; a refresh failure keeps the last-good table and never reverts to the static file once a network table has been adopted.

Index

Constants

View Source
const MicroUSDPerUSD = 1_000_000

MicroUSDPerUSD is the fixed-point scale: 1 USD = 1_000_000 µUSD.

Variables

View Source
var ErrEmptyTable = errors.New("pricing: fetch succeeded but produced no priced models")

ErrEmptyTable marks a fetch that succeeded syntactically but produced no priced models. The registry treats it as a refresh failure (D7) rather than swapping in an empty table.

Functions

func Cost

func Cost(mp ModelPrice, promptTok, completionTok int) int64

Cost returns the cost in µUSD of a completed call: (promptTok*InputUSD + completionTok*OutputUSD) / 1_000_000.

func EstimatePromptTokens

func EstimatePromptTokens(body []byte) int

EstimatePromptTokens estimates token count using the default chars-per-token heuristic (4.0). Prefer (*Pricing).EstimatePromptTokens to honor the loaded prompt_estimation_chars_per_token knob.

func ExtractModel

func ExtractModel(body []byte) string

ExtractModel returns the "model" field of a chat-completions request body, "" if absent or unparseable.

Types

type Config

type Config struct {
	PricingPath     string        // static baseline file (config/pricing.json)
	Endpoint        string        // "" => static-only mode (no fetcher)
	APIKey          string        // optional Bearer token
	UserAgent       string        // default "finops-proxy/1.0 (openrouter-fetcher)"
	RefreshInterval time.Duration // default 6h; <=0 disables background refresh
	StartupTimeout  time.Duration // default 3s (bootstrap budget)
	HTTPTimeout     time.Duration // default 30s
	BackoffBase     time.Duration // default 10s
	BackoffMax      time.Duration // default 5m
	StaleAfter      time.Duration // default 24h
	HardFailOnEmpty bool          // default true
	MaxBodyBytes    int64         // default 8 MiB
}

Config is the single, consolidated configuration for the pricing layer. It is stdlib-only and carries no dependencies. Zero fields get their documented defaults via DefaultConfig (or a defensive normalize inside the constructors); the two fields with meaningful zero values are Endpoint ("" ⇒ static-only) and RefreshInterval (<=0 ⇒ no background refresh).

func DefaultConfig

func DefaultConfig() Config

DefaultConfig returns the §10 defaults. Endpoint defaults to the OpenRouter catalog; an operator who wants static-only mode overrides it to "".

type FetchResult

type FetchResult struct {
	Models  map[string]ModelPrice
	Default ModelPrice // informative seed default (the registry keeps base's)
	Skipped []SkippedModel
	Updated time.Time
}

FetchResult is the outcome of one catalog fetch: the priced models, an informative (seed) default, the models skipped for being unpriceable, and when the table was pulled.

type Fetcher

type Fetcher struct {
	// contains filtered or unexported fields
}

Fetcher is a stateless HTTP client for the model catalog. It owns no table and no loop; callers drive it.

func NewFetcher

func NewFetcher(cfg Config) *Fetcher

NewFetcher builds a Fetcher for cfg. It returns nil in static-only mode (cfg.Endpoint == ""), in which case there is nothing to fetch from.

func (*Fetcher) Fetch

func (f *Fetcher) Fetch(ctx context.Context) (FetchResult, error)

Fetch is FetchModels plus skip diagnostics and an Updated timestamp.

func (*Fetcher) FetchModels

func (f *Fetcher) FetchModels(ctx context.Context, endpoint string) (map[string]ModelPrice, error)

FetchModels fetches the catalog and returns the priced models, dropping unpriceable entries silently. endpoint == "" falls back to f.cfg.Endpoint. An empty map is not an error here — the registry decides (D7).

type ModelPrice

type ModelPrice struct {
	InputUSD  int64 // µUSD per 1M input (prompt) tokens
	OutputUSD int64 // µUSD per 1M output (completion) tokens
}

ModelPrice is one model's price in µUSD per 1,000,000 tokens.

type PriceSource

type PriceSource interface {
	// PriceFor resolves one model's price (exact key, else longest configured
	// prefix, else the default). ok=false means "fell back to default".
	PriceFor(model string) (ModelPrice, bool)
	// EstimatePromptTokens estimates the prompt token count of a request body
	// using the table's chars-per-token knob.
	EstimatePromptTokens(body []byte) int
	// SavedCost estimates the µUSD spend avoided by blocking this request.
	SavedCost(body []byte) int64
	// AssumedCompletionTokens is the completion token count credited to a
	// blocked request's saved-cost estimate.
	AssumedCompletionTokens() int
}

PriceSource is the single contract the proxy's cost layer consumes. Both the static *Pricing and the dynamic *Registry satisfy it, so the handler never knows — or needs to know — where the table came from.

Implementations MUST be safe for concurrent reads and MUST NOT return an empty or partial table: every model resolves (exact key → longest prefix → default), and no read ever blocks on the network.

type Pricing

type Pricing struct {

	// PromptEstimationCharsPerToken is the chars-per-token heuristic used to
	// estimate prompt tokens. Default 4.0.
	PromptEstimationCharsPerToken float64
	// contains filtered or unexported fields
}

Pricing is an immutable price table. Safe for concurrent reads (no mutex — the model map is never mutated after Load returns).

func Build

func Build(models map[string]ModelPrice, def ModelPrice, assumedCompletionTokens int, promptEstimationCharsPerToken float64) *Pricing

Build constructs an immutable serving *Pricing from a persisted price book: the exact-key model map plus the shared default price and estimation knobs that a remote catalog or database overlay never supplies. It is the rebuild seam the enterprise module and the reference stores use to turn a persisted price book (SQLite model_prices rows, or any in-memory map) into a live serving table.

models is used as-is (not copied): the caller must not mutate it after the call, mirroring the immutability contract of *Pricing. promptEstimationCharsPerToken is clamped to the default (4.0) when non-positive.

func Load

func Load(path string) (*Pricing, error)

Load reads, validates and normalizes config/pricing.json. It errors on any invalid entry — a proxy that cannot price its traffic must not start silently with a wrong table.

func (*Pricing) AssumedCompletionTokens

func (p *Pricing) AssumedCompletionTokens() int

AssumedCompletionTokens returns the completion token count credited to a blocked request's saved-cost estimate. It exists so *Pricing satisfies the PriceSource contract; the registry delegates to the same knob on its current table. The backing value is stored unexported because a Go type cannot have a field and a method with the same name.

func (*Pricing) Default

func (p *Pricing) Default() ModelPrice

Default returns the fallback price used when a model resolves to neither an exact key nor a configured prefix. It pairs with Build so a caller can carry a table's default price and estimation knobs across a rebuild. A nil receiver yields the zero price.

func (*Pricing) EstimatePromptTokens

func (p *Pricing) EstimatePromptTokens(body []byte) int

EstimatePromptTokens estimates token count using p's chars-per-token knob (prompt_estimation_chars_per_token; default 4.0). For JSON bodies it slightly overestimates, which is the safe direction for a saved-cost estimate.

func (*Pricing) Models

func (p *Pricing) Models() map[string]ModelPrice

Models returns a shallow copy of the exact-key price map. It exists for boot seeding: main seeds the SQLite price book from the static config table (with INSERT OR IGNORE) so a DB-first Rebuild preserves the guaranteed seed. A nil receiver yields nil.

func (*Pricing) PriceFor

func (p *Pricing) PriceFor(model string) (ModelPrice, bool)

PriceFor returns the price for model: exact key, else longest configured key that is a prefix of model, else the default. ok=false means "fell back to default".

func (*Pricing) SavedCost

func (p *Pricing) SavedCost(body []byte) int64

SavedCost estimates (µUSD) the spend avoided by blocking this request: Cost(PriceFor(ExtractModel(body)), EstimatePromptTokens(body), AssumedCompletionTokens). It is an estimate, never an exact charge — the request never reached upstream, so there is no real usage.

type RefreshError

type RefreshError struct {
	Endpoint string
	Status   int // 0 if no HTTP response
	Snippet  string
	Err      error
}

RefreshError describes a failed table refresh: which endpoint, the HTTP status (0 when no HTTP response was received — network/DNS/timeout), a short response snippet, and the underlying error. Status lets a caller tell a permanent 401/403 from a transient 5xx or network failure.

func (*RefreshError) Error

func (e *RefreshError) Error() string

func (*RefreshError) Unwrap

func (e *RefreshError) Unwrap() error

Unwrap exposes the underlying error for errors.Is/As.

type Registry

type Registry struct {
	// contains filtered or unexported fields
}

Registry is a thread-safe PriceSource. See the package comment for the reader guarantees and the fallback ordering (live > stale-last-good > static).

func NewRegistry

func NewRegistry(base *Pricing, fetcher *Fetcher, cfg Config) *Registry

NewRegistry builds a registry seeded with base (the loaded static table). base must be non-nil — it is the guaranteed seed served until the first successful network fetch. fetcher may be nil for static-only mode.

func (*Registry) AssumedCompletionTokens

func (r *Registry) AssumedCompletionTokens() int

func (*Registry) Bootstrap

func (r *Registry) Bootstrap(ctx context.Context)

Bootstrap performs one synchronous, bounded startup fetch so the proxy can serve a live table immediately on boot. On failure it logs a WARN and keeps the static table — a network outage must not prevent startup (8.2). It is a no-op in static-only mode.

func (*Registry) EstimatePromptTokens

func (r *Registry) EstimatePromptTokens(body []byte) int

func (*Registry) LastRefresh

func (r *Registry) LastRefresh() (time.Time, error)

LastRefresh reports the time of the last successful refresh and the last refresh error (nil on success, or when a fetch was never attempted).

func (*Registry) PriceFor

func (r *Registry) PriceFor(model string) (ModelPrice, bool)

func (*Registry) RefreshNow

func (r *Registry) RefreshNow(ctx context.Context) error

RefreshNow performs one fetch+merge synchronously and returns the outcome. It is safe to call concurrently with the background loop — refreshMu serializes.

func (*Registry) Replace

func (r *Registry) Replace(tbl *Pricing)

Replace swaps in tbl as the current serving table, preserving the current kind, last-sync time and failure counter. It is the manual-PUT path: the caller has already persisted the row and rebuilt a fresh *Pricing. A nil tbl is a no-op (defensive).

func (*Registry) ReplaceLive

func (r *Registry) ReplaceLive(tbl *Pricing, lastSync time.Time)

ReplaceLive swaps in tbl and marks the table live, resetting the failure counter, last error and last-sync time. It is the manual-sync path (POST /api/v1/pricing/sync). A nil tbl is a no-op (defensive).

func (*Registry) SavedCost

func (r *Registry) SavedCost(body []byte) int64

func (*Registry) SetOnSync

func (r *Registry) SetOnSync(fn func(ctx context.Context, models map[string]ModelPrice) (*Pricing, error))

SetOnSync installs the persistence hook for the background refresh loop. When set, refresh calls fn with the freshly fetched models instead of merge, and swaps in the *Pricing fn returns. The hook is the Service.OnSync seam; it is installed once at startup before Bootstrap/Start. Safe for concurrent use.

func (*Registry) Snapshot

func (r *Registry) Snapshot() *Pricing

Snapshot returns the current immutable table. The registry never mutates the returned *Pricing, so a caller may hold it indefinitely.

func (*Registry) SourceKind

func (r *Registry) SourceKind() SourceKind

SourceKind reports the current table source (static/live/stale).

func (*Registry) Start

func (r *Registry) Start(ctx context.Context)

Start launches the background refresh loop and returns immediately (7.3). The loop does an immediate first fetch — unless Bootstrap already populated the table (lastSync non-zero), which avoids a redundant double-fetch at startup — then re-fetches on a timer with exponential backoff on failure. Start is a no-op in static-only mode or when background refresh is disabled (RefreshInterval <= 0).

func (*Registry) Stop

func (r *Registry) Stop()

Stop stops the background loop and waits for any in-flight refresh to finish (bounded by HTTPTimeout). It is idempotent and safe to call multiple times. Call it after Start; the wg counter is set by Start, so Stop without Start is a no-op that just closes stopCh.

type SkippedModel

type SkippedModel struct {
	ID, Reason string
}

SkippedModel is one catalog entry dropped for a non-fatal reason (empty id or unpriceable pricing). It is diagnostic only — a bad entry never fails a fetch.

type SourceKind

type SourceKind int

SourceKind identifies the current state of the registry's table.

const (
	// SourceStatic: the static config/pricing.json table; no successful network
	// fetch has ever replaced it (never a successful fetch).
	SourceStatic SourceKind = iota
	// SourceLive: a network table, last refresh OK.
	SourceLive
	// SourceStale: a network table was adopted, but the last refresh FAILED.
	SourceStale
)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL