Documentation
¶
Overview ¶
Package pricing implements list-price estimation for qualification runs. It is llm-free: token counting arrives behind Counter, supplied by the CLI module. Costs are estimates, never invoices (design: "Preflight token and cost estimate").
Three things live here: Snapshot, a frozen models.dev price table with provenance (snapshot.go); Cost, the pure calculation that prices one Usage against one Rates row, honest about anything it cannot know (cost.go); and Preflight, which turns a set of runnable qual.TablePlan values into the call-count and token-cost plan a caller prints before spending a paid call (preflight.go).
Index ¶
Constants ¶
const MaxSnapshotBytes = 8 << 20 // 8 MiB
MaxSnapshotBytes bounds a models.dev price snapshot, whether it arrives over the network (FetchSnapshot) or is handed to ParseSnapshot directly (e.g. a caller-cached copy on disk). It rejects an absurd or hostile response before it is decoded, in either path.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Amount ¶
Amount is a cost subtotal that is honest about unknowns. Known is false when some usage dimension had no matching rate, the reasoning-subset invariant was violated, or the usage itself was incomplete; Reason then explains why, and USD is not a meaningful value.
func Cost ¶
Cost prices u against r: (Output-Reasoning)×output_rate + Reasoning× reasoning_rate when r.Reasoning is set, else Output×output_rate once. It never double-counts reasoning tokens as both output and reasoning.
Cost returns Known=false, never a fabricated number, when: u.Complete is false (the provider reported no usage at all); u.Reasoning exceeds u.Output (the reasoning-subset invariant is violated — this package has no error return here, so a violation surfaces as an unknown Amount, not a silent correction); any usage dimension carries a negative count; or any dimension with nonzero usage has a nil rate. A dimension with zero usage never blocks the calculation, whether or not it is priced — nothing was consumed, so there is nothing to be unsure about. A dimension explicitly priced at zero (a non-nil rate pointing at 0.0) is Known=true and contributes zero cost; that is a real (free) price, not a missing one.
type Counter ¶
type Counter interface {
Count(ctx context.Context, req inference.Request) (tokens int, quality string, err error)
}
Counter abstracts the llm context counter (design: llm-free root module). A real implementation lives in the CLI module, which alone may import an llm tokenizer; pricing only depends on this narrow interface. quality is a free-form, implementation-defined label (e.g. the tokenizer's name/version) surfaced to the caller via Plan.CounterQuality so an estimate's provenance is never silently lost.
type Plan ¶
type Plan struct {
TargetCalls, JudgeCalls int
InputTokens [2]int // [expected, max]
OutputTokens [2]int // [expected, max]
Expected, Max Amount
CounterQuality string
Unknowns []string
}
Plan is the preflight cost plan printed before paid inference. TargetCalls and JudgeCalls are exact (derived from scenario/trial/evaluator counts, never estimated); InputTokens and OutputTokens are [expected, max] estimates in whole tokens; Expected and Max price those totals against the caller's Rates; CounterQuality names the token-counting method actually used ("heuristic" when counter was nil); Unknowns lists, in encounter order, every table or judge evaluator whose token contribution could not be estimated (missing template, no declared output cap) — never silently dropped.
func Preflight ¶
func Preflight(ctx context.Context, plans []qual.TablePlan, cfg eval.RunConfig, rates Rates, counter Counter, templates map[eval.Name]inference.Request) (Plan, error)
Preflight builds the plan for a set of runnable table plans: target calls are scenarios × trials per table, summed across plans; judge calls are scenarios × trials × (evaluators on that table whose Descriptor().Method is not eval.MethodProgrammatic), summed across plans. Method is the signal used to identify a "judge" evaluator because it is the one thing eval.Evaluator exposes for this purpose (Method's own doc calls it out for "cost accounting"): every model judge built by eval/judge.New declares MethodModel, and every programmatic evaluator in eval/exact declares MethodProgrammatic; there is no exported marker or type assertion that distinguishes "judge" more directly without depending on the judge package's unexported concrete type. Checking "!= MethodProgrammatic" rather than "== MethodModel" is deliberate: a future eval.MethodComposite evaluator (none exists yet) would still involve a model judge call and must be counted, not silently ignored — see judgeDescriptors.
A non-runnable plan (Runnable=false) contributes nothing: qual.Plan leaves its Suite and Evaluators at their zero value, so it has no scenarios to count and is skipped explicitly here for clarity.
templates supplies the inference.Request used to estimate token cost for one call, keyed by the identity of whatever makes that call: a table's own Table name for its target calls, and a judge evaluator's Descriptor().Name for its judge calls (the CLI, which constructs both the live target and each judge in Task 12, is positioned to supply both keys from the same templates it already built). A missing key means that portion of the plan cannot be estimated; Preflight records it in Unknowns and excludes it from the token totals rather than guessing.
counter nil ⇒ every per-call input-token estimate falls back to a heuristic (content bytes / 4) and Plan.CounterQuality is "heuristic". A non-nil counter's own reported quality is used instead, and any error it returns aborts Preflight: a broken counter is not a number to route around silently before a paid run.
The per-call output-token estimate (used for both the expected and max arms — no generation has happened yet to observe a real distribution) is read from the request's own declared cap: Override.MaxTokens, else Model.Sampling.MaxTokens, else Model.Limits.MaxOutputTokens. When none of those is set, the call's output contribution is recorded as an Unknown rather than invented. The per-call max *input* estimate additionally widens to Model.Limits.MaxInputTokens when that is a known, larger ceiling than the counted/heuristic estimate.
type Rates ¶
type Rates struct {
Input, Output, Reasoning, CacheRead, CacheWrite *float64
}
Rates are USD per million tokens. A nil field means that dimension is not priced in the catalog (unknown), never zero; a dimension explicitly priced at zero is a non-nil pointer to 0.0. See Cost for how this distinction is used.
type Snapshot ¶
type Snapshot struct {
SourceURL string
FetchedAt time.Time
Digest string // sha256 hex of the raw snapshot bytes
Rows map[string]Rates
}
Snapshot is a frozen models.dev price table with provenance: where it came from, when it was fetched, and a digest of the exact bytes it was parsed from, so a caller can prove which price table priced a given run.
func FetchSnapshot ¶
FetchSnapshot fetches a models.dev price document over HTTP and parses it with ParseSnapshot. It builds the request with ctx (every I/O call here carries the caller's deadline — there is no unbounded blocking) and bounds the response body to MaxSnapshotBytes with io.LimitReader, reading one byte past the bound so an oversized body is detected and rejected rather than silently truncated. client may be nil, in which case a default client is used.
url is validated before use: it must be a syntactically safe https URL (or http restricted to a loopback host, for local testing), with a host present and no embedded userinfo. This is the same rule the inference module applies to a Model's BaseURL, applied here because url is caller-supplied and this function makes it the target of a real network request.
That validation only covers the URL actually passed in: without more, the real HTTP request would still follow Go's default redirect behavior (up to 10 redirects, none of them re-validated), so a validated https://models.dev/api.json that later 302s to an internal or unsafe host would sail through unchecked. To close that gap, FetchSnapshot disables redirect-following — mirroring the same CheckRedirect: http.ErrUseLastResponse policy the inference module's own transport client uses (see inference/transport/client.go's newHTTPClient) — on whichever client ends up making the request: when client is nil, the default client it builds never follows redirects; when the caller supplies their own client, that same policy is applied to a copy of it (the caller's original client is never mutated) unless the caller has already set their own CheckRedirect, in which case that explicit choice is respected as-is. Either way, a 3xx response is surfaced as its own (non-200) status and rejected below, rather than silently chased.
func ParseSnapshot ¶
ParseSnapshot decodes raw as a models.dev price document and returns the resulting Snapshot, stamped with sourceURL and fetchedAt for provenance and a sha256 digest of raw itself (not a re-encoding — the digest proves which exact bytes were parsed). raw is bounded to MaxSnapshotBytes before decoding: an oversized document is rejected outright, whether it reached here from the network or a local cache.
Decoding tolerates unknown fields (the live document carries many fields this package does not need, such as ids, names, and context limits) and extracts only the cost object of each model. A model with no cost object at all is omitted from Rows entirely: nothing is known about it, so no row is worth reporting. A model with a cost object is always included, even if every individual dimension is absent (nil) — that is itself informative (the catalog knows about the model but prices none of its dimensions).
type Usage ¶
Usage is one call's normalized token usage. Reasoning is a subset of Output (a design invariant: reasoning tokens are billed as output tokens that happen to be reasoning, never as an addition to it). Complete is false when the provider did not report usage for the call at all; Cost treats that as wholly unknown rather than guessing zero.
Source Files