regen-builtin-pricing

command
v0.13.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 15, 2026 License: Apache-2.0 Imports: 15 Imported by: 0

Documentation

Overview

Generator for pricing/builtin.go, the compiled-in table core-models' pricing package ships to mast and core-agent alike.

Reads BerriAI/litellm's model_prices_and_context_window.json (from the URL by default, or a local file via --source), selects the Gemini and Anthropic models that can actually drive an agent loop (plus the OpenAI, xAI and Vertex AI partner models reached through a profile), and emits a fresh builtin.go carrying their rates, their context windows, and a generation-time UpdatedAt on every entry.

Selection is a RULE, not a list — see eligible() below. There used to be a hand-curated `allowlist` here, and it had no discovery path: the generator warned about listed-but-missing models, so a model nobody had thought to add was invisible forever. That is how six models reachable from the /model picker (both Gemini 2.5 entries, the two 3.1 pro variants, gemini-3-flash-preview and gemini-3.1-flash-image-preview) ended up with no builtin rate at all — which silently disables the --max-*-cost-usd ceilings, since an unpriced model contributes $0 to every budget check.

Motivation: issue #259 showed that hand-authored builtin rates drift silently — the demo's gemini-3.5-flash entry was 20× too low on input, 30× too low on output. Regenerating from LiteLLM removes that class of drift; the UpdatedAt field lets operators see how old the current builtin snapshot is.

Backend-qualified rates ride along too, in a second map. A rate is a property of the (backend, model) pair, not of the model id: the same claude-opus-5 is reachable through api.anthropic.com and through Vertex, and the same gemini-3.7-flash through the Gemini Developer API and through Vertex. LiteLLM prices those separately, under prefixed keys ("vertex_ai/claude-opus-5", "gemini/gemini-3.7-flash"), and this generator used to throw every one of them away — leaving a lookup keyed on a bare id that cannot say which backend billed the tokens. It got the right answer only because the two tables happen to agree today, which nothing checked and nothing enforced.

See backends below for the four pairs core-models' adapters resolve. Emitting a row per pair, even when it duplicates the bare row, is the point: the key names the thing being priced, and a future divergence shows up as a one-row diff instead of as a wrong ceiling.

Context windows ride along in the same file for the same reason. The parent project's hand-maintained window table said gemini-2.5-pro held 2,000,000 input tokens (a carry-over from Gemini 1.5 Pro) when the real cap is 1,048,576, so mid-tier compaction was scheduled to fire at ~1.3M — past a hard limit the session would have died on first. LiteLLM publishes max_input_tokens for every model we select, so the number is generated rather than remembered; core-agent's compaction reads it, after a profile's declared window (profile.ContextWindowFor).

Long-context rates ride along on the rows that have them: LiteLLM's "<bucket>_above_<N>k_tokens" fields become Rates.LongContext, read by internal/litellm so pricing's Refresh reads them the same way.

Usage:

# Regenerate from LiteLLM's live master:
go run ./dev/regen-builtin-pricing

# From a pinned local snapshot (e.g. reviewing what would change
# without hitting the network):
go run ./dev/regen-builtin-pricing --source=/tmp/litellm.json

# Preview to stdout without writing:
go run ./dev/regen-builtin-pricing --stdout

# Ask "have any RATES moved?" without writing anything:
go run ./dev/regen-builtin-pricing --check

# Same, plus a markdown "what moved" fragment for a PR body:
go run ./dev/regen-builtin-pricing --check --report=/tmp/moved.md

--check reports its answer on STDOUT as a single line, either `drift=true` or `drift=false`, and exits 0 either way. Human-readable detail (which models moved, and how) goes to stderr. A non-zero exit from --check always means the generator itself failed — bad JSON, unreachable network, unreadable --out.

Drift is deliberately NOT signalled through the exit code, because `go run` collapses every non-zero child status to 1: a program that exits 2 makes `go run` print "exit status 2" and then exit 1 itself. All four callers invoke this through `go run`, so an exit-code convention would make a failed fetch indistinguishable from a real price change — and the weekly workflow would open a pull request every time GitHub's egress hiccuped.

Ownership: regenerate before every release, and REVIEW THE DIFF. Because selection is a rule, a regen can add or remove models on its own — not just move rates. `added` / `removed` lines in the --check report are the ones to read carefully: an addition means LiteLLM started publishing a model that satisfies eligible(), and a removal means one was deprecated upstream or lost its tool-calling flag. Both are usually correct and occasionally are upstream mistakes.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL