regen-builtin-pricing

Overview ¶

Generator for pkg/pricing/builtin.go.

Reads BerriAI/litellm's model_prices_and_context_window.json (from the URL by default, or a local file via --source), selects the Gemini and Anthropic models that can actually drive an agent loop, and emits a fresh builtin.go carrying their rates, their context windows, and a generation-time UpdatedAt on every entry.

Selection is a RULE, not a list — see eligible() below. There used to be a hand-curated `allowlist` here, and it had no discovery path: the generator warned about listed-but-missing models, so a model nobody had thought to add was invisible forever. That is how six models reachable from the /model picker (both Gemini 2.5 entries, the two 3.1 pro variants, gemini-3-flash-preview and gemini-3.1-flash-image-preview) ended up with no builtin rate at all — which silently disables the --max-*-cost-usd ceilings, since an unpriced model contributes $0 to every budget check.

Motivation: issue #259 showed that hand-authored builtin rates drift silently — the demo's gemini-3.5-flash entry was 20× too low on input, 30× too low on output. Regenerating from LiteLLM removes that class of drift; the UpdatedAt field lets operators see how old the current builtin snapshot is.

Context windows ride along in the same file for the same reason. pkg/usage's hand-maintained table said gemini-2.5-pro held 2,000,000 input tokens (a carry-over from Gemini 1.5 Pro) when the real cap is 1,048,576, so mid-tier compaction was scheduled to fire at ~1.3M — past a hard limit the session would have died on first. LiteLLM publishes max_input_tokens for every model we select, so the number is now generated rather than remembered.

Usage:

# Regenerate from LiteLLM's live master:
go run ./dev/regen-builtin-pricing

# From a pinned local snapshot (e.g. reviewing what would change
# without hitting the network):
go run ./dev/regen-builtin-pricing --source=/tmp/litellm.json

# Preview to stdout without writing:
go run ./dev/regen-builtin-pricing --stdout

# Ask "have any RATES moved?" without writing anything:
go run ./dev/regen-builtin-pricing --check

--check reports its answer on STDOUT as a single line, either `drift=true` or `drift=false`, and exits 0 either way. Human-readable detail (which models moved, and how) goes to stderr. A non-zero exit from --check always means the generator itself failed — bad JSON, unreachable network, unreadable --out.

Drift is deliberately NOT signalled through the exit code, because `go run` collapses every non-zero child status to 1: a program that exits 2 makes `go run` print "exit status 2" and then exit 1 itself. All four callers invoke this through `go run`, so an exit-code convention would make a failed fetch indistinguishable from a real price change — and the weekly workflow would open a pull request every time GitHub's egress hiccuped.

Ownership: regenerate before every release, and REVIEW THE DIFF. Because selection is a rule, a regen can add or remove models on its own — not just move rates. `added` / `removed` lines in the --check report are the ones to read carefully: an addition means LiteLLM started publishing a model that satisfies eligible(), and a removal means one was deprecated upstream or lost its tool-calling flag. Both are usually correct and occasionally are upstream mistakes.