Documentation
¶
Overview ¶
Package litellm reads the parts of LiteLLM's model_prices_and_context_window.json that more than one caller in this module needs: the pricing package's Refresh and the dev/regen-builtin-pricing generator. One reader, so the two can't disagree about what a field means.
Index ¶
- Variables
- func BareID(model string) (string, bool)
- func IsDateSuffix(s string) bool
- func PerMTok(perToken float64) float64
- func Qualify(priced map[string]string) map[string]string
- func Round6(x float64) float64
- func UpstreamKeys(prefix, id string) []string
- type Backend
- type LongContext
- type ProfileBackend
Constants ¶
This section is empty.
Variables ¶
var Backends = []Backend{ {Name: "anthropic", Prefix: "", Family: "claude-", Convention: true}, {Name: "anthropic-vertex", Prefix: "vertex_ai/", Family: "claude-", Discover: "vertex_ai/"}, {Name: "gemini", Prefix: "gemini/", Family: "gemini-", Discover: "gemini/"}, {Name: "vertex", Prefix: "", Family: "gemini-", Discover: "vertex_ai/", Convention: true}, }
Backends are the four family backends. The generator (dev/regen-builtin-pricing) and the pricing package's Refresh both read them, so the built-in table and a refreshed one spell every (backend, model) pair the same way.
var ProfileBackends = []ProfileBackend{ {Name: "vertex-maas", Prefix: "vertex_ai/", Suffix: "-maas", Slash: true}, {Name: "openai", Provider: "openai"}, {Name: "xai", Prefix: "xai/", Provider: "xai"}, }
ProfileBackends are the profile backends, shared like Backends.
Functions ¶
func BareID ¶ added in v0.13.2
BareID turns a prefixed key's model part into the id the bare table would use: Vertex's "<base>@<YYYYMMDD>" becomes "<base>-<YYYYMMDD>" and "<base>@default" becomes "<base>" (the inverse of UpstreamKeys). Any other "@" suffix has no bare spelling, so it reports false.
func IsDateSuffix ¶ added in v0.13.2
IsDateSuffix reports whether s is a bare YYYYMMDD stamp.
func PerMTok ¶ added in v0.13.2
PerMTok converts LiteLLM's per-token price to the per-million-token rate core-models stores, through Round6.
func Qualify ¶ added in v0.13.2
Qualify returns the rows a price table keyed in core-models' backend vocabulary adds to a LiteLLM catalog: each result maps the key to emit to the LiteLLM key whose rates it carries.
priced holds every LiteLLM key the caller can price, with its litellm_provider. Qualify applies the generator's key rules (the generator additionally screens each model for agent-loop eligibility; Qualify does not):
- "<backend>/<id>" for every family model on every family backend that lists it, from that backend's own entry (Vertex's dated "@" spellings included). A model with no unprefixed Vertex entry takes its Vertex rate from vertex_ai/.
- The bare id, for a family model LiteLLM lists only under a prefix, from the family's convention backend (Vertex for gemini-*) — never from another backend, because a bare claude-* row is Anthropic first-party's rate by convention.
- "<backend>/<model>" for every profile backend's model.
Keys come back lowercase, as the catalog folds every lookup.
func Round6 ¶ added in v0.13.2
Round6 rounds x to 6 decimal places. Undoes binary-repr noise introduced when LiteLLM's per-token rates (like 0.0000001) are multiplied by 1M and end up as 0.09999999999999999 instead of 0.1. Six decimals = one-millionth-of-a-dollar per Mtok, way finer than any real rate we care about.
func UpstreamKeys ¶ added in v0.13.2
UpstreamKeys spells one model id the ways a backend's table might carry it, most specific first.
Vertex publishes a dated model as "<base>@<date>" where the unprefixed Anthropic table spells it "<base>-<date>", and tags the undated pointer "@default". Both forms have to be tried or every dated id would miss and silently fall back to the bare row.
Types ¶
type Backend ¶ added in v0.13.2
Backend is one (backend, family) pair a built-in profile resolves to, paired with how LiteLLM spells the same pair.
Name is core-models' name for the backend — the built-in profile's backend (profile.Profile.BackendName), which is what usage.Detail.Backend reports and both products key prices on — and it is what a price table emits as the "<backend>/<model>" key prefix. Using that vocabulary rather than LiteLLM's keeps upstream's spelling ("vertex_ai-anthropic_models") out of the key space, so a caller that has resolved a backend can build the key without a translation table.
Prefix is LiteLLM's key prefix for that backend; empty means the backend's rates live under the unprefixed id. Note the asymmetry that makes this worth writing down: LiteLLM's unprefixed claude-* keys are ANTHROPIC first-party, while its unprefixed gemini-* keys are VERTEX. A bare row is therefore a mixture of two backends, which is exactly the ambiguity the qualified rows remove.
Convention marks the backend whose rate a family's bare row carries: Anthropic for claude-*, Vertex for gemini-*. Discover is where LiteLLM lists a model of the family that has NO bare key at all, which happens: gemini-4-argon arrived as gemini/gemini-4-argon and vertex_ai/gemini-4-argon only. Empty means a model missing from the bare table cannot turn up on that backend (Anthropic's first-party rows ARE the bare table).
type LongContext ¶
type LongContext struct {
// ThresholdTokens is the prompt size above which these rates apply:
// LiteLLM's "_above_<N>k_tokens" suffix, with k = 1,000 (OpenAI's
// "272K" is 272,000 tokens).
ThresholdTokens int
Input float64
CacheRead float64
CacheCreation float64
CacheCreation1hr float64
Output float64
}
LongContext is one entry's above-threshold prices, per token, as LiteLLM publishes them. A zero rate means the field was absent.
func LongContextOf ¶
func LongContextOf(entry json.RawMessage) (LongContext, bool)
LongContextOf returns an entry's above-threshold prices, or false when it publishes none or publishes no above-threshold input rate (the one field the others are relative to). LiteLLM publishes at most one threshold per model; if an entry ever carried two, the lowest wins, because billing the higher tier from the lower threshold would overstate and the reverse would understate — the lower threshold is the one the provider applies first.
type ProfileBackend ¶ added in v0.13.2
ProfileBackend is a backend reached through a core-models built-in profile whose models have no bare row: an id like zai-org/glm-5.2-maas or grok-4.7 names a model on one backend, so the (backend, model) row is the only row, and the context-window table (keyed on bare ids) does not apply.
Name is the profile's backend name, Prefix is LiteLLM's key prefix (empty for OpenAI, whose rows are unprefixed), Provider narrows by litellm_provider where the prefix cannot (OpenAI's unprefixed keys sit beside every other vendor's), Suffix narrows to the profile's models, and Slash requires a publisher-qualified id.