litellm

package
v0.13.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 19, 2026 License: Apache-2.0 Imports: 4 Imported by: 0

Documentation

Overview

Package litellm reads the parts of LiteLLM's model_prices_and_context_window.json that more than one caller in this module needs: the pricing package's Refresh and the dev/regen-builtin-pricing generator. One reader, so the two can't disagree about what a field means.

Index

Constants

This section is empty.

Variables

View Source
var Backends = []Backend{
	{Name: "anthropic", Prefix: "", Family: "claude-", Convention: true},
	{Name: "anthropic-vertex", Prefix: "vertex_ai/", Family: "claude-", Discover: "vertex_ai/"},
	{Name: "gemini", Prefix: "gemini/", Family: "gemini-", Discover: "gemini/"},
	{Name: "vertex", Prefix: "", Family: "gemini-", Discover: "vertex_ai/", Convention: true},
}

Backends are the four family backends. The generator (dev/regen-builtin-pricing) and the pricing package's Refresh both read them, so the built-in table and a refreshed one spell every (backend, model) pair the same way.

View Source
var ProfileBackends = []ProfileBackend{

	{Name: "vertex-maas", Prefix: "vertex_ai/", Suffix: "-maas", Slash: true},

	{Name: "openai", Provider: "openai"},

	{Name: "xai", Prefix: "xai/", Provider: "xai"},
}

ProfileBackends are the profile backends, shared like Backends.

Functions

func BareID added in v0.13.2

func BareID(model string) (string, bool)

BareID turns a prefixed key's model part into the id the bare table would use: Vertex's "<base>@<YYYYMMDD>" becomes "<base>-<YYYYMMDD>" and "<base>@default" becomes "<base>" (the inverse of UpstreamKeys). Any other "@" suffix has no bare spelling, so it reports false.

func IsDateSuffix added in v0.13.2

func IsDateSuffix(s string) bool

IsDateSuffix reports whether s is a bare YYYYMMDD stamp.

func PerMTok added in v0.13.2

func PerMTok(perToken float64) float64

PerMTok converts LiteLLM's per-token price to the per-million-token rate core-models stores, through Round6.

func Qualify added in v0.13.2

func Qualify(priced map[string]string) map[string]string

Qualify returns the rows a price table keyed in core-models' backend vocabulary adds to a LiteLLM catalog: each result maps the key to emit to the LiteLLM key whose rates it carries.

priced holds every LiteLLM key the caller can price, with its litellm_provider. Qualify applies the generator's key rules (the generator additionally screens each model for agent-loop eligibility; Qualify does not):

  • "<backend>/<id>" for every family model on every family backend that lists it, from that backend's own entry (Vertex's dated "@" spellings included). A model with no unprefixed Vertex entry takes its Vertex rate from vertex_ai/.
  • The bare id, for a family model LiteLLM lists only under a prefix, from the family's convention backend (Vertex for gemini-*) — never from another backend, because a bare claude-* row is Anthropic first-party's rate by convention.
  • "<backend>/<model>" for every profile backend's model.

Keys come back lowercase, as the catalog folds every lookup.

func Round6 added in v0.13.2

func Round6(x float64) float64

Round6 rounds x to 6 decimal places. Undoes binary-repr noise introduced when LiteLLM's per-token rates (like 0.0000001) are multiplied by 1M and end up as 0.09999999999999999 instead of 0.1. Six decimals = one-millionth-of-a-dollar per Mtok, way finer than any real rate we care about.

func UpstreamKeys added in v0.13.2

func UpstreamKeys(prefix, id string) []string

UpstreamKeys spells one model id the ways a backend's table might carry it, most specific first.

Vertex publishes a dated model as "<base>@<date>" where the unprefixed Anthropic table spells it "<base>-<date>", and tags the undated pointer "@default". Both forms have to be tried or every dated id would miss and silently fall back to the bare row.

Types

type Backend added in v0.13.2

type Backend struct {
	Name, Prefix, Family, Discover string
	Convention                     bool
}

Backend is one (backend, family) pair a built-in profile resolves to, paired with how LiteLLM spells the same pair.

Name is core-models' name for the backend — the built-in profile's backend (profile.Profile.BackendName), which is what usage.Detail.Backend reports and both products key prices on — and it is what a price table emits as the "<backend>/<model>" key prefix. Using that vocabulary rather than LiteLLM's keeps upstream's spelling ("vertex_ai-anthropic_models") out of the key space, so a caller that has resolved a backend can build the key without a translation table.

Prefix is LiteLLM's key prefix for that backend; empty means the backend's rates live under the unprefixed id. Note the asymmetry that makes this worth writing down: LiteLLM's unprefixed claude-* keys are ANTHROPIC first-party, while its unprefixed gemini-* keys are VERTEX. A bare row is therefore a mixture of two backends, which is exactly the ambiguity the qualified rows remove.

Convention marks the backend whose rate a family's bare row carries: Anthropic for claude-*, Vertex for gemini-*. Discover is where LiteLLM lists a model of the family that has NO bare key at all, which happens: gemini-4-argon arrived as gemini/gemini-4-argon and vertex_ai/gemini-4-argon only. Empty means a model missing from the bare table cannot turn up on that backend (Anthropic's first-party rows ARE the bare table).

type LongContext

type LongContext struct {
	// ThresholdTokens is the prompt size above which these rates apply:
	// LiteLLM's "_above_<N>k_tokens" suffix, with k = 1,000 (OpenAI's
	// "272K" is 272,000 tokens).
	ThresholdTokens  int
	Input            float64
	CacheRead        float64
	CacheCreation    float64
	CacheCreation1hr float64
	Output           float64
}

LongContext is one entry's above-threshold prices, per token, as LiteLLM publishes them. A zero rate means the field was absent.

func LongContextOf

func LongContextOf(entry json.RawMessage) (LongContext, bool)

LongContextOf returns an entry's above-threshold prices, or false when it publishes none or publishes no above-threshold input rate (the one field the others are relative to). LiteLLM publishes at most one threshold per model; if an entry ever carried two, the lowest wins, because billing the higher tier from the lower threshold would overstate and the reverse would understate — the lower threshold is the one the provider applies first.

type ProfileBackend added in v0.13.2

type ProfileBackend struct {
	Name, Prefix, Provider, Suffix string
	Slash                          bool
}

ProfileBackend is a backend reached through a core-models built-in profile whose models have no bare row: an id like zai-org/glm-5.2-maas or grok-4.7 names a model on one backend, so the (backend, model) row is the only row, and the context-window table (keyed on bare ids) does not apply.

Name is the profile's backend name, Prefix is LiteLLM's key prefix (empty for OpenAI, whose rows are unprefixed), Provider narrows by litellm_provider where the prefix cannot (OpenAI's unprefixed keys sit beside every other vendor's), Suffix narrows to the profile's models, and Slash requires a publisher-qualified id.

func (ProfileBackend) ModelOf added in v0.13.2

func (b ProfileBackend) ModelOf(key, provider string) (string, bool)

ModelOf reports the model id LiteLLM key names on b, given the entry's litellm_provider, or false when the key is not one of b's.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL