Documentation
¶
Overview ¶
Package catalog holds model facts (context window, output limit, reasoning support and per-token prices) loaded from LiteLLM's model list or set explicitly. Providers never read it; applications use it to choose limits and to price usage.
Names are the list's keys, whose vendor prefixes follow LiteLLM's provider names: "xai/grok-4", not "grok/grok-4". The catalog does not translate names, since a model listed under several sites may be priced differently on each; an application keeps the catalog name of each model it prices.
Index ¶
Constants ¶
const DefaultURL = "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"
DefaultURL is LiteLLM's community-maintained model list.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Catalog ¶
type Catalog struct {
// contains filtered or unexported fields
}
Catalog is a concurrency-safe model table. The zero value is empty and ready to use.
func (*Catalog) Get ¶
Get returns the facts for the exact catalog key name. A missing key returns a zero Model and false; provider prefixes are never added or removed.
func (*Catalog) LoadFromReader ¶
LoadFromReader replaces the table with the chat and responses models of a model list in LiteLLM's format. A model is priced when it has both input and output rates. Its long-input rates, keys such as input_cost_per_token_above_200k_tokens or a tiered_pricing table, become Tiers; a rate a tier lacks is the model's, or in a tiered_pricing table the tier's input rate, as LiteLLM prices them. Invalid models fail the load without changing the table.
func (*Catalog) LoadFromURL ¶
LoadFromURL replaces the table with the model list at url, such as DefaultURL.
type Cost ¶
type Cost struct {
Input float64 `json:"input"`
Output float64 `json:"output"`
CacheRead float64 `json:"cache_read,omitempty"`
CacheWrite float64 `json:"cache_write,omitempty"`
Total float64 `json:"total"`
}
Cost is a cost breakdown in the currency of the rates.
type Model ¶
type Model struct {
// Provider is the provider the list files the model under, such as
// "anthropic" or "vertex_ai-language-models".
Provider string `json:"provider,omitempty"`
// MaxInputTokens is the context window.
MaxInputTokens int `json:"max_input_tokens,omitempty"`
MaxOutputTokens int `json:"max_output_tokens,omitempty"`
// Reasoning is nil when support is unknown; false means unsupported.
Reasoning *bool `json:"reasoning,omitempty"`
// Pricing is nil when the model is unpriced.
Pricing *Pricing `json:"pricing,omitempty"`
}
Model holds what is known about a model; zero values are unknown.
type Pricing ¶
type Pricing struct {
Rates
// Tiers, in increasing order of AboveInputTokens, replace Rates for a
// call whose input tokens exceed AboveInputTokens: the last such tier.
Tiers []Tier `json:"tiers,omitempty"`
}
Pricing holds a model's rates; LiteLLM's list is in USD. Vendors that charge more for long inputs price every token of such a call at a tier's rates.
func (Pricing) Cost ¶
Cost prices usage at the rates of the last tier its input tokens are above, or else at Rates. Cache reads and writes are priced at their rates and the rest of the input at the input rate; a cache count the vendor did not report is zero, its tokens priced as input. Usage without input tokens is an error: every call has some, so the vendor reported none. So are writes cached for an hour without their rate. p must be valid, as a Catalog's pricing is; check any other with Validate first.
type Rates ¶ added in v1.9.4
type Rates struct {
Input float64 `json:"input_cost_per_token"`
Output float64 `json:"output_cost_per_token"`
CacheRead *float64 `json:"cache_read_input_token_cost,omitempty"`
CacheWrite *float64 `json:"cache_creation_input_token_cost,omitempty"`
CacheWrite1h *float64 `json:"cache_creation_input_token_cost_above_1hr,omitempty"`
}
Rates are per-token rates. A nil cache rate inherits the input rate; a non-nil zero means free cache usage. Writes cached for an hour are priced at CacheWrite1h alone, since vendors charge more for them.