Documentation
¶
Overview ¶
Package catalog holds model facts (context window, output limit, reasoning support and per-token prices) loaded from LiteLLM's model list or set explicitly. Providers never read it; applications use it to choose limits and to price usage.
Names are the list's keys, whose vendor prefixes follow LiteLLM's provider names: "xai/grok-4", not "grok/grok-4". The catalog does not translate names, since a model listed under several sites may be priced differently on each; an application keeps the catalog name of each model it prices.
Index ¶
Constants ¶
const DefaultURL = "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"
DefaultURL is LiteLLM's community-maintained model list.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Catalog ¶
type Catalog struct {
// contains filtered or unexported fields
}
Catalog is a concurrency-safe model table. The zero value is empty and ready to use.
func (*Catalog) Get ¶
Get returns the facts for the exact catalog key name. A missing key returns a zero Model and false; provider prefixes are never added or removed.
func (*Catalog) LoadFromReader ¶
LoadFromReader replaces the table with the chat and responses models of a model list in LiteLLM's format. A model is priced when it has both input and output rates. Invalid models fail the load without changing the table.
func (*Catalog) LoadFromURL ¶
LoadFromURL replaces the table with the model list at url, such as DefaultURL.
type Cost ¶
type Cost struct {
Input float64 `json:"input"`
Output float64 `json:"output"`
CacheRead float64 `json:"cache_read,omitempty"`
CacheWrite float64 `json:"cache_write,omitempty"`
Total float64 `json:"total"`
}
Cost is a cost breakdown in the currency of the rates.
type Model ¶
type Model struct {
// Provider is the provider the list files the model under, such as
// "anthropic" or "vertex_ai-language-models".
Provider string `json:"provider,omitempty"`
// MaxInputTokens is the context window.
MaxInputTokens int `json:"max_input_tokens,omitempty"`
MaxOutputTokens int `json:"max_output_tokens,omitempty"`
// Reasoning is nil when support is unknown; false means unsupported.
Reasoning *bool `json:"reasoning,omitempty"`
// Pricing is nil when the model is unpriced.
Pricing *Pricing `json:"pricing,omitempty"`
}
Model holds what is known about a model; zero values are unknown.
type Pricing ¶
type Pricing struct {
InputCostPerToken float64 `json:"input_cost_per_token"`
OutputCostPerToken float64 `json:"output_cost_per_token"`
CacheReadCostPerToken *float64 `json:"cache_read_input_token_cost,omitempty"`
CacheWriteCostPerToken *float64 `json:"cache_creation_input_token_cost,omitempty"`
CacheWrite1hCostPerToken *float64 `json:"cache_creation_input_token_cost_above_1hr,omitempty"`
}
Pricing holds per-token rates; LiteLLM's list is in USD. A nil cache rate inherits the input rate; a non-nil zero means free cache usage. Writes cached for an hour are priced at CacheWrite1hCostPerToken alone, since vendors charge more for them.
func (Pricing) Cost ¶
Cost prices usage. Cache reads and writes are priced at their rates and the rest of the input at the input rate; a cache count the vendor did not report is zero, its tokens priced as input. Usage without input tokens is an error: every call has some, so the vendor reported none. So are writes cached for an hour without their rate.