catalog

package
v1.9.5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 10, 2026 License: Apache-2.0 Imports: 16 Imported by: 0

Documentation

Overview

Package catalog holds model facts (context window, output limit, reasoning support and per-token prices) loaded from LiteLLM's model list or set explicitly. Providers never read it; applications use it to choose limits and to price usage.

Names are the list's keys, whose vendor prefixes follow LiteLLM's provider names: "xai/grok-4", not "grok/grok-4". The catalog does not translate names, since a model listed under several sites may be priced differently on each; an application keeps the catalog name of each model it prices.

Index

Constants

View Source
const DefaultURL = "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"

DefaultURL is LiteLLM's community-maintained model list.

Variables

This section is empty.

Functions

This section is empty.

Types

type Catalog

type Catalog struct {
	// contains filtered or unexported fields
}

Catalog is a concurrency-safe model table. The zero value is empty and ready to use.

func (*Catalog) All

func (c *Catalog) All() iter.Seq2[string, Model]

All yields the models in name order, as of the start of the iteration.

func (*Catalog) Get

func (c *Catalog) Get(name string) (Model, bool)

Get returns the facts for the exact catalog key name. A missing key returns a zero Model and false; provider prefixes are never added or removed.

func (*Catalog) LoadFromReader

func (c *Catalog) LoadFromReader(reader io.Reader) error

LoadFromReader replaces the table with the chat and responses models of a model list in LiteLLM's format. A model is priced when it has both input and output rates. Its long-input rates, keys such as input_cost_per_token_above_200k_tokens or a tiered_pricing table, become Tiers; a rate a tier lacks is the model's, or in a tiered_pricing table the tier's input rate, as LiteLLM prices them. Invalid models fail the load without changing the table.

func (*Catalog) LoadFromURL

func (c *Catalog) LoadFromURL(ctx context.Context, url string) error

LoadFromURL replaces the table with the model list at url, such as DefaultURL.

func (*Catalog) Set

func (c *Catalog) Set(name string, model Model) error

Set adds or replaces the facts for the model called name.

type Cost

type Cost struct {
	Input      float64 `json:"input"`
	Output     float64 `json:"output"`
	CacheRead  float64 `json:"cache_read,omitempty"`
	CacheWrite float64 `json:"cache_write,omitempty"`
	Total      float64 `json:"total"`
}

Cost is a cost breakdown in the currency of the rates.

func (*Cost) Add added in v1.9.4

func (c *Cost) Add(o Cost)

Add adds o to c, as a running total.

type Model

type Model struct {
	// Provider is the provider the list files the model under, such as
	// "anthropic" or "vertex_ai-language-models".
	Provider string `json:"provider,omitempty"`
	// MaxInputTokens is the context window.
	MaxInputTokens  int `json:"max_input_tokens,omitempty"`
	MaxOutputTokens int `json:"max_output_tokens,omitempty"`
	// Reasoning is nil when support is unknown; false means unsupported.
	Reasoning *bool `json:"reasoning,omitempty"`
	// Pricing is nil when the model is unpriced.
	Pricing *Pricing `json:"pricing,omitempty"`
}

Model holds what is known about a model; zero values are unknown.

type Pricing

type Pricing struct {
	Rates
	// Tiers, in increasing order of AboveInputTokens, replace Rates for a
	// call whose input tokens exceed AboveInputTokens: the last such tier.
	Tiers []Tier `json:"tiers,omitempty"`
}

Pricing holds a model's rates; LiteLLM's list is in USD. Vendors that charge more for long inputs price every token of such a call at a tier's rates.

func (Pricing) Cost

func (p Pricing) Cost(usage litellm.Usage) (Cost, error)

Cost prices usage at the rates of the last tier its input tokens are above, or else at Rates. Cache reads and writes are priced at their rates and the rest of the input at the input rate; a cache count the vendor did not report is zero, its tokens priced as input. Usage without input tokens is an error: every call has some, so the vendor reported none. So are writes cached for an hour without their rate. p must be valid, as a Catalog's pricing is; check any other with Validate first.

func (Pricing) Validate added in v1.9.4

func (p Pricing) Validate() error

Validate reports rates that are negative or not finite, and tiers out of order. A Catalog validates the pricing it holds.

type Rates added in v1.9.4

type Rates struct {
	Input        float64  `json:"input_cost_per_token"`
	Output       float64  `json:"output_cost_per_token"`
	CacheRead    *float64 `json:"cache_read_input_token_cost,omitempty"`
	CacheWrite   *float64 `json:"cache_creation_input_token_cost,omitempty"`
	CacheWrite1h *float64 `json:"cache_creation_input_token_cost_above_1hr,omitempty"`
}

Rates are per-token rates. A nil cache rate inherits the input rate; a non-nil zero means free cache usage. Writes cached for an hour are priced at CacheWrite1h alone, since vendors charge more for them.

type Tier added in v1.9.4

type Tier struct {
	AboveInputTokens int `json:"above_input_tokens"`
	Rates
}

Tier holds the rates of calls with more than AboveInputTokens input tokens.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL