catalog

package
v1.9.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 2, 2026 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Overview

Package catalog holds model facts (context window, output limit, reasoning support and per-token prices) loaded from LiteLLM's model list or set explicitly. Providers never read it; applications use it to choose limits and to price usage.

Names are the list's keys, whose vendor prefixes follow LiteLLM's provider names: "xai/grok-4", not "grok/grok-4". The catalog does not translate names, since a model listed under several sites may be priced differently on each; an application keeps the catalog name of each model it prices.

Index

Constants

View Source
const DefaultURL = "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"

DefaultURL is LiteLLM's community-maintained model list.

Variables

This section is empty.

Functions

This section is empty.

Types

type Catalog

type Catalog struct {
	// contains filtered or unexported fields
}

Catalog is a concurrency-safe model table. The zero value is empty and ready to use.

func (*Catalog) All

func (c *Catalog) All() iter.Seq2[string, Model]

All yields the models in name order, as of the start of the iteration.

func (*Catalog) Get

func (c *Catalog) Get(name string) (Model, bool)

Get returns the facts for the exact catalog key name. A missing key returns a zero Model and false; provider prefixes are never added or removed.

func (*Catalog) LoadFromReader

func (c *Catalog) LoadFromReader(reader io.Reader) error

LoadFromReader replaces the table with the chat and responses models of a model list in LiteLLM's format. A model is priced when it has both input and output rates. Invalid models fail the load without changing the table.

func (*Catalog) LoadFromURL

func (c *Catalog) LoadFromURL(ctx context.Context, url string) error

LoadFromURL replaces the table with the model list at url, such as DefaultURL.

func (*Catalog) Set

func (c *Catalog) Set(name string, model Model) error

Set adds or replaces the facts for the model called name.

type Cost

type Cost struct {
	Input      float64 `json:"input"`
	Output     float64 `json:"output"`
	CacheRead  float64 `json:"cache_read,omitempty"`
	CacheWrite float64 `json:"cache_write,omitempty"`
	Total      float64 `json:"total"`
}

Cost is a cost breakdown in the currency of the rates.

type Model

type Model struct {
	// Provider is the provider the list files the model under, such as
	// "anthropic" or "vertex_ai-language-models".
	Provider string `json:"provider,omitempty"`
	// MaxInputTokens is the context window.
	MaxInputTokens  int `json:"max_input_tokens,omitempty"`
	MaxOutputTokens int `json:"max_output_tokens,omitempty"`
	// Reasoning is nil when support is unknown; false means unsupported.
	Reasoning *bool `json:"reasoning,omitempty"`
	// Pricing is nil when the model is unpriced.
	Pricing *Pricing `json:"pricing,omitempty"`
}

Model holds what is known about a model; zero values are unknown.

type Pricing

type Pricing struct {
	InputCostPerToken        float64  `json:"input_cost_per_token"`
	OutputCostPerToken       float64  `json:"output_cost_per_token"`
	CacheReadCostPerToken    *float64 `json:"cache_read_input_token_cost,omitempty"`
	CacheWriteCostPerToken   *float64 `json:"cache_creation_input_token_cost,omitempty"`
	CacheWrite1hCostPerToken *float64 `json:"cache_creation_input_token_cost_above_1hr,omitempty"`
}

Pricing holds per-token rates; LiteLLM's list is in USD. A nil cache rate inherits the input rate; a non-nil zero means free cache usage. Writes cached for an hour are priced at CacheWrite1hCostPerToken alone, since vendors charge more for them.

func (Pricing) Cost

func (p Pricing) Cost(usage litellm.Usage) (Cost, error)

Cost prices usage. Cache reads and writes are priced at their rates and the rest of the input at the input rate; a cache count the vendor did not report is zero, its tokens priced as input. Usage without input tokens is an error: every call has some, so the vendor reported none. So are writes cached for an hour without their rate.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL