catalog

package
v1.9.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 29, 2026 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Overview

Package catalog holds model facts (context window, output limit, reasoning support and per-token prices) loaded from LiteLLM's model list or set explicitly. Providers never read it; applications use it to choose limits and to price usage.

Names are the list's keys, whose vendor prefixes follow LiteLLM's provider names: "xai/grok-4", not "grok/grok-4". The catalog does not translate names, since a model listed under several sites may be priced differently on each; an application keeps the catalog name of each model it prices.

Index

Constants

View Source
const DefaultURL = "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"

DefaultURL is LiteLLM's community-maintained model list.

Variables

This section is empty.

Functions

This section is empty.

Types

type Catalog

type Catalog struct {
	// contains filtered or unexported fields
}

Catalog is a concurrency-safe model table. The zero value is empty and ready to use.

func (*Catalog) All

func (c *Catalog) All() iter.Seq2[string, Model]

All yields the models in name order, as of the start of the iteration.

func (*Catalog) Get

func (c *Catalog) Get(name string) (Model, bool)

Get returns the facts for the exact catalog key name. A missing key returns a zero Model and false; provider prefixes are never added or removed.

func (*Catalog) LoadFromReader

func (c *Catalog) LoadFromReader(reader io.Reader) error

LoadFromReader replaces the table with the chat and responses models of a model list in LiteLLM's format. A model is priced when it has both input and output rates. Invalid models fail the load without changing the table.

func (*Catalog) LoadFromURL

func (c *Catalog) LoadFromURL(ctx context.Context, url string) error

LoadFromURL replaces the table with the model list at url, such as DefaultURL.

func (*Catalog) Set

func (c *Catalog) Set(name string, model Model) error

Set adds or replaces the facts for the model called name.

type Cost

type Cost struct {
	Input      float64 `json:"input"`
	Output     float64 `json:"output"`
	CacheRead  float64 `json:"cache_read,omitempty"`
	CacheWrite float64 `json:"cache_write,omitempty"`
	Total      float64 `json:"total"`
}

Cost is a cost breakdown in the currency of the rates.

type Model

type Model struct {
	// Provider is the provider the list files the model under, such as
	// "anthropic" or "vertex_ai-language-models".
	Provider string `json:"provider,omitempty"`
	// MaxInputTokens is the context window.
	MaxInputTokens  int `json:"max_input_tokens,omitempty"`
	MaxOutputTokens int `json:"max_output_tokens,omitempty"`
	// Reasoning is nil when support is unknown; false means unsupported.
	Reasoning *bool `json:"reasoning,omitempty"`
	// Pricing is nil when the model is unpriced.
	Pricing *Pricing `json:"pricing,omitempty"`
}

Model holds what is known about a model; zero values are unknown.

type Pricing

type Pricing struct {
	InputCostPerToken      float64  `json:"input_cost_per_token"`
	OutputCostPerToken     float64  `json:"output_cost_per_token"`
	CacheReadCostPerToken  *float64 `json:"cache_read_input_token_cost,omitempty"`
	CacheWriteCostPerToken *float64 `json:"cache_creation_input_token_cost,omitempty"`
}

Pricing holds per-token rates; LiteLLM's list is in USD. A nil cache rate inherits the input rate; a non-nil zero means free cache usage.

func (Pricing) Cost

func (p Pricing) Cost(usage litellm.Usage) (Cost, error)

Cost prices usage. Input and output counts must be known; cache counts may be unknown only when their rate equals the input rate.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL