pricing

package
v1.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 25, 2026 License: Apache-2.0 Imports: 10 Imported by: 0

Documentation

Overview

Package pricing computes request cost from Usage and per-token rates, set explicitly or loaded into a Registry from LiteLLM's price list.

Index

Constants

View Source
const DefaultURL = "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"

DefaultURL is LiteLLM's community-maintained model price list.

Variables

This section is empty.

Functions

This section is empty.

Types

type Cost

type Cost struct {
	Input      float64 `json:"input"`
	Output     float64 `json:"output"`
	CacheRead  float64 `json:"cache_read,omitempty"`
	CacheWrite float64 `json:"cache_write,omitempty"`
	Total      float64 `json:"total"`
}

Cost is a cost breakdown in the currency of the rates.

type ModelPricing

type ModelPricing struct {
	InputCostPerToken      float64  `json:"input_cost_per_token"`
	OutputCostPerToken     float64  `json:"output_cost_per_token"`
	CacheReadCostPerToken  *float64 `json:"cache_read_input_token_cost,omitempty"`
	CacheWriteCostPerToken *float64 `json:"cache_creation_input_token_cost,omitempty"`
}

ModelPricing holds per-token rates; LiteLLM's list is in USD. A nil cache rate inherits the input rate; a non-nil zero means free cache usage.

func (ModelPricing) Cost added in v1.9.0

func (price ModelPricing) Cost(usage litellm.Usage) (Cost, error)

Cost prices usage. Input and output counts must be known; cache counts may be unknown only when their rate equals the input rate.

type Registry

type Registry struct {
	// contains filtered or unexported fields
}

Registry is a concurrency-safe price table. The zero value is ready to use.

func NewRegistry

func NewRegistry() *Registry

NewRegistry returns an empty Registry.

func (*Registry) Cost added in v1.9.0

func (r *Registry) Cost(model string, usage litellm.Usage) (Cost, error)

Cost prices usage with the rates for model.

func (*Registry) Get

func (r *Registry) Get(model string) (ModelPricing, bool)

Get returns the rates for model. A "vendor/model" name falls back to "model" when it has no entry of its own.

func (*Registry) LoadFromReader

func (r *Registry) LoadFromReader(reader io.Reader) error

LoadFromReader replaces the table with a JSON price list in LiteLLM's format. Models without both input and output rates are skipped.

func (*Registry) LoadFromURL

func (r *Registry) LoadFromURL(ctx context.Context, url string) error

LoadFromURL replaces the table with the JSON price list at url, such as DefaultURL.

func (*Registry) Set

func (r *Registry) Set(model string, price ModelPricing) error

Set adds or replaces the rates for model.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL