Documentation
¶
Overview ¶
Package pricing computes request cost from Usage and per-token rates, set explicitly or loaded into a Registry from LiteLLM's price list.
Index ¶
- Constants
- type Cost
- type ModelPricing
- type Registry
- func (r *Registry) Cost(model string, usage litellm.Usage) (Cost, error)
- func (r *Registry) Get(model string) (ModelPricing, bool)
- func (r *Registry) LoadFromReader(reader io.Reader) error
- func (r *Registry) LoadFromURL(ctx context.Context, url string) error
- func (r *Registry) Set(model string, price ModelPricing) error
Constants ¶
const DefaultURL = "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"
DefaultURL is LiteLLM's community-maintained model price list.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Cost ¶
type Cost struct {
Input float64 `json:"input"`
Output float64 `json:"output"`
CacheRead float64 `json:"cache_read,omitempty"`
CacheWrite float64 `json:"cache_write,omitempty"`
Total float64 `json:"total"`
}
Cost is a cost breakdown in the currency of the rates.
type ModelPricing ¶
type ModelPricing struct {
InputCostPerToken float64 `json:"input_cost_per_token"`
OutputCostPerToken float64 `json:"output_cost_per_token"`
CacheReadCostPerToken *float64 `json:"cache_read_input_token_cost,omitempty"`
CacheWriteCostPerToken *float64 `json:"cache_creation_input_token_cost,omitempty"`
}
ModelPricing holds per-token rates; LiteLLM's list is in USD. A nil cache rate inherits the input rate; a non-nil zero means free cache usage.
type Registry ¶
type Registry struct {
// contains filtered or unexported fields
}
Registry is a concurrency-safe price table. The zero value is ready to use.
func (*Registry) Get ¶
func (r *Registry) Get(model string) (ModelPricing, bool)
Get returns the rates for model. A "vendor/model" name falls back to "model" when it has no entry of its own.
func (*Registry) LoadFromReader ¶
LoadFromReader replaces the table with a JSON price list in LiteLLM's format. Models without both input and output rates are skipped.
func (*Registry) LoadFromURL ¶
LoadFromURL replaces the table with the JSON price list at url, such as DefaultURL.