Documentation
¶
Overview ¶
Package litellm reads the parts of LiteLLM's model_prices_and_context_window.json that more than one caller in this module needs: the pricing package's Refresh and the dev/regen-builtin-pricing generator. One reader, so the two can't disagree about what a field means.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type LongContext ¶
type LongContext struct {
// ThresholdTokens is the prompt size above which these rates apply:
// LiteLLM's "_above_<N>k_tokens" suffix, with k = 1,000 (OpenAI's
// "272K" is 272,000 tokens).
ThresholdTokens int
Input float64
CacheRead float64
CacheCreation float64
CacheCreation1hr float64
Output float64
}
LongContext is one entry's above-threshold prices, per token, as LiteLLM publishes them. A zero rate means the field was absent.
func LongContextOf ¶
func LongContextOf(entry json.RawMessage) (LongContext, bool)
LongContextOf returns an entry's above-threshold prices, or false when it publishes none or publishes no above-threshold input rate (the one field the others are relative to). LiteLLM publishes at most one threshold per model; if an entry ever carried two, the lowest wins, because billing the higher tier from the lower threshold would overstate and the reverse would understate — the lower threshold is the one the provider applies first.