litellm

package
v0.13.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 13, 2026 License: Apache-2.0 Imports: 3 Imported by: 0

Documentation

Overview

Package litellm reads the parts of LiteLLM's model_prices_and_context_window.json that more than one caller in this module needs: the pricing package's Refresh and the dev/regen-builtin-pricing generator. One reader, so the two can't disagree about what a field means.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type LongContext

type LongContext struct {
	// ThresholdTokens is the prompt size above which these rates apply:
	// LiteLLM's "_above_<N>k_tokens" suffix, with k = 1,000 (OpenAI's
	// "272K" is 272,000 tokens).
	ThresholdTokens  int
	Input            float64
	CacheRead        float64
	CacheCreation    float64
	CacheCreation1hr float64
	Output           float64
}

LongContext is one entry's above-threshold prices, per token, as LiteLLM publishes them. A zero rate means the field was absent.

func LongContextOf

func LongContextOf(entry json.RawMessage) (LongContext, bool)

LongContextOf returns an entry's above-threshold prices, or false when it publishes none or publishes no above-threshold input rate (the one field the others are relative to). LiteLLM publishes at most one threshold per model; if an entry ever carried two, the lowest wins, because billing the higher tier from the lower threshold would overstate and the reverse would understate — the lower threshold is the one the provider applies first.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL