Documentation
¶
Overview ¶
Package pricing resolves a model's per-million-token rates across a layered set of sources so usage costs stay accurate as new models ship and operators add overrides.
Lookup chain (first exact-match wins; longest-prefix only at the end):
- cfg.Model.Pricing[name] — operator override in .agents/config.json, keyed by model name (case-insensitive). Survives /model switches.
- .agents/pricing.json — project-local additions (team-internal model variants, project-specific routing).
- ~/.core-agent/pricing.json — user-global file. Two sections: `manual` (operator-curated, hand-edited or set via /pricing set) and `external` (auto-fetched from LiteLLM in PR B; absent in PR A).
- builtin — the compiled-in fallback table; the zero-config baseline for common Gemini models. Lives in internal/pricing/builtin.go.
- longest-prefix match across the merge of (1)..(4) — handles `gemini-3.1-pro-preview-customtools`-style suffixes.
- (Rates{}, false) — rate unknown; callers (e.g. the TUI's cost displays) should render "$—" rather than "$0".
The catalog is built once at startup from these sources (see NewCatalog) and consulted on every per-turn cost append; lookups are read-only and lock-free.
Index ¶
Constants ¶
const ( ProjectFileName = "pricing.json" UserFileName = "pricing.json" )
File names. Exposed so callers can reference them in error messages and tests.
const ( SourceCfgOverride = "cfg-override" SourceProjectFile = "project-file" SourceUserManual = "user-manual" SourceUserExternal = "user-external" SourceBuiltin = "builtin" )
Layer source names surfaced via LookupWithSource + the attach /pricing endpoint. Stable strings — operators grep for them, docs reference them. Don't rename without a deprecation cycle.
const DefaultRefreshInterval = 24 * time.Hour
DefaultRefreshInterval is the daily cadence Refresh enforces: if the cache's FetchedAt is younger than this, Refresh is a no-op. 24h is long enough that the network cost is amortized but short enough that rate changes propagate within an operator's next session.
const DefaultRefreshSource = "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"
DefaultRefreshSource is the canonical LiteLLM pricing JSON URL. Community-maintained, MIT-licensed, structured, covers hundreds of models from every major provider. Overridable via RefreshOptions for tests + air-gapped mirrors.
const SchemaVersion = 1
SchemaVersion is the on-disk schema version for pricing files. Bumped only on incompatible schema changes; new fields with `omitempty` don't require a bump.
Variables ¶
This section is empty.
Functions ¶
func Builtin ¶
Builtin returns a defensive copy of the compiled-in table. Used by tests + by tools that want to inspect what shipped (e.g. a future `/pricing list builtin` view).
func SaveUserFile ¶
SaveUserFile writes uf to <userHome>/<UserFileName> atomically. Used by PR B's refresher and PR C's /pricing set slash. Empty userHome is an error (caller should resolve a real home first).
Types ¶
type Catalog ¶
type Catalog struct {
// contains filtered or unexported fields
}
Catalog is the merged view of all pricing sources, queried by model name. Construct with NewCatalog; consult with Lookup.
Layers are stored separately so PR B's daily refresh can rewrite the external slice without touching the others, and so the precedence chain stays explicit (no "where did this rate come from" mystery).
func NewCatalog ¶
NewCatalog reads every configured source and returns the merged catalog. Missing files are not errors (the common case); only I/O failures and malformed JSON return non-nil.
The returned catalog is read-only after construction. Callers that want a refresh (PR B's daily fetch, or after /reload) build a new Catalog and swap atomically via the consumer's chosen pointer-store mechanism.
func (*Catalog) Counts ¶
func (c *Catalog) Counts() CountByLayer
Counts returns per-layer entry counts.
func (*Catalog) Lookup ¶
Lookup returns the resolved rates for modelID plus a found flag. !found means the caller should treat the cost as unknown ($—) rather than zero.
Resolution: exact match scan across layers in precedence order, then a longest-prefix scan across the union of all layers.
func (*Catalog) LookupWithSource ¶
LookupWithSource is Lookup + the name of the catalog layer that served the rate (SourceCfgOverride / SourceProjectFile / SourceUserManual / SourceUserExternal / SourceBuiltin). Empty source string when !ok. Used by /pricing so operators can spot stale builtin rates that should have been overridden by a fresh LiteLLM refresh but weren't — the visibility that #259 asked for.
Resolution matches Lookup: exact match by precedence first, then longest-prefix across the union. The prefix-fallback path returns the source of the LAYER that held the winning prefix entry.
type CountByLayer ¶
type CountByLayer struct {
CfgOverride int
ProjectFile int
UserManual int
UserExternal int
Builtin int
}
CountByLayer reports how many model entries each layer holds. Surfaced via /pricing list (PR C) and useful for tests that want to assert the expected number of rows landed in each layer.
type ExternalSource ¶
type ExternalSource struct {
FetchedAt time.Time `json:"fetched_at"`
Source string `json:"source"` // canonical URL the data was pulled from
ETag string `json:"etag,omitempty"`
Models map[string]ModelRates `json:"models,omitempty"`
}
ExternalSource is the auto-fetched section. Populated by PR B's LiteLLM refresh; unused in PR A. The fetched_at + etag fields drive cache-validity logic (skip refresh if <24h, send If-None-Match on revalidation).
type ManualSection ¶
type ManualSection struct {
Models map[string]ModelRates `json:"models,omitempty"`
}
ManualSection is the operator-curated section. Round-trips intact across refreshes (PR B's fetcher only rewrites External).
type ModelRates ¶
type ModelRates struct {
InputPerMTok float64 `json:"input_per_mtok,omitempty"`
CachedInputPerMTok float64 `json:"cached_input_per_mtok,omitempty"`
OutputPerMTok float64 `json:"output_per_mtok,omitempty"`
UpdatedAt time.Time `json:"updated_at,omitempty"`
}
ModelRates is the on-disk per-model rate, mirroring config's PricingConfig field names so operator-edited files use the same spelling regardless of whether they live in pricing.json or in .agents/config.json's `model.pricing` override map.
type Options ¶
type Options struct {
// CfgOverride is the operator's per-model override from
// .agents/config.json's `model.pricing` map. Highest precedence.
CfgOverride map[string]ModelRates
// AgentsDir is the resolved .agents/ directory (empty when no
// project root was found). Catalog construction reads
// <agentsDir>/pricing.json if present.
AgentsDir string
// UserHome is the per-user core-agent state directory (usually
// ~/.core-agent). Catalog construction reads <UserHome>/pricing.json
// if present (both manual + external sections).
UserHome string
}
Options bundle the inputs NewCatalog needs to assemble the layered view. All fields are optional — an empty Options yields a catalog with only the compiled-in builtin layer (useful for tests + as the default before main.go wires the files in).
type ProjectFile ¶
type ProjectFile struct {
Version int `json:"version"`
Models map[string]ModelRates `json:"models,omitempty"`
}
ProjectFile is the .agents/pricing.json shape — flat models map. Project files are always operator-curated (never auto-fetched), so no manual/external split is needed.
func LoadProjectFile ¶
func LoadProjectFile(agentsDir string) (*ProjectFile, error)
LoadProjectFile reads .agents/pricing.json from agentsDir, or returns an empty file when the file is missing (a missing project file is the common case, not an error). Returns an error for I/O failures or malformed JSON.
type Rates ¶
type Rates struct {
InputPerMTok float64
CachedInputPerMTok float64
OutputPerMTok float64
UpdatedAt time.Time
}
Rates is the per-million-token cost for one model. CachedInputPerMTok is the rate applied to input tokens served from the provider's prompt cache (Gemini's `cachedContentTokenCount`, Anthropic's `cache_read_input_tokens`); a zero value means the cache-read rate isn't known and callers should bill cached tokens at InputPerMTok.
UpdatedAt records when the rate was last verified against its source (LiteLLM refresh time, generator run time for builtin entries, operator edit time for manual overrides). Zero when unknown. Surfaced through /pricing so operators can spot stale entries at a glance — issue #259 called out that hand-authored rates drift silently, and staleness visibility is the mitigation baked into the "regenerate builtin from LiteLLM" workflow that followed.
func (Rates) CostUSD ¶
CostUSD returns the dollar cost of (input, output) tokens at r. Treats every input token as uncached — see CostUSDWithCache for the cached-vs-uncached split.
func (Rates) CostUSDWithCache ¶
CostUSDWithCache returns the dollar cost with cache-hit tokens billed at CachedInputPerMTok. When CachedInputPerMTok is zero (rate unknown) cached tokens fall back to InputPerMTok — no silent free-riding.
func (Rates) IsZero ¶
IsZero reports whether the rates carry no useful pricing. Used by callers to distinguish "free model" from "rate unknown" — only the latter should render "$—". CachedInputPerMTok isn't part of this check: a row that carries only a cache rate but no base input/output rates is still "unpriced" in the useful sense.
type RefreshOptions ¶
type RefreshOptions struct {
// Source is the URL to fetch. Defaults to DefaultRefreshSource.
Source string
// Client is the HTTP client used. Defaults to a fresh one with
// a 30s timeout — long enough for slow links, short enough that
// startup doesn't hang forever.
Client *http.Client
// MinInterval is the minimum age the cache must reach before
// Refresh actually fetches. Zero defaults to
// DefaultRefreshInterval (24h). Set to a negative duration to
// force a fetch regardless of cache age (used by PR C's
// /pricing refresh slash command).
MinInterval time.Duration
// Now is the clock Refresh uses to compute cache age. Tests
// override to deterministically drive stale-vs-fresh. Defaults
// to time.Now.
Now func() time.Time
}
RefreshOptions controls the refresh fetch. All fields optional; defaults match a production launch (LiteLLM upstream, 24h cadence, no max-size cap, 30s timeout). Tests + air-gapped mirrors override Source; integration tests override Now for deterministic stale-vs-fresh decisions.
type RefreshOutcome ¶
type RefreshOutcome struct {
Skipped bool // true when the cache was fresh enough to skip the fetch
NotModified bool // true when the server replied 304 (cache still authoritative)
NetworkFailed bool // true when the fetch itself failed; cache was preserved
NetworkError error // populated when NetworkFailed
StaleAge time.Duration // age of the cache at decision time; zero for fresh writes
ModelCount int // number of models written to the external section (zero on Skipped/NotModified)
FetchedAt time.Time // timestamp stored in the cache file (or pre-existing one for Skipped)
}
RefreshOutcome is the result of a Refresh call. Surfaced so the caller (cmd/core-agent startup + PR C's /pricing refresh slash) can render a meaningful one-liner: "Refreshed 247 models from LiteLLM" / "Cache is 4h old; skipped" / "Using 6-day-old cache; network unreachable: connection refused".
func Refresh ¶
func Refresh(ctx context.Context, userHome string, opts RefreshOptions) (RefreshOutcome, error)
Refresh fetches the LiteLLM pricing table and writes it into the `external` section of <userHome>/pricing.json, preserving the `manual` section operators curate by hand. Daily cadence is enforced via MinInterval — if the cache is younger, the fetch is skipped.
Network failures are non-fatal: existing cached data stays in place, the caller gets a populated RefreshOutcome.NetworkError, and the surfaced one-liner ("using N-day-old cache") lets the operator know the rates may be stale without breaking the session. Same fall-through for HTTP 5xx / malformed bodies.
userHome must resolve to a writable directory (typically ~/.core-agent). Empty userHome is an error.
type UserFile ¶
type UserFile struct {
Version int `json:"version"`
External *ExternalSource `json:"external,omitempty"`
Manual *ManualSection `json:"manual,omitempty"`
}
UserFile is the ~/.core-agent/pricing.json shape — sectioned so PR B's daily refresh can overwrite the `external` section without touching `manual` entries the operator hand-edited (or set via /pricing set in PR C).
func LoadUserFile ¶
LoadUserFile reads ~/.core-agent/pricing.json (or whatever userHome resolves to), or returns an empty file when missing. Same not-an-error treatment for missing files as LoadProjectFile.