Documentation
¶
Overview ¶
Package pricing resolves a model's per-million-token rates across a layered set of sources, so usage costs stay accurate as new models ship and operators add overrides. It is the one catalog mast and core-agent share (design §9, phase L6).
Layers, highest precedence first (LookupForWithSource has the exact order; in short, an exact key in any layer beats every prefix, and a higher layer beats a lower one):
- Options.CfgOverride — the product's own per-model override (its config file), keyed by model name, case-insensitive.
- Options.AgentsDir/pricing.json — project-local additions.
- Options.UserHome/pricing.json, section `manual` — operator-curated.
- Options.UserHome/pricing.json, section `external` — fetched from LiteLLM by Refresh.
- builtin — the compiled-in table, generated from LiteLLM's catalog by dev/regen-builtin-pricing into ./builtin.go.
- Options.Declared — rates a provider profile declares (models[].rates), labelled SourceDeclared. Lowest, so a model any catalog prices is never billed at an operator's estimate.
- longest-prefix match, layer by layer in the same order — handles `gemini-3.1-pro-preview-customtools`-style suffixes.
- (Rates{}, false) — rate unknown. Callers render "$—", never "$0": a self-hosted model nobody priced is unpriced, and nothing here invents a default rate for it.
Where those files live, whether and how often Refresh runs, and how a product's config feeds CfgOverride are the product's (AGENTS.md rule 12). This package takes them as parameters.
The catalog is built once (NewCatalog) and consulted on every cost append; lookups are read-only and lock-free.
Index ¶
- Constants
- func Builtin() map[string]Rates
- func BuiltinByBackend() map[string]Rates
- func BuiltinContextWindow(modelID string) (int, bool)
- func BuiltinContextWindows() map[string]int
- func DeclaredFromProfiles(ps []profile.Profile) map[string]ModelRates
- func SaveUserFile(userHome string, uf *UserFile) error
- type Catalog
- func (c *Catalog) Counts() CountByLayer
- func (c *Catalog) Lookup(modelID string) (Rates, bool)
- func (c *Catalog) LookupFor(backend, modelID string) (Rates, bool)
- func (c *Catalog) LookupForWithSource(backend, modelID string) (Rates, string, bool)
- func (c *Catalog) LookupWithSource(modelID string) (Rates, string, bool)
- type CountByLayer
- type ExternalSource
- type IntroductoryRate
- type LongContext
- type ManualSection
- type ModelRates
- type Options
- type ProjectFile
- type Rates
- func (r Rates) At(promptTokens int) Rates
- func (r Rates) CostUSD(inputTokens, outputTokens int) float64
- func (r Rates) CostUSDWithCache(uncachedInputTokens, cachedInputTokens, outputTokens int) float64
- func (r Rates) CostUSDWithCacheTTLs(...) float64
- func (r Rates) CostUSDWithCacheWrites(uncachedInputTokens, cacheReadTokens, cacheWriteTokens, outputTokens int) float64
- func (r Rates) IsZero() bool
- type RefreshOptions
- type RefreshOutcome
- type UserFile
- type Verdict
Constants ¶
const ( ProjectFileName = "pricing.json" UserFileName = "pricing.json" )
File names. Exposed so callers can reference them in error messages and tests.
const ( SourceCfgOverride = "cfg-override" SourceProjectFile = "project-file" SourceUserManual = "user-manual" SourceUserExternal = "user-external" SourceBuiltin = "builtin" // SourceDeclared is a rate a provider profile declares for a model // (models[].rates): the operator's own price for a model no catalog // covers, such as a self-hosted one. core-agent's spelling, kept. SourceDeclared = "profile-declared" )
Layer source names surfaced via LookupWithSource and the products' pricing views. Stable strings — operators grep for them, docs reference them. Don't rename without a deprecation cycle.
const DefaultMaxRefreshBodyBytes int64 = 32 << 20
DefaultMaxRefreshBodyBytes is RefreshOptions.MaxBodyBytes' default.
The two products disagreed: mast capped at 8 MiB, core-agent at 32 MiB. LiteLLM's catalog measured 1.67 MiB on 2026-08-17 and 2.65 MiB on 2026-10-13: it grows, and a refresh that fails on the cap keeps serving a stale table silently. 32 MiB is still an OOM guard, which is all the cap is for, and a product that wants tighter sets the option.
const DefaultRefreshInterval = 24 * time.Hour
DefaultRefreshInterval is the daily cadence Refresh enforces: if the cache's FetchedAt is younger than this, Refresh is a no-op. 24h is long enough that the network cost is amortized but short enough that rate changes propagate within an operator's next session.
const DefaultRefreshSource = "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"
DefaultRefreshSource is the canonical LiteLLM pricing JSON URL. Community-maintained, MIT-licensed, structured, covers hundreds of models from every major provider. Overridable via RefreshOptions for tests + air-gapped mirrors.
const ExternalFormat = 2
ExternalFormat is the ExternalSource.Format Refresh writes. Format 2 adds the backend-qualified rows litellm.Qualify derives ("anthropic-vertex/claude-…", "vertex/gemini-…", "vertex-maas/…") beside LiteLLM's own keys.
A section in an older format is handled twice over. Loading it derives the qualified rows from the LiteLLM keys it already holds, so its bare rows can't shadow the built-in table's qualified ones even offline. And Refresh treats it as stale: it fetches in full, ignoring the cadence and the cached ETag, because a 304 would otherwise keep the old section until LiteLLM's file next changed upstream.
const SchemaVersion = 1
SchemaVersion is the on-disk schema version for pricing files. Bumped only on incompatible schema changes; new fields with `omitempty` don't require a bump.
Variables ¶
This section is empty.
Functions ¶
func Builtin ¶
Builtin returns a defensive copy of the compiled-in table. Used by tests + by tools that want to inspect what shipped (e.g. a future `/pricing list builtin` view).
Bare model ids only. The backend-qualified rows are a separate table with a separate accessor, because the invariants differ: every entry here is a bare model id a product runs, and so must carry a context window, while a qualified row is a price for a (backend, model) pair and carries none.
func BuiltinByBackend ¶
BuiltinByBackend returns a defensive copy of the compiled-in (backend, model) table, keyed "<backend>/<model>". See LookupFor for the resolution order that consults it.
func BuiltinContextWindow ¶
BuiltinContextWindow returns the compiled-in max input window for modelID and whether one is known. Keys are lowercase, matching the rest of this package's case-insensitive lookup contract; callers holding an operator-typed id should lower it first.
Separate from Rates because a context window is a capability, not a price: operator pricing overrides (.agents/pricing.json, `/pricing set`) must not be able to move a model's window, and a model can be repriced without its window changing.
func BuiltinContextWindows ¶
BuiltinContextWindows returns a defensive copy of the whole window table, for tests and cross-table invariant checks.
func DeclaredFromProfiles ¶
func DeclaredFromProfiles(ps []profile.Profile) map[string]ModelRates
DeclaredFromProfiles collects the rates provider profiles declare (models[].rates) into Options.Declared, keyed "<backend>/<model>" with the profile's backend name (profile.Profile.BackendName). LookupFor reaches them through that qualified key, so two profiles that declare different prices for the same model id each get their own, and a declaration never prices the same id on another backend. Pass expanded profiles (profile.Expand), so a profile that inherits its rates through extends declares them too.
A later profile overrides an earlier one with the same backend and model. Within one profile, the first model entry with rates wins when an id repeats, as profile.Profile.RatesFor reads it.
func SaveUserFile ¶
SaveUserFile writes uf to <userHome>/<UserFileName> atomically. Used by the LiteLLM refresher and the /pricing set slash. Empty userHome is an error (caller should resolve a real home first).
Types ¶
type Catalog ¶
type Catalog struct {
// contains filtered or unexported fields
}
Catalog is the merged view of all pricing sources, queried by model name. Construct with NewCatalog; consult with Lookup.
Layers are stored separately so the daily LiteLLM refresh can rewrite the external slice without touching the others, and so the precedence chain stays explicit (no "where did this rate come from" mystery).
func NewCatalog ¶
NewCatalog reads every configured source and returns the merged catalog. Missing files are not errors (the common case); only I/O failures and malformed JSON return non-nil.
The returned catalog is read-only after construction. Callers that want a refresh (after Refresh, or a config reload) build a new Catalog and swap atomically via the consumer's chosen pointer-store mechanism.
func (*Catalog) Counts ¶
func (c *Catalog) Counts() CountByLayer
Counts returns per-layer entry counts.
func (*Catalog) Lookup ¶
Lookup returns the resolved rates for modelID plus a found flag. !found means the caller should treat the cost as unknown ($—) rather than zero.
Lookup is LookupFor with no backend; see LookupForWithSource for the resolution order.
func (*Catalog) LookupFor ¶
LookupFor returns the rates for a model as served by a named backend — a core-models backend name, as profile.Profile.BackendName and usage.Detail.Backend report it: "anthropic", "anthropic-vertex", "gemini", "vertex", "vertex-maas", "openai", "xai", or a profile's own — and is the lookup a caller should prefer whenever it knows which backend will bill the tokens.
A rate is a property of the (backend, model) pair. The same claude-opus-5 is reachable first-party and through Vertex; the same gemini-3.7-flash through the Developer API and through Vertex. Lookup cannot tell those apart, and the bare table it consults is itself a mixture — LiteLLM's unprefixed claude-* rates are first-party Anthropic while its unprefixed gemini-* rates are Vertex.
Both the "<backend>/<model>" key and the bare model id are consulted, layer by layer, so an operator's bare override still beats a builtin qualified row; see LookupForWithSource for the exact order. The bare fallback is what makes this safe to call everywhere — a pair no table qualifies resolves exactly as Lookup would, rather than reporting the model unpriced and dropping the session's cost to zero.
An empty backend is exactly Lookup, which is the right behavior for an offline fake or a model whose backend could not be resolved.
func (*Catalog) LookupForWithSource ¶
LookupForWithSource is LookupFor plus the layer that served the rate. See LookupWithSource for the source-name vocabulary.
Resolution runs in two phases, and the first hit wins:
- Exact. For each layer in precedence order (cfg-override, project-file, user-manual, user-external, builtin, declared): the qualified key "<backend>/<model>", then the bare model id.
- Prefix, only when no layer holds an exact key. For each layer in the same order: the longest key that prefixes the qualified key (and is longer than "<backend>/"), then the longest key that prefixes the bare id.
Layer precedence therefore outranks qualification: an operator's bare override beats a builtin qualified row, because the operator wrote it and the builtin did not. Exactness outranks both: a dated id with its own bare row is never billed at a qualified family prefix. Within one layer, qualified beats bare at the same phase, which is the point of a qualified key.
An isolated backend (a built-in or Options.Profiles profile with pricing "isolated", such as sap-ai-core) skips every bare key, in every layer and both phases: only "<backend>/<model>" rows price it. A gateway that bills in its own units would otherwise be billed at the first-party list price of every model it also offers.
func (*Catalog) LookupWithSource ¶
LookupWithSource is Lookup + the name of the catalog layer that served the rate (SourceCfgOverride / SourceProjectFile / SourceUserManual / SourceUserExternal / SourceBuiltin / SourceDeclared). Empty source string when !ok. Products show it so operators can spot stale builtin rates that should have been overridden by a fresh LiteLLM refresh but weren't — the visibility that #259 asked for.
Resolution is LookupForWithSource's with no qualified key: an exact match by layer precedence, then the longest prefix within the first layer that holds one.
type CountByLayer ¶
type CountByLayer struct {
CfgOverride int
ProjectFile int
UserManual int
UserExternal int
Builtin int
Declared int
}
CountByLayer reports how many model entries each layer holds. Surfaced by the products' pricing views and useful for tests that want to assert the expected number of rows landed in each layer.
type ExternalSource ¶
type ExternalSource struct {
// Format is the shape of Models: ExternalFormat for a section
// Refresh writes now, zero for one written before backend-qualified
// rows existed (v0.13.1 and earlier). See ExternalFormat.
Format int `json:"format,omitempty"`
FetchedAt time.Time `json:"fetched_at"`
Source string `json:"source"` // canonical URL the data was pulled from
ETag string `json:"etag,omitempty"`
Models map[string]ModelRates `json:"models,omitempty"`
}
ExternalSource is the auto-fetched section, populated by Refresh from LiteLLM's catalog. The fetched_at + etag fields drive cache-validity logic (skip refresh if <24h, send If-None-Match on revalidation).
type IntroductoryRate ¶ added in v0.12.0
type IntroductoryRate struct {
Model string
// NotBefore is the first instant at which still carrying the
// introductory rate is a defect: the day after the published expiry,
// since "expires the 31st" doesn't say which hour.
NotBefore time.Time
// IntroInputPerMTok and IntroOutputPerMTok are the promotional rates.
// The check fires only while a row carries exactly these numbers.
IntroInputPerMTok, IntroOutputPerMTok float64
// Standard is what to expect after the expiry, for the message.
Standard string
// Why it matters, for the message.
Why string
// Resolved, when set, records that the increase won't happen: the
// check inverts and pins the row AT the former promotional rate. The
// text says what was checked, and where.
Resolved string
}
IntroductoryRate is a promotional rate with a known end date, which the builtin table cannot express: LiteLLM's catalog carries no expiry on any row, so a promotional number keeps being re-read, freshly stamped, after it has become false. Rates.UpdatedAt measures freshness, not truth. This list is where the date lives, and Check turns it into a test failure.
Both directions happen. A rate can double on schedule (gemini-3.x-flash on 2027-01-01) or the increase can be cancelled and the promotional number made standard (claude-sonnet-5, 2026-09-01), so an entry either expires or is Resolved.
func IntroductoryRates ¶ added in v0.12.0
func IntroductoryRates() []IntroductoryRate
IntroductoryRates returns the known promotional rates. Products run Check over each, against the rows they actually bill from (their backend-qualified rows too), so their own builds go red when a rate they depend on lapses, whatever core-models version they pin.
func (IntroductoryRate) Check ¶ added in v0.12.0
func (ir IntroductoryRate) Check(r Rates, now time.Time) Verdict
Check judges one row against the entry at time now. Its outcomes:
- Resolved, row still at the former promotional rate: fine.
- Resolved, row moved: Fail (the provider repriced again, or the table is wrong).
- Before NotBefore: fine; a note if the row moved early.
- After NotBefore, row moved: fine; a note that the entry is spent.
- After NotBefore, row unchanged: Fail.
type LongContext ¶
type LongContext struct {
ThresholdTokens int `json:"threshold_tokens"`
InputPerMTok float64 `json:"input_per_mtok,omitempty"`
CachedInputPerMTok float64 `json:"cached_input_per_mtok,omitempty"`
CacheCreationInputPerMTok float64 `json:"cache_creation_input_per_mtok,omitempty"`
CacheCreation1hInputPerMTok float64 `json:"cache_creation_1h_input_per_mtok,omitempty"`
OutputPerMTok float64 `json:"output_per_mtok,omitempty"`
}
LongContext is a model's above-threshold price. LiteLLM publishes absolute rates per bucket ("input_cost_per_token_above_272k_tokens", …), not a multiplier, and the buckets don't move together — gemini-2.5-pro's cache reads double above 200K while its output goes 1.5x — so the rates are carried as published.
A call is long when its whole prompt (uncached + cache reads + cache writes) is above ThresholdTokens; then every token of it, output included, bills at these rates, which is how the providers charge.
type ManualSection ¶
type ManualSection struct {
Models map[string]ModelRates `json:"models,omitempty"`
}
ManualSection is the operator-curated section. Round-trips intact across refreshes (the fetcher only rewrites External).
type ModelRates ¶
type ModelRates struct {
InputPerMTok float64 `json:"input_per_mtok,omitempty"`
CachedInputPerMTok float64 `json:"cached_input_per_mtok,omitempty"`
CacheCreationInputPerMTok float64 `json:"cache_creation_input_per_mtok,omitempty"`
// CacheCreation1hInputPerMTok is the 1-hour breakpoint's write rate,
// spelled as core-agent's pricing files spell it.
CacheCreation1hInputPerMTok float64 `json:"cache_creation_1h_input_per_mtok,omitempty"`
OutputPerMTok float64 `json:"output_per_mtok,omitempty"`
UpdatedAt time.Time `json:"updated_at,omitempty"`
// LongContext is the above-threshold price, as Rates.LongContext.
LongContext *LongContext `json:"long_context,omitempty"`
}
ModelRates is the on-disk per-model rate, mirroring config's PricingConfig field names so operator-edited files use the same spelling regardless of whether they live in pricing.json or in .agents/config.json's `model.pricing` override map.
func (ModelRates) Rates ¶
func (m ModelRates) Rates() Rates
Rates converts the JSON-tagged form into the Rates type. Field order + names are identical so a direct conversion suffices. Exported because embedders that build a UserFile/ProjectFile in memory (rather than on disk) need a way to feed those rows into Options without re-declaring the struct.
type Options ¶
type Options struct {
// CfgOverride is the operator's per-model override from the
// product's own config (core-agent's `model.pricing`). Highest
// precedence.
CfgOverride map[string]ModelRates
// AgentsDir is the product's project directory (both use .agents/;
// empty when there is none). Catalog construction reads
// <AgentsDir>/pricing.json if present.
AgentsDir string
// UserHome is the product's per-user state directory (~/.mast,
// ~/.core-agent). Catalog construction reads <UserHome>/pricing.json
// if present (both manual + external sections).
UserHome string
// Declared is the per-model rates provider profiles declare
// (models[].rates): an operator's price for a model no published
// catalog covers, such as a self-hosted one. Lowest precedence —
// every other layer, builtin included, outranks it, so a model a
// catalog does price is never billed at a guess. Build it with
// DeclaredFromProfiles, which keys each row "<backend>/<model>" so
// LookupFor scopes it to the profile that declared it.
Declared map[string]ModelRates
// Profiles are the provider profiles the product declares. NewCatalog
// expands each (profile.Expand) and reads two things from it: the
// rates it declares (models[].rates), added to Declared as
// DeclaredFromProfiles would; and its pricing mode. A backend whose
// profile sets pricing "isolated" is priced only from rows keyed
// "<backend>/<model>", never from a model's bare row: a gateway that
// bills in its own units. The built-in profiles' modes apply without
// being passed (sap-ai-core is isolated); a profile passed here that
// keeps a built-in's backend and sets pricing "shared" undoes it. Two
// profiles on one backend that disagree are an error, since the
// catalog prices by backend and cannot tell them apart.
Profiles []profile.Profile
}
Options bundle the inputs NewCatalog needs to assemble the layered view. All fields are optional — an empty Options yields a catalog with only the compiled-in builtin layer (useful for tests, and for a product that wires none of the files).
type ProjectFile ¶
type ProjectFile struct {
Version int `json:"version"`
Models map[string]ModelRates `json:"models,omitempty"`
}
ProjectFile is the <AgentsDir>/pricing.json shape — flat models map. Project files are always operator-curated (never auto-fetched), so no manual/external split is needed.
func LoadProjectFile ¶
func LoadProjectFile(agentsDir string) (*ProjectFile, error)
LoadProjectFile reads .agents/pricing.json from agentsDir, or returns an empty file when the file is missing (a missing project file is the common case, not an error). Returns an error for I/O failures or malformed JSON.
type Rates ¶
type Rates struct {
InputPerMTok float64
CachedInputPerMTok float64
CacheCreationInputPerMTok float64
// CacheCreation1hInputPerMTok is the write rate for a 1-hour
// breakpoint (2x base input on Anthropic, against 1.25x for the
// 5-minute one CacheCreationInputPerMTok holds). Zero when the
// source publishes none; see CostUSDWithCacheTTLs for the fallback.
CacheCreation1hInputPerMTok float64
OutputPerMTok float64
UpdatedAt time.Time
// LongContext, when set, replaces these rates for a call whose prompt
// exceeds its threshold: OpenAI's 272K cliff, Gemini's 200K, some
// Claude models' 100K. Nil for a model with one price at every size.
LongContext *LongContext
}
Rates is the per-million-token cost for one model. CachedInputPerMTok is the rate applied to input tokens served from the provider's prompt cache (Gemini's `cachedContentTokenCount`, Anthropic's `cache_read_input_tokens`); a zero value means the cache-read rate isn't known and callers should bill cached tokens at InputPerMTok.
CacheCreationInputPerMTok is the rate for input tokens that WRITE a cache entry — Anthropic's `cache_creation_input_tokens`, billed at a premium over base input rather than a discount. It holds the 5-minute-TTL rate (1.25x base input), LiteLLM's cache_creation_input_token_cost. Gemini has no equivalent bucket: its explicit caches bill storage per hour, not per written token, so the field stays zero for Gemini rows. A zero value means the cache-write rate isn't known and callers bill written tokens at InputPerMTok, which UNDERCOUNTS, so the builtin table is kept populated (go-steer/core-agent#263).
CacheCreation1hInputPerMTok is the same bucket at Anthropic's 1-hour breakpoint TTL, which bills 2x base input rather than 1.25x — LiteLLM's cache_creation_input_token_cost_above_1hr. Two rates rather than one because a request may mix both TTLs and the response reports them separately, so there is a right answer to bill. Zero means the model publishes no 1-hour rate, and CostUSDWithCacheTTLs falls back to CacheCreationInputPerMTok (go-steer/core-agent#770).
UpdatedAt records when the rate was last verified against its source (LiteLLM refresh time, generator run time for builtin entries, operator edit time for manual overrides). Zero when unknown. Surfaced through the products' pricing views so operators can spot stale entries at a glance — issue #259 called out that hand-authored rates drift silently, and staleness visibility is the mitigation baked into the "regenerate builtin from LiteLLM" workflow that followed.
func (Rates) At ¶
At returns the rates that bill a call whose prompt is promptTokens long: r itself, or r with its LongContext rates in force when the prompt is above the threshold. The result has no LongContext of its own.
A bucket the long-context row leaves at zero is not free: it falls back to the base rate scaled by the input multiplier (long input over base input), so an unpublished above-threshold cache rate tracks the published input increase instead of silently staying at the cheaper base price. Output falls back to the base output rate unscaled, since output and input move independently.
func (Rates) CostUSD ¶
CostUSD returns the dollar cost of (input, output) tokens at r. Treats every input token as uncached — see CostUSDWithCache for the cached-vs-uncached split.
func (Rates) CostUSDWithCache ¶
CostUSDWithCache returns the dollar cost with cache-hit tokens billed at CachedInputPerMTok. When CachedInputPerMTok is zero (rate unknown) cached tokens fall back to InputPerMTok — no silent free-riding.
Providers that also report cache-WRITE tokens should call CostUSDWithCacheWrites instead; this signature folds them into the uncached bucket, which undercounts (go-steer/core-agent#263).
func (Rates) CostUSDWithCacheTTLs ¶
func (r Rates) CostUSDWithCacheTTLs(uncachedInputTokens, cacheReadTokens, cacheWriteTokens, cacheWrite1hTokens, outputTokens int) float64
CostUSDWithCacheTTLs is CostUSDWithCacheWrites with the write bucket split by breakpoint TTL. cacheWrite1hTokens is a SUBSET of cacheWriteTokens — the share Anthropic reports under usage.cache_creation.ephemeral_1h_input_tokens — and is billed at CacheCreation1hInputPerMTok; the remainder goes at the 5-minute rate.
A 1-hour rate of zero falls back to the 5-minute rate rather than to base input, as core-agent's pricing does (core-agent #770): a model that publishes one write rate and not the other is far likelier to be missing a catalog field than to bill 1h writes at base, so the nearer neighbour is the better estimate.
func (Rates) CostUSDWithCacheWrites ¶
func (r Rates) CostUSDWithCacheWrites(uncachedInputTokens, cacheReadTokens, cacheWriteTokens, outputTokens int) float64
CostUSDWithCacheWrites is CostUSDWithCache plus the cache-write bucket: tokens that created a cache entry this turn, billed at CacheCreationInputPerMTok.
The three input buckets are mutually exclusive and must not overlap — pass uncached = total prompt - cache reads - cache writes. Unknown rates fall back to InputPerMTok for both cache buckets rather than to zero, so a missing catalog entry degrades to the old (understated) number instead of billing cached or written tokens as free.
Every write token is billed at the 5-minute rate. Callers that know how the write bucket split across TTLs should use CostUSDWithCacheTTLs; this signature has no bucket for the 1-hour share, so it understates a 1h write by the gap between the two rates.
func (Rates) IsZero ¶
IsZero reports whether the rates carry no useful pricing. Used by callers to distinguish "free model" from "rate unknown" — only the latter should render "$—". CachedInputPerMTok isn't part of this check: a row that carries only a cache rate but no base input/output rates is still "unpriced" in the useful sense.
type RefreshOptions ¶
type RefreshOptions struct {
// Source is the URL to fetch. Defaults to DefaultRefreshSource.
Source string
// Client is the HTTP client used. Defaults to a fresh one with
// a 30s timeout — long enough for slow links, short enough that
// startup doesn't hang forever.
Client *http.Client
// MinInterval is the minimum age the cache must reach before
// Refresh actually fetches. Zero defaults to
// DefaultRefreshInterval (24h). Set to a negative duration to
// force a fetch regardless of cache age (used by the
// /pricing refresh slash command).
MinInterval time.Duration
// Now is the clock Refresh uses to compute cache age. Tests
// override to deterministically drive stale-vs-fresh. Defaults
// to time.Now.
Now func() time.Time
// MaxBodyBytes caps how much of the response Refresh buffers; a
// larger body is a fetch failure and the cache is kept. Zero means
// DefaultMaxRefreshBodyBytes.
MaxBodyBytes int64
}
RefreshOptions controls the refresh fetch. All fields optional; defaults match a production launch (LiteLLM upstream, 24h cadence, no max-size cap, 30s timeout). Tests + air-gapped mirrors override Source; integration tests override Now for deterministic stale-vs-fresh decisions.
type RefreshOutcome ¶
type RefreshOutcome struct {
Skipped bool // true when the cache was fresh enough to skip the fetch
NotModified bool // true when the server replied 304 (cache still authoritative)
NetworkFailed bool // true when the fetch itself failed; cache was preserved
NetworkError error // populated when NetworkFailed
StaleAge time.Duration // age of the cache at decision time; zero for fresh writes
ModelCount int // number of rows written to the external section, backend-qualified rows included (zero on Skipped/NotModified)
FetchedAt time.Time // timestamp stored in the cache file (or pre-existing one for Skipped)
}
RefreshOutcome is the result of a Refresh call. Surfaced so the caller (daemon startup + a future /pricing refresh surface) can render a meaningful one-liner: "Refreshed 247 models from LiteLLM" / "Cache is 4h old; skipped" / "Using 6-day-old cache; network unreachable: connection refused".
func Refresh ¶
func Refresh(ctx context.Context, userHome string, opts RefreshOptions) (RefreshOutcome, error)
Refresh fetches the LiteLLM pricing table and writes it into the `external` section of <userHome>/pricing.json, preserving the `manual` section operators curate by hand. Daily cadence is enforced via MinInterval — if the cache is younger, the fetch is skipped.
Network failures are non-fatal: existing cached data stays in place, the caller gets a populated RefreshOutcome.NetworkError, and the surfaced one-liner ("using N-day-old cache") lets the operator know the rates may be stale without breaking the session. Same fall-through for HTTP 5xx / malformed bodies. An existing pricing.json that cannot be read or parsed is the exception: Refresh returns the error and writes nothing.
userHome must resolve to a writable directory (typically ~/.mast or ~/.core-agent). Empty userHome is an error.
type UserFile ¶
type UserFile struct {
Version int `json:"version"`
External *ExternalSource `json:"external,omitempty"`
Manual *ManualSection `json:"manual,omitempty"`
}
UserFile is the <UserHome>/pricing.json shape — sectioned so the LiteLLM refresh can overwrite the `external` section without touching `manual` entries the operator hand-edited (or set via /pricing set).
func LoadUserFile ¶
LoadUserFile reads <userHome>/pricing.json (or whatever userHome resolves to), or returns an empty file when missing. Same not-an-error treatment for missing files as LoadProjectFile.
type Verdict ¶ added in v0.12.0
type Verdict struct {
// Fail is true when the row is wrong: past expiry and still on the
// promotional rate, or a Resolved entry whose row has moved.
Fail bool
// Message explains the outcome, failing or not; empty when there is
// nothing to say.
Message string
}
Verdict is the outcome of IntroductoryRate.Check.