Documentation
¶
Overview ¶
Package llmtier gives the learning pipelines their model access: an explicit Cheap/Strong generator pair (plan §cost control) behind the multi-backend llmgen seam, gated by a persistent daily spend ledger. Every pipeline call goes through Tier.Generate, so no learning code can bypass the caps.
Index ¶
Constants ¶
This section is empty.
Variables ¶
ErrBackendUnavailable reports the learning backend could not be reached in a way retrying this run won't fix: the claude CLI is logged out, or the API key is missing/invalid. Workers stop the run and keep jobs queued (like ErrCapped) — and, crucially, Generate does NOT fold such a failure into the daily cap: it spent nothing and every queued job would fail identically, so counting it would let a logged-out CLI burn the whole daily_call_cap and block learning even after the user logs back in.
var ErrCapped = errors.New("llmtier: daily learning cap reached")
ErrCapped reports today's spend cap is reached. Workers treat it as "stop now, keep everything queued" — the queue drains after midnight UTC.
Functions ¶
Types ¶
type DaySpend ¶
type DaySpend struct {
Calls int `json:"calls"`
Prompt int `json:"prompt_tokens"`
Completion int `json:"completion_tokens"`
USD float64 `json:"usd"`
}
DaySpend is one UTC day's usage bucket.
type Ledger ¶
type Ledger struct {
Days map[string]DaySpend `json:"days"`
// contains filtered or unexported fields
}
Ledger is the persistent daily spend record (state/spend.json). Its methods are safe for concurrent use so a --no-cap parallel drain can record spend from several mining goroutines at once.
func LoadLedger ¶
LoadLedger reads the ledger; missing or corrupt files start empty (worst case today's caps reset — bounded by one day's caps, never unbounded).
func (*Ledger) Allow ¶
Allow returns ErrCapped (wrapped with the reason) when today's bucket is at either cap. Caps ≤ 0 mean "no cap of that kind".
type Tier ¶
type Tier struct {
Cheap llmgen.Generator
Strong llmgen.Generator
// contains filtered or unexported fields
}
Tier is the resolved Cheap/Strong pair plus its spend gate.
func NewTier ¶
func NewTier(cheap, strong llmgen.Generator, stateDir string, usdCap float64, callCap int, priced bool) *Tier
NewTier assembles a tier directly. Resolve is the config-driven path; this exists for tests and pipelines with custom backends.
func Resolve ¶
Resolve builds the tier from config, mirroring the import backend policy: "auto" prefers the strongest thing the user already has and never errors — a nil Tier with a reason means learning quietly cannot run. Explicit providers DO error when their prerequisite is missing.
func (*Tier) Generate ¶
func (t *Tier) Generate(ctx context.Context, strong bool, system, user, name string, schema map[string]any, out any) (llmgen.Usage, error)
Generate runs one capped structured call on the chosen tier. Usage is recorded even for failed calls — the user paid for every attempt — EXCEPT deterministic backend-unavailable failures (auth/config), which spent nothing and would otherwise let identical retries drain the cap (see isBackendUnavailable / ErrBackendUnavailable). A cap hit returns ErrCapped before any model call; the caller leaves its work queued.
Source Files