budget

package
v0.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 30, 2026 License: Apache-2.0 Imports: 7 Imported by: 0

Documentation

Overview

Package budget meters model usage against workload budget ceilings.

Spike-2 probe for the "where does cost accounting come from" question (docs/orchestration-design.md budget composition): ADK v2 carries genai UsageMetadata on every model event (session.Event embeds model.LLMResponse), so a meter over the runner's event stream sees token counts per call with no ADK patching. What ADK does NOT provide is pricing or enforcement — both are mast-side. This package is the minimal mast-side shape: per-session cumulative token/cost meter, checked as events stream; the caller aborts the run when Observe reports the ceiling is crossed.

Scopes: per-specialist ceilings under the session's

A workload budget bounds the session; a specialist's own budget bounds that specialist. Config.Scopes composes the two by attributing each usage event to the agent that authored it — session.Event.Author is the agent's name on every dispatch shape mast builds, which is what makes one attribution rule enough for all of them.

It does not follow that one seam is enough to SEE them. A coordinator's sub-agent tool and a workflow-graph node funnel their events up the root runner's stream, so observing that stream catches both. A planner's invoke_specialist does not: it runs the specialist on a private runner, whose events reach the host only through planner.SubRunObserver (#226). A host that meters must feed this package from both — the arithmetic is identical either way, and the Author on a sub-run event is still the specialist's name, so a declared ceiling binds on the planner's door exactly as it does on a coordinator's.

A scope carries its own ceilings and, when the specialist declares a `model:` override, its own price, so a cheap analyst's tokens are not billed at the synthesizer's rate.

Composition is tightest-cap-wins by construction rather than by arithmetic: every event is checked against its scope and against the session, and whichever ceiling is reached first stops what it holds. A scope's ceiling is reported ahead of the session's on the event that crosses both, because the specialist is the more specific fact and the workload's cap would have been crossed on a later call anyway.

Who a ceiling stops

A ceiling holder is either the workload or one specialist, and since v0.6 the two have different consequences. Scope(err) reports which an enforcement error belonged to: a scoped one closes that specialist's path, and the turn drivers log it, count it, and route on; an unscoped one ends the turn. Through v0.5 both ended the turn, which was never a decision — Observe runs on the event stream, outside the specialist's own run, where the run context was the only lever available. Allow is the lever that was missing, and it is per-call and per-agent (see allow.go).

The counting follows the same split. Refusals reports every refusal, for the metric and the log line; SessionRefusals reports only the workload's own, for the driver deciding whether to stop. Each keeps its own first reason on purpose: the first refusal of a turn is usually a specialist's, and a driver that stopped on the session's cap while quoting a specialist's reason would send an operator to raise the wrong ceiling.

Known limitations (findings, not TODOs)

Metering at the event stream is enforcement-after-the-call — a single runaway call is only priced once its usage event lands. This meter stays the ledger; Allow is asked in front of it, and refuses only where the arithmetic is a proof rather than an estimate, so the two disagree by at most one call.

Index

Constants

View Source
const (
	DimensionTurns   = "turns"
	DimensionTokens  = "tokens"
	DimensionCostUSD = "cost_usd"
)

Budget dimensions, as reported on Trip.Dimension. One per ceiling in Limits; a session can be past more than one at once.

View Source
const DetailKey = "mast.usage_detail"

DetailKey is the model.LLMResponse.CustomMetadata key a provider adapter attaches its usage sidecar under. The meter reads the value there when it implements Detailer and ignores it otherwise.

The key is stable and namespaced because the sidecar rides on an ADK-owned map that anything in the process may write to.

Variables

View Source
var ErrExceeded = errors.New("budget exceeded")

ErrExceeded is returned by Observe once cumulative usage crosses a ceiling. Callers should abort the run — unless Scope reports the ceiling was one specialist's, in which case one path is closed and the run can continue without it.

View Source
var ErrRefused = errors.New("budget refused the call")

ErrRefused is returned by Allow when a ceiling makes the requested call impossible. It is distinct from ErrExceeded on purpose: the two describe different events, and a caller that conflated them would report money spent that was not.

ErrExceeded means a call happened and pushed the meter past a cap. ErrRefused means a call did not happen, because it could not have finished under one.

View Source
var ErrRestored = errors.New("budget: meter has already been restored")

ErrRestored is returned by a second Restore on the same meter. Prior spend is additive, so folding it twice doubles it — and the sessions this matters for are the ones already near a ceiling, where the double count is the difference between running and refused.

Functions

func Scope added in v0.6.0

func Scope(err error) (string, bool)

Scope reports the specialist whose own ceiling produced err, and false for a session-level ceiling or any error that is not one of ours.

The distinction is the whole of W10.3: a specialist that is out of budget is one path closed, and the coordinator can try another; a workload that is out of budget has nothing left to try. A caller that cannot tell them apart has to assume the harsher one, which is what mast did through v0.5.

Types

type Buckets added in v0.9.0

type Buckets struct {
	// CacheReadTokens is the prompt subset served from cache —
	// Anthropic's cache_read_input_tokens, Gemini's
	// cachedContentTokenCount. Also reachable through
	// UsageMetadata.CachedContentTokenCount, so a nil here is not a
	// gap; the field exists so a provider whose genai projection is
	// lossy can correct it.
	CacheReadTokens *int64

	// CacheWriteTokens is the prompt subset that created a cache entry —
	// Anthropic's cache_creation_input_tokens. genai's usage metadata
	// has nowhere to carry this, which is why the sidecar exists: folded
	// into the uncached bucket it is billed at 1x instead of 1.25x and
	// every cache-warming turn is undercounted (#352).
	CacheWriteTokens *int64

	// CacheWrite1hTokens is the SUBSET of CacheWriteTokens written with
	// a 1-hour breakpoint — Anthropic's
	// usage.cache_creation.ephemeral_1h_input_tokens. Those bill at 2x
	// base input against the 5-minute 1.25x, so a meter that folded them
	// into the write bucket would understate every 1h turn. Nil when the
	// provider did not split the bucket, which reads as "all 5-minute".
	CacheWrite1hTokens *int64
}

Buckets is a provider's own statement of the token counts behind one call, for the counts genai's UsageMetadata cannot carry.

Every count is a pointer because nil means the provider did not say, which is not the same as zero: an Anthropic turn that wrote no cache entry reports cache_creation_input_tokens = 0, while a provider that has no such concept reports nothing at all, and billing those two the same way is how an undercount reports success.

A stated count wins over the genai projection of the same bucket. Absent fields fall back to the genai fields, so an event with no sidecar prices exactly as it did before this type existed.

type Call added in v0.8.0

type Call struct {
	// UncachedInputTokens is prompt tokens billed at the full input
	// rate — the prompt less whatever the provider served from cache.
	UncachedInputTokens int

	// CachedInputTokens is the prompt subset served from cache, billed
	// at the cache-read rate.
	CachedInputTokens int

	// CacheWriteTokens is the prompt subset that CREATED a cache entry
	// this call, billed at the cache-write rate — a premium over fresh
	// input (Anthropic's 5-minute TTL is 1.25x), not a discount. Zero
	// for a provider that has no such bucket: Gemini's explicit caches
	// bill storage per hour rather than per written token.
	//
	// The three input buckets are mutually exclusive and sum to the
	// prompt.
	CacheWriteTokens int

	// CacheWrite1hTokens is the subset of CacheWriteTokens written with
	// a 1-hour breakpoint, billed at the 1-hour write rate; the rest of
	// CacheWriteTokens bills at the 5-minute rate. A subset, not a fourth
	// bucket, so the three above still sum to the prompt.
	CacheWrite1hTokens int

	// OutputTokens is everything the model generated, billed at the
	// output rate. On a reasoning model this includes thinking tokens.
	OutputTokens int
}

Call is one model call's billable token counts, already normalized out of the provider's usage record.

A struct rather than three int parameters because the buckets are the part expected to grow: reasoning tokens billed apart from output, and cache-*write* tokens billed apart from cache reads, are both on model-support-design's list, and the pricing Rates said 1h-TTL cache support "means adding a second rate here" (it did: CacheWrite1hTokens). Each of those arrives as a new field, which a pricer that does not know about it ignores.

type Config added in v0.3.0

type Config struct {
	// Limits are the session-wide ceilings (the workload budget).
	Limits Limits

	// Scopes are per-agent ceilings and prices, keyed by the agent name
	// that authors the event — for a specialist, its spec name. An
	// agent with no scope is metered into the session totals only.
	Scopes map[string]Limits

	// OnSpend, when set, is called once per priced call with what that
	// call added — the write half of the durability seam described in
	// durable.go. It runs on the caller's goroutine, outside the meter's
	// lock, after the fold and before Observe returns, including when
	// the fold reported a crossed ceiling: the call happened and the
	// money is spent whether or not it was the one that stopped the run.
	//
	// It must not call back into the meter (Snapshot and friends take
	// the same lock the fold just released, so a re-entrant caller would
	// read a different meter than the one it was told about, and a
	// re-entrant Observe would recurse). Keep it to handing the Spend
	// somewhere durable.
	OnSpend func(Spend)

	// FinalReport lets an agent that has already spent something buy one
	// model call past its ceiling, to write the report it was stopped
	// before finishing. Off by default: a cap that can be overshot by one
	// call is not what every operator declared. See finalreport.go for
	// what the grant is and the three bounds on it.
	FinalReport bool
}

Config is the full meter shape: the session's ceilings plus the per-agent scopes composed under them.

type Detailer added in v0.9.0

type Detailer interface {
	UsageBuckets() Buckets
}

Detailer is the sidecar contract: what a provider adapter attaches under DetailKey so the meter can see buckets genai's usage metadata has no field for. mast's implementation is internal/providers/usage.Detail.

An interface rather than a named struct type because pkg/budget imports nothing else in this module — a provider package naming a budget type is the right direction for that dependency, and the reverse would drag an unsupported package into the v1.0 freeze through a supported one (#338).

The method returns a struct for the same reason Call is one: the buckets are the part expected to grow, and a new field is additive where a new method is not.

type Limits

type Limits struct {
	MaxCostUSD float64
	MaxTokens  int64

	// MaxTurns caps the number of model calls in the session.
	//
	// Vocabulary: mast counts one "turn" per model call — the same
	// unit as the meter's calls counter (one streamed event carrying
	// UsageMetadata). This matches docs/orchestration-design.md's
	// "budget.max_turns remains mast-side turn counting (ADK has no
	// turn cap)": a Task specialist that loops through five model
	// calls before finish_task has spent five turns, not one.
	MaxTurns int

	// Pricer prices each model call exactly, against the per-model
	// input / cache-read / output rates rather than one blended number.
	//
	// Optional, and strictly better than RatePer1K where a caller can
	// supply it. The flat rate exists because this meter originally saw
	// only UsageMetadata.TotalTokenCount, so internal/compose derives it
	// as the plain average of a model's input and output rates — an
	// approximation that "overcharges input-heavy sessions and
	// undercharges output-heavy ones". That premise stopped being true
	// once the event carried the input/output split and the cache-read
	// subset: the call can be priced against the same core-models pricing catalog
	// everything else uses. The error is not small on a real agent — an
	// input-heavy, cache-warm session measured here ran 5.9x over its
	// flat-rate figure, and a cost ceiling that wrong is a ceiling that
	// fires on the wrong sessions.
	//
	// Unknown models fall through to RatePer1K, so a pricing miss never
	// silently drops a session's cost to zero. Unpriced counts them.
	//
	// On a scope, nil means "inherit the session's pricer", matching
	// Backend, Model and RatePer1K below.
	Pricer Pricer

	// Backend and Model are the (backend, model) pair the pricer is
	// keyed by, resolved by the caller. Ignored unless Pricer is set.
	//
	// This is what the field above used to say it owed, paid. A rate is
	// a property of the pair and not of the model — mast's bare table has
	// Claude rows that are first-party and Gemini rows that are Vertex's
	// — so the pricer is asked for the pair, whose backend half comes
	// from internal/compose.Backend and is not on the event. An empty
	// Backend asks for the bare id, which is right for an offline fake or
	// a backend that could not be resolved.
	//
	// Model is what that comment expected to read off ModelVersion, and
	// it has to be carried for a second reason the comment did not know:
	// the event frequently does not name the model at all. ADK's
	// streaming aggregator rebuilds the final response — the only one
	// carrying UsageMetadata — without ModelVersion
	// (internal/llminternal/stream_aggregator.go, adk/v2 v2.2.0), so on
	// a streaming Gemini run every priced call reaches the pricer with
	// an empty key. Its database session service has no column for the
	// field either, which is one of the reasons durable.go keeps a spend
	// ledger rather than replaying events.
	//
	// The event is still asked first, because a server echo names the
	// model that was actually billed and can be more specific than the
	// id the run was started with; an echo the pricer cannot resolve
	// falls through to Model rather than ending the search. Without this
	// pair a configured Pricer would miss on every Gemini call and
	// quietly serve the flat rate it was configured to replace.
	//
	// Model is the name internal/compose already resolved —
	// SpecModelName for a specialist, the run's model for the session.
	Backend string
	Model   string

	// RatePer1K is the flat USD price per 1K total tokens (spike
	// pricing model), and the fallback for a call Pricer cannot price.
	//
	// On a scope, zero means "inherit the session's rate" — the right
	// default for a specialist that declares no model of its own, and
	// the reason an un-tiered roster prices exactly as it did before
	// scopes existed.
	RatePer1K float64
}

Limits are the ceilings for one session. Zero values mean unlimited.

func (Limits) IsZero added in v0.8.0

func (l Limits) IsZero() bool

IsZero reports whether l declares nothing at all — no ceiling and no price. Scope composition uses it to drop a specialist that asked for neither.

It exists because Limits stopped being safely comparable when Pricer became an interface: == on two Limits that both carry a pricer whose dynamic type is uncomparable — a func, a map-backed table, a mock with a recorded call log — panics at runtime. The two call sites this replaced compared against the zero Limits, which is the one shape that cannot panic (the dynamic types differ, so the comparison short-circuits), so nothing was broken. But the operator being usable only against one specific operand is not a property anyone should have to know, and "declares nothing" is what those call sites were asking anyway.

type Meter

type Meter struct {
	// contains filtered or unexported fields
}

Meter accumulates usage for one session, and for each scoped agent within it.

func New added in v0.3.0

func New(cfg Config) *Meter

New constructs a Meter from a full config.

func NewMeter

func NewMeter(limits Limits) *Meter

NewMeter constructs a Meter with the given session limits and no per-agent scopes.

func (*Meter) Allow added in v0.6.0

func (m *Meter) Allow(author string) error

Allow reports whether author may make another model call now, or the reason it may not.

It checks author's own scope first and then the session, matching the order fold reports a crossing in, so the refusal names the tighter of the two ceilings rather than whichever was tested first. An author the meter carries no scope for is checked against the session alone.

A nil Meter allows everything. That is not a convenience: it is the statement "no budget was configured for this call", which is what a host says when the caller runs outside a metered session, and it is deliberately different from an empty Meter, which says "metered, with no ceilings".

func (*Meter) AllowFinalReport added in v0.7.0

func (m *Meter) AllowFinalReport(author string) bool

AllowFinalReport reports whether author may make one last model call to write the report it was stopped before finishing — and consumes the grant, so a second ask returns false.

False is the answer for an unconfigured meter, a nil meter, an author that has already taken its grant, and an author with no spend on the record. The caller treats false as the ordinary refusal.

The grant does NOT relax any ceiling. It is a statement about one call at the seam that asks; Observe still folds what that call costs, the overshoot lands in the reported total, and the next Allow refuses on arithmetic that now includes it.

func (*Meter) FinalReportsTaken added in v0.7.0

func (m *Meter) FinalReportsTaken() int

FinalReportsTaken counts the grants handed out, for the surfaces that have to report an overshoot rather than let it arrive unannounced in a total.

func (*Meter) Grant added in v0.4.0

func (m *Meter) Grant(scope string, add Limits) (Limits, error)

Grant raises scope's ceilings by add and returns the resulting limits. scope "" targets the session; an unknown scope name is an error rather than a silent no-op, since the caller believes it just bought runway.

A dimension that was already unlimited stays unlimited: adding budget must never be able to *impose* a ceiling. Handing 5 turns to a session with no turn cap would otherwise cap it at 5 — a reset that halts the session it was called to unwedge.

func (*Meter) Observe

func (m *Meter) Observe(ev *session.Event) error

Observe folds one event's usage into the meter and reports whether a ceiling has been crossed. Events without UsageMetadata (function responses, control events) are free.

func (*Meter) Precluded added in v0.6.0

func (m *Meter) Precluded(author string) []Trip

Precluded lists every ceiling that already makes another call by author impossible — the pre-call counterpart of Trips, and reported in the same shape so an operator surface can render one the way it renders the other.

Empty means the call may proceed. It does not mean the call will fit: nothing here knows how large it will be.

func (*Meter) PrecludedAfter added in v0.6.0

func (m *Meter) PrecludedAfter(scope string, add Limits) []Trip

PrecludedAfter is TripsAfter's pre-call counterpart: the ceilings that would still make another call impossible if scope's were raised by add. scope "" targets the session.

A reset needs this rather than TripsAfter for the reason the turn door does. A grant that leaves the target sitting *exactly* on its new cap has crossed nothing and cleared nothing — the operator would get a 200, the next turn would be refused, and the only visible difference is that their grant is gone.

func (*Meter) PrecludedAll added in v0.6.0

func (m *Meter) PrecludedAll() []Trip

PrecludedAll lists every ceiling on the meter — the session's and each scope's — that already makes another call impossible. It is Trips's pre-call counterpart in the same way Precluded is crossed's, and it reports the whole meter because that is what an operator surface renders: a specialist that can no longer be dispatched is worth showing even when the session as a whole is fine.

func (*Meter) PrecludedSession added in v0.6.0

func (m *Meter) PrecludedSession() []Trip

PrecludedSession lists the *workload's* own ceilings that already make another call impossible, ignoring every specialist's.

This is the question a turn door asks (W10.3). A precluded specialist closes one path and the coordinator can route around it, so a session whose roster is half spent still has work it can do; a precluded workload has nothing left to try, and starting a turn on it buys a refusal and a log line. PrecludedAll is the reporting view and answers a different question — "what is this operator looking at".

func (*Meter) Refusals added in v0.6.0

func (m *Meter) Refusals() (n int, first error)

Refusals reports how many calls Allow has refused and the reason it gave for the first of them.

A refusal has to be legible somewhere, and the meter is the only place that sees every one: the refusal itself is a synthesized answer, so a turn that ended because its ceiling stopped it otherwise looks exactly like a turn that finished. A driver snapshots the count before a turn and compares after — that, rather than asking whether a ceiling is *now* precluding, because a run that finished its work and happened to land exactly on its cap did not stop for the budget and must not be reported as though it had.

The first reason rather than the last: a fan-out refused ten times says the same thing ten times, and the first one is the ceiling that actually stopped the work.

func (*Meter) Restore added in v0.5.0

func (m *Meter) Restore(p Prior) error

Restore seeds the meter with spend from before this process.

It adds rather than assigns, and it deliberately does not check the ceilings it may have just crossed. Both follow from where enforcement lives: there is no trip flag (see trips.go), so a restored meter that is already over its cap reports that through Trips() and stops the next call through Observe, by the same comparison a meter that never restarted would use. Restoring is not an enforcement point; it is the accumulator catching up with the facts.

Calling it twice is an error, not a no-op — see ErrRestored. Calling it on a meter that has already observed events is allowed: a caller whose first read failed and who retried a turn later has real spend in this process and real spend in the ledger, and they sum.

func (*Meter) Restored added in v0.5.0

func (m *Meter) Restored() bool

Restored reports whether prior spend has been folded into this meter.

func (*Meter) ScopeLimits added in v0.4.0

func (m *Meter) ScopeLimits(name string) (Limits, bool)

ScopeLimits returns one scope's ceilings. ok is false for an agent the meter carries no scope for, matching ScopeSnapshot.

func (*Meter) ScopeNames added in v0.4.0

func (m *Meter) ScopeNames() []string

ScopeNames lists the scoped agents this meter meters, sorted, so a projection over them is stable between reads.

func (*Meter) ScopeSnapshot added in v0.3.0

func (m *Meter) ScopeSnapshot(name string) (tokens int64, costUSD float64, calls int, ok bool)

ScopeSnapshot returns one scoped agent's cumulative usage. ok is false for an agent the meter carries no scope for — which is not the same as an agent that has spent nothing.

func (*Meter) SessionLimits added in v0.4.0

func (m *Meter) SessionLimits() Limits

SessionLimits returns the session's ceilings as they stand now, including any raised by Grant.

func (*Meter) SessionRefusals added in v0.6.0

func (m *Meter) SessionRefusals() (n int, first error)

SessionRefusals is Refusals counting only the refusals no specialist can be blamed for — the workload's own ceilings.

The two differ in what a driver may do about them (W10.3). A refused specialist is one path closed and the coordinator can try another, so the turn goes on; a refused *workload* has nothing left to try, and the turn has to stop. Refusals is therefore the right count for "say something happened" — the trip metric, the log line — and this is the right one for "stop".

func (*Meter) Snapshot

func (m *Meter) Snapshot() (tokens int64, costUSD float64, calls int)

Snapshot returns the session's cumulative usage so far.

func (*Meter) Trips added in v0.4.0

func (m *Meter) Trips() []Trip

Trips reports every ceiling the meter is currently past — the session's and each scope's. Empty means the next turn will not be stopped by this meter.

func (*Meter) TripsAfter added in v0.4.0

func (m *Meter) TripsAfter(scope string, add Limits) []Trip

TripsAfter reports the trips that would remain if scope's ceilings were raised by add — the check an operator reset runs before spending the operator's grant on a raise that provably would not clear anything.

It reports the whole meter, not just scope, because any crossed ceiling wedges the session: raising the workload's cap while a specialist's own cap is still crossed buys nothing, and a 200 that buys nothing is what this reporting exists to prevent.

scope "" targets the session. An unknown scope name raises nothing, which correctly reports the trips as unclearable by that request.

func (*Meter) Unpriced added in v0.3.0

func (m *Meter) Unpriced() int

Unpriced reports how many calls a configured Catalog could not price and that fell back to RatePer1K. Non-zero means the cost figure is a mix of two pricing models and should be read as approximate — a caller that displays cost should surface it rather than let a stale catalog quietly downgrade an exact number.

type Pricer added in v0.8.0

type Pricer interface {
	PriceCall(backend, modelID string, c Call) (usd float64, ok bool)
}

Pricer prices one model call. ok is false when the call cannot be priced — an unknown (backend, model) pair, or a known one whose rates are all zero — and the meter then falls back to Limits.RatePer1K and counts the call in Unpriced.

This is deliberately narrower than the rate table behind it. A pricer is asked for a number, never for rates: mast's own implementation wraps core-models' pricing catalog, which owed a re-key from the backend name to the more general notion of a provider profile (docs/model-support-design.md M2), and a third backend must not be a breaking change to the meter. Naming the catalog here would have frozen the table's shape through pkg/budget, which is one of the six paths v1.0 covers.

type Prior added in v0.5.0

type Prior struct {
	Session  Totals
	ByAuthor map[string]Totals
	Unpriced int
}

Prior is what a session spent before this process started.

Session is the whole session's total, including every call that ByAuthor also accounts for; the two are projections of the same rows and a producer must not let them disagree. ByAuthor is keyed by Spend.Author, and Restore ignores an author the meter carries no scope for — the same rule Observe applies to a live event.

func (Prior) IsZero added in v0.5.0

func (p Prior) IsZero() bool

IsZero reports whether there is nothing to restore, which is the common case: most sessions are new.

type Spend added in v0.5.0

type Spend struct {
	Author   string
	Tokens   int64
	CostUSD  float64
	Unpriced bool

	// At is the event's own timestamp, which is when the call landed
	// rather than when the hook ran. Zero for an event a producer left
	// unstamped; a consumer that needs an ordering has the arrival order
	// of the hook itself.
	At time.Time

	// Model is the id the price was resolved against — see priceOf. It
	// is empty only when the event named no model and the meter was
	// configured with none, which on a streaming Gemini run is the
	// common case for the first half of that pair (Limits.Model exists
	// for exactly that).
	Model string

	// Call is the billable split of this call: the three mutually
	// exclusive input buckets and the output total. Output includes
	// thinking, because thinking bills at the output rate.
	Call Call

	// ThoughtsTokens is the thinking subset of Call.OutputTokens, and
	// ToolUseTokens the tool-definition subset of the prompt. Both are
	// breakdowns of a bucket already counted above, never additions to
	// it — adding either to a total double-counts it.
	//
	// Both come from genai's usage metadata rather than from the
	// provider sidecar. The sidecar's equivalents (usage.Detail's
	// ReasoningTokens and ToolUseTokens) are the same two Gemini
	// counters, and the sidecar exists for the buckets genai has no
	// field for, which these are not.
	ThoughtsTokens int64
	ToolUseTokens  int64

	// CostUSDUncachedReference is what this call would have cost on a
	// backend with no prompt cache at all: every prompt token fresh, at
	// the plain input rate.
	//
	// Usually above CostUSD, and the gap is what the cache saved — but
	// deliberately not floored there. A turn that WRITES a cache entry
	// costs more than one that does not (Anthropic's 5-minute TTL bills
	// writes at 1.25x fresh input), so on a warming turn this figure is
	// below the cost, and a workload that warms caches it never reuses
	// should be able to see that it is paying for nothing.
	//
	// Equal to CostUSD when no saving can be computed rather than when
	// none was made: a call the rate card could not price falls back to
	// the flat rate, which has no notion of a cache.
	CostUSDUncachedReference float64
}

Spend is one metered model call's contribution, handed to Config.OnSpend after the meter has folded it.

Author is the event's author verbatim, not the scope it resolved to. The distinction matters across a restart: a roster edit can add or remove a specialist's scope between processes, and a ledger that recorded "session" for an author the old config did not scope could never be re-attributed. Recording the author lets Restore attribute against whatever scopes the *current* config carries, which is exactly what a live Observe does.

The breakdown half

The first four fields are what a durable ledger needs. The rest are what a usage report needs, and they are here rather than derived by a second reader of the event because deriving them a second time is how the two disagree: the split of a prompt into cached, written and fresh is not a copy of the provider's counters but the meter's own reading of them (see callOf and fitBucket), and it is the reading the money was computed from. A report keyed off anything else is a breakdown of a different call than the one that was billed.

Everything below the line is reported, not priced, except Call and CostUSDUncachedReference. A consumer that only wants the ledger keeps ignoring them — the four original fields have not moved.

type Totals added in v0.5.0

type Totals struct {
	Tokens  int64
	CostUSD float64
	Calls   int
}

Totals is one accumulator's durable form — the exported shape of the package's internal usage counter.

func (Totals) String added in v0.5.0

func (t Totals) String() string

String renders totals for a log line.

type Trip added in v0.4.0

type Trip struct {
	// Scope is the specialist whose own ceiling was crossed, or "" for
	// the session's.
	Scope string
	// Dimension is which bound was crossed (turns / tokens / cost_usd).
	Dimension string
	// Reason is the same detail string the enforcement error carries,
	// e.g. "$0.0612 > cap $0.0500 (30600 tokens over 4 calls)".
	Reason string
}

Trip is one crossed ceiling: which accumulator crossed which bound, and the operator-facing arithmetic behind it.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL