budget

package
v0.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 3, 2026 License: Apache-2.0 Imports: 6 Imported by: 0

Documentation

Overview

Package budget meters model usage against workload budget ceilings.

Spike-2 probe for the "where does cost accounting come from" question (docs/orchestration-design.md budget composition): ADK v2 carries genai UsageMetadata on every model event (session.Event embeds model.LLMResponse), so a meter over the runner's event stream sees token counts per call with no ADK patching. What ADK does NOT provide is pricing or enforcement — both are mast-side. This package is the minimal mast-side shape: per-session cumulative token/cost meter, checked as events stream; the caller aborts the run when Observe reports the ceiling is crossed.

Scopes: per-specialist ceilings under the session's

A workload budget bounds the session; a specialist's own budget bounds that specialist. Config.Scopes composes the two by attributing each usage event to the agent that authored it — session.Event.Author is the agent's name on every dispatch shape mast builds, which is what makes one attribution rule enough for all of them.

It does not follow that one seam is enough to SEE them. A coordinator's sub-agent tool and a workflow-graph node funnel their events up the root runner's stream, so observing that stream catches both. A planner's invoke_specialist does not: it runs the specialist on a private runner, whose events reach the host only through planner.SubRunObserver (#226). A host that meters must feed this package from both — the arithmetic is identical either way, and the Author on a sub-run event is still the specialist's name, so a declared ceiling binds on the planner's door exactly as it does on a coordinator's.

A scope carries its own ceilings and, when the specialist declares a `model:` override, its own price, so a cheap analyst's tokens are not billed at the synthesizer's rate.

Composition is tightest-cap-wins by construction rather than by arithmetic: every event is checked against its scope and against the session, and whichever ceiling is reached first stops what it holds. A scope's ceiling is reported ahead of the session's on the event that crosses both, because the specialist is the more specific fact and the workload's cap would have been crossed on a later call anyway.

Who a ceiling stops

A ceiling holder is either the workload or one specialist, and since v0.6 the two have different consequences. Scope(err) reports which an enforcement error belonged to: a scoped one closes that specialist's path, and the turn drivers log it, count it, and route on; an unscoped one ends the turn. Through v0.5 both ended the turn, which was never a decision — Observe runs on the event stream, outside the specialist's own run, where the run context was the only lever available. Allow is the lever that was missing, and it is per-call and per-agent (see allow.go).

The counting follows the same split. Refusals reports every refusal, for the metric and the log line; SessionRefusals reports only the workload's own, for the driver deciding whether to stop. Each keeps its own first reason on purpose: the first refusal of a turn is usually a specialist's, and a driver that stopped on the session's cap while quoting a specialist's reason would send an operator to raise the wrong ceiling.

Known limitations (findings, not TODOs)

Metering at the event stream is enforcement-after-the-call — a single runaway call is only priced once its usage event lands. This meter stays the ledger; Allow is asked in front of it, and refuses only where the arithmetic is a proof rather than an estimate, so the two disagree by at most one call.

Index

Constants

View Source
const (
	DimensionTurns   = "turns"
	DimensionTokens  = "tokens"
	DimensionCostUSD = "cost_usd"
)

Budget dimensions, as reported on Trip.Dimension. One per ceiling in Limits; a session can be past more than one at once.

Variables

View Source
var ErrExceeded = errors.New("budget exceeded")

ErrExceeded is returned by Observe once cumulative usage crosses a ceiling. Callers should abort the run — unless Scope reports the ceiling was one specialist's, in which case one path is closed and the run can continue without it.

View Source
var ErrRefused = errors.New("budget refused the call")

ErrRefused is returned by Allow when a ceiling makes the requested call impossible. It is distinct from ErrExceeded on purpose: the two describe different events, and a caller that conflated them would report money spent that was not.

ErrExceeded means a call happened and pushed the meter past a cap. ErrRefused means a call did not happen, because it could not have finished under one.

View Source
var ErrRestored = errors.New("budget: meter has already been restored")

ErrRestored is returned by a second Restore on the same meter. Prior spend is additive, so folding it twice doubles it — and the sessions this matters for are the ones already near a ceiling, where the double count is the difference between running and refused.

Functions

func Scope added in v0.6.0

func Scope(err error) (string, bool)

Scope reports the specialist whose own ceiling produced err, and false for a session-level ceiling or any error that is not one of ours.

The distinction is the whole of W10.3: a specialist that is out of budget is one path closed, and the coordinator can try another; a workload that is out of budget has nothing left to try. A caller that cannot tell them apart has to assume the harsher one, which is what mast did through v0.5.

Types

type Config added in v0.3.0

type Config struct {
	// Limits are the session-wide ceilings (the workload budget).
	Limits Limits

	// Scopes are per-agent ceilings and prices, keyed by the agent name
	// that authors the event — for a specialist, its spec name. An
	// agent with no scope is metered into the session totals only.
	Scopes map[string]Limits

	// OnSpend, when set, is called once per priced call with what that
	// call added — the write half of the durability seam described in
	// durable.go. It runs on the caller's goroutine, outside the meter's
	// lock, after the fold and before Observe returns, including when
	// the fold reported a crossed ceiling: the call happened and the
	// money is spent whether or not it was the one that stopped the run.
	//
	// It must not call back into the meter (Snapshot and friends take
	// the same lock the fold just released, so a re-entrant caller would
	// read a different meter than the one it was told about, and a
	// re-entrant Observe would recurse). Keep it to handing the Spend
	// somewhere durable.
	OnSpend func(Spend)
}

Config is the full meter shape: the session's ceilings plus the per-agent scopes composed under them.

type Limits

type Limits struct {
	MaxCostUSD float64
	MaxTokens  int64

	// MaxTurns caps the number of model calls in the session.
	//
	// Vocabulary: mast counts one "turn" per model call — the same
	// unit as the meter's calls counter (one streamed event carrying
	// UsageMetadata). This matches docs/orchestration-design.md's
	// "budget.max_turns remains mast-side turn counting (ADK has no
	// turn cap)": a Task specialist that loops through five model
	// calls before finish_task has spent five turns, not one.
	MaxTurns int

	// Catalog prices each model call exactly, from the model the event
	// says it was billed against.
	//
	// Optional, and strictly better than RatePer1K where a caller can
	// supply it. The flat rate exists because this meter originally saw
	// only UsageMetadata.TotalTokenCount, so internal/compose derives it
	// as the plain average of a model's input and output rates — an
	// approximation that "overcharges input-heavy sessions and
	// undercharges output-heavy ones". Both halves of that premise have
	// since stopped being true: the event carries the input/output split
	// and the cache-read subset, and it carries ModelVersion, so the call
	// can be priced against the same pkg/pricing catalog everything else
	// uses. The error is not small on a real agent — an input-heavy,
	// cache-warm session measured here ran 5.9x over its flat-rate
	// figure, and a cost ceiling that wrong is a ceiling that fires on
	// the wrong sessions.
	//
	// Unknown models fall through to RatePer1K, so a catalog miss never
	// silently drops a session's cost to zero. Unpriced counts them.
	//
	// Nothing in mast sets this yet; RatePer1K is the live path. Before
	// it becomes one, it owes the backend: a rate is a property of the
	// (backend, model) pair, not of the model, and ModelVersion names
	// only the model. Priced here it would resolve through
	// Catalog.Lookup, which is the bare-id half of a table whose bare
	// Claude rows are first-party and whose bare Gemini rows are
	// Vertex's. The pair-keyed entry point is pricing.LookupFor, and
	// what it needs is the backend internal/compose.Backend resolved for
	// the call — which is not on the event and would have to be carried
	// here.
	//
	// On a scope, nil means "inherit the session's catalog", matching
	// RatePer1K's rule below. A per-scope catalog is unusual — the model
	// is on the event — but the inherit rule costs nothing and keeps the
	// two price knobs behaving alike.
	Catalog *pricing.Catalog

	// RatePer1K is the flat USD price per 1K total tokens (spike
	// pricing model), and the fallback for a call Catalog cannot price.
	//
	// On a scope, zero means "inherit the session's rate" — the right
	// default for a specialist that declares no model of its own, and
	// the reason an un-tiered roster prices exactly as it did before
	// scopes existed.
	RatePer1K float64
}

Limits are the ceilings for one session. Zero values mean unlimited.

type Meter

type Meter struct {
	// contains filtered or unexported fields
}

Meter accumulates usage for one session, and for each scoped agent within it.

func New added in v0.3.0

func New(cfg Config) *Meter

New constructs a Meter from a full config.

func NewMeter

func NewMeter(limits Limits) *Meter

NewMeter constructs a Meter with the given session limits and no per-agent scopes.

func (*Meter) Allow added in v0.6.0

func (m *Meter) Allow(author string) error

Allow reports whether author may make another model call now, or the reason it may not.

It checks author's own scope first and then the session, matching the order fold reports a crossing in, so the refusal names the tighter of the two ceilings rather than whichever was tested first. An author the meter carries no scope for is checked against the session alone.

A nil Meter allows everything. That is not a convenience: it is the statement "no budget was configured for this call", which is what a host says when the caller runs outside a metered session, and it is deliberately different from an empty Meter, which says "metered, with no ceilings".

func (*Meter) Grant added in v0.4.0

func (m *Meter) Grant(scope string, add Limits) (Limits, error)

Grant raises scope's ceilings by add and returns the resulting limits. scope "" targets the session; an unknown scope name is an error rather than a silent no-op, since the caller believes it just bought runway.

A dimension that was already unlimited stays unlimited: adding budget must never be able to *impose* a ceiling. Handing 5 turns to a session with no turn cap would otherwise cap it at 5 — a reset that halts the session it was called to unwedge.

func (*Meter) Observe

func (m *Meter) Observe(ev *session.Event) error

Observe folds one event's usage into the meter and reports whether a ceiling has been crossed. Events without UsageMetadata (function responses, control events) are free.

func (*Meter) Precluded added in v0.6.0

func (m *Meter) Precluded(author string) []Trip

Precluded lists every ceiling that already makes another call by author impossible — the pre-call counterpart of Trips, and reported in the same shape so an operator surface can render one the way it renders the other.

Empty means the call may proceed. It does not mean the call will fit: nothing here knows how large it will be.

func (*Meter) PrecludedAfter added in v0.6.0

func (m *Meter) PrecludedAfter(scope string, add Limits) []Trip

PrecludedAfter is TripsAfter's pre-call counterpart: the ceilings that would still make another call impossible if scope's were raised by add. scope "" targets the session.

A reset needs this rather than TripsAfter for the reason the turn door does. A grant that leaves the target sitting *exactly* on its new cap has crossed nothing and cleared nothing — the operator would get a 200, the next turn would be refused, and the only visible difference is that their grant is gone.

func (*Meter) PrecludedAll added in v0.6.0

func (m *Meter) PrecludedAll() []Trip

PrecludedAll lists every ceiling on the meter — the session's and each scope's — that already makes another call impossible. It is Trips's pre-call counterpart in the same way Precluded is crossed's, and it reports the whole meter because that is what an operator surface renders: a specialist that can no longer be dispatched is worth showing even when the session as a whole is fine.

func (*Meter) PrecludedSession added in v0.6.0

func (m *Meter) PrecludedSession() []Trip

PrecludedSession lists the *workload's* own ceilings that already make another call impossible, ignoring every specialist's.

This is the question a turn door asks (W10.3). A precluded specialist closes one path and the coordinator can route around it, so a session whose roster is half spent still has work it can do; a precluded workload has nothing left to try, and starting a turn on it buys a refusal and a log line. PrecludedAll is the reporting view and answers a different question — "what is this operator looking at".

func (*Meter) Refusals added in v0.6.0

func (m *Meter) Refusals() (n int, first error)

Refusals reports how many calls Allow has refused and the reason it gave for the first of them.

A refusal has to be legible somewhere, and the meter is the only place that sees every one: the refusal itself is a synthesized answer, so a turn that ended because its ceiling stopped it otherwise looks exactly like a turn that finished. A driver snapshots the count before a turn and compares after — that, rather than asking whether a ceiling is *now* precluding, because a run that finished its work and happened to land exactly on its cap did not stop for the budget and must not be reported as though it had.

The first reason rather than the last: a fan-out refused ten times says the same thing ten times, and the first one is the ceiling that actually stopped the work.

func (*Meter) Restore added in v0.5.0

func (m *Meter) Restore(p Prior) error

Restore seeds the meter with spend from before this process.

It adds rather than assigns, and it deliberately does not check the ceilings it may have just crossed. Both follow from where enforcement lives: there is no trip flag (see trips.go), so a restored meter that is already over its cap reports that through Trips() and stops the next call through Observe, by the same comparison a meter that never restarted would use. Restoring is not an enforcement point; it is the accumulator catching up with the facts.

Calling it twice is an error, not a no-op — see ErrRestored. Calling it on a meter that has already observed events is allowed: a caller whose first read failed and who retried a turn later has real spend in this process and real spend in the ledger, and they sum.

func (*Meter) Restored added in v0.5.0

func (m *Meter) Restored() bool

Restored reports whether prior spend has been folded into this meter.

func (*Meter) ScopeLimits added in v0.4.0

func (m *Meter) ScopeLimits(name string) (Limits, bool)

ScopeLimits returns one scope's ceilings. ok is false for an agent the meter carries no scope for, matching ScopeSnapshot.

func (*Meter) ScopeNames added in v0.4.0

func (m *Meter) ScopeNames() []string

ScopeNames lists the scoped agents this meter meters, sorted, so a projection over them is stable between reads.

func (*Meter) ScopeSnapshot added in v0.3.0

func (m *Meter) ScopeSnapshot(name string) (tokens int64, costUSD float64, calls int, ok bool)

ScopeSnapshot returns one scoped agent's cumulative usage. ok is false for an agent the meter carries no scope for — which is not the same as an agent that has spent nothing.

func (*Meter) SessionLimits added in v0.4.0

func (m *Meter) SessionLimits() Limits

SessionLimits returns the session's ceilings as they stand now, including any raised by Grant.

func (*Meter) SessionRefusals added in v0.6.0

func (m *Meter) SessionRefusals() (n int, first error)

SessionRefusals is Refusals counting only the refusals no specialist can be blamed for — the workload's own ceilings.

The two differ in what a driver may do about them (W10.3). A refused specialist is one path closed and the coordinator can try another, so the turn goes on; a refused *workload* has nothing left to try, and the turn has to stop. Refusals is therefore the right count for "say something happened" — the trip metric, the log line — and this is the right one for "stop".

func (*Meter) Snapshot

func (m *Meter) Snapshot() (tokens int64, costUSD float64, calls int)

Snapshot returns the session's cumulative usage so far.

func (*Meter) Trips added in v0.4.0

func (m *Meter) Trips() []Trip

Trips reports every ceiling the meter is currently past — the session's and each scope's. Empty means the next turn will not be stopped by this meter.

func (*Meter) TripsAfter added in v0.4.0

func (m *Meter) TripsAfter(scope string, add Limits) []Trip

TripsAfter reports the trips that would remain if scope's ceilings were raised by add — the check an operator reset runs before spending the operator's grant on a raise that provably would not clear anything.

It reports the whole meter, not just scope, because any crossed ceiling wedges the session: raising the workload's cap while a specialist's own cap is still crossed buys nothing, and a 200 that buys nothing is what this reporting exists to prevent.

scope "" targets the session. An unknown scope name raises nothing, which correctly reports the trips as unclearable by that request.

func (*Meter) Unpriced added in v0.3.0

func (m *Meter) Unpriced() int

Unpriced reports how many calls a configured Catalog could not price and that fell back to RatePer1K. Non-zero means the cost figure is a mix of two pricing models and should be read as approximate — a caller that displays cost should surface it rather than let a stale catalog quietly downgrade an exact number.

type Prior added in v0.5.0

type Prior struct {
	Session  Totals
	ByAuthor map[string]Totals
	Unpriced int
}

Prior is what a session spent before this process started.

Session is the whole session's total, including every call that ByAuthor also accounts for; the two are projections of the same rows and a producer must not let them disagree. ByAuthor is keyed by Spend.Author, and Restore ignores an author the meter carries no scope for — the same rule Observe applies to a live event.

func (Prior) IsZero added in v0.5.0

func (p Prior) IsZero() bool

IsZero reports whether there is nothing to restore, which is the common case: most sessions are new.

type Spend added in v0.5.0

type Spend struct {
	Author   string
	Tokens   int64
	CostUSD  float64
	Unpriced bool
}

Spend is one metered model call's contribution, handed to Config.OnSpend after the meter has folded it.

Author is the event's author verbatim, not the scope it resolved to. The distinction matters across a restart: a roster edit can add or remove a specialist's scope between processes, and a ledger that recorded "session" for an author the old config did not scope could never be re-attributed. Recording the author lets Restore attribute against whatever scopes the *current* config carries, which is exactly what a live Observe does.

type Totals added in v0.5.0

type Totals struct {
	Tokens  int64
	CostUSD float64
	Calls   int
}

Totals is one accumulator's durable form — the exported shape of the package's internal usage counter.

func (Totals) String added in v0.5.0

func (t Totals) String() string

String renders totals for a log line.

type Trip added in v0.4.0

type Trip struct {
	// Scope is the specialist whose own ceiling was crossed, or "" for
	// the session's.
	Scope string
	// Dimension is which bound was crossed (turns / tokens / cost_usd).
	Dimension string
	// Reason is the same detail string the enforcement error carries,
	// e.g. "$0.0612 > cap $0.0500 (30600 tokens over 4 calls)".
	Reason string
}

Trip is one crossed ceiling: which accumulator crossed which bound, and the operator-facing arithmetic behind it.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL