budget

package
v0.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 30, 2026 License: Apache-2.0 Imports: 6 Imported by: 0

Documentation

Overview

Package budget meters model usage against workload budget ceilings.

Spike-2 probe for the "where does cost accounting come from" question (docs/orchestration-design.md budget composition): ADK v2 carries genai UsageMetadata on every model event (session.Event embeds model.LLMResponse), so a meter over the runner's event stream sees token counts per call with no ADK patching. What ADK does NOT provide is pricing or enforcement — both are mast-side. This package is the minimal mast-side shape: per-session cumulative token/cost meter, checked as events stream; the caller aborts the run when Observe reports the ceiling is crossed.

Scopes: per-specialist ceilings under the session's

A workload budget bounds the session; a specialist's own budget bounds that specialist. Config.Scopes composes the two by attributing each usage event to the agent that authored it — session.Event.Author is the agent's name on every dispatch shape mast builds, which is what makes one attribution rule enough for all of them.

It does not follow that one seam is enough to SEE them. A coordinator's sub-agent tool and a workflow-graph node funnel their events up the root runner's stream, so observing that stream catches both. A planner's invoke_specialist does not: it runs the specialist on a private runner, whose events reach the host only through planner.SubRunObserver (#226). A host that meters must feed this package from both — the arithmetic is identical either way, and the Author on a sub-run event is still the specialist's name, so a declared ceiling binds on the planner's door exactly as it does on a coordinator's.

A scope carries its own ceilings and, when the specialist declares a `model:` override, its own price, so a cheap analyst's tokens are not billed at the synthesizer's rate.

Composition is tightest-cap-wins by construction rather than by arithmetic: every event is checked against its scope and against the session, and whichever ceiling is crossed first stops the run. A scope's ceiling is reported ahead of the session's on the event that crosses both, because the specialist is the more specific fact and the workload's cap would have been crossed on a later call anyway.

Known limitations (findings, not TODOs)

Metering at the event stream is enforcement-after-the-call — a single runaway call is only caught once its usage event lands. Pre-call gating needs a model-layer interceptor (wrap model.LLM) or ADK's BeforeModel plugin callback; both compose with this meter rather than replacing it.

A crossed scope ceiling stops the session, not just the specialist, because the event stream is outside the specialist's own run and the only lever there is the run context. Stopping one specialist and handing the coordinator a refusal it can route around is the better shape, and it needs the pre-call seam above.

One dispatch shape already has that better shape, by accident of where its seam sits: a planner sub-run is observed from inside the tool call that started it, so an error from Observe there stops the specialist and returns the planner a labelled partial, leaving the session alive (see planner.SubRunObserver). Coordinator and graph dispatch still stop the session, and will until the pre-call seam exists.

Index

Constants

View Source
const (
	DimensionTurns   = "turns"
	DimensionTokens  = "tokens"
	DimensionCostUSD = "cost_usd"
)

Budget dimensions, as reported on Trip.Dimension. One per ceiling in Limits; a session can be past more than one at once.

Variables

View Source
var ErrExceeded = errors.New("budget exceeded")

ErrExceeded is returned by Observe once the session's cumulative usage crosses a ceiling. Callers should abort the run.

View Source
var ErrRestored = errors.New("budget: meter has already been restored")

ErrRestored is returned by a second Restore on the same meter. Prior spend is additive, so folding it twice doubles it — and the sessions this matters for are the ones already near a ceiling, where the double count is the difference between running and refused.

Functions

This section is empty.

Types

type Config added in v0.3.0

type Config struct {
	// Limits are the session-wide ceilings (the workload budget).
	Limits Limits

	// Scopes are per-agent ceilings and prices, keyed by the agent name
	// that authors the event — for a specialist, its spec name. An
	// agent with no scope is metered into the session totals only.
	Scopes map[string]Limits

	// OnSpend, when set, is called once per priced call with what that
	// call added — the write half of the durability seam described in
	// durable.go. It runs on the caller's goroutine, outside the meter's
	// lock, after the fold and before Observe returns, including when
	// the fold reported a crossed ceiling: the call happened and the
	// money is spent whether or not it was the one that stopped the run.
	//
	// It must not call back into the meter (Snapshot and friends take
	// the same lock the fold just released, so a re-entrant caller would
	// read a different meter than the one it was told about, and a
	// re-entrant Observe would recurse). Keep it to handing the Spend
	// somewhere durable.
	OnSpend func(Spend)
}

Config is the full meter shape: the session's ceilings plus the per-agent scopes composed under them.

type Limits

type Limits struct {
	MaxCostUSD float64
	MaxTokens  int64

	// MaxTurns caps the number of model calls in the session.
	//
	// Vocabulary: mast counts one "turn" per model call — the same
	// unit as the meter's calls counter (one streamed event carrying
	// UsageMetadata). This matches docs/orchestration-design.md's
	// "budget.max_turns remains mast-side turn counting (ADK has no
	// turn cap)": a Task specialist that loops through five model
	// calls before finish_task has spent five turns, not one.
	MaxTurns int

	// Catalog prices each model call exactly, from the model the event
	// says it was billed against.
	//
	// Optional, and strictly better than RatePer1K where a caller can
	// supply it. The flat rate exists because this meter originally saw
	// only UsageMetadata.TotalTokenCount, so internal/compose derives it
	// as the plain average of a model's input and output rates — an
	// approximation that "overcharges input-heavy sessions and
	// undercharges output-heavy ones". Both halves of that premise have
	// since stopped being true: the event carries the input/output split
	// and the cache-read subset, and it carries ModelVersion, so the call
	// can be priced against the same pkg/pricing catalog everything else
	// uses. The error is not small on a real agent — an input-heavy,
	// cache-warm session measured here ran 5.9x over its flat-rate
	// figure, and a cost ceiling that wrong is a ceiling that fires on
	// the wrong sessions.
	//
	// Unknown models fall through to RatePer1K, so a catalog miss never
	// silently drops a session's cost to zero. Unpriced counts them.
	//
	// On a scope, nil means "inherit the session's catalog", matching
	// RatePer1K's rule below. A per-scope catalog is unusual — a rate is
	// a property of the model, and the model is on the event — but the
	// inherit rule costs nothing and keeps the two price knobs behaving
	// alike.
	Catalog *pricing.Catalog

	// RatePer1K is the flat USD price per 1K total tokens (spike
	// pricing model), and the fallback for a call Catalog cannot price.
	//
	// On a scope, zero means "inherit the session's rate" — the right
	// default for a specialist that declares no model of its own, and
	// the reason an un-tiered roster prices exactly as it did before
	// scopes existed.
	RatePer1K float64
}

Limits are the ceilings for one session. Zero values mean unlimited.

type Meter

type Meter struct {
	// contains filtered or unexported fields
}

Meter accumulates usage for one session, and for each scoped agent within it.

func New added in v0.3.0

func New(cfg Config) *Meter

New constructs a Meter from a full config.

func NewMeter

func NewMeter(limits Limits) *Meter

NewMeter constructs a Meter with the given session limits and no per-agent scopes.

func (*Meter) Grant added in v0.4.0

func (m *Meter) Grant(scope string, add Limits) (Limits, error)

Grant raises scope's ceilings by add and returns the resulting limits. scope "" targets the session; an unknown scope name is an error rather than a silent no-op, since the caller believes it just bought runway.

A dimension that was already unlimited stays unlimited: adding budget must never be able to *impose* a ceiling. Handing 5 turns to a session with no turn cap would otherwise cap it at 5 — a reset that halts the session it was called to unwedge.

func (*Meter) Observe

func (m *Meter) Observe(ev *session.Event) error

Observe folds one event's usage into the meter and reports whether a ceiling has been crossed. Events without UsageMetadata (function responses, control events) are free.

func (*Meter) Restore added in v0.5.0

func (m *Meter) Restore(p Prior) error

Restore seeds the meter with spend from before this process.

It adds rather than assigns, and it deliberately does not check the ceilings it may have just crossed. Both follow from where enforcement lives: there is no trip flag (see trips.go), so a restored meter that is already over its cap reports that through Trips() and stops the next call through Observe, by the same comparison a meter that never restarted would use. Restoring is not an enforcement point; it is the accumulator catching up with the facts.

Calling it twice is an error, not a no-op — see ErrRestored. Calling it on a meter that has already observed events is allowed: a caller whose first read failed and who retried a turn later has real spend in this process and real spend in the ledger, and they sum.

func (*Meter) Restored added in v0.5.0

func (m *Meter) Restored() bool

Restored reports whether prior spend has been folded into this meter.

func (*Meter) ScopeLimits added in v0.4.0

func (m *Meter) ScopeLimits(name string) (Limits, bool)

ScopeLimits returns one scope's ceilings. ok is false for an agent the meter carries no scope for, matching ScopeSnapshot.

func (*Meter) ScopeNames added in v0.4.0

func (m *Meter) ScopeNames() []string

ScopeNames lists the scoped agents this meter meters, sorted, so a projection over them is stable between reads.

func (*Meter) ScopeSnapshot added in v0.3.0

func (m *Meter) ScopeSnapshot(name string) (tokens int64, costUSD float64, calls int, ok bool)

ScopeSnapshot returns one scoped agent's cumulative usage. ok is false for an agent the meter carries no scope for — which is not the same as an agent that has spent nothing.

func (*Meter) SessionLimits added in v0.4.0

func (m *Meter) SessionLimits() Limits

SessionLimits returns the session's ceilings as they stand now, including any raised by Grant.

func (*Meter) Snapshot

func (m *Meter) Snapshot() (tokens int64, costUSD float64, calls int)

Snapshot returns the session's cumulative usage so far.

func (*Meter) Trips added in v0.4.0

func (m *Meter) Trips() []Trip

Trips reports every ceiling the meter is currently past — the session's and each scope's. Empty means the next turn will not be stopped by this meter.

func (*Meter) TripsAfter added in v0.4.0

func (m *Meter) TripsAfter(scope string, add Limits) []Trip

TripsAfter reports the trips that would remain if scope's ceilings were raised by add — the check an operator reset runs before spending the operator's grant on a raise that provably would not clear anything.

It reports the whole meter, not just scope, because any crossed ceiling wedges the session: raising the workload's cap while a specialist's own cap is still crossed buys nothing, and a 200 that buys nothing is what this reporting exists to prevent.

scope "" targets the session. An unknown scope name raises nothing, which correctly reports the trips as unclearable by that request.

func (*Meter) Unpriced added in v0.3.0

func (m *Meter) Unpriced() int

Unpriced reports how many calls a configured Catalog could not price and that fell back to RatePer1K. Non-zero means the cost figure is a mix of two pricing models and should be read as approximate — a caller that displays cost should surface it rather than let a stale catalog quietly downgrade an exact number.

type Prior added in v0.5.0

type Prior struct {
	Session  Totals
	ByAuthor map[string]Totals
	Unpriced int
}

Prior is what a session spent before this process started.

Session is the whole session's total, including every call that ByAuthor also accounts for; the two are projections of the same rows and a producer must not let them disagree. ByAuthor is keyed by Spend.Author, and Restore ignores an author the meter carries no scope for — the same rule Observe applies to a live event.

func (Prior) IsZero added in v0.5.0

func (p Prior) IsZero() bool

IsZero reports whether there is nothing to restore, which is the common case: most sessions are new.

type Spend added in v0.5.0

type Spend struct {
	Author   string
	Tokens   int64
	CostUSD  float64
	Unpriced bool
}

Spend is one metered model call's contribution, handed to Config.OnSpend after the meter has folded it.

Author is the event's author verbatim, not the scope it resolved to. The distinction matters across a restart: a roster edit can add or remove a specialist's scope between processes, and a ledger that recorded "session" for an author the old config did not scope could never be re-attributed. Recording the author lets Restore attribute against whatever scopes the *current* config carries, which is exactly what a live Observe does.

type Totals added in v0.5.0

type Totals struct {
	Tokens  int64
	CostUSD float64
	Calls   int
}

Totals is one accumulator's durable form — the exported shape of the package's internal usage counter.

func (Totals) String added in v0.5.0

func (t Totals) String() string

String renders totals for a log line.

type Trip added in v0.4.0

type Trip struct {
	// Scope is the specialist whose own ceiling was crossed, or "" for
	// the session's.
	Scope string
	// Dimension is which bound was crossed (turns / tokens / cost_usd).
	Dimension string
	// Reason is the same detail string the enforcement error carries,
	// e.g. "$0.0612 > cap $0.0500 (30600 tokens over 4 calls)".
	Reason string
}

Trip is one crossed ceiling: which accumulator crossed which bound, and the operator-facing arithmetic behind it.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL