budget

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 17, 2026 License: Apache-2.0 Imports: 6 Imported by: 0

Documentation

Overview

Package budget meters model usage against workload budget ceilings.

Spike-2 probe for the "where does cost accounting come from" question (docs/orchestration-design.md budget composition): ADK v2 carries genai UsageMetadata on every model event (session.Event embeds model.LLMResponse), so a meter over the runner's event stream sees token counts per call with no ADK patching. What ADK does NOT provide is pricing or enforcement — both are mast-side. This package is the minimal mast-side shape: per-session cumulative token/cost meter, checked as events stream; the caller aborts the run when Observe reports the ceiling is crossed.

Scopes: per-specialist ceilings under the session's

A workload budget bounds the session; a specialist's own budget bounds that specialist. Config.Scopes composes the two by attributing each usage event to the agent that authored it — session.Event.Author is the agent's name on every dispatch shape mast builds (a coordinator's sub-agent tool, a workflow-graph node, a planner's invoke_specialist), which is what makes one seam enough. A scope carries its own ceilings and, when the specialist declares a `model:` override, its own price, so a cheap analyst's tokens are not billed at the synthesizer's rate.

Composition is tightest-cap-wins by construction rather than by arithmetic: every event is checked against its scope and against the session, and whichever ceiling is crossed first stops the run. A scope's ceiling is reported ahead of the session's on the event that crosses both, because the specialist is the more specific fact and the workload's cap would have been crossed on a later call anyway.

Known limitations (findings, not TODOs)

Metering at the event stream is enforcement-after-the-call — a single runaway call is only caught once its usage event lands. Pre-call gating needs a model-layer interceptor (wrap model.LLM) or ADK's BeforeModel plugin callback; both compose with this meter rather than replacing it.

A crossed scope ceiling stops the session, not just the specialist, because the event stream is outside the specialist's own run and the only lever there is the run context. Stopping one specialist and handing the coordinator a refusal it can route around is the better shape, and it needs the pre-call seam above.

Index

Constants

View Source
const (
	DimensionTurns   = "turns"
	DimensionTokens  = "tokens"
	DimensionCostUSD = "cost_usd"
)

Budget dimensions, as reported on Trip.Dimension. One per ceiling in Limits; a session can be past more than one at once.

Variables

View Source
var ErrExceeded = errors.New("budget exceeded")

ErrExceeded is returned by Observe once the session's cumulative usage crosses a ceiling. Callers should abort the run.

Functions

This section is empty.

Types

type Config added in v0.3.0

type Config struct {
	// Limits are the session-wide ceilings (the workload budget).
	Limits Limits

	// Scopes are per-agent ceilings and prices, keyed by the agent name
	// that authors the event — for a specialist, its spec name. An
	// agent with no scope is metered into the session totals only.
	Scopes map[string]Limits
}

Config is the full meter shape: the session's ceilings plus the per-agent scopes composed under them.

type Limits

type Limits struct {
	MaxCostUSD float64
	MaxTokens  int64

	// MaxTurns caps the number of model calls in the session.
	//
	// Vocabulary: mast counts one "turn" per model call — the same
	// unit as the meter's calls counter (one streamed event carrying
	// UsageMetadata). This matches docs/orchestration-design.md's
	// "budget.max_turns remains mast-side turn counting (ADK has no
	// turn cap)": a Task specialist that loops through five model
	// calls before finish_task has spent five turns, not one.
	MaxTurns int

	// Catalog prices each model call exactly, from the model the event
	// says it was billed against.
	//
	// Optional, and strictly better than RatePer1K where a caller can
	// supply it. The flat rate exists because this meter originally saw
	// only UsageMetadata.TotalTokenCount, so internal/compose derives it
	// as the plain average of a model's input and output rates — an
	// approximation that "overcharges input-heavy sessions and
	// undercharges output-heavy ones". Both halves of that premise have
	// since stopped being true: the event carries the input/output split
	// and the cache-read subset, and it carries ModelVersion, so the call
	// can be priced against the same pkg/pricing catalog everything else
	// uses. The error is not small on a real agent — an input-heavy,
	// cache-warm session measured here ran 5.9x over its flat-rate
	// figure, and a cost ceiling that wrong is a ceiling that fires on
	// the wrong sessions.
	//
	// Unknown models fall through to RatePer1K, so a catalog miss never
	// silently drops a session's cost to zero. Unpriced counts them.
	//
	// On a scope, nil means "inherit the session's catalog", matching
	// RatePer1K's rule below. A per-scope catalog is unusual — a rate is
	// a property of the model, and the model is on the event — but the
	// inherit rule costs nothing and keeps the two price knobs behaving
	// alike.
	Catalog *pricing.Catalog

	// RatePer1K is the flat USD price per 1K total tokens (spike
	// pricing model), and the fallback for a call Catalog cannot price.
	//
	// On a scope, zero means "inherit the session's rate" — the right
	// default for a specialist that declares no model of its own, and
	// the reason an un-tiered roster prices exactly as it did before
	// scopes existed.
	RatePer1K float64
}

Limits are the ceilings for one session. Zero values mean unlimited.

type Meter

type Meter struct {
	// contains filtered or unexported fields
}

Meter accumulates usage for one session, and for each scoped agent within it.

func New added in v0.3.0

func New(cfg Config) *Meter

New constructs a Meter from a full config.

func NewMeter

func NewMeter(limits Limits) *Meter

NewMeter constructs a Meter with the given session limits and no per-agent scopes.

func (*Meter) Grant added in v0.4.0

func (m *Meter) Grant(scope string, add Limits) (Limits, error)

Grant raises scope's ceilings by add and returns the resulting limits. scope "" targets the session; an unknown scope name is an error rather than a silent no-op, since the caller believes it just bought runway.

A dimension that was already unlimited stays unlimited: adding budget must never be able to *impose* a ceiling. Handing 5 turns to a session with no turn cap would otherwise cap it at 5 — a reset that halts the session it was called to unwedge.

func (*Meter) Observe

func (m *Meter) Observe(ev *session.Event) error

Observe folds one event's usage into the meter and reports whether a ceiling has been crossed. Events without UsageMetadata (function responses, control events) are free.

func (*Meter) ScopeLimits added in v0.4.0

func (m *Meter) ScopeLimits(name string) (Limits, bool)

ScopeLimits returns one scope's ceilings. ok is false for an agent the meter carries no scope for, matching ScopeSnapshot.

func (*Meter) ScopeNames added in v0.4.0

func (m *Meter) ScopeNames() []string

ScopeNames lists the scoped agents this meter meters, sorted, so a projection over them is stable between reads.

func (*Meter) ScopeSnapshot added in v0.3.0

func (m *Meter) ScopeSnapshot(name string) (tokens int64, costUSD float64, calls int, ok bool)

ScopeSnapshot returns one scoped agent's cumulative usage. ok is false for an agent the meter carries no scope for — which is not the same as an agent that has spent nothing.

func (*Meter) SessionLimits added in v0.4.0

func (m *Meter) SessionLimits() Limits

SessionLimits returns the session's ceilings as they stand now, including any raised by Grant.

func (*Meter) Snapshot

func (m *Meter) Snapshot() (tokens int64, costUSD float64, calls int)

Snapshot returns the session's cumulative usage so far.

func (*Meter) Trips added in v0.4.0

func (m *Meter) Trips() []Trip

Trips reports every ceiling the meter is currently past — the session's and each scope's. Empty means the next turn will not be stopped by this meter.

func (*Meter) TripsAfter added in v0.4.0

func (m *Meter) TripsAfter(scope string, add Limits) []Trip

TripsAfter reports the trips that would remain if scope's ceilings were raised by add — the check an operator reset runs before spending the operator's grant on a raise that provably would not clear anything.

It reports the whole meter, not just scope, because any crossed ceiling wedges the session: raising the workload's cap while a specialist's own cap is still crossed buys nothing, and a 200 that buys nothing is what this reporting exists to prevent.

scope "" targets the session. An unknown scope name raises nothing, which correctly reports the trips as unclearable by that request.

func (*Meter) Unpriced added in v0.3.0

func (m *Meter) Unpriced() int

Unpriced reports how many calls a configured Catalog could not price and that fell back to RatePer1K. Non-zero means the cost figure is a mix of two pricing models and should be read as approximate — a caller that displays cost should surface it rather than let a stale catalog quietly downgrade an exact number.

type Trip added in v0.4.0

type Trip struct {
	// Scope is the specialist whose own ceiling was crossed, or "" for
	// the session's.
	Scope string
	// Dimension is which bound was crossed (turns / tokens / cost_usd).
	Dimension string
	// Reason is the same detail string the enforcement error carries,
	// e.g. "$0.0612 > cap $0.0500 (30600 tokens over 4 calls)".
	Reason string
}

Trip is one crossed ceiling: which accumulator crossed which bound, and the operator-facing arithmetic behind it.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL