budget

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 14, 2026 License: Apache-2.0 Imports: 5 Imported by: 0

Documentation

Overview

Package budget meters model usage against workload budget ceilings.

Spike-2 probe for the "where does cost accounting come from" question (docs/orchestration-design.md budget composition): ADK v2 carries genai UsageMetadata on every model event (session.Event embeds model.LLMResponse), so a meter over the runner's event stream sees token counts per call with no ADK patching. What ADK does NOT provide is pricing or enforcement — both are mast-side. This package is the minimal mast-side shape: per-session cumulative token/cost meter, checked as events stream; the caller aborts the run when Observe reports the ceiling is crossed.

Scopes: per-specialist ceilings under the session's

A workload budget bounds the session; a specialist's own budget bounds that specialist. Config.Scopes composes the two by attributing each usage event to the agent that authored it — session.Event.Author is the agent's name on every dispatch shape mast builds (a coordinator's sub-agent tool, a workflow-graph node, a planner's invoke_specialist), which is what makes one seam enough. A scope carries its own ceilings and, when the specialist declares a `model:` override, its own price, so a cheap analyst's tokens are not billed at the synthesizer's rate.

Composition is tightest-cap-wins by construction rather than by arithmetic: every event is checked against its scope and against the session, and whichever ceiling is crossed first stops the run. A scope's ceiling is reported ahead of the session's on the event that crosses both, because the specialist is the more specific fact and the workload's cap would have been crossed on a later call anyway.

Known limitations (findings, not TODOs)

Metering at the event stream is enforcement-after-the-call — a single runaway call is only caught once its usage event lands. Pre-call gating needs a model-layer interceptor (wrap model.LLM) or ADK's BeforeModel plugin callback; both compose with this meter rather than replacing it.

A crossed scope ceiling stops the session, not just the specialist, because the event stream is outside the specialist's own run and the only lever there is the run context. Stopping one specialist and handing the coordinator a refusal it can route around is the better shape, and it needs the pre-call seam above.

Index

Constants

This section is empty.

Variables

View Source
var ErrExceeded = errors.New("budget exceeded")

ErrExceeded is returned by Observe once the session's cumulative usage crosses a ceiling. Callers should abort the run.

Functions

This section is empty.

Types

type Config added in v0.3.0

type Config struct {
	// Limits are the session-wide ceilings (the workload budget).
	Limits Limits

	// Scopes are per-agent ceilings and prices, keyed by the agent name
	// that authors the event — for a specialist, its spec name. An
	// agent with no scope is metered into the session totals only.
	Scopes map[string]Limits
}

Config is the full meter shape: the session's ceilings plus the per-agent scopes composed under them.

type Limits

type Limits struct {
	MaxCostUSD float64
	MaxTokens  int64

	// MaxTurns caps the number of model calls in the session.
	//
	// Vocabulary: mast counts one "turn" per model call — the same
	// unit as the meter's calls counter (one streamed event carrying
	// UsageMetadata). This matches docs/orchestration-design.md's
	// "budget.max_turns remains mast-side turn counting (ADK has no
	// turn cap)": a Task specialist that loops through five model
	// calls before finish_task has spent five turns, not one.
	MaxTurns int

	// Catalog prices each model call exactly, from the model the event
	// says it was billed against.
	//
	// Optional, and strictly better than RatePer1K where a caller can
	// supply it. The flat rate exists because this meter originally saw
	// only UsageMetadata.TotalTokenCount, so internal/compose derives it
	// as the plain average of a model's input and output rates — an
	// approximation that "overcharges input-heavy sessions and
	// undercharges output-heavy ones". Both halves of that premise have
	// since stopped being true: the event carries the input/output split
	// and the cache-read subset, and it carries ModelVersion, so the call
	// can be priced against the same pkg/pricing catalog everything else
	// uses. The error is not small on a real agent — an input-heavy,
	// cache-warm session measured here ran 5.9x over its flat-rate
	// figure, and a cost ceiling that wrong is a ceiling that fires on
	// the wrong sessions.
	//
	// Unknown models fall through to RatePer1K, so a catalog miss never
	// silently drops a session's cost to zero. Unpriced counts them.
	//
	// On a scope, nil means "inherit the session's catalog", matching
	// RatePer1K's rule below. A per-scope catalog is unusual — a rate is
	// a property of the model, and the model is on the event — but the
	// inherit rule costs nothing and keeps the two price knobs behaving
	// alike.
	Catalog *pricing.Catalog

	// RatePer1K is the flat USD price per 1K total tokens (spike
	// pricing model), and the fallback for a call Catalog cannot price.
	//
	// On a scope, zero means "inherit the session's rate" — the right
	// default for a specialist that declares no model of its own, and
	// the reason an un-tiered roster prices exactly as it did before
	// scopes existed.
	RatePer1K float64
}

Limits are the ceilings for one session. Zero values mean unlimited.

type Meter

type Meter struct {
	// contains filtered or unexported fields
}

Meter accumulates usage for one session, and for each scoped agent within it.

func New added in v0.3.0

func New(cfg Config) *Meter

New constructs a Meter from a full config.

func NewMeter

func NewMeter(limits Limits) *Meter

NewMeter constructs a Meter with the given session limits and no per-agent scopes.

func (*Meter) Observe

func (m *Meter) Observe(ev *session.Event) error

Observe folds one event's usage into the meter and reports whether a ceiling has been crossed. Events without UsageMetadata (function responses, control events) are free.

func (*Meter) ScopeSnapshot added in v0.3.0

func (m *Meter) ScopeSnapshot(name string) (tokens int64, costUSD float64, calls int, ok bool)

ScopeSnapshot returns one scoped agent's cumulative usage. ok is false for an agent the meter carries no scope for — which is not the same as an agent that has spent nothing.

func (*Meter) Snapshot

func (m *Meter) Snapshot() (tokens int64, costUSD float64, calls int)

Snapshot returns the session's cumulative usage so far.

func (*Meter) Unpriced added in v0.3.0

func (m *Meter) Unpriced() int

Unpriced reports how many calls a configured Catalog could not price and that fell back to RatePer1K. Non-zero means the cost figure is a mix of two pricing models and should be read as approximate — a caller that displays cost should surface it rather than let a stale catalog quietly downgrade an exact number.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL