guardrail

package
v0.2.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 23, 2026 License: MIT Imports: 9 Imported by: 0

Documentation

Overview

Package guardrail tracks daily token usage and enforces limits.

Index

Constants

View Source
const DefaultMaxDailyTokens = 50000000

DefaultMaxDailyTokens is the default daily token limit.

Variables

This section is empty.

Functions

func FormatUsage

func FormatUsage(u Usage, limit int64) string

FormatUsage returns a human-readable usage summary.

func WrapModel

func WrapModel(llm model.LLM, tracker *Tracker) model.LLM

WrapModel wraps an LLM model to track token usage via the guardrail tracker. Each response's UsageMetadata is recorded. If the daily limit is exceeded, subsequent calls return an error.

Types

type LimitExceededError

type LimitExceededError struct {
	Limit int64
	Used  int64
	Asked int64
}

LimitExceededError is returned when the daily token limit is reached.

func (*LimitExceededError) Error

func (e *LimitExceededError) Error() string

type Tracker

type Tracker struct {
	// contains filtered or unexported fields
}

Tracker tracks daily token usage and enforces a configurable limit.

func New

func New(maxDailyTokens int64) *Tracker

New creates a tracker with the given daily token limit. It loads any existing usage for today from ~/.pi-go/usage.json.

func NewWithPath

func NewWithPath(maxDailyTokens int64, path string) *Tracker

NewWithPath creates a tracker with a custom file path (for testing).

func (*Tracker) Add

func (t *Tracker) Add(inputTokens, outputTokens int32) error

Add records token usage from an LLM response. Returns an error if the daily limit would be exceeded.

func (*Tracker) AddWithCache added in v0.0.48

func (t *Tracker) AddWithCache(inputTokens, outputTokens, cachedTokens int32) error

AddWithCache records token usage from an LLM response, separating the portion of inputTokens that the provider served from its prompt cache. cachedTokens must be a subset of inputTokens; providers report the prompt count inclusive of cache reads.

Returns an error if the daily limit would be exceeded.

func (*Tracker) BodyTokens added in v0.0.48

func (t *Tracker) BodyTokens() int64

BodyTokens returns the tokens accumulated after the stable cached prefix. This is the figure the BodyAfterPrefix compaction scope measures, so a large but fully-cached system prompt never pushes a session toward compaction.

func (*Tracker) CacheHitRateToday added in v0.0.48

func (t *Tracker) CacheHitRateToday() float64

CacheHitRateToday returns the share of today's prompt tokens served from cache (0-100). Returns 0 when no prompt tokens have been recorded.

func (*Tracker) CachePrefixTokens added in v0.0.48

func (t *Tracker) CachePrefixTokens() int64

CachePrefixTokens returns the stable prefix baseline for the current context window — the prompt size of the window's first request.

func (*Tracker) CachedTokensToday added in v0.0.48

func (t *Tracker) CachedTokensToday() int64

CachedTokensToday returns prompt tokens served from cache today.

func (*Tracker) Check

func (t *Tracker) Check() error

Check returns an error if the daily limit is already exceeded.

func (*Tracker) ContextPercentUsed added in v0.0.18

func (t *Tracker) ContextPercentUsed() float64

ContextPercentUsed returns the percentage of the context window used (0-100+). Returns 0 if context window size is unknown.

func (*Tracker) ContextWindowSize added in v0.0.18

func (t *Tracker) ContextWindowSize() int64

ContextWindowSize returns the model's context window size.

func (*Tracker) Current

func (t *Tracker) Current() Usage

Current returns the current usage snapshot.

func (*Tracker) LastCachedTokens added in v0.0.48

func (t *Tracker) LastCachedTokens() int64

LastCachedTokens returns the cache-read token count of the most recent LLM response. Zero means the last request missed the prompt cache entirely.

func (*Tracker) LastPromptTokens added in v0.0.18

func (t *Tracker) LastPromptTokens() int64

LastPromptTokens returns the most recent prompt token count from an LLM response. This represents how much of the context window is currently used.

func (*Tracker) Limit

func (t *Tracker) Limit() int64

Limit returns the configured daily token limit.

func (*Tracker) PercentUsed

func (t *Tracker) PercentUsed() float64

PercentUsed returns the percentage of daily limit consumed (0-100+). Returns 0 if unlimited.

func (*Tracker) Remaining

func (t *Tracker) Remaining() int64

Remaining returns how many tokens are left today. Returns -1 if unlimited.

func (*Tracker) ResetContextWindow added in v0.0.48

func (t *Tracker) ResetContextWindow()

ResetContextWindow clears the per-window baselines. Call this after compaction installs a fresh context window, so the next request re-establishes the prefix baseline.

func (*Tracker) SetContextWindowSize added in v0.0.18

func (t *Tracker) SetContextWindowSize(size int64)

SetContextWindowSize sets the model's context window size for display.

func (*Tracker) SetLastPromptTokens added in v0.0.48

func (t *Tracker) SetLastPromptTokens(n int64)

SetLastPromptTokens overwrites the last-prompt-token baseline so the context gauge can reflect a freshly-compacted window without waiting for the next LLM response to overwrite it.

A compaction pass knows the post-pass token count exactly (it just rewrote the events); pushing it back here makes that knowledge visible immediately instead of leaving the gauge stuck on the pre-compaction number for the duration of a turn.

The cache-prefix baseline is also reset: the new window has, by definition, a new stable prefix, and any dedup pointers into the old prefix are stale. Pass 0 to clear the prefix without setting a new prompt count — useful when the caller knows the window is empty.

func (*Tracker) SetLimit

func (t *Tracker) SetLimit(limit int64)

SetLimit updates the daily token limit.

func (*Tracker) TotalUsed

func (t *Tracker) TotalUsed() int64

TotalUsed returns total tokens consumed today.

type Usage

type Usage struct {
	Date         string `json:"date"`          // YYYY-MM-DD
	InputTokens  int64  `json:"input_tokens"`  // total prompt tokens (includes cached)
	OutputTokens int64  `json:"output_tokens"` // total completion tokens
	Requests     int64  `json:"requests"`      // number of LLM calls

	// CachedInputTokens is the portion of InputTokens served from a provider
	// prompt cache. Providers bill these at a steep discount (Anthropic: 0.1x),
	// so InputTokens alone overstates cost whenever caching is active.
	//
	// Cache *write* tokens are not broken out separately: the genai usage
	// metadata has no field for them, so they remain folded into the
	// non-cached remainder. Writes bill at 1.25x, so the fresh-token figure
	// is a slight underestimate of true cost when a cache is first populated.
	CachedInputTokens int64 `json:"cached_input_tokens"`
}

Usage tracks token consumption for the current day.

func (*Usage) CacheHitRate added in v0.0.48

func (u *Usage) CacheHitRate() float64

CacheHitRate returns the share of prompt tokens served from cache (0-100). Returns 0 when no prompt tokens have been recorded.

func (*Usage) FreshInputTokens added in v0.0.48

func (u *Usage) FreshInputTokens() int64

FreshInputTokens returns prompt tokens that were not served from cache.

func (*Usage) TotalTokens

func (u *Usage) TotalTokens() int64

TotalTokens returns the combined input + output token count.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL