Documentation
¶
Overview ¶
Package guardrail tracks daily token usage and enforces limits.
Index ¶
- Constants
- func FormatUsage(u Usage, limit int64) string
- func WrapModel(llm model.LLM, tracker *Tracker) model.LLM
- type LimitExceededError
- type Tracker
- func (t *Tracker) Add(inputTokens, outputTokens int32) error
- func (t *Tracker) AddWithCache(inputTokens, outputTokens, cachedTokens int32) error
- func (t *Tracker) BodyTokens() int64
- func (t *Tracker) CacheHitRateToday() float64
- func (t *Tracker) CachePrefixTokens() int64
- func (t *Tracker) CachedTokensToday() int64
- func (t *Tracker) Check() error
- func (t *Tracker) ContextPercentUsed() float64
- func (t *Tracker) ContextWindowSize() int64
- func (t *Tracker) Current() Usage
- func (t *Tracker) LastCachedTokens() int64
- func (t *Tracker) LastPromptTokens() int64
- func (t *Tracker) Limit() int64
- func (t *Tracker) PercentUsed() float64
- func (t *Tracker) Remaining() int64
- func (t *Tracker) ResetContextWindow()
- func (t *Tracker) SetContextWindowSize(size int64)
- func (t *Tracker) SetLastPromptTokens(n int64)
- func (t *Tracker) SetLimit(limit int64)
- func (t *Tracker) TotalUsed() int64
- type Usage
Constants ¶
const DefaultMaxDailyTokens = 50000000
DefaultMaxDailyTokens is the default daily token limit.
Variables ¶
This section is empty.
Functions ¶
func FormatUsage ¶
FormatUsage returns a human-readable usage summary.
Types ¶
type LimitExceededError ¶
LimitExceededError is returned when the daily token limit is reached.
func (*LimitExceededError) Error ¶
func (e *LimitExceededError) Error() string
type Tracker ¶
type Tracker struct {
// contains filtered or unexported fields
}
Tracker tracks daily token usage and enforces a configurable limit.
func New ¶
New creates a tracker with the given daily token limit. It loads any existing usage for today from ~/.pi-go/usage.json.
func NewWithPath ¶
NewWithPath creates a tracker with a custom file path (for testing).
func (*Tracker) Add ¶
Add records token usage from an LLM response. Returns an error if the daily limit would be exceeded.
func (*Tracker) AddWithCache ¶ added in v0.0.48
AddWithCache records token usage from an LLM response, separating the portion of inputTokens that the provider served from its prompt cache. cachedTokens must be a subset of inputTokens; providers report the prompt count inclusive of cache reads.
Returns an error if the daily limit would be exceeded.
func (*Tracker) BodyTokens ¶ added in v0.0.48
BodyTokens returns the tokens accumulated after the stable cached prefix. This is the figure the BodyAfterPrefix compaction scope measures, so a large but fully-cached system prompt never pushes a session toward compaction.
func (*Tracker) CacheHitRateToday ¶ added in v0.0.48
CacheHitRateToday returns the share of today's prompt tokens served from cache (0-100). Returns 0 when no prompt tokens have been recorded.
func (*Tracker) CachePrefixTokens ¶ added in v0.0.48
CachePrefixTokens returns the stable prefix baseline for the current context window — the prompt size of the window's first request.
func (*Tracker) CachedTokensToday ¶ added in v0.0.48
CachedTokensToday returns prompt tokens served from cache today.
func (*Tracker) ContextPercentUsed ¶ added in v0.0.18
ContextPercentUsed returns the percentage of the context window used (0-100+). Returns 0 if context window size is unknown.
func (*Tracker) ContextWindowSize ¶ added in v0.0.18
ContextWindowSize returns the model's context window size.
func (*Tracker) LastCachedTokens ¶ added in v0.0.48
LastCachedTokens returns the cache-read token count of the most recent LLM response. Zero means the last request missed the prompt cache entirely.
func (*Tracker) LastPromptTokens ¶ added in v0.0.18
LastPromptTokens returns the most recent prompt token count from an LLM response. This represents how much of the context window is currently used.
func (*Tracker) PercentUsed ¶
PercentUsed returns the percentage of daily limit consumed (0-100+). Returns 0 if unlimited.
func (*Tracker) Remaining ¶
Remaining returns how many tokens are left today. Returns -1 if unlimited.
func (*Tracker) ResetContextWindow ¶ added in v0.0.48
func (t *Tracker) ResetContextWindow()
ResetContextWindow clears the per-window baselines. Call this after compaction installs a fresh context window, so the next request re-establishes the prefix baseline.
func (*Tracker) SetContextWindowSize ¶ added in v0.0.18
SetContextWindowSize sets the model's context window size for display.
func (*Tracker) SetLastPromptTokens ¶ added in v0.0.48
SetLastPromptTokens overwrites the last-prompt-token baseline so the context gauge can reflect a freshly-compacted window without waiting for the next LLM response to overwrite it.
A compaction pass knows the post-pass token count exactly (it just rewrote the events); pushing it back here makes that knowledge visible immediately instead of leaving the gauge stuck on the pre-compaction number for the duration of a turn.
The cache-prefix baseline is also reset: the new window has, by definition, a new stable prefix, and any dedup pointers into the old prefix are stale. Pass 0 to clear the prefix without setting a new prompt count — useful when the caller knows the window is empty.
type Usage ¶
type Usage struct {
Date string `json:"date"` // YYYY-MM-DD
InputTokens int64 `json:"input_tokens"` // total prompt tokens (includes cached)
OutputTokens int64 `json:"output_tokens"` // total completion tokens
Requests int64 `json:"requests"` // number of LLM calls
// CachedInputTokens is the portion of InputTokens served from a provider
// prompt cache. Providers bill these at a steep discount (Anthropic: 0.1x),
// so InputTokens alone overstates cost whenever caching is active.
//
// Cache *write* tokens are not broken out separately: the genai usage
// metadata has no field for them, so they remain folded into the
// non-cached remainder. Writes bill at 1.25x, so the fresh-token figure
// is a slight underestimate of true cost when a cache is first populated.
CachedInputTokens int64 `json:"cached_input_tokens"`
}
Usage tracks token consumption for the current day.
func (*Usage) CacheHitRate ¶ added in v0.0.48
CacheHitRate returns the share of prompt tokens served from cache (0-100). Returns 0 when no prompt tokens have been recorded.
func (*Usage) FreshInputTokens ¶ added in v0.0.48
FreshInputTokens returns prompt tokens that were not served from cache.
func (*Usage) TotalTokens ¶
TotalTokens returns the combined input + output token count.