Documentation
¶
Index ¶
- func ObserveTokenUsage(burn *BurnRateTracker, store *StateStore, provider string, tokens int, ...)
- type BurnRateTracker
- func (t *BurnRateTracker) Budget(provider string) int
- func (t *BurnRateTracker) Record(provider string, tokens int, now time.Time) (exhausted bool, retryAfter time.Time)
- func (t *BurnRateTracker) Reset()
- func (t *BurnRateTracker) SetBudget(provider string, budget int)
- func (t *BurnRateTracker) Used(provider string, now time.Time) int
- type State
- type StateStore
- func (s *StateStore) AllExhausted() map[string]time.Time
- func (s *StateStore) ExhaustedAt(now time.Time) map[string]time.Time
- func (s *StateStore) MarkAvailable(provider string)
- func (s *StateStore) MarkQuotaExhausted(provider string, retryAfter time.Time)
- func (s *StateStore) State(provider string, now time.Time) (State, time.Time)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ObserveTokenUsage ¶
func ObserveTokenUsage(burn *BurnRateTracker, store *StateStore, provider string, tokens int, now time.Time)
ObserveTokenUsage funnels post-execution token counts into the burn-rate tracker and cascades a predictive transition into the quota state store when the daily token budget is projected to be exceeded.
Types ¶
type BurnRateTracker ¶
type BurnRateTracker struct {
// contains filtered or unexported fields
}
BurnRateTracker maintains a per-provider rolling window of token usage and compares observed burn-rate against an operator-configured daily token budget. The tracker is safe for concurrent use.
func NewBurnRateTracker ¶
func NewBurnRateTracker() *BurnRateTracker
NewBurnRateTracker returns an empty tracker with no configured budgets.
func (*BurnRateTracker) Budget ¶
func (t *BurnRateTracker) Budget(provider string) int
Budget returns the configured daily budget for provider, or 0 when none is set.
func (*BurnRateTracker) Record ¶
func (t *BurnRateTracker) Record(provider string, tokens int, now time.Time) (exhausted bool, retryAfter time.Time)
Record adds tokens to provider's rolling-window usage at now and evaluates whether predictive exhaustion has been triggered.
func (*BurnRateTracker) Reset ¶
func (t *BurnRateTracker) Reset()
Reset drops all per-provider windows. Budgets are preserved.
func (*BurnRateTracker) SetBudget ¶
func (t *BurnRateTracker) SetBudget(provider string, budget int)
SetBudget installs or replaces the daily token budget for provider. A non-positive budget disables predictive exhaustion for that provider.
type State ¶
type State string
State is the state of one provider in the quota state machine.
const ( // StateAvailable means the provider has no known quota exhaustion and is // eligible for routing. StateAvailable State = "available" // StateQuotaExhausted means the provider returned a quota signal and // should be excluded from routing until RetryAfter has elapsed. StateQuotaExhausted State = "quota_exhausted" )
type StateStore ¶
type StateStore struct {
// contains filtered or unexported fields
}
StateStore is the per-provider quota state machine.
The store is safe for concurrent use.
func NewStateStore ¶
func NewStateStore() *StateStore
NewStateStore returns an empty store. Every provider is implicitly available until MarkQuotaExhausted is called.
func (*StateStore) AllExhausted ¶
func (s *StateStore) AllExhausted() map[string]time.Time
AllExhausted returns provider->retry_after for every provider whose entry is currently in quota_exhausted state, including entries whose retry_after has elapsed. Recovery probing relies on seeing elapsed entries.
func (*StateStore) ExhaustedAt ¶
ExhaustedAt returns provider->retry_after for every provider currently in quota_exhausted state with retry_after > now. The returned map is a copy safe for the caller to mutate.
func (*StateStore) MarkAvailable ¶
func (s *StateStore) MarkAvailable(provider string)
MarkAvailable forces provider back to available, dropping any pending retry_after.
func (*StateStore) MarkQuotaExhausted ¶
func (s *StateStore) MarkQuotaExhausted(provider string, retryAfter time.Time)
MarkQuotaExhausted transitions provider into quota_exhausted with the given retry_after. A zero retryAfter is normalized to available.
Source Files