Documentation
¶
Overview ¶
Package ratelimit holds provider-agnostic rate-limiting helpers shared by clients that talk to LLM/embedding APIs directly.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ParseRetryDelay ¶
ParseRetryDelay extracts a server-declared retry delay for a throttled response, preferring the most authoritative source available:
- A google.rpc.RetryInfo retryDelay in the JSON error body (e.g. "31s").
- The HTTP Retry-After header (seconds, integer or float).
Returns 0 when neither is present or parseable; callers should fall back to their own exponential backoff.
Types ¶
type QuotaScope ¶
type QuotaScope int
QuotaScope classifies which quota dimension a 429 response exhausted. The distinction drives fundamentally different reactions:
- Minute-scoped exhaustion (RPM/TPM) is transient: back off, shrink the request budget, retry within seconds-to-minutes.
- Day-scoped exhaustion (RPD) is terminal for the window: every retry until the provider's daily reset is guaranteed to fail and — on metered APIs — costs money. Callers must stop and reschedule.
const ( // QuotaScopeNone means the body carries no recognizable quota details. QuotaScopeNone QuotaScope = iota // QuotaScopeMinute means a per-minute budget (RPM or TPM) was exceeded. QuotaScopeMinute // QuotaScopeDay means a per-day budget (RPD) was exceeded. QuotaScopeDay )
func ClassifyQuotaError ¶
func ClassifyQuotaError(body []byte) QuotaScope
ClassifyQuotaError inspects a 429 response body for google.rpc.QuotaFailure details and reports the strictest (longest) quota scope it names. Bodies without recognizable details classify as QuotaScopeNone — callers should treat that as minute-scoped (transient) since per-day exhaustion is practically always annotated.
Note: this is Google-specific today (google.rpc.QuotaFailure detail type). Any handler keying off QuotaScopeDay inherits Google's reset semantics (midnight Pacific — see nextQuotaReset in the controller); extend this alongside a per-provider reset policy when supporting other providers.
Source Files