Documentation
¶
Overview ¶
Package ratelimit provides request and token rate limiting for LLM sources. Supports both Redis-backed shared counters and in-memory per-node fallback.
Index ¶
Constants ¶
This section is empty.
Variables ¶
View Source
var ErrRateLimitExceeded = errors.New("rate limit exceeded")
Functions ¶
This section is empty.
Types ¶
type Limiter ¶
type Limiter struct {
// contains filtered or unexported fields
}
Limiter enforces per-minute request and token limits for an LLM source. When a StoreSource is provided, counters are shared via Redis keys. Otherwise, counters are tracked in-memory (per-node only).
func New ¶
func New(sourceName string, rpm, tpm int, store sources.StoreSource) *Limiter
New creates a rate limiter for the given source. If store is nil, uses in-memory counters.
Source Files
Click to show internal directories.
Click to hide internal directories.