Documentation
¶
Overview ¶
Package ratelimiter provides rate limiting primitives for LLM calls, mirroring langchain_core.rate_limiters.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func RateLimited ¶
RateLimited reports whether err is the rate-limited sentinel.
Types ¶
type InMemoryRateLimiter ¶
type InMemoryRateLimiter struct {
// contains filtered or unexported fields
}
InMemoryRateLimiter is a token-bucket style rate limiter, mirroring langchain_core.rate_limiters.InMemoryRateLimiter.
It enforces a maximum number of requests per time window. The implementation uses a simple sliding window with a mutex and is safe for concurrent use.
func NewInMemoryRateLimiter ¶
func NewInMemoryRateLimiter(cfg InMemoryRateLimiterConfig) *InMemoryRateLimiter
NewInMemoryRateLimiter builds an InMemoryRateLimiter.
type InMemoryRateLimiterConfig ¶
type InMemoryRateLimiterConfig struct {
// RequestsPerSecond is the maximum average number of requests per second.
RequestsPerSecond float64
// CheckEvery is how often the bucket is refilled. Defaults to 0.1s.
CheckEvery time.Duration
// MaxBucketSize is the maximum burst size. Defaults to the same as
// RequestsPerSecond.
MaxBucketSize float64
}
InMemoryRateLimiterConfig configures an InMemoryRateLimiter.
type RateLimiter ¶
type RateLimiter interface {
// Acquire blocks until a request is permitted, or the context is done.
Acquire(ctx context.Context) error
}
RateLimiter is the interface for rate limiters that can gate requests.
Source Files
Click to show internal directories.
Click to hide internal directories.