Documentation
¶
Overview ¶
Package ratelimit implements Backyard's per-instance request limits (SPEC §11): token buckets per category and key, and a daily budget of AI tokens.
Keys identify the caller: the user id for authenticated users, or the client IP for anonymous ones (see Key). Every category has its own bucket per key whose capacity is the per-minute limit and which refills at limit/60 tokens per second, so a client may burst up to one minute's worth of requests and then continues at the sustained rate.
Buckets that have refilled completely are indistinguishable from new ones and are evicted lazily, so memory stays proportional to the number of recently active callers. The daily AI token counter resets at midnight UTC.
A Limiter is safe for concurrent use. Limits are per instance: with N server instances behind a load balancer the effective limit is up to N times higher.
Index ¶
- Constants
- Variables
- func Key(userID string, anonymous bool, clientIP string) string
- func RetryAfterSeconds(d time.Duration) int
- type Category
- type Limiter
- func (l *Limiter) AddTokens(key string, n int64)
- func (l *Limiter) Allow(cat Category, key string) (bool, time.Duration)
- func (l *Limiter) CheckTokens(key string) (bool, time.Duration)
- func (l *Limiter) Enabled() bool
- func (l *Limiter) Limit(cat Category) int
- func (l *Limiter) TokensUsed(key string) int64
- func (l *Limiter) Tracked(cat Category) int
- type Limits
- type Option
Constants ¶
const ( // DefaultMaxKeys bounds the number of tracked keys per category (and for // the daily token counter). DefaultMaxKeys = 1 << 20 )
Variables ¶
Categories lists every category.
Functions ¶
func Key ¶
Key returns the rate-limit key of a caller: the user id, or the client IP for anonymous users (anonymous ids are cheap to mint by dropping cookies).
func RetryAfterSeconds ¶
RetryAfterSeconds converts a wait duration into a Retry-After header value (whole seconds, rounded up, at least 1).
Types ¶
type Category ¶
type Category string
Category is a class of requests sharing one limit.
const ( // API counts every site/hub API request (ratelimit.api_per_minute). API Category = "api" // DBWrite counts database writes (ratelimit.db_writes_per_minute). DBWrite Category = "db_write" // AI counts AI calls: chat, image, embed (ratelimit.ai_per_minute). AI Category = "ai" // Warehouse counts warehouse queries (ratelimit.warehouse_per_minute). Warehouse Category = "warehouse" // Upload counts file uploads (ratelimit.uploads_per_minute). Upload Category = "upload" )
Rate-limited categories.
type Limiter ¶
type Limiter struct {
// contains filtered or unexported fields
}
Limiter enforces Limits. The zero value is not usable; call New.
func New ¶
func New(cfg config.RateLimitConfig, opts ...Option) *Limiter
New returns a limiter for cfg (cfg.Disabled turns every check into a no-op).
func NewWithLimits ¶
NewWithLimits returns a limiter for explicit limits.
func (*Limiter) Allow ¶
Allow consumes one token of cat for key. When the bucket is empty it returns false and how long the caller should wait before retrying.
func (*Limiter) CheckTokens ¶
CheckTokens reports whether key still has AI token budget left today. When the budget is exhausted it returns false and the time until it resets (midnight UTC).
func (*Limiter) TokensUsed ¶
TokensUsed returns the AI tokens consumed by key today.
type Limits ¶
Limits are the configured limits. A per-minute limit <= 0 and an AITokensPerDay <= 0 mean "unlimited".
func LimitsFromConfig ¶
func LimitsFromConfig(c config.RateLimitConfig) Limits
LimitsFromConfig extracts the request limits from the configuration.
type Option ¶
type Option func(*Limiter)
Option configures a Limiter.
func WithMaxKeys ¶
WithMaxKeys bounds the number of keys tracked per category. When the bound is reached, idle entries are evicted first and then arbitrary ones (which only ever makes those callers less limited, never more).
Source Files