Documentation
¶
Overview ¶
Package ratelimit implements the global upstream token bucket gating outbound calls to Rivian's GraphQL gateway.
Why a token bucket and not a leaky bucket / fixed window:
- Bursty work is fine: a UI page that fires 4 parallel queries should sail through the bucket if there's budget, not get slow-walked by a leaky bucket.
- 429s come in waves; we want to absorb the wave with our budget then refill, not run a tight rolling window that punishes the recovery as much as the spike.
Why Redis and not in-process: at N>1 pods, every pod independently classifying its own outbound rate produces N× the real rate against Rivian. A shared bucket forces the whole fleet to honour a single budget.
Storage shape: one Redis hash per class (main + priority) with fields {tokens, ts}. The check-and-decrement runs as a Lua script so the whole "read → refill based on elapsed → maybe decrement → write" sequence is atomic with no client-side round trips.
Two classes:
- "priority" — user-initiated calls that block the UI (Login, RefreshToken, getUserInformation on first paint). Smaller bucket, refilled separately so background work can never starve a logged-in human.
- "main" — everything else (state polls, vehicle metadata, subscriptions' REST seeds). Bigger bucket; absorbs reconnect storms.
Allow returns (allowed, retryAfter). When allowed is false, retryAfter is the time until at least one token will be available, suitable for an HTTP Retry-After header. Callers should respect this — we'll plumb the breaker / api 503 path to use it in the next change.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Class ¶
type Class string
Class is the token-bucket selector. Stable on the wire — the strings appear in Redis keys and Prometheus labels.
const ( // ClassMain is the default for unattended, retry-friendly // upstream calls (state polls, REST seeds, metadata). ClassMain Class = "main" // ClassPriority is for user-blocking calls (Login, // RefreshToken). Reserved budget so a reconnect storm in // ClassMain can't lock new users out. ClassPriority Class = "priority" // ClassBackground is for timer-driven work no one waits on (schedule, OTA and receipts reads). It // draws from the main bucket but may not spend the last Config.BackgroundFloor tokens, so a burst of // background reads at a handover cannot leave the owner's live path (seeds, subscribes) empty-handed. ClassBackground Class = "background" )
type Config ¶
type Config struct {
MainCapacity int
MainRefillPerSec float64
PriorityCapacity int
PriorityRefillPerSec float64
// BackgroundFloor is how many of main's tokens a ClassBackground call must leave alone (zero or less:
// 20 of the 60 burst). The live class may spend them; priority has its own bucket.
BackgroundFloor int
// KeyPrefix lets multiple rivolt deployments share one Redis.
// Defaults to "rivolt:rl".
KeyPrefix string
// Now is injectable for tests; defaults to time.Now.
Now func() time.Time
// FallbackDivisor sizes the per-process bucket used while Redis cannot answer: the
// configured capacity and refill divided by it (zero or less: 3, the number of
// replicas we run). With up to that many replicas the outage budget stays within
// the configured one; it is never "no limit".
FallbackDivisor float64
// OnFallback, when set, is called for each call served from the per-process bucket.
OnFallback func(class Class)
}
Config holds bucket parameters per class. Capacity is the burst allowance; RefillPerSec tunes steady-state throughput.
func DefaultConfig ¶
func DefaultConfig() Config
DefaultConfig returns sane defaults sized for a single-region deployment against Rivian's gateway. Tuned conservatively — the breaker still trips on bursts that exceed these.
type Limiter ¶
type Limiter struct {
// contains filtered or unexported fields
}
Limiter is the public type. Build with New, call Allow on every outbound Rivian call. Safe for concurrent use; all state lives in Redis.
func New ¶
New constructs a Limiter wrapping rdb. Pings Redis up to ctx's deadline and returns an error if the cluster is unreachable — callers can decide whether that's fatal at boot or whether to fall back to a no-op limiter.
func (*Limiter) Allow ¶
Allow attempts to take one token from the named class's bucket. Returns (true, 0) on success and (false, retryAfter) when out of budget. retryAfter is the time until at least one token would be available; suitable for HTTP Retry-After.
On Redis errors Allow does not fail open: it draws from a per-process bucket sized as a share of the global one (Config.FallbackDivisor) and logs at warn. A Redis blip then costs the shared budget its accuracy, not its existence: the upstream path is not black-holed, and not unlimited either.
type Retrying ¶ added in v0.37.6
type Retrying struct {
// contains filtered or unexported fields
}
Retrying is a Limiter that may not exist yet. Redis can be unreachable when the process boots (a rollout that starts Redis and the app together); the limiter used to be disabled for the life of the process in that case, so a later rollout fired Rivian calls with no global budget at all. Retrying keeps trying in the background and starts limiting as soon as Redis answers. Until then, like a Redis error in Allow, it uses the per-process bucket.
func NewRetrying ¶ added in v0.37.6
func NewRetrying(ctx context.Context, rdb *redis.Client, cfg Config, logger *slog.Logger, pingTimeout, retryEvery time.Duration, onReady func()) *Retrying
NewRetrying tries New once (bounded by pingTimeout) and, if that fails, keeps trying every retryEvery in the background until it works or ctx is done. onReady, when set, is called once when the connection is made later.
Source Files