Documentation
¶
Overview ¶
Package ratelimit is the domain layer for the RateLimit entity — a named rule set (requests / tokens / concurrency caps) attachable to a Policy, ProviderKey, or Model.
The Attachment type is also defined here because the entities that attach to a RateLimit depend on this package; reverse imports would create cycles.
store.go is the data-access layer for RateLimit. Mirrors the other entities.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func PerModelScope ¶
PerModelScope adds the request model id to a bucket key so per-model rules partition correctly. Used when a Policy.RLBinding has non-empty Models (otherwise the binding is "any model" and gets one shared bucket). The modelID suffix lives in the key, not the namespace, so the Lua hash-tag boundary still groups all of a key's buckets.
func ResolveWithScope ¶
func ResolveWithScope(namespace, subject string, rl *RateLimit) []pkgratelimit.Rule
ResolveWithScope is the policy-less variant used by proxy mode, where the rate-limit subject is not a Policy but a per-key hash or per-IP identifier. namespace identifies the bucket family ("proxy", "proxy-anon"); subject is the request's bucket key (relay-key hash, client IP, etc.). Key construction:
"{namespace}:{subject}:{rule-index}:{meter}"
Returns nil when rl is nil, disabled, or has no Rules.
Types ¶
type Meter ¶
type Meter string
Meter is the dimension a Rule counts.
const ( MeterRequests Meter = "requests" MeterConcurrency Meter = "concurrency" MeterTokens Meter = "tokens" MeterTokensInput Meter = "tokens.input" MeterTokensOutput Meter = "tokens.output" MeterTokensCacheRead Meter = "tokens.cache_read" MeterTokensCacheCreation Meter = "tokens.cache_creation" MeterTokensReasoning Meter = "tokens.reasoning" MeterTokensServerToolUseIn Meter = "tokens.server_tool_use_input" MeterTokensServerToolUseOut Meter = "tokens.server_tool_use_output" )
Closed set of recognised meters. Bare "tokens" sums every token sub-meter; "tokens.<key>" targets one. "concurrency" ignores Strategy.
type RateLimit ¶
type RateLimit struct {
Meta meta.Metadata `json:"metadata" yaml:"metadata"`
Spec Spec `json:"spec" yaml:"spec"`
}
RateLimit is a named rule set. All three OwnerKinds are valid:
- system: bundled, operator-immutable (e.g. inference-api, control-api).
- provider: auto-injected upstream-tier mirror; Owner.ID = Provider id.
- user: operator-defined.
func (*RateLimit) Validate ¶
Validate runs intra-row rules via the shared meta.Validator and enforces the RateLimit-specific owner shape:
- Owner.Kind is required (any of system/provider/user).
- Owner.Kind=provider requires Owner.ID (the Provider id).
Cross-entity checks (provider-owned RLs reference an existing Provider; system mirrors are unique per tier) live in the composition layer.
type Rule ¶
type Rule struct {
Meter Meter `` /* 228-byte string literal not displayed */
Amount int64 `json:"amount" yaml:"amount" validate:"required,gt=0"`
Window time.Duration `json:"window" yaml:"window" validate:"required,gt=0"`
Strategy Strategy `` /* 126-byte string literal not displayed */
}
Rule is one cap. A RateLimit with N rules produces N concurrent buckets at request time. Strategy is per-rule — there is no spec-level default fallback.
Window is a time.Duration; on the wire it accepts a human-readable string ("30s", "1m") or an int64 nanosecond count for round-tripping with the storage format.
type Spec ¶
type Spec struct {
Rules []Rule `json:"rules" yaml:"rules" validate:"required,min=1,dive"`
Enabled *bool `json:"enabled,omitempty" yaml:"enabled,omitempty"` // nil = true
}
Spec carries the rule list and an enable flag.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store is the RateLimit data-access type.
type Strategy ¶
type Strategy string
Strategy is the algorithm used to enforce a Rule.
const ( StrategyTokenBucket Strategy = "token-bucket" StrategySlidingWindow Strategy = "sliding-window" StrategyFixedWindow Strategy = "fixed-window" StrategyLeakyBucket Strategy = "leaky-bucket" // StrategySessionWindow anchors on first request, runs for `window`, // then idles until the next request anchors a fresh window. Used for // session-quota patterns like Anthropic's 5-hour limit. StrategySessionWindow Strategy = "session-window" )
Source Files