Documentation
¶
Overview ¶
Package ratelimit slows reconciles down automatically when an external API signals a rate limit, per endpoint key, or when the controller's own container is CPU-saturated. It replaces hand-tuning --max-reconcile-rate and --poll per installation.
Design: docs/design-rate-limit-governor.md. The Signal shape and its meaning are shared with unstructured-runtime's pkg/ratelimit, which declares the same type. The two are deliberately duplicated rather than imported from a shared module (design §10, open question 1), so a change here must be made there too.
The runtime makes no outbound calls of its own: the provider does. So a rate limit is visible only to the provider, which reports it with Report; the runtime acts on it through a Governor. Without a Governor, Report is a no-op and nothing changes.
Index ¶
- Constants
- func Report(ctx context.Context, s Signal)
- func WithSink(ctx context.Context, s Sink) context.Context
- type DeferReason
- type Deferral
- type Governor
- func (g *Governor) Acquire(ctx context.Context, key string) (release func(), d *Deferral)
- func (g *Governor) AcquireAll(ctx context.Context, keys []string) (release func(), d *Deferral)
- func (g *Governor) Capacity(key string) int
- func (g *Governor) Ceiling() int
- func (g *Governor) LocalCapacity() int
- func (g *Governor) NeedLeaderElection() bool
- func (g *Governor) Observe(s Signal)
- func (g *Governor) RegisterReconciler()
- func (g *Governor) SaturationEnabled() bool
- func (g *Governor) Snapshot() Snapshot
- func (g *Governor) Start(ctx context.Context) error
- type InputSnapshot
- type KeySnapshot
- type MetricsRecorder
- type Options
- type SaturationOptions
- type Signal
- type SignalKind
- type Sink
- type Snapshot
Constants ¶
const ( // DefaultSamplePeriod is how often the sampler reads the cgroup files. CPU // pressure gains nothing from a faster sample, because "avg10" is already // a 10-second average, and a faster sample costs CPU in the container it // protects. A shorter period only matters to the experimental memory // input. DefaultSamplePeriod = time.Second // DefaultStartupRampInterval is how long local capacity waits between // steps of +1 from the start until its first cut, so 16 workers are all // admitted after about 30s. It is not tied to the sample period. DefaultStartupRampInterval = 2 * time.Second // DefaultCPUPressureThreshold is the cpu.pressure "some avg10" percentage // above which the controller counts as CPU-saturated. PROVISIONAL. DefaultCPUPressureThreshold = 50.0 // DefaultWorkingSetSoftRatio is the working-set share of memory.max at // which the experimental memory input halves local capacity. PROVISIONAL. DefaultWorkingSetSoftRatio = 0.80 // DefaultWorkingSetHardRatio is the working-set share of memory.max at // which the experimental memory input drops local capacity straight to 1. // PROVISIONAL. DefaultWorkingSetHardRatio = 0.90 )
Saturation defaults (design §7), decided by the user on 2026-11-05 from the kind measurement in docs/configuration.md.
The CPU pressure threshold is PROVISIONAL: the measurement supports 50, and core-provider is validating it. The working-set ratios belong to the experimental memory input, which is off by default: re-measured, that input did not prevent OOMKills (design §11).
const ( InputCPUPressure = "cpu_pressure" InputMemoryWorkingSet = "memory_working_set" )
Saturation inputs, as named in the metrics.
const DefaultRecoveryInterval = 30 * time.Second
DefaultRecoveryInterval is how long a key, or the local capacity after a cut, must go clean before its capacity rises by one.
Variables ¶
This section is empty.
Functions ¶
func Report ¶
Report records s on the sink ctx carries. It is a no-op when ctx carries no sink, which is the case whenever no Governor is configured, so a provider can report unconditionally. A Signal with an empty Key is dropped: the empty key is not a scope.
Report from the provider's own code (or let pkg/ratelimit/httpsignal do it) with the context the runtime passed in, on success and on error.
Types ¶
type DeferReason ¶
type DeferReason string
DeferReason says why a call was not let through.
const ( // DeferThrottled: the server refused a call for this key and named a // deadline. Deferral.Until is that deadline. DeferThrottled DeferReason = "Throttled" // DeferAtCapacity: the key, or the local capacity while it ramps up after // a start or a saturation, has as many calls in flight as it allows. There // is no server deadline. DeferAtCapacity DeferReason = "AtCapacity" // DeferSaturated: the controller's own container is saturated and the // process-wide local capacity is full. There is no server deadline. DeferSaturated DeferReason = "Saturated" // DeferRamping: the local capacity is full and still on its startup ramp, // before its first cut, with nothing saturated. The controller has only // just started, which says nothing about the object, so the reconciler // requeues quietly: it doesn't touch Synced or record an event. It is the // one deferral a provider's users don't see on the object; the deferrals // metric counts it under cause "ramping". DeferRamping DeferReason = "Ramping" )
func (DeferReason) Cause ¶
func (r DeferReason) Cause() string
Cause is the value of the cause attribute on the deferrals metric: throttled, capacity, saturated or ramping.
type Deferral ¶
type Deferral struct {
// Key is the key that stopped the call. For a Local deferral no key stopped
// it: Key is then the call's key, or empty when the call has no key or
// several, since naming one of several would blame the wrong endpoint.
Key string
// Reason says why.
Reason DeferReason
// Until is the server's own deadline. It is set only for DeferThrottled;
// the governor never invents a deadline.
Until time.Time
// RequeueAfter is how long the caller waits before it tries again. For
// DeferThrottled it is the time left until Until, never less. For
// DeferRamping it is a random delay between 1s and the time the ramp still
// needs, at most the RecoveryInterval, so the object is back before the
// ramp ends. Otherwise it is a random delay between 1s and the
// RecoveryInterval.
RequeueAfter time.Duration
// Capacity is the capacity that was full: the key's, or the local one when
// Local is set.
Capacity int
// Local is set when the process-wide local capacity was full, rather than
// the key's. Its message names the cause and no key: the controller is
// saturated (DeferSaturated), recovering after a cut (DeferAtCapacity), or
// on its startup ramp (DeferRamping).
Local bool
// Cause names the saturated inputs for DeferSaturated, for example
// "memory working set 93% of limit".
Cause string
}
A Deferral says a call must not be made now, why, and when to try again.
func (*Deferral) Message ¶
Message is the Synced condition message for a deferred pass. It names the key or the cause, and the deadline as an RFC 3339 time when there is one, or says that it is waiting for capacity.
The throttled wording is the same as unstructured-runtime's, so two controllers deferring against the same endpoint say it in the same words.
type Governor ¶
type Governor struct {
// contains filtered or unexported fields
}
A Governor gates external calls per endpoint key, and across all keys by the controller's own saturation. Create it with New, share one per process, pass it to each reconciler with reconciler.WithRateLimitGovernor, and add it to the manager (mgr.Add) so its saturation sampler runs.
A Governor is safe for concurrent use.
func New ¶
New returns a Governor. It returns an error if MaxConcurrentReconciles is below 1 or a saturation setting is out of range: a governor that silently capped every key at 1 would be a slow-down nobody configured.
func (*Governor) Acquire ¶
Acquire asks to make one gated call for key. It never blocks and does no I/O: it either grants a slot in both the key's capacity and the local capacity, returning the release func to call when the call returns, or it returns a Deferral at once. An empty key asks for a local slot only: the call of a provider without a RateLimitKeyer, which saturation still bounds.
A blocking gate would park workers: with MaxConcurrentReconciles workers and one throttled key, every worker could end up waiting on that key while objects on healthy endpoints stop reconciling (design §5).
release is nil exactly when the Deferral is not. Calling release more than once is safe.
func (*Governor) AcquireAll ¶ added in v1.10.0
AcquireAll is Acquire for a call that talks to several rate-limited scopes, such as a sync between two endpoints. The call needs a slot in every key plus one local slot, all or nothing: it either holds them all, or holds none and gets a Deferral. Nothing is held while deferred, so two calls that share keys can't wait on each other.
Empty keys are ignored and repeated keys count once, so two names for the same scope take one slot. With no keys left it asks for a local slot only.
A hold on any key outranks a full capacity, as it does for one key. With several keys held the Deferral names the latest deadline: the call can't run before that one lifts, so waking at an earlier one would only defer it again.
func (*Governor) Capacity ¶
Capacity reports key's current capacity: the most calls for it that may be in flight at once. It is the number operators used to tune by hand.
func (*Governor) Ceiling ¶
Ceiling reports the capacity every key and the local capacity recover to.
func (*Governor) LocalCapacity ¶
LocalCapacity reports the process-wide local capacity.
func (*Governor) NeedLeaderElection ¶
NeedLeaderElection is false: every replica samples its own container.
func (*Governor) Observe ¶
Observe folds a provider's signal into its key's state. The Governor is the Sink the reconciler installs, so Report reaches it. A signal is applied to its own key, even when that differs from the key the call was gated on.
- Throttled: capacity drops to 1 and the gate closes until the signal's deadline, max(RetryAfter, ResetAt). A later deadline extends the hold; an earlier one never shortens it.
- Advisory: capacity halves, with a minimum of 1.
- Either restarts the key's clean RecoveryInterval.
- Anything else (a healthy quota report) changes nothing.
func (*Governor) RegisterReconciler ¶
func (g *Governor) RegisterReconciler()
RegisterReconciler tells the Governor one more reconciler, running MaxConcurrentReconciles workers, shares it, and raises the ceiling to match. Key capacities still at the old ceiling move to the new one, and so does the local capacity when saturation is off; with saturation on, local capacity recovers to the new ceiling at its own pace. reconciler. WithRateLimitGovernor calls it; call it yourself only if you gate calls through Acquire outside pkg/reconciler.
func (*Governor) SaturationEnabled ¶
SaturationEnabled reports whether the local (saturation) input is on, in which case every gated call needs a local slot, keyed or not.
type InputSnapshot ¶
type InputSnapshot struct {
// Input is cpu_pressure or memory_working_set.
Input string
// Enabled is false when the input is missing (no cgroup v2, no PSI, no
// memory limit) or saturation is off. A disabled input has no Percent:
// zero would read as healthy.
Enabled bool
// Percent is the input's value: PSI some avg10 for cpu_pressure, the working
// set as a share of memory.max for the working set.
Percent float64
}
InputSnapshot is one saturation input's last sample, for metrics.
type KeySnapshot ¶
type KeySnapshot struct {
Key string
Capacity int
// SecondsToReset is the time left until the key's hold lifts, 0 when it
// isn't held.
SecondsToReset float64
}
KeySnapshot is one key's state, for metrics.
type MetricsRecorder ¶
type MetricsRecorder interface {
// IncRateLimitDeferral counts one deferral of a call for key.
IncRateLimitDeferral(ctx context.Context, key string, reason DeferReason)
// IncRateLimitSignal counts one throttled or advisory signal for key.
IncRateLimitSignal(ctx context.Context, key string, kind SignalKind)
// ObserveRateLimit registers snapshot as the source of the governor's
// gauges: per-key capacity and seconds to reset, local capacity, and each
// saturation input.
ObserveRateLimit(snapshot func() Snapshot)
}
MetricsRecorder records the governor's metrics. *telemetry.Metrics implements it.
type Options ¶
type Options struct {
// MaxConcurrentReconciles is each controller's worker count (a provider's
// --max-reconcile-rate). The ceiling every capacity recovers to, per key
// and for the process-wide local capacity, is this times the number of
// reconcilers sharing the Governor (each reconciler.WithRateLimitGovernor
// registers one), that is the process's total workers: enabling the
// governor never holds a healthy endpoint below what the workers could do
// today. Required, at least 1; controller.Options.SetupRateLimitGovernor
// fills it from controller.Options.MaxConcurrentReconciles.
MaxConcurrentReconciles int
// IgnoreSignals makes Observe drop every Signal, so only the saturation
// input gates calls. controller.Options.SetupRateLimitGovernor sets it
// when saturation is enabled and the rate-limit input is not. Default
// false: signals are honoured.
IgnoreSignals bool
// RecoveryInterval is how long a key, or the local capacity after a cut,
// must go clean before its capacity rises by one, up to the ceiling. It is
// also the upper bound of the at-capacity requeue delay. Defaults to 30s.
RecoveryInterval time.Duration
// Saturation configures the local input (design §7). Off unless Enabled.
Saturation SaturationOptions
// Logger logs disabled saturation inputs (once each) and capacity
// changes. Defaults to a no-op logger.
Logger logging.Logger
// Metrics records deferrals, signals and the gauges. Optional.
Metrics MetricsRecorder
}
Options configures a Governor.
type SaturationOptions ¶
type SaturationOptions struct {
// Enabled turns the local input on, independently of the rate-limit input.
// It reads CPU pressure only.
Enabled bool
// ExperimentalMemoryWorkingSet adds the memory working set to the local
// input. EXPERIMENTAL, off by default, and not memory protection: on the
// kind re-measurement it did not prevent OOMKills, in 0 of 19 runs on an
// overcommitted workload (docs/configuration.md). The calls admitted
// between two samples, and the single call the floor always lets through,
// can still cross the limit.
ExperimentalMemoryWorkingSet bool
// CgroupDir is the cgroup v2 directory to read. Empty means the process's
// own cgroup: /sys/fs/cgroup joined with the path in /proc/self/cgroup
// when that directory exists, else /sys/fs/cgroup (which, with a private
// cgroup namespace, as Kubernetes gives containers on cgroup v2, is the
// container's own).
CgroupDir string
// SamplePeriod is how often the files are read. Defaults to 1s. A shorter
// period only matters to the experimental memory input: CPU pressure's
// "avg10" is already a 10-second average, so sampling it faster gains
// nothing and costs CPU in the container it protects.
SamplePeriod time.Duration
// StartupRampInterval is the time between steps of +1 from the start until
// the first cut. Defaults to 2s. A step happens on the first sample after
// the interval has passed, so a SamplePeriod above it sets the pace.
StartupRampInterval time.Duration
// CPUPressureThreshold is the cpu.pressure "some avg10" percentage (0-100)
// above which the controller is saturated. Defaults to
// DefaultCPUPressureThreshold (provisional).
CPUPressureThreshold float64
// WorkingSetSoftRatio is the share of memory.max (0-1) at or above which
// the working set (memory.current minus inactive_file) halves local
// capacity. Read only with ExperimentalMemoryWorkingSet. Defaults to 0.80
// (provisional).
WorkingSetSoftRatio float64
// WorkingSetHardRatio is the share of memory.max (0-1) at or above which
// local capacity drops straight to 1. Read only with
// ExperimentalMemoryWorkingSet. Defaults to 0.90 (provisional).
WorkingSetHardRatio float64
}
SaturationOptions configures the local input: the controller's own CPU saturation, read from its container's cgroup v2 files. With it enabled, every gated call (Connect and each ExternalClient method) needs a slot in the process-wide local capacity, whether or not the provider implements a RateLimitKeyer. Local capacity starts at 1, rises by one per StartupRampInterval until its first cut, and after any cut rises by one per clean RecoveryInterval, up to the ceiling.
Memory is not part of this unless ExperimentalMemoryWorkingSet is set.
type Signal ¶
type Signal struct {
// Key identifies the rate-limited scope: normally the endpoint host, or
// host|credential-reference when the quota is per identity. It must be the
// same string the provider's RateLimitKeyer returns for the object, or the
// signal throttles a key no call is gated on. A Signal with an empty Key is
// dropped.
Key string
// RetryAfter is how long the server asked the client to wait, when it said
// so as a delay. Zero when it gave an instant instead.
RetryAfter time.Duration
// ResetAt is when the server's window resets, when it gave an instant.
// When both RetryAfter and ResetAt are set, the later of the two wins.
ResetAt time.Time
// Remaining and Limit are the quota counters, when the server publishes
// them. HasQuota tells "Remaining is 0" apart from "no counters were sent".
Remaining, Limit int
HasQuota bool
// Throttled means the server refused this request. A Signal that is not
// Throttled is advisory at most: the call happened, and the server says the
// client is close to, or already being slowed by, a limit.
Throttled bool
}
Signal is one observation of an endpoint's rate-limit state, as the provider saw it. Report it on a successful response as well as on a failed one: Azure DevOps announces a delay on a successful response before it starts refusing.
Field names and meaning are identical to unstructured-runtime's Signal.
type SignalKind ¶
type SignalKind string
SignalKind is what a Signal means to the governor.
const ( // SignalThrottled closes the gate for the key until the signal's deadline, // and drops the key's capacity to 1. SignalThrottled SignalKind = "throttled" // SignalAdvisory halves the key's capacity, with a minimum of 1. SignalAdvisory SignalKind = "advisory" // SignalNone changes nothing. A healthy quota report, such as GitHub's // X-RateLimit-* headers on every response, is SignalNone. SignalNone SignalKind = "none" )
func Classify ¶
func Classify(s Signal, now time.Time) SignalKind
Classify says what s means at now. The governor acts on it, and the conformance helpers in pkg/reconciler/reconcilertest use it to tell an advisory signal from a healthy one, so both read a Signal the same way.
- Throttled is SignalThrottled.
- Otherwise, quota counters under a quarter of the limit are SignalAdvisory, as is a RetryAfter (a server asking for a delay on a call it served, as Azure DevOps does), or a future ResetAt sent without quota counters.
- A future ResetAt sent next to a healthy quota is SignalNone: it is only the end of the current window, and GitHub sends it on every response.
type Sink ¶
type Sink interface {
Observe(Signal)
}
Sink receives signals. The reconciler installs the Governor as the sink of the context it passes to Connect and to every ExternalClient call.
type Snapshot ¶
type Snapshot struct {
Keys []KeySnapshot
LocalCapacity int
Inputs []InputSnapshot
}
Snapshot is the governor's state, for metrics.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package httpsignal reports rate-limit signals from HTTP responses, so a provider that talks to GitHub, Azure DevOps or any API using the same headers doesn't rewrite the parsing.
|
Package httpsignal reports rate-limit signals from HTTP responses, so a provider that talks to GitHub, Azure DevOps or any API using the same headers doesn't rewrite the parsing. |