Documentation
¶
Overview ¶
Package overload is a 5-stage CPU degradation ladder driven by observe.Collector.
A background polling goroutine samples collector.Snapshot().CPUUtilization every [Config.PollInterval] and transitions the stage atomically. Hot-path handlers read the stage via a single atomic load — no locks, no map lookups — so the Normal path adds only a few nanoseconds of overhead.
Stages (thresholds configurable, hysteresis applied to downward transitions):
Normal — pass through unchanged
Expand — signal best-effort worker widening; pass through
Reap — opt-in runtime.GC() then pass through
Reorder — low-priority requests return 503; others pass
Backpressure — low-priority requests sleep BackpressureDelay; others
return 503; exempt pass through
Reject — all non-exempt requests return 503 + Retry-After
Priority is application-defined via [Config.PriorityFunc]. Without it, Reorder passes everything and Backpressure delays everything.
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func New ¶
func New(config ...Config) celeris.HandlerFunc
New returns an overload middleware that uses the configured [Config.CollectorProvider] to sample CPU. A background goroutine is started; cancel it via [Config.StopContext] or use NewWithController for explicit control.
Types ¶
type Config ¶
type Config struct {
// CollectorProvider returns the [observe.Collector] from which
// CPU utilization is sampled. Required.
CollectorProvider func() *observe.Collector
// Thresholds sets the per-stage CPU fractions. Zero-valued fields
// fall back to the defaults documented on [Thresholds].
Thresholds Thresholds
// DepthThresholds sets the per-stage in-flight request count that
// escalates the driver. Optional; zero-valued fields are ignored.
// See [DepthThresholds] for guidance.
DepthThresholds DepthThresholds
// LatencyThresholds sets the per-stage EMA tail-latency target that
// escalates the driver. Optional; zero-valued fields are ignored.
LatencyThresholds LatencyThresholds
// LatencyEMAAlpha is the smoothing factor for the in-flight latency
// EMA. 0..1 range; higher weights recent samples more aggressively.
// Default: 0.1 (10% weight to current sample, 90% to history).
LatencyEMAAlpha float64
// PollInterval is how often the background goroutine samples CPU
// and updates the stage. Default: 1 second.
PollInterval time.Duration
// ExemptPaths are request paths that never get degraded.
ExemptPaths []string
// ExemptFunc, when non-nil, short-circuits the degradation logic
// for requests where it returns true.
ExemptFunc func(*celeris.Context) bool
// PriorityFunc classifies request priority for Reorder and
// Backpressure stages. Higher values win. When nil, all requests
// share the same priority (0) so Reorder passes everything and
// Backpressure delays everything.
PriorityFunc func(*celeris.Context) int
// PriorityThreshold is the cutoff below which requests are rejected
// at StageReorder and delayed/rejected at StageBackpressure.
// Default: 0 (priority < 0 is "low").
PriorityThreshold int
// BackpressureDelay is the artificial latency added at
// StageBackpressure for non-exempt low-priority requests.
// Default: 50 ms.
BackpressureDelay time.Duration
// BackpressureStatus is the status code returned at StageBackpressure
// for non-delayable requests. Default: 503.
BackpressureStatus int
// RejectStatus is the status code returned at StageReject.
// Default: 503.
RejectStatus int
// RetryAfter is the Retry-After header value accompanying 503s.
// Default: 5 seconds.
RetryAfter time.Duration
// EnableReap opts into calling runtime.GC() at StageReap. Default:
// false. Forced GC can hurt tail latency; enable only when you
// can measure the effect.
EnableReap bool
// ReapAggressiveness: 1=GC hint, 2=GC + encourage pool drain.
// Default: 1.
ReapAggressiveness int
// Skip defines a function to skip this middleware for certain
// requests (bypasses all stage logic).
Skip func(*celeris.Context) bool
// SkipPaths lists paths to skip entirely.
SkipPaths []string
// StopContext, when cancelled, stops the background sampling
// goroutine. Default: context.Background (never cancels).
StopContext context.Context
}
Config defines the overload middleware configuration.
Example ¶
ExampleConfig — the middleware needs a [Config.CollectorProvider] to sample CPU/queue/latency signals. A celeris.Server's observe.Collector is the typical source; any function returning a `*observe.Collector` works.
In production wire it as `app.Use(overload.New(cfg))` after the server is constructed and its collector is reachable.
package main
import (
"github.com/goceleris/celeris/middleware/overload"
"github.com/goceleris/celeris/observe"
)
func main() {
_ = overload.Config{
CollectorProvider: func() *observe.Collector { return nil },
}
}
Output:
type Controller ¶
type Controller struct {
// contains filtered or unexported fields
}
Controller exposes runtime introspection and control of a running overload middleware instance.
func NewWithController ¶
func NewWithController(config ...Config) (celeris.HandlerFunc, *Controller)
NewWithController is like New but also returns a Controller for querying the current stage and stopping the poll goroutine.
func (*Controller) CPUSample ¶
func (c *Controller) CPUSample() float64
CPUSample returns the CPU utilization (0..1) from the last completed poll tick. Returns -1 if no sample has been taken yet.
func (*Controller) InFlight ¶
func (c *Controller) InFlight() int32
InFlight returns the current in-flight request count observed by the middleware. Not affected by SkipPaths / ExemptPaths (they never touch the counter).
func (*Controller) LatencyEMA ¶
func (c *Controller) LatencyEMA() time.Duration
LatencyEMA returns the current smoothed (EMA) tail latency across observed non-exempt requests. Zero if no requests have completed yet.
func (*Controller) Stop ¶
func (c *Controller) Stop()
Stop cancels the background sampling goroutine. Safe to call multiple times. After Stop the stage stays frozen at whatever value it last observed.
type DepthThresholds ¶
type DepthThresholds struct {
Expand int32 // default 0 (disabled)
Reap int32 // default 0 (disabled)
Reorder int32 // default 0 (disabled)
Backpressure int32 // default 0 (disabled)
Reject int32 // default 0 (disabled)
Hysteresis int32 // default 0 (no hysteresis on depth)
}
DepthThresholds defines the per-stage in-flight request count that escalates the driver independent of CPU. Zero-valued fields disable the depth signal for that stage (CPU-only). Each threshold is an absolute count — if the middleware observes N >= threshold in-flight requests, the effective stage is at least that one.
Depth is the most reliable overload signal when handlers block on upstream I/O: CPU stays low while requests queue up. Typical choice: Reorder = 2×NumWorkers, Backpressure = 4×NumWorkers, Reject = 8×NumWorkers.
type LatencyThresholds ¶
type LatencyThresholds struct {
Expand time.Duration // default 0 (disabled)
Reap time.Duration // default 0 (disabled)
Reorder time.Duration // default 0 (disabled)
Backpressure time.Duration // default 0 (disabled)
Reject time.Duration // default 0 (disabled)
Hysteresis time.Duration // default 0 (no hysteresis on latency)
}
LatencyThresholds defines the per-stage EMA tail-latency target that escalates the driver independent of CPU or depth. When the smoothed latency EMA exceeds a threshold, the effective stage is at least that one. Zero-valued fields disable the latency signal for that stage.
Latency is the right signal for SLO-aware degradation: at fixed CPU and fixed load, a sudden jump in latency usually means an upstream has slowed down — applying backpressure early preserves overall SLO.
type Stage ¶
type Stage int32
Stage identifies one of the five CPU-pressure stages the driver can occupy. Stages rise as CPU utilization crosses upward thresholds and fall after CPU drops below (upper - Hysteresis).
type Thresholds ¶
type Thresholds struct {
Expand float64 // default 0.70
Reap float64 // default 0.80
Reorder float64 // default 0.85
Backpressure float64 // default 0.90
Reject float64 // default 0.95
Hysteresis float64 // default 0.05
}
Thresholds defines the upward transition CPU fractions (0.0..1.0). Each field is the lower bound for entering that stage. The downward transition threshold is (upper - Hysteresis).
Source Files