Documentation
¶
Overview ¶
Package resilience provides circuit breaker, rate limiting, and query timeout utilities.
Index ¶
- Constants
- func CircuitBreakerMiddleware(cb *gobreaker.CircuitBreaker) func(http.Handler) http.Handler
- func IsOpen(cb *gobreaker.CircuitBreaker) bool
- func NewQueryBreaker(name string) *gobreaker.CircuitBreaker
- func NewQueryBreakerWith(name string, isFailure func(error) bool) *gobreaker.CircuitBreaker
- func RateLimit(tl *TenantLimiter) func(http.Handler) http.Handler
- func RateLimitMiddleware(tl *TenantLimiter, tier Tier) func(http.Handler) http.Handler
- func WithQueryTimeout(ctx context.Context, fn func(ctx context.Context) error) error
- type MemoryGuard
- type RateLimitConfig
- type TenantLimiter
- type Tier
Constants ¶
const MemoryGuardEnvVar = "APPXIMO_MEMORY_GUARD_MIN_MB"
MemoryGuardEnvVar is the operator knob (MiB floor; 0 disables).
Variables ¶
This section is empty.
Functions ¶
func CircuitBreakerMiddleware ¶
CircuitBreakerMiddleware wraps a handler with a gobreaker circuit breaker. When the circuit is open it returns 503 with Retry-After: 8. Failures are counted when the handler writes a 5xx response.
func IsOpen ¶
func IsOpen(cb *gobreaker.CircuitBreaker) bool
IsOpen is the hot-path state check — an O(1) mutex read with zero allocations. Call this before executing a query to decide whether to serve from cache or return 503.
func NewQueryBreaker ¶
func NewQueryBreaker(name string) *gobreaker.CircuitBreaker
NewQueryBreaker returns a circuit breaker tuned for PostgreSQL query protection.
Opens when ≥10 requests have a ≥60% failure rate. Transitions open→half-open after 8 s; allows 2 probe requests in half-open. Every non-nil error counts as a failure — the gobreaker default. Production callers use NewQueryBreakerWith, which decides what a failure IS.
func NewQueryBreakerWith ¶ added in v0.1.10
func NewQueryBreakerWith(name string, isFailure func(error) bool) *gobreaker.CircuitBreaker
NewQueryBreakerWith is NewQueryBreaker with an explicit definition of failure: isFailure(err) reports whether a non-nil error means the database could not serve the request. When nil, every error counts.
WHY (ENG-49, MOTOR-TIPO-JSON-S1). The breaker exists to shed load when PostgreSQL is DOWN. With the default "every error is a failure", a unique violation, an unknown column (a plain 422), a class-22 value, a driver encode error — all produced by CLIENT INPUT, none an outage — were counted, and six 422s in a row opened the breaker: every write of the process (every tenant of the app) answered 503 for 8 s, renewably, to any caller with `create` on any resource. A statement the database REJECTED is proof the database is up. pkg/db passes the SAME predicate that already decides the 503 (timeouts, connection failures, class 08/53/57P0x), so "counted by the breaker" and "answered 503" can never disagree.
func RateLimit ¶
func RateLimit(tl *TenantLimiter) func(http.Handler) http.Handler
RateLimit enforces a configured TenantLimiter's per-tenant policy. It is the tier-less entry point used by the server: the limiter already carries its RPS/Burst config, so the tier argument is irrelevant and passed as TierPro.
func RateLimitMiddleware ¶
RateLimitMiddleware returns a chi-compatible middleware that enforces per-tenant rate limits. Tenants not found in context are skipped (e.g., health checks). Returns 429 Too Many Requests when the token bucket is empty.
Types ¶
type MemoryGuard ¶ added in v0.1.12
type MemoryGuard struct {
// contains filtered or unexported fields
}
MemoryGuard — the minimal, honest write-admission guard (MIGRACION-CONFIANZA-S1, D-ter).
WHAT IT IS NOT: capacity. It does not make the engine "hold" a bulk load on a small box. It makes the engine STOP ACCEPTING NEW WRITES when the HOST is about to run out of memory, answering a 503 that says why, instead of accepting until the kernel's OOM killer takes PostgreSQL — and with it every app that shares that PostgreSQL. Measured in the field (a Symfony migration, 46k rows, 957 MiB box, no swap, five apps on one Postgres): the engine kept accepting writes until `postgresql@14-main.service: Failed with result 'oom-kill'` and all five apps went down. The audit before this guard: the engine had NO notion of host memory pressure anywhere — GOMEMLIMIT bounds its OWN heap (which was never the problem: the memory that grew was Postgres's backends), the pool is a fixed 10 connections, the rate limiter counts requests, not bytes.
WHAT IT MEASURES — and why not MemAvailable alone: on a box that runs PostgreSQL, `shared_buffers` shows up as Cached but is NOT reclaimable, so MemAvailable at rest sits at a few tens of MiB however much RAM the box has. A guard on MemAvailable alone would trip permanently and be switched off on day one. The signal is MemAvailable + SwapFree: what the kernel can still hand out before it has to kill something. (On a box with NO swap the two coincide — and the installer now warns loudly about exactly that box, scripts/install.sh.)
COST: one atomic load per write request; /proc/meminfo is read at most once per second, by the first writer to notice the sample is stale (others keep the previous sample — never a stampede, never a lock on the hot path). Reads never consult it: a saturated host still serves what it can.
KNOBS: APPXIMO_MEMORY_GUARD_MIN_MB — the floor of MemAvailable + SwapFree in MiB under which writes are refused. Default: max(32, 2 % of MemTotal) MiB — deliberately LOW, so it fires only when the kernel is genuinely about to run out, never as a permanent false positive on a busy but healthy box. `0` disables the guard. A non-integer value refuses to boot (a safety knob never falls back silently — the ENG-47 rule).
func NewMemoryGuard ¶ added in v0.1.12
func NewMemoryGuard(minBytes int64, meminfoPath string, interval time.Duration) *MemoryGuard
NewMemoryGuard is the testable constructor: floor in bytes, the meminfo path, and the minimum interval between two samples.
func NewMemoryGuardFromEnv ¶ added in v0.1.12
func NewMemoryGuardFromEnv() (*MemoryGuard, error)
NewMemoryGuardFromEnv builds the guard from APPXIMO_MEMORY_GUARD_MIN_MB and /proc/meminfo. It returns (nil, nil) when disabled (0, or a host without /proc/meminfo) and an error for a value that is not a non-negative integer.
func (*MemoryGuard) Allow ¶ added in v0.1.12
func (g *MemoryGuard) Allow() (ok bool, availableBytes int64)
Allow reports whether a write may proceed, with the measured value.
func (*MemoryGuard) Middleware ¶ added in v0.1.12
func (g *MemoryGuard) Middleware(next http.Handler) http.Handler
Middleware refuses data-plane writes while the host is under memory pressure: 503 + Retry-After + a body that names the measurement, the floor and the knob.
func (*MemoryGuard) MinBytes ¶ added in v0.1.12
func (g *MemoryGuard) MinBytes() int64
MinBytes reports the configured floor.
func (*MemoryGuard) Sample ¶ added in v0.1.12
func (g *MemoryGuard) Sample() (int64, error)
Sample reads MemAvailable + SwapFree now and returns it (bytes).
type RateLimitConfig ¶
type RateLimitConfig struct {
RPS float64 // sustained requests per second, per tenant
Burst int // bucket capacity for short spikes
}
RateLimitConfig sets an explicit per-tenant token-bucket policy, independent of subscription tier. Used to wire the limiter from environment configuration.
type TenantLimiter ¶
type TenantLimiter struct {
// contains filtered or unexported fields
}
TenantLimiter provides per-tenant token-bucket rate limiting. Each tenant gets an independent limiter keyed by tenantID. With a nil cfg the burst size equals the tier rate (1-second burst capacity); with cfg set, every tenant uses cfg.RPS / cfg.Burst and the tier is ignored.
Past maxLimiters distinct tenants, additional (unknown) tenants share a single overflow bucket: memory stays bounded and the rate limit still applies.
func NewConfiguredLimiter ¶
func NewConfiguredLimiter(cfg RateLimitConfig) *TenantLimiter
NewConfiguredLimiter creates a TenantLimiter that applies cfg to every tenant.
func NewTenantLimiter ¶
func NewTenantLimiter() *TenantLimiter
NewTenantLimiter creates an empty tier-based TenantLimiter.
Source Files