Documentation
¶
Overview ¶
Package deps supervises a process's external dependencies. Each dependency is probed forever: on a fixed interval while up, with full-jitter capped exponential backoff while down. Callers read Up() to choose between a primary backend and a local fallback, and Report data-path errors so an outage is noticed on the first failed call instead of the next probe.
Index ¶
- Constants
- func Backoff(attempt int, base, max time.Duration) time.Duration
- func Call[T, R any](s *Switch[T], fn func(T) (R, error)) (R, error)
- func Do[T any](s *Switch[T], fn func(T) error) error
- func IsConnectivity(err error) bool
- func NewRedis(c RedisConfig) redis.UniversalClient
- func PostgresUnavailable(err error) bool
- func RedisUnavailable(err error) bool
- type Class
- type Counter
- type DepOption
- type Dependency
- func (d *Dependency) Class() Class
- func (d *Dependency) Done() <-chan struct{}
- func (d *Dependency) Name() string
- func (d *Dependency) OnDown(fn func(context.Context))
- func (d *Dependency) OnUp(fn func(context.Context))
- func (d *Dependency) Report(err error) bool
- func (d *Dependency) Status() Status
- func (d *Dependency) Up() bool
- type Gate
- type Label
- type Option
- type Probe
- type RedisConfig
- type Sample
- type Status
- type Supervisor
- func (s *Supervisor) Add(name string, class Class, probe Probe, unavailable func(error) bool, ...) *Dependency
- func (s *Supervisor) AddPostgres(name string, cfg *pgx.ConnConfig, opts ...DepOption) *Dependency
- func (s *Supervisor) AddRedis(name string, client redis.UniversalClient) *Dependency
- func (s *Supervisor) Counter(name, help string) *Counter
- func (s *Supervisor) Drain()
- func (s *Supervisor) Gate() *Gate
- func (s *Supervisor) GaugeFunc(name, help string, fn func() []Sample) error
- func (s *Supervisor) Get(name string) *Dependency
- func (s *Supervisor) Livez(w http.ResponseWriter, _ *http.Request)
- func (s *Supervisor) Metrics(w http.ResponseWriter, _ *http.Request)
- func (s *Supervisor) OpsHandler() http.Handler
- func (s *Supervisor) Ready() bool
- func (s *Supervisor) Readyz(w http.ResponseWriter, _ *http.Request)
- func (s *Supervisor) Retry(ctx context.Context, name string, fn func(context.Context) error) error
- func (s *Supervisor) SetReady()
- func (s *Supervisor) Start(ctx context.Context)
- func (s *Supervisor) Statuses() []Status
- func (s *Supervisor) Statusz(w http.ResponseWriter, _ *http.Request)
- type Switch
- type Timing
Constants ¶
const ( LivePath = "/livez" ReadyPath = "/readyz" StatusPath = "/statusz" MetricsPath = "/metrics" )
Probe endpoint paths.
Variables ¶
This section is empty.
Functions ¶
func Backoff ¶
Backoff returns a full-jitter delay for the given zero-based attempt: uniform in [0, min(max, base·2^attempt)].
func Call ¶
Call runs fn on the current backend. When the primary fails with an unavailability error the dependency is marked down and fn is retried once on the fallback, so the caller never sees the outage.
func IsConnectivity ¶
IsConnectivity reports whether err means the remote could not be reached or did not answer in time (as opposed to an application-level error).
func NewRedis ¶
func NewRedis(c RedisConfig) redis.UniversalClient
NewRedis builds a client without contacting the server. Timeouts are short and retries are off: the supervisor, not each command, owns reconnection.
func PostgresUnavailable ¶
PostgresUnavailable reports whether err means Postgres could not be reached or is not accepting work right now (connection failures, timeouts, server shutdown/startup), as opposed to a query or schema error.
func RedisUnavailable ¶
RedisUnavailable reports whether err from a Redis command means the server is unreachable, as opposed to a miss (redis.Nil) or a command error.
Types ¶
type Class ¶
type Class int
Class says whether the process can do meaningful work without a dependency.
type Counter ¶
type Counter struct {
// contains filtered or unexported fields
}
Counter is a monotonically increasing value exported on /metrics.
type DepOption ¶
type DepOption func(*Dependency)
DepOption tunes one dependency.
func DownInterval ¶ added in v0.5.0
DownInterval sets the least time between probes while this dependency is failing (±10% jitter), for probes that cost something: the capped backoff applies above it. A non-positive t leaves the backoff alone.
func OnClose ¶
func OnClose(fn func()) DepOption
OnClose releases resources the probe holds (such as a dedicated connection). It runs once, after the dependency stops: when it is replaced by a same-named Add or the supervisor's context ends.
func ProbeInterval ¶ added in v0.5.0
ProbeInterval overrides the probe period while the dependency is up, for probes that cost something (third-party APIs); a non-positive t keeps the supervisor's interval. Backoff while down is unchanged.
func ProbeTimeout ¶
ProbeTimeout overrides the per-probe timeout for this dependency.
type Dependency ¶
type Dependency struct {
// contains filtered or unexported fields
}
Dependency is one supervised external system.
func (*Dependency) Class ¶
func (d *Dependency) Class() Class
func (*Dependency) Done ¶
func (d *Dependency) Done() <-chan struct{}
Done is closed once the dependency has stopped and released its resources.
func (*Dependency) Name ¶
func (d *Dependency) Name() string
func (*Dependency) OnDown ¶
func (d *Dependency) OnDown(fn func(context.Context))
OnDown registers fn to run after each up→down transition.
func (*Dependency) OnUp ¶
func (d *Dependency) OnUp(fn func(context.Context))
OnUp registers fn to run after each down→up transition (not the first up). Hooks run in order on a goroutine of their own, never on the probe loop, with a context that ends when the supervisor stops.
func (*Dependency) Report ¶
func (d *Dependency) Report(err error) bool
Report feeds a data-path result back. It returns true when err means the dependency is unavailable, in which case the dependency is marked down immediately and probed with backoff until it recovers.
func (*Dependency) Status ¶
func (d *Dependency) Status() Status
func (*Dependency) Up ¶
func (d *Dependency) Up() bool
Up reports whether the dependency is currently usable. A nil Dependency is never up, so an unconfigured dependency reads as permanently down.
type Gate ¶
type Gate struct {
// contains filtered or unexported fields
}
Gate is the handler for the application listener. It answers /livez and /readyz itself and returns 503 for everything else until Open installs the application, so the listener can bind before the application is built.
type Label ¶ added in v0.5.0
type Label struct{ Name, Value string }
Label is one Prometheus label of a Sample.
type Option ¶
type Option func(*Supervisor)
Option configures a Supervisor.
func WithLogger ¶
func WithTiming ¶
type Probe ¶
Probe checks a dependency once. It must honor ctx.
func PostgresProbe ¶
func PostgresProbe(cfg *pgx.ConnConfig) (probe Probe, close func())
PostgresProbe checks Postgres over one dedicated connection, so a saturated application pool never reads as Postgres being down. The connection is re-dialed after any failure. Pass close to Add as OnClose.
type RedisConfig ¶
type RedisConfig struct {
Addrs []string `koanf:"addrs" yaml:"addrs" json:"addrs"`
SentinelAddrs []string `koanf:"sentinel_addrs" yaml:"sentinel_addrs" json:"sentinel_addrs"`
MasterName string `koanf:"master_name" yaml:"master_name" json:"master_name"`
Username string `koanf:"username" yaml:"username" json:"username"`
Password string `koanf:"password" yaml:"password" json:"password"`
SentinelUsername string `koanf:"sentinel_username" yaml:"sentinel_username" json:"sentinel_username"`
// SentinelPassword defaults to Password.
SentinelPassword string `koanf:"sentinel_password" yaml:"sentinel_password" json:"sentinel_password"`
DB int `koanf:"db" yaml:"db" json:"db"`
}
RedisConfig selects the Redis topology without code changes: MasterName with SentinelAddrs is a Sentinel failover client; otherwise one address is a plain client and several addresses a cluster client.
func (RedisConfig) Configured ¶
func (c RedisConfig) Configured() bool
Configured reports whether Redis is configured at all.
func (RedisConfig) Validate ¶
func (c RedisConfig) Validate() error
Validate rejects a half-configured Sentinel setup.
type Status ¶
type Status struct {
Name string `json:"name"`
Class string `json:"class"`
Up bool `json:"up"`
Since time.Time `json:"since"`
LastError string `json:"last_error,omitempty"`
ConsecutiveFailures int `json:"consecutive_failures"`
TransitionsUp int64 `json:"transitions_up"`
TransitionsDown int64 `json:"transitions_down"`
}
Status is a point-in-time view of one dependency.
type Supervisor ¶
type Supervisor struct {
// contains filtered or unexported fields
}
Supervisor owns a set of dependencies and the process readiness state.
func New ¶
func New(opts ...Option) *Supervisor
func (*Supervisor) Add ¶
func (s *Supervisor) Add(name string, class Class, probe Probe, unavailable func(error) bool, opts ...DepOption) *Dependency
Add registers a dependency, replacing (and stopping) any earlier one with the same name. unavailable classifies data-path errors passed to Report; nil uses IsConnectivity. Dependencies added after Start start at once.
func (*Supervisor) AddPostgres ¶
func (s *Supervisor) AddPostgres(name string, cfg *pgx.ConnConfig, opts ...DepOption) *Dependency
AddPostgres supervises Postgres as a required dependency over a dedicated connection that is closed when the dependency stops.
func (*Supervisor) AddRedis ¶
func (s *Supervisor) AddRedis(name string, client redis.UniversalClient) *Dependency
AddRedis supervises client as an optional dependency. The probe writes a short-lived key, so a primary that refuses writes (no replica connected, read-only after failover) counts as down, not only an unreachable one.
func (*Supervisor) Counter ¶
func (s *Supervisor) Counter(name, help string) *Counter
Counter registers (or returns the existing) counter exported on /metrics. name must be a valid Prometheus metric name ending in _total.
func (*Supervisor) Drain ¶
func (s *Supervisor) Drain()
Drain marks the process as shutting down; readiness fails from now on.
func (*Supervisor) Gate ¶
func (s *Supervisor) Gate() *Gate
func (*Supervisor) GaugeFunc ¶ added in v0.5.0
func (s *Supervisor) GaugeFunc(name, help string, fn func() []Sample) error
GaugeFunc exports fn's samples as a gauge on /metrics, read at scrape time (e.g. the age of a peer's cached keys). name must be a valid Prometheus metric name not already exported; fn must be cheap and safe for concurrent use. Samples with an invalid label name are skipped.
func (*Supervisor) Get ¶
func (s *Supervisor) Get(name string) *Dependency
Get returns a registered dependency by name, or nil.
func (*Supervisor) Livez ¶
func (s *Supervisor) Livez(w http.ResponseWriter, _ *http.Request)
Livez answers 200 whenever the process can serve HTTP. It never checks dependencies: restarting a process does not fix a missing dependency.
func (*Supervisor) Metrics ¶
func (s *Supervisor) Metrics(w http.ResponseWriter, _ *http.Request)
Metrics writes Prometheus text exposition.
func (*Supervisor) OpsHandler ¶
func (s *Supervisor) OpsHandler() http.Handler
OpsHandler serves /livez, /readyz, /statusz and /metrics, for a listener that is not exposed through the ingress.
func (*Supervisor) Ready ¶
func (s *Supervisor) Ready() bool
Ready reports whether the process should receive traffic.
func (*Supervisor) Readyz ¶
func (s *Supervisor) Readyz(w http.ResponseWriter, _ *http.Request)
Readyz answers 200 once the process is built and until it starts draining. Shared dependencies are deliberately not consulted: when one is down every replica would fail together and the ingress would have nothing to route to.
func (*Supervisor) Retry ¶
Retry calls fn until it succeeds or ctx ends, sleeping with full-jitter capped exponential backoff between attempts. Use it for required dependencies at startup instead of exiting.
func (*Supervisor) SetReady ¶
func (s *Supervisor) SetReady()
SetReady marks the process as built and able to serve.
func (*Supervisor) Start ¶
func (s *Supervisor) Start(ctx context.Context)
Start probes every dependency in the background until ctx ends.
func (*Supervisor) Statuses ¶
func (s *Supervisor) Statuses() []Status
Statuses returns every dependency's status in registration order.
func (*Supervisor) Statusz ¶
func (s *Supervisor) Statusz(w http.ResponseWriter, _ *http.Request)
Statusz reports every dependency as JSON. Always 200.
type Switch ¶
type Switch[T any] struct { // contains filtered or unexported fields }
Switch serves a primary backend while its dependency is up and a local fallback otherwise. Fallback state is never merged back: every down→up transition replaces the fallback with a fresh one.
func NewSwitch ¶
func NewSwitch[T any](dep *Dependency, primary T, newFallback func() T) *Switch[T]
NewSwitch builds a Switch. A nil dep means no primary is configured and the fallback is always used.
func (*Switch[T]) Dependency ¶
func (s *Switch[T]) Dependency() *Dependency
Dependency returns the dependency that gates the primary (nil if none).
type Timing ¶
type Timing struct {
Interval time.Duration // probe period while up (10s)
Timeout time.Duration // per-probe timeout (1s)
BackoffBase time.Duration // first backoff ceiling while down (500ms)
BackoffMax time.Duration // backoff ceiling cap (30s)
RecoverAfter int // consecutive successes before down→up (2)
// RequiredDownAfter is how many consecutive failed probes mark a required
// dependency down (3); optional ones go down on the first.
RequiredDownAfter int
}
Timing controls probing. Zero fields take the defaults.
Source Files