Documentation
¶
Overview ¶
Package controls provides a lifecycle controller for managing concurrent, long-running services such as HTTP servers, background workers, and schedulers.
A Controller starts a fixed set of registered services concurrently, aggregates the health probes they supply into HealthReport values, and drives a bounded graceful shutdown in reverse registration order, with an optional RestartPolicy. It installs no OS signal handler unless asked to with WithSignals, and it serves no health endpoint: the reports are plain values for a transport to expose.
A Supervisor runs children that attach and detach while the process is running, and a Generational owns one generation of a single-use resource at a time for a service that has to survive a restart.
The Controllable interface, and the narrower role interfaces it is composed of, let code that depends on a controller substitute a fake.
Index ¶
- Constants
- Variables
- type ChannelProvider
- type CheckResult
- type CheckStatus
- type CheckType
- type Child
- type ChildState
- type ChildStatus
- type Configurable
- type Controllable
- type Controller
- func (c *Controller) Errors() chan error
- func (c *Controller) GetCheckResult(name string) (CheckResult, bool)
- func (c *Controller) GetContext() context.Context
- func (c *Controller) GetLogger() *slog.Logger
- func (c *Controller) GetServiceInfo(name string) (ServiceInfo, bool)
- func (c *Controller) GetState() State
- func (c *Controller) IsRunning() bool
- func (c *Controller) IsStopped() bool
- func (c *Controller) IsStopping() bool
- func (c *Controller) Liveness() HealthReport
- func (c *Controller) Messages() chan Message
- func (c *Controller) Readiness() HealthReport
- func (c *Controller) Register(id string, opts ...ServiceOption)
- func (c *Controller) RegisterHealthCheck(check HealthCheck) error
- func (c *Controller) SetErrorsChannel(errs chan error)
- func (c *Controller) SetLogger(l *slog.Logger)
- func (c *Controller) SetMessageChannel(messages chan Message)
- func (c *Controller) SetShutdownTimeout(d time.Duration)
- func (c *Controller) SetSignalsChannel(signals chan os.Signal)
- func (c *Controller) SetState(state State)
- func (c *Controller) SetWaitGroup(wg *sync.WaitGroup)
- func (c *Controller) Signals() chan os.Signal
- func (c *Controller) Start()
- func (c *Controller) Status() HealthReport
- func (c *Controller) Stop()
- func (c *Controller) Wait()
- func (c *Controller) WaitContext(ctx context.Context) error
- func (c *Controller) WaitGroup() *sync.WaitGroup
- type ControllerOpt
- type Failure
- type Generational
- type HealthCheck
- type HealthCheckReporter
- type HealthReport
- type HealthReporter
- type Message
- type ProbeFunc
- type RestartPolicy
- type Runner
- type Service
- type ServiceInfo
- type ServiceOption
- func WithLiveness(fn ProbeFunc) ServiceOption
- func WithReadiness(fn ProbeFunc) ServiceOption
- func WithRestartPolicy(policy RestartPolicy) ServiceOption
- func WithRestartResetInterval(d time.Duration) ServiceOption
- func WithStart(fn StartFunc) ServiceOption
- func WithStatus(fn StatusFunc) ServiceOption
- func WithStop(fn StopFunc) ServiceOption
- func WithStopErr(fn StopErrFunc) ServiceOption
- type ServiceStatus
- type Services
- type StartFunc
- type State
- type StateAccessor
- type StatusFunc
- type StopErrFunc
- type StopFunc
- type Supervisor
- func (s *Supervisor) Attach(c Child) error
- func (s *Supervisor) Detach(ctx context.Context, name string) error
- func (s *Supervisor) DroppedReports() int64
- func (s *Supervisor) Failures() <-chan Failure
- func (s *Supervisor) Health() map[string]ChildStatus
- func (s *Supervisor) HealthCheck(name string) HealthCheck
- func (s *Supervisor) Readiness() error
- func (s *Supervisor) Start(ctx context.Context) error
- func (s *Supervisor) Stop(ctx context.Context)
- type SupervisorOption
- type ValidErrorFunc
Examples ¶
Constants ¶
const DefaultFailureBufferSize = 16
DefaultFailureBufferSize bounds the channel returned by Supervisor.Failures.
Bounded because a supervisor that blocks on an undrained notification channel has let its reporting stall the thing it reports on. Sixteen is generous for terminal failures, which are rare by construction — a child reaches one only after exhausting its restart policy.
const ( // DefaultRestartResetInterval is the duration a service must run healthily // before its consecutive-failure restart counter resets to zero. DefaultRestartResetInterval = 30 * time.Second )
const DefaultShutdownTimeout = 5 * time.Second
DefaultShutdownTimeout is the time allowed for graceful shutdown before services are force-stopped.
Variables ¶
var ( // ErrNoGeneration is returned by Use when no generation is live: before the // first Start, during a stop, and after one. It is deliberately one error // rather than three, because the caller's response is the same. ErrNoGeneration = errors.NewSentinel("controls.no_generation", "controls: no live generation") // ErrGenerationRunning is returned by a Start that would have built a rival // generation alongside a live one. ErrGenerationRunning = errors.NewSentinel("controls.generation_running", "controls: a generation is already running") // ErrPredecessorLive is returned by a Start whose predecessor has not // finished releasing. It never reflects consumer code: a lease is bounded // by its own call, so only a Release that ignores its context can hold it. ErrPredecessorLive = errors.NewSentinel("controls.predecessor_live", "controls: the previous generation still holds resources") // ErrStopTimeout is returned by a Stop whose budget expired before the // generation's resources were released. The generation remains un-released // and a later Start is refused until it is: the error is semantic, not // decoration. ErrStopTimeout = errors.NewSentinel("controls.stop_timeout", "controls: stop budget expired before release completed") )
Errors returned by Generational. They are sentinels because a caller needs to distinguish "there is nothing to talk to" from "the thing said no", and that distinction must survive a process boundary.
var ErrChildAttached = errors.NewSentinel("controls.child_attached",
"controls: a child is already attached under that name")
ErrChildAttached is returned when a name is already in use.
var ErrChildNotAttached = errors.NewSentinel("controls.child_not_attached",
"controls: no child is attached under that name")
ErrChildNotAttached is returned when a name is not known.
var ErrDetachTimeout = errors.NewSentinel("controls.detach_timeout",
"controls: the child was still running when the detach budget expired")
ErrDetachTimeout is returned when a child outlives the context given to Supervisor.Detach.
The child is still forgotten — it is gone from the supervisor's bookkeeping either way. This error is how the caller learns that its goroutine had not yet returned, which is the whole difference between a bounded detach and a fire-and-forget one.
var ErrRestartsExhausted = errors.NewSentinel("controls.restarts_exhausted", "max restarts exceeded")
ErrRestartsExhausted is in the error a service or a child leaves behind when it has used up its restart policy: on ServiceInfo.Error and Errors() for a service, on Failure.Err for a child. Test for it with errors.Is; the service's last error is reachable the same way beside it.
Nothing is stopped on its account (spec 0006 D4). A consumer that wants the process to end when a service will not come back tests for this on Errors() and calls Stop.
var ErrShutdown = errors.NewSentinel("controls.shutdown", "controller shutdown")
ErrShutdown is the cause attached to the controller context when a graceful shutdown is initiated. Callers can distinguish a controlled stop from an upstream cancellation via context.Cause(ctx) == controls.ErrShutdown.
var ErrSupervisorNotStarted = errors.NewSentinel("controls.supervisor_not_started",
"controls: the supervisor has not started")
ErrSupervisorNotStarted is returned by Supervisor.Readiness before Start.
var ErrSupervisorStopped = errors.NewSentinel("controls.supervisor_stopped",
"controls: the supervisor has stopped")
ErrSupervisorStopped is returned once shutdown has begun, by Supervisor.Attach, Supervisor.Start and Supervisor.Readiness.
A Supervisor is single-use. Accepting a child it will never supervise is worse than refusing one, because the caller goes away believing it is running.
Functions ¶
This section is empty.
Types ¶
type ChannelProvider ¶
type ChannelProvider interface {
Messages() chan Message
Errors() chan error
Signals() chan os.Signal
}
ChannelProvider provides access to controller channels.
type CheckResult ¶
type CheckResult struct {
// Status is the health status.
Status CheckStatus
// Message provides human-readable detail about the check result.
Message string
// Timestamp is when this result was produced.
Timestamp time.Time
}
CheckResult represents the outcome of a health check.
type CheckStatus ¶
type CheckStatus int
CheckStatus represents the health state of a check.
const ( // CheckHealthy indicates the check passed. CheckHealthy CheckStatus = iota // CheckDegraded indicates the check passed but with warnings. // Maps to OverallHealthy: true with Status: "DEGRADED". CheckDegraded // CheckUnhealthy indicates the check failed. // Maps to OverallHealthy: false with Status: "ERROR". CheckUnhealthy )
type CheckType ¶
type CheckType int
CheckType determines which health endpoint(s) a check contributes to.
type Child ¶ added in v0.4.0
type Child struct {
// Name identifies the child within its supervisor.
Name string
// Start runs the child. Returning nil means it finished cleanly and will
// not be restarted; returning an error starts the restart policy.
Start StartFunc
// Stop is called when the child is detached or the supervisor stops. It is
// optional: cancelling the context passed to Start is the primary mechanism.
Stop StopFunc
// ValidError identifies errors that mean a clean stop rather than a failure,
// http.ErrServerClosed being the case it exists for. An error it accepts ends
// the child without a restart and without a [Failure], exactly as a
// [Controller]'s WithValidError does for a Service.
//
// Per child rather than per supervisor, because a supervisor may hold children
// of very different kinds and one predicate for all of them is the wrong unit.
// Nil means no error is exempt.
ValidError ValidErrorFunc
// RestartPolicy governs restarts. Nil means never restart: the child runs
// once and its outcome is final.
//
// The restart rules are a Service's rules, read through the same helpers,
// including that MaxRestarts <= 0 means UNLIMITED rather than none. The same
// type is used deliberately: a caller should configure restart behaviour one
// way whether the thing being restarted is registered or attached, and two
// rules with the same fields is the divergence nobody notices until they
// disagree.
//
// Two of the type's fields are inert here. HealthFailureThreshold and
// HealthCheckInterval drive a Service's health-based restarts through its
// Status probe, and a Child has no probe to read. Setting them changes
// nothing rather than failing, which is why it is said here.
//
// An expected terminal error is [Child.ValidError] rather than a policy field,
// matching where a Service declares it.
//
// The value is copied at [Supervisor.Attach], so a caller may reuse or edit
// its policy struct afterwards without racing the supervision goroutine.
RestartPolicy *RestartPolicy
}
Child is one supervised unit.
Unlike a Service, a child is not a requirement for the process to operate: see Supervisor for what that distinction buys.
type ChildState ¶ added in v0.4.0
type ChildState string
ChildState is what a supervisor can say about a child.
const ( // ChildPending means the child is attached and its Start has not been called, // because the supervisor has not started yet. ChildPending ChildState = "pending" // ChildRunning means the child's Start has not returned. ChildRunning ChildState = "running" // ChildBackoff means the child failed and is waiting out its restart backoff. // Reporting it as running would describe a dead child as a live one for as // long as MaxBackoff. ChildBackoff ChildState = "backoff" // ChildFailed means the child exhausted its restart policy. ChildFailed ChildState = "failed" // ChildStopped means the child returned cleanly or was stopped. ChildStopped ChildState = "stopped" )
type ChildStatus ¶ added in v0.4.0
type ChildStatus struct {
State ChildState
Restarts int
Panics int
LastErr error
}
ChildStatus is a child's observable state.
type Configurable ¶
type Configurable interface {
SetErrorsChannel(errs chan error)
SetMessageChannel(control chan Message)
SetSignalsChannel(sigs chan os.Signal)
SetWaitGroup(wg *sync.WaitGroup)
SetShutdownTimeout(d time.Duration)
SetLogger(l *slog.Logger)
}
Configurable provides controller configuration setters.
These setters mutate channel and logger fields that controller goroutines read after Start. They carry no internal synchronization and must only be called during construction — before Start — which is how the WithX ControllerOpt options apply them inside NewController. Calling any setter after Start races the running goroutines and is a programming error.
type Controllable ¶
type Controllable interface {
Runner
HealthReporter
StateAccessor
Configurable
ChannelProvider
}
Controllable is the full controller interface, composed of all role-based interfaces. Prefer using the narrower interfaces (Runner, HealthReporter, Configurable, etc.) where possible.
type Controller ¶
type Controller struct {
// contains filtered or unexported fields
}
Controller orchestrates the lifecycle of registered services: concurrent startup, health monitoring, ordered (reverse-registration) shutdown, and signal handling.
func NewController ¶
func NewController(ctx context.Context, opts ...ControllerOpt) *Controller
NewController creates a Controller with the given context and options.
It does NOT install an OS signal handler. Signal disposition is process-global state and belongs to whichever layer is outermost — typically the CLI framework or main. Pass WithSignals when the controller genuinely is that outermost layer.
The caller's context is watched but not inherited for cancellation: its completion, by cancel or deadline, triggers a graceful Stop, so every service observes ErrShutdown as its context cause. See docs/how-to/graceful-shutdown.md.
Example ¶
package main
import (
"context"
"fmt"
"time"
"gitlab.com/phpboyscout/go/controls"
)
func main() {
ctx := context.Background()
// Create a controller. No OS signal handler is installed by default; a
// standalone daemon that owns signals adds controls.WithSignals().
controller := controls.NewController(ctx)
// Register an HTTP service
controller.Register("http-api",
controls.WithStart(func(ctx context.Context) error {
fmt.Println("HTTP server starting")
return nil
}),
controls.WithStop(func(ctx context.Context) {
fmt.Println("HTTP server stopping")
}),
controls.WithStatus(func() error {
return nil // healthy
}),
)
// Start all services
controller.Start()
// Graceful shutdown
time.Sleep(10 * time.Millisecond)
controller.Stop()
controller.Wait()
}
Output:
func (*Controller) Errors ¶
func (c *Controller) Errors() chan error
func (*Controller) GetCheckResult ¶
func (c *Controller) GetCheckResult(name string) (CheckResult, bool)
GetCheckResult returns the latest result for a named health check.
func (*Controller) GetContext ¶
func (c *Controller) GetContext() context.Context
func (*Controller) GetLogger ¶
func (c *Controller) GetLogger() *slog.Logger
func (*Controller) GetServiceInfo ¶
func (c *Controller) GetServiceInfo(name string) (ServiceInfo, bool)
GetServiceInfo returns the runtime information and statistics for a specific service.
func (*Controller) GetState ¶
func (c *Controller) GetState() State
func (*Controller) IsRunning ¶
func (c *Controller) IsRunning() bool
func (*Controller) IsStopped ¶
func (c *Controller) IsStopped() bool
func (*Controller) IsStopping ¶
func (c *Controller) IsStopping() bool
func (*Controller) Liveness ¶
func (c *Controller) Liveness() HealthReport
Liveness returns an aggregate liveness report for all registered services and health checks.
func (*Controller) Messages ¶
func (c *Controller) Messages() chan Message
func (*Controller) Readiness ¶
func (c *Controller) Readiness() HealthReport
Readiness returns an aggregate readiness report for all registered services and health checks.
func (*Controller) Register ¶
func (c *Controller) Register(id string, opts ...ServiceOption)
func (*Controller) RegisterHealthCheck ¶
func (c *Controller) RegisterHealthCheck(check HealthCheck) error
RegisterHealthCheck adds a standalone health check to the controller. Must be called before Start(). The check name must be unique among health checks; it is not checked against service names, so a check that shares a name with a service is accepted and the report carries both entries.
func (*Controller) SetErrorsChannel ¶
func (c *Controller) SetErrorsChannel(errs chan error)
func (*Controller) SetLogger ¶
func (c *Controller) SetLogger(l *slog.Logger)
func (*Controller) SetMessageChannel ¶
func (c *Controller) SetMessageChannel(messages chan Message)
func (*Controller) SetShutdownTimeout ¶
func (c *Controller) SetShutdownTimeout(d time.Duration)
func (*Controller) SetSignalsChannel ¶
func (c *Controller) SetSignalsChannel(signals chan os.Signal)
func (*Controller) SetState ¶
func (c *Controller) SetState(state State)
func (*Controller) SetWaitGroup ¶
func (c *Controller) SetWaitGroup(wg *sync.WaitGroup)
func (*Controller) Signals ¶
func (c *Controller) Signals() chan os.Signal
func (*Controller) Start ¶
func (c *Controller) Start()
Start launches all registered services. It is idempotent: a second call while already running (or stopping/stopped) returns early without double-starting services or double-counting the wait group (D3).
func (*Controller) Status ¶
func (c *Controller) Status() HealthReport
Status returns an aggregate health report for all registered services and health checks.
func (*Controller) Stop ¶
func (c *Controller) Stop()
Stop initiates a graceful shutdown. Duplicate calls while already stopping or stopped are safely ignored.
func (*Controller) Wait ¶
func (c *Controller) Wait()
Wait blocks until every supervisor goroutine and the shutdown sequence have finished. It is unbounded and REQUIRES context-respecting StartFuncs: a StartFunc that never returns after cancellation pins its supervisor goroutine, and Wait blocks forever — even though the controller itself has completed shutdown and reports Stopped. When a service wraps third-party code that may ignore cancellation, use WaitContext to bound the wait (D10).
func (*Controller) WaitContext ¶ added in v0.1.3
func (c *Controller) WaitContext(ctx context.Context) error
WaitContext blocks until all supervisor goroutines have exited, or until ctx is done, whichever comes first. It returns nil on a clean drain and ctx.Err() when the wait is abandoned. On the abandon path the internal helper goroutine (and any stuck supervisors pinning the wait group) are deliberately leaked — the same abandon-at-deadline tradeoff the shutdown sequence applies to context-ignoring StopFuncs (D10).
func (*Controller) WaitGroup ¶
func (c *Controller) WaitGroup() *sync.WaitGroup
type ControllerOpt ¶
type ControllerOpt func(Configurable)
ControllerOpt is a functional option for configuring a Controller.
func WithLogger ¶
func WithLogger(l *slog.Logger) ControllerOpt
WithLogger sets the controller logger.
func WithShutdownTimeout ¶
func WithShutdownTimeout(d time.Duration) ControllerOpt
WithShutdownTimeout sets the graceful shutdown timeout.
func WithSignals ¶ added in v0.2.0
func WithSignals() ControllerOpt
WithSignals gives the controller ownership of SIGINT/SIGTERM, so the first signal drives a graceful Stop.
Signal disposition is process-global, so it belongs to whichever layer is outermost — which is why it is opt-in. In a CLI framework that already translates signals into context cancellation (as go-tool-base's root command does), do NOT use this: the controller observes the parent context instead, and a second handler would race the framework's own. Reach for it in a standalone main where the controller genuinely is the outermost thing.
func WithValidError ¶
func WithValidError(fn ValidErrorFunc) ControllerOpt
WithValidError registers a predicate that identifies expected terminal errors (e.g. http.ErrServerClosed, context.Canceled). The restart supervisor treats a matching error as a graceful end-of-run rather than a failure, so it neither counts toward the restart total nor is forwarded on the error channel (D7).
type Failure ¶ added in v0.4.0
type Failure struct {
// Name is the child that failed.
Name string
// Err is the error it last returned, wrapped to say restarts were exhausted:
// errors.Is matches [ErrRestartsExhausted] and the child's own error.
Err error
// Restarts is how many were attempted before giving up.
Restarts int
// Panicked reports whether the last run ended in a recovered panic rather
// than a returned error. A bug and a failure are different things, and a
// consumer deciding how to proceed wants to know which it has.
Panicked bool
}
Failure is a child that has exhausted its restart policy.
It is delivered to the consumer because whether that child mattered is the consumer's judgement, not the supervisor's.
type Generational ¶ added in v0.5.0
type Generational[R any] struct { // Build creates everything one generation needs. It is called once per // Start, and anything it acquires that needs releasing must be reachable // from the returned R, or it leaks. That obligation is the whole contract // this type places on a consumer. Build func(ctx context.Context) (R, error) // Release frees everything Build acquired. It must be idempotent, and it // must be safe to call concurrently with in-flight work that still holds a // lease, because a lease that outlives the stop budget is disowned rather // than waited for. // // A nil error means every resource is gone, and that is the only thing that // permits a new generation to start. Release func(ctx context.Context, r R) error // Probe reports the health of the current generation. Optional; nil means // always healthy while running. Probe func(r R) error // ReleaseAttempt is the budget each call to Release gets. The disposer // retries on this interval until Release returns nil, so it is a retry // interval rather than a deadline for the whole release. Zero or negative // selects 250ms (spec 0007 D1, D2). ReleaseAttempt time.Duration // contains filtered or unexported fields }
Generational owns one generation of R at a time, for a service that can be stopped and started again in-process.
Why this exists ¶
A restart re-invokes a service's StartFunc, so anything that closure captured is shared across generations. Five times across this estate a module has captured something single-use — a supervisor, a gRPC server, an http.Server, an exit status, a per-tenant handle — and then failed in one of two ways: the restart could not work, or the stopped thing carried on looking healthy. Two properties answer those two failures, and neither subsumes the other:
- a new generation is built whole, from one recipe, never revived; and
- a stopped generation refuses further use, loudly.
This type provides both. A Supervisor is single-use by design, so a service that owns one and must survive a restart wraps it here rather than reaching for a restartable supervisor.
Using it ¶
g := &controls.Generational[*run]{
Build: func(ctx context.Context) (*run, error) { return open(ctx) },
Release: func(ctx context.Context, r *run) error { return r.close(ctx) },
}
controller.Register("thing",
controls.WithStart(g.Start),
controls.WithStopErr(g.Stop),
controls.WithStatus(g.Healthy),
)
The zero value is not usable: Build and Release are required. It is safe for concurrent use, and Use is on the hot path — it takes no mutex.
Example ¶
A Generational hands each Start a value built fresh, refuses a second Start while one is live, and refuses Use once stopped rather than serving a stale handle.
package main
import (
"context"
"fmt"
"gitlab.com/phpboyscout/go/errors"
"gitlab.com/phpboyscout/go/controls"
)
type conn struct{ id int }
func main() {
built := 0
g := &controls.Generational[*conn]{
Build: func(context.Context) (*conn, error) {
built++
return &conn{id: built}, nil
},
Release: func(_ context.Context, c *conn) error {
fmt.Println("released", c.id)
return nil
},
}
ctx := context.Background()
show := func(c *conn) error {
fmt.Println("using", c.id, "in generation", g.Generation())
return nil
}
fmt.Println("use before start:", errors.Is(g.Use(show), controls.ErrNoGeneration))
_ = g.Start(ctx)
_ = g.Use(show)
fmt.Println("second start:", errors.Is(g.Start(ctx), controls.ErrGenerationRunning))
_ = g.Stop(ctx)
fmt.Println("use after stop:", errors.Is(g.Use(show), controls.ErrNoGeneration), "generation", g.Generation())
_ = g.Start(ctx)
_ = g.Use(show)
_ = g.Stop(ctx)
}
Output: use before start: true using 1 in generation 1 second start: true released 1 use after stop: true generation 0 using 2 in generation 2 released 2
func (*Generational[R]) Generation ¶ added in v0.5.0
func (g *Generational[R]) Generation() uint64
Generation is the current generation's number, or zero when none is live. It increases by one per successful Start and never repeats.
func (*Generational[R]) Healthy ¶ added in v0.5.0
func (g *Generational[R]) Healthy() error
Healthy reports the current generation's health, or ErrNoGeneration when there is none. A stopped generation is never reported healthy.
func (*Generational[R]) Start ¶ added in v0.5.0
func (g *Generational[R]) Start(ctx context.Context) error
Start builds a generation and installs it.
It refuses rather than overlapping: a second Start while one is live returns ErrGenerationRunning, and a Start whose predecessor is still releasing waits for it within ctx and then returns ErrPredecessorLive.
func (*Generational[R]) Stop ¶ added in v0.5.0
func (g *Generational[R]) Stop(ctx context.Context) error
Stop closes admission, drains outstanding leases within ctx, and releases the generation's resources.
It is bounded: it returns within the caller's budget whatever the consumer's code does. What it does not promise is that every leased call has returned — a call that ignores its own cancellation is disowned rather than waited for, because Go cannot end a goroutine. Resources are still released, and a later Start is refused until they are.
It is idempotent, and stopping when nothing is running is not an error.
func (*Generational[R]) Use ¶ added in v0.5.0
func (g *Generational[R]) Use(fn func(R) error) error
Use runs fn against the live generation, holding a lease so Release cannot begin while fn runs.
It is the ONLY accessor. There is deliberately no method returning R for a caller to keep: a retained handle with no staleness protection is the defect this type exists to prevent, and it has already happened once in this estate.
type HealthCheck ¶
type HealthCheck struct {
// Name is the unique identifier for this check.
Name string
// Check is the function that performs the health check.
// It receives a context with the check's timeout applied.
Check func(ctx context.Context) CheckResult
// Timeout is the maximum duration for a single check execution.
// Default: 5s.
Timeout time.Duration
// Interval is the polling interval for async checks.
// Zero means synchronous (run on every health request).
Interval time.Duration
// Type determines which health endpoints this check feeds into.
// Default: CheckTypeReadiness.
Type CheckType
}
HealthCheck defines a named health check function.
type HealthCheckReporter ¶
type HealthCheckReporter interface {
HealthReporter
// GetCheckResult returns the latest result for a named health check.
GetCheckResult(name string) (CheckResult, bool)
}
HealthCheckReporter extends HealthReporter with check-specific queries.
type HealthReport ¶
type HealthReport struct {
OverallHealthy bool `json:"overall_healthy"`
// State is the controller's lifecycle state when the report was taken.
//
// Carried on every report, including Status, which is explicitly not a gate.
// A reader that only sees OverallHealthy cannot tell an unhealthy service
// from a controller that has begun shutting down, and those want different
// responses.
State State `json:"state"`
Services []ServiceStatus `json:"services"`
}
HealthReport is the aggregate health status across all registered services.
type HealthReporter ¶
type HealthReporter interface {
Status() HealthReport
Liveness() HealthReport
Readiness() HealthReport
GetServiceInfo(name string) (ServiceInfo, bool)
}
HealthReporter provides read access to service health, liveness, and readiness reports, and to per-service runtime information. Handlers and transports that only need to query health should depend on this interface rather than the full Controllable.
type Message ¶
type Message string
Message represents a control message sent to the controller (e.g. "stop").
const (
Stop Message = "stop"
)
type ProbeFunc ¶
type ProbeFunc func() error
ProbeFunc is a health check function for liveness or readiness probes.
type RestartPolicy ¶
type RestartPolicy struct {
MaxRestarts int
InitialBackoff time.Duration
MaxBackoff time.Duration
HealthFailureThreshold int
HealthCheckInterval time.Duration
// RestartResetInterval is how long a service must run healthily before its
// consecutive-failure restart counter is reset to zero. Zero selects
// DefaultRestartResetInterval. The count therefore measures consecutive
// failures, not lifetime restarts.
RestartResetInterval time.Duration
}
RestartPolicy defines how a service should be restarted on failure.
type Runner ¶
type Runner interface {
Start()
Stop()
IsRunning() bool
IsStopped() bool
IsStopping() bool
Register(id string, opts ...ServiceOption)
}
Runner provides service lifecycle operations.
type Service ¶
type Service struct {
Name string
Start StartFunc
Stop StopFunc
StopErr StopErrFunc
Status StatusFunc
Liveness ProbeFunc
Readiness ProbeFunc
RestartPolicy *RestartPolicy
}
Service represents a managed background service with start/stop lifecycle, health probes, and optional restart policy.
type ServiceInfo ¶
type ServiceInfo struct {
Name string
RestartCount int
LastStarted time.Time
LastStopped time.Time
Error error
// StopErr is how the last stop ended: nil when every resource was
// released, non-nil when it was not, and always nil for a service
// registered with [WithStop], which cannot report either way.
//
// A panic inside a stop is contained and recorded here rather than ending
// the process.
StopErr error
}
ServiceInfo holds runtime metadata about a registered service.
type ServiceOption ¶
type ServiceOption func(*Service)
ServiceOption is a functional option for configuring a Service.
func WithLiveness ¶
func WithLiveness(fn ProbeFunc) ServiceOption
WithLiveness sets a liveness probe for the service.
Example ¶
package main
import (
"context"
"net/http"
"gitlab.com/phpboyscout/go/controls"
)
func main() {
controller := controls.NewController(context.Background())
controller.Register("api",
controls.WithStart(func(ctx context.Context) error { return nil }),
controls.WithLiveness(func() error {
// Check if the service can respond
resp, err := http.Get("http://localhost:8080/healthz")
if err != nil {
return err
}
_ = resp.Body.Close()
return nil
}),
)
_ = controller
}
Output:
func WithReadiness ¶
func WithReadiness(fn ProbeFunc) ServiceOption
WithReadiness sets a readiness probe for the service.
func WithRestartPolicy ¶
func WithRestartPolicy(policy RestartPolicy) ServiceOption
WithRestartPolicy configures automatic restart behaviour for a service.
Example ¶
package main
import (
"context"
"time"
"gitlab.com/phpboyscout/go/controls"
)
func main() {
controller := controls.NewController(context.Background())
controller.Register("worker",
controls.WithStart(func(ctx context.Context) error {
return nil
}),
controls.WithRestartPolicy(controls.RestartPolicy{
MaxRestarts: 3,
InitialBackoff: time.Second,
MaxBackoff: 30 * time.Second,
}),
)
_ = controller
}
Output:
func WithRestartResetInterval ¶
func WithRestartResetInterval(d time.Duration) ServiceOption
WithRestartResetInterval sets how long a service must run healthily before its consecutive-failure restart counter resets. It implies a restart policy: if the service has none, a default policy is created so the interval takes effect.
func WithStart ¶
func WithStart(fn StartFunc) ServiceOption
WithStart sets the service's start function.
func WithStatus ¶
func WithStatus(fn StatusFunc) ServiceOption
WithStatus sets the service's health check function.
func WithStop ¶
func WithStop(fn StopFunc) ServiceOption
WithStop sets the service's graceful shutdown function.
func WithStopErr ¶ added in v0.5.0
func WithStopErr(fn StopErrFunc) ServiceOption
WithStopErr registers a stop function that can report failure to release.
It is purely additive: WithStop is unchanged and is equivalent to a StopErrFunc that always returns nil, so nothing that already works needs to move. Setting both is a mistake rather than a merge — the error-reporting one wins, because the alternative is calling a service's stop twice.
Example ¶
A stop that reports how it ended lands on ServiceInfo.StopErr, and the controller changes nothing else on the strength of it.
package main
import (
"context"
"fmt"
"gitlab.com/phpboyscout/go/errors"
"gitlab.com/phpboyscout/go/controls"
)
func main() {
controller := controls.NewController(context.Background())
controller.Register("holder",
controls.WithStart(func(ctx context.Context) error {
<-ctx.Done()
return ctx.Err()
}),
controls.WithStopErr(func(context.Context) error {
return errors.New("listener still open")
}),
)
controller.Start()
controller.Stop()
controller.Wait()
info, _ := controller.GetServiceInfo("holder")
fmt.Println("stop error:", info.StopErr)
}
Output: stop error: listener still open
type ServiceStatus ¶
type ServiceStatus struct {
Name string `json:"name"`
Status string `json:"status"` // "OK", "ERROR"
Error string `json:"error,omitempty"`
}
ServiceStatus is the health status of a single service, used in HealthReport.
type Services ¶
type Services struct {
// contains filtered or unexported fields
}
Services manages the collection of registered services and their lifecycle.
type StartFunc ¶
StartFunc is the callback invoked to start a service. It receives a context that is cancelled when the controller shuts down.
A restart calls this again, so what the closure captured is shared ¶
Everything this function's closure captured is shared across generations and must be safe to use again. Anything single-use or generation-scoped — a listener, a server, a supervisor, an exit status, a per-tenant handle — must be built INSIDE the run that consumes it, or the second run gets the first run's corpse.
The two failures this causes are worth naming, because they look nothing alike. Either the restart cannot work at all, or the stopped thing carries on looking healthy — a status cell that was never cleared, a handle whose backing is gone. So: a stopped generation must refuse further use, loudly, and a new generation is built whole, from one recipe, never revived and never partially reused.
Generational provides both for a service that needs them. A service that genuinely captures nothing single-use needs neither, and most do not.
type State ¶
type State string
State represents the lifecycle state of the controller.
It moves in one direction: NeverStarted to Running, then either to Stopping and Stopped or to UnableToStart and then Stopping and Stopped. Unknown is not part of that sequence; it means the value could not be determined.
const ( // Unknown means the state could not be determined. It is what [Controller.GetState] // reports for a zero-valued Controller, i.e. one built without NewController, // which is otherwise undetectable because State is a string type and its zero // value is the empty string. Unknown State = "unknown" // NeverStarted means the controller was constructed and Start has not been // called. Registration is only honoured in this state, and a Stop here leaves // it unchanged: stopping something that never started is a no-op, so the // controller stays startable. NeverStarted State = "never_started" // Running means Start has been called and shutdown has not begun. It is the // only state in which the controller reports ready. Running State = "running" // UnableToStart means a registered service has proven it will never start: it // has failed without ever starting cleanly and has exhausted its restart // policy. Nothing is stopped, and the error still reaches the error channel; // what changes is that the controller stops reporting ready, so an // orchestrator routes no traffic to a process that cannot do its job. // // A service that fails and then recovers on a restart never reaches this, and // neither does one that started cleanly and failed later. Both conditions are // required, so the state is terminal rather than something readiness flaps on // through a slow boot. UnableToStart State = "unable_to_start" Stopping State = "stopping" Stopped State = "stopped" )
type StateAccessor ¶
type StateAccessor interface {
GetState() State
SetState(state State)
GetContext() context.Context
GetLogger() *slog.Logger
}
StateAccessor provides access to controller state and context, and is meant to be consumed AND implemented outside this module.
The setter is deliberate, and is why this is not a read-only interface: a consumer driving a controller it owns needs to be able to say what state it is in, and a consumer implementing this interface over its own type needs the same surface the Controller has. It was proposed for removal once on the reading that this was a read-side view, which the doc comment then said it was; see wiki spec 0003 D7.
SetState mutates a field the control goroutines read, so the contract is the one Configurable states: call it during construction, or on a controller you own. Reaching into a running controller from elsewhere races those goroutines, and since readiness is gated on the state (0003 D2) it can also take a healthy process out of rotation.
type StatusFunc ¶
type StatusFunc func() error
StatusFunc is the callback invoked to check a service's health. Returns nil if healthy, an error otherwise.
It must report the CURRENT run. A status cell captured by the closure and never cleared between generations keeps reporting the previous run's failure, which fails the health threshold, which restarts the service, which reports the same stale failure — a service churning to restart exhaustion without one log line naming the cause. See StartFunc on what a restart shares.
type StopErrFunc ¶ added in v0.5.0
StopErrFunc stops a service and reports how the stop ended.
A nil error means every resource the run acquired has been released. A non-nil one means the budget expired or release failed, and the service may still be holding something — a listener, a connection, a subscription.
It is recorded, not acted upon ¶
The Controller stores the result in ServiceInfo and changes nothing else: no restart is refused and no policy is altered on the strength of it. Knowing the answer is the prerequisite, and making a restart depend on it is a behaviour change for every consumer and a separate decision.
Before this existed a stop that ignored its context was abandoned at the deadline silently, and nothing anywhere recorded that it had happened.
type StopFunc ¶
StopFunc is the callback invoked to stop a service gracefully. The context carries the shutdown timeout.
type Supervisor ¶ added in v0.4.0
type Supervisor struct {
// contains filtered or unexported fields
}
Supervisor runs children that attach and detach while the process is running.
Registered means required; attached does not ¶
A Service registered with a Controller is a requirement for operation: if its probe fails, the whole process reports unready. A child attached to a Supervisor is not. A failed child never makes the supervisor unready — at any proportion, including all of them — because whether that child mattered is the consumer's judgement and the consumer has the context to make it.
What the supervisor does instead is report CheckDegraded while any child has terminally failed, and hand the consumer a Failure to act on.
Which of the two to use ¶
A Controller manages a fixed set registered before Start, with ordered shutdown. A Supervisor manages a set that changes, with concurrent shutdown and no ordering between children — they must not depend on each other. A process has one Controller and may have several Supervisors.
A Supervisor is itself a Service: register it with a Controller and its children are supervised beneath it.
Single use, and what each call does out of order ¶
A Supervisor moves through new, running, stopping and stopped, and never goes back. Once shutdown begins, Supervisor.Attach and Supervisor.Start return ErrSupervisorStopped and Supervisor.Readiness reports it, rather than accepting work that will never be supervised.
Supervisor.Stop and Supervisor.Detach are bounded by the context they are given, including across a Child.Stop that blocks. A child that outlives the budget is abandoned rather than allowed to hold shutdown open, which is the same bargain Controller strikes at its own shutdown deadline.
Example ¶
A Supervisor accepts children before and after Start. A child that returns nil has finished; one with no restart policy that returns an error is reported to the consumer once, as a Failure.
package main
import (
"context"
"fmt"
"time"
"gitlab.com/phpboyscout/go/errors"
"gitlab.com/phpboyscout/go/controls"
)
func main() {
sup := controls.NewSupervisor(
controls.WithOnFailure(func(f controls.Failure) {
fmt.Println("failed:", f.Name, "after", f.Restarts, "restarts")
}),
)
_ = sup.Attach(controls.Child{
Name: "once",
Start: func(context.Context) error {
fmt.Println("once ran")
return nil
},
})
if err := sup.Start(context.Background()); err != nil {
fmt.Println("start:", err)
return
}
// Attaching after Start is the point: the child is supervised identically.
_ = sup.Attach(controls.Child{
Name: "worker",
Start: func(ctx context.Context) error {
<-ctx.Done()
return ctx.Err()
},
})
_ = sup.Attach(controls.Child{
Name: "broken",
Start: func(context.Context) error { return errors.New("no such queue") },
})
for {
h := sup.Health()
if h["once"].State == controls.ChildStopped &&
h["worker"].State == controls.ChildRunning &&
h["broken"].State == controls.ChildFailed {
break
}
time.Sleep(time.Millisecond)
}
// The failed child does not make the supervisor unready.
fmt.Println("ready:", sup.Readiness() == nil)
fmt.Println("check:", sup.HealthCheck("children").Check(context.Background()).Message)
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
sup.Stop(ctx)
fmt.Println("after stop:", errors.Is(sup.Readiness(), controls.ErrSupervisorStopped))
}
Output: once ran failed: broken after 0 restarts ready: true check: 1 of 3 supervised children have failed after stop: true
func NewSupervisor ¶ added in v0.4.0
func NewSupervisor(opts ...SupervisorOption) *Supervisor
NewSupervisor returns a supervisor with no children.
func (*Supervisor) Attach ¶ added in v0.4.0
func (s *Supervisor) Attach(c Child) error
Attach adds a child, before or after Supervisor.Start.
Attaching after Start is the point of this type: a Controller cannot do it, and says so — a late registration there "is never started, monitored, or stopped". A child attached here is supervised identically whenever it arrives.
Once shutdown has begun it returns ErrSupervisorStopped.
func (*Supervisor) Detach ¶ added in v0.4.0
func (s *Supervisor) Detach(ctx context.Context, name string) error
Detach stops one child and forgets it. Every other child is untouched.
It waits for the child to stop, bounded by ctx across both the child's own goroutine and its Child.Stop. If the budget expires the child is still forgotten and ErrDetachTimeout is returned, so a caller learns that a goroutine outlived its detach rather than discovering it later as a leak nothing reports.
Detaching a child the supervisor never started is immediate and returns nil. Its Start was never called, so there is nothing to stop and nothing to wait for, and reporting a timeout for it would be a lie about what happened.
func (*Supervisor) DroppedReports ¶ added in v0.4.0
func (s *Supervisor) DroppedReports() int64
DroppedReports is how many failures could not be delivered to the channel returned by Supervisor.Failures because it was full.
It should be zero, and it counts that channel alone: the callback registered by WithOnFailure has its own unbounded queue and loses nothing. A drop nobody counts is indistinguishable from a system with nothing to report, which is the failure this whole type is arranged to avoid.
func (*Supervisor) Failures ¶ added in v0.4.0
func (s *Supervisor) Failures() <-chan Failure
Failures returns a channel of children that have exhausted their restart policy.
The channel is created on first call and bounded at DefaultFailureBufferSize. A consumer that never calls this never has one to fill; a consumer that does is expected to drain it, and sends that would block are dropped and counted rather than stalling the supervisor. See Supervisor.DroppedReports.
It is never closed. Ranging over it does not terminate at shutdown, so a consumer that wants a loop to end should select on it alongside its own done channel.
func (*Supervisor) Health ¶ added in v0.4.0
func (s *Supervisor) Health() map[string]ChildStatus
Health reports every attached child's state.
This is data, not health. What a consumer does about a failed child is its judgement — see Supervisor.Failures and WithOnFailure.
func (*Supervisor) HealthCheck ¶ added in v0.4.0
func (s *Supervisor) HealthCheck(name string) HealthCheck
HealthCheck returns a check reporting DEGRADED while any child has terminally failed, and healthy otherwise.
Register it with a Controller alongside the supervisor itself. It never returns CheckUnhealthy: a supervisor with failed children is working, and whether an empty working set is a crisis belongs to whoever attached them.
func (*Supervisor) Readiness ¶ added in v0.4.0
func (s *Supervisor) Readiness() error
Readiness reports whether the SUPERVISOR is working. It never fails because a child has failed.
This is the point of the register/attach boundary and it is load-bearing. A Supervisor registered with a Controller is a Service, and a registered service whose probe fails sets OverallHealthy to false for the whole process. If a failed child could fail this probe, one dead child would take the process out of rotation — exactly the coupling attaching rather than registering exists to avoid, arriving through the registration in the back door.
A failed child is reported by Supervisor.HealthCheck as DEGRADED, which is visible to an operator and inert to a probe, and delivered to the consumer as a Failure to judge.
What it does fail on is the supervisor's own lifecycle: ErrSupervisorNotStarted before Start and ErrSupervisorStopped once shutdown has begun. A stopped supervisor reporting ready is the same boundary failing in the other direction.
func (*Supervisor) Start ¶ added in v0.4.0
func (s *Supervisor) Start(ctx context.Context) error
Start begins supervising, and is a StartFunc so a Supervisor registers with a Controller like any other service.
It returns once children are launched; the supervisor serves in the background from then on. A Supervisor is single use: after Stop it returns ErrSupervisorStopped rather than starting again.
func (*Supervisor) Stop ¶ added in v0.4.0
func (s *Supervisor) Stop(ctx context.Context)
Stop cancels every child at once and waits for all of them, bounded by ctx.
Concurrent, not ordered: shutdown is bounded by the slowest child rather than by their number. Ten children taking 100ms each stop in about 100ms, where one at a time would take a second. The price is that children must not depend on each other — a consumer whose units do wants a Controller, which provides reverse-registration ordering deliberately.
A child that outlives ctx is abandoned, the same bargain a Controller strikes at its shutdown deadline, because a stop that cannot end is worse than one that reports an incomplete result. Supervisor.Health still lists it.
Calling Stop before Start stops nothing: no child's Start was called, so there is nothing to cancel and no Child.Stop to run. A second Stop waits for the first to finish rather than reporting a completion that has not happened.
type SupervisorOption ¶ added in v0.4.0
type SupervisorOption func(*Supervisor)
SupervisorOption configures a Supervisor.
func WithOnFailure ¶ added in v0.4.0
func WithOnFailure(fn func(Failure)) SupervisorOption
WithOnFailure registers a callback for a child that has exhausted its restart policy.
It runs on a dedicated dispatch goroutine, behind a recover, from an ordered queue. That buys three things: a slow callback cannot stall supervision, failures arrive in order, and none is lost while the callback is busy. The queue is unbounded because a terminal failure is rare by construction — a child reaches one only after exhausting its restart policy — and because a callback is not opt-in the way Supervisor.Failures is, so shedding from it would lose notifications a consumer never agreed to lose.
A callback may call back into the supervisor, including Supervisor.Stop. Nothing is held while it runs, and Stop does not wait for the dispatch goroutine: a callback still executing when Stop returns runs to completion, and the goroutine exits once the queue drains.
The callback returns nothing. A consumer that wants the supervisor to act calls its API; an instruction returned from a notification would make the callback load-bearing, synchronous and re-entrant.
type ValidErrorFunc ¶
ValidErrorFunc determines whether an error from a service is expected (e.g. http.ErrServerClosed) and should not trigger a restart.