Documentation
¶
Overview ¶
Package supervisor runs a child process and keeps it running.
An Engine asks its Source for a Spec before every start, including the first, and supervises the child started from it until it exits. The Source then decides whether the child runs again; a restart waits Options.RestartDelay and asks for a fresh Spec, so a consumer that builds the Spec from its configuration re-reads it on every start.
Serialization ¶
Everything that changes the state of the supervision happens on the goroutine that called Engine.Run: starting, signalling, stopping and killing the child, deciding on and waiting for a restart, writing and removing pid files. Every callback but Options.Check runs there too (Source.Next, Source.Restart, Options.OnEvent, Options.OnReload, Options.Cleanup), so the callbacks never run concurrently with each other. Three helper goroutines only feed the loop: one relays OS signals, one waits for the child and one runs the periodic check while a child runs.
Pid files ¶
The engine owns its pid files through package pidfile: one owner among any number of processes racing for the file, a flock held for as long as the pid in the file is current, and a file removed only by its owner and only while the path still names it. That package states the protocol and what it assumes of other writers of the same path.
Options.PidFile is the supervisor's own pid file. Run takes it after it has subscribed to signals, so whoever finds the pid there can already stop the supervisor with a signal. If Run cannot take it, it returns before starting anything and does not run Options.Cleanup: whatever is out there belongs to someone else. Once taken, it is removed when Run returns, after the child is gone and Cleanup has run, so its disappearance means the supervision is over.
Options.ChildPidFile is written after every start, before the Started event, and removed after every exit. If it cannot be written, the child is killed and Run returns.
Signals ¶
Run catches every signal but SIGURG, which the Go runtime sends itself for preemption, and SIGCHLD, which reports the supervisor's own children. Each received signal is handled exactly once, in the order received. That is not the order or the number of the signals sent: the kernel merges a signal sent again before the first one was delivered, and the Go runtime drops one it cannot buffer. While a child runs:
- A stop signal (Options.StopSignals) sends the child Spec.StopSignal, or the received signal itself with Spec.ForwardStopSignal. A child still alive Spec.StopTimeout after the first stop signal is killed with SIGKILL. Once the child has exited, Run returns without asking about a restart. A further stop signal is sent to the child as well but does not restart the timeout.
- The reload signal (Options.ReloadSignal) runs Options.OnReload, and then sends the child Spec.ReloadSignal, or the received signal itself with Spec.ForwardReloadSignal, or nothing when neither is set.
- An ignored signal (Options.IgnoreSignals) is reported and dropped, with a child or without.
- Any other signal is forwarded to the child.
Signals go to the child, or to its whole process group with Spec.ProcessGroup.
While no child runs, a stop signal ends Run, the reload hook runs, and a signal that would have been forwarded is dropped. That holds during the restart delay and for the signals that have arrived by each decision: before the Source is asked for a Spec, before the child is started with it, and before the Source is asked about a restart. So a pending stop always wins: no child is started and the Source is not asked anything after it, even when the child's exit or the end of the restart delay is ready at the same moment. A signal that arrives while the child is being started waits in a buffer and is handled once the child runs; one that finds the buffer full is lost.
Cancelling the context passed to Run is a stop with Spec.StopSignal; it wins over a start or a restart in the same way.
Checks ¶
With Options.CheckPeriod set, Options.Check runs that often while a child runs, and a Checked event reports each result. When it returns an error, the child (its group, with Spec.ProcessGroup) is killed with SIGKILL, and once the child has exited Run returns the error. A failed check is a hard stop: the Source is not asked about a restart and no child is started again, whatever the Source would answer. A check that panics has failed: the panic, on the checker's goroutine where nothing around Run could recover it, becomes an error wrapping ErrCheckPanicked.
When the child exits while a check runs, the engine cancels the check's context and waits for it. An error it returns is still a failed check, reported by a Checked event and ending Run, unless every error at the leaves of its tree, through Unwrap and through joined errors, is context.Canceled. So errors.Join(finding, ctx.Err()) is a failed check.
Starting ¶
Every child is started by Options.Start, which receives a fully prepared exec.Cmd and has to start it. It is the one place to change how a child is started, for example to execute bytes that were verified in memory rather than whatever the path names by the time of the exec. A process it started but did not hand over, because it returned an error or panicked afterwards, is killed with SIGKILL (its group, with Spec.ProcessGroup) and waited for before the error or the panic goes on.
Detach starts a process that outlives its caller. While the caller lives, it waits for the process in the background, so none is left a zombie.
Guarantees ¶
When Run returns, the child it started last has exited and has been waited for, the checks have stopped, the child's pid file is removed, Options.Cleanup has run if the supervisor pid file was taken (or none was configured), and the supervisor pid file is removed.
The same holds when a callback panics: the child is killed with SIGKILL and waited for, the checks stop, the pid files are removed, Cleanup runs (the supervisor pid file goes even if Cleanup panics as well), the signals are no longer caught, and then the panic goes on to the caller of Run.
Limits ¶
These are part of the contract rather than defects:
- A member of the child's process group that outlives the child is neither killed nor waited for: only the stop timeout and a failed check kill the whole group, and only while the child still runs.
- A signal sent to the group after the child has been waited for, in the moment before the engine sees the exit, goes to whatever process group has that id by then; the id can be reused once the whole group is gone.
- A Check that ignores the cancellation of its context, or an io.Writer given as Spec.Stdout or Spec.Stderr that blocks, keeps Run from returning until it returns.
- Callbacks must not rely on being called after a panic in another callback: only Cleanup is.
Index ¶
- Variables
- func Detach(path string, args []string, start StartFunc) (int, error)
- func StartCmd(cmd *exec.Cmd) error
- type Checked
- type Engine
- type Error
- type Event
- type Exit
- type KillReason
- type Killed
- type Op
- type Options
- type PidFileWritten
- type Restarting
- type SignalAction
- type SignalReceived
- type Source
- type Spec
- type StartFunc
- type Started
Constants ¶
This section is empty.
Variables ¶
var ( // ErrInvalidOptions is wrapped by the error New returns for options it // refuses. ErrInvalidOptions = errors.New("invalid supervisor options") // ErrAlreadyRun is returned by a second call to Engine.Run. ErrAlreadyRun = errors.New("the supervisor has already run") // ErrCheckPanicked is wrapped by the error of a check that panicked; it // counts as a failed check. ErrCheckPanicked = errors.New("the check panicked") )
var ErrInvalidSpec = errors.New("invalid spec")
ErrInvalidSpec is wrapped by the error for a Spec that cannot be started.
var ErrNotStarted = errors.New("the start function returned without a process")
ErrNotStarted is returned when a StartFunc reports success without having started the process.
Functions ¶
func Detach ¶
Detach starts path with args in a process group of its own and does not wait for it: the process keeps running after the caller exits and is not stopped by signals sent to the caller's group. Its standard streams are connected to the null device and it inherits the caller's environment. start starts the process; nil means StartCmd. Detach returns the pid.
While the caller lives, a goroutine waits for the process, so one that exits before the caller does not stay behind as a zombie.
Types ¶
type Checked ¶
type Checked struct {
Err error
}
Checked reports a finished periodic check. A non-nil Err is a failed check.
type Engine ¶
type Engine struct {
// contains filtered or unexported fields
}
Engine supervises one child at a time. See the package documentation for the model.
func (*Engine) Run ¶
Run supervises until a stop, a cancelled ctx, an exit the Source does not restart, or a failure. It returns nil in the first three cases and an error that unwraps to *Error otherwise. Run can be called once.
A panic in a callback tears the supervision down as described in the package documentation and then goes on up to the caller of Run.
type Event ¶
type Event interface {
// contains filtered or unexported methods
}
Event is something that happened during the supervision. Options.OnEvent receives one of the types below; a consumer switches on the type to write its own log lines.
type Exit ¶
type Exit struct {
Pid int
State *os.ProcessState
Err error
}
Exit reports a child that exited and was waited for. State is the state of the exited process, nil only if waiting for it failed. Err is what exec.Cmd.Wait returned: nil for a zero exit status, an *exec.ExitError for any other exit, or the failure to wait or to copy the child's input or output.
type KillReason ¶
type KillReason int
KillReason tells why the child was killed with SIGKILL.
const ( // KillStopTimeout: the child outlived Spec.StopTimeout after a stop. KillStopTimeout KillReason = iota + 1 // KillCheckFailed: Options.Check returned an error. KillCheckFailed // KillChildPidFile: the child pid file could not be written. KillChildPidFile )
func (KillReason) String ¶
func (reason KillReason) String() string
type Killed ¶
type Killed struct {
Pid int
Reason KillReason
Err error
}
Killed reports SIGKILL sent to the child, or to its process group. Err is the failure to send it.
type Op ¶
type Op int
Op is the step of the supervision an Error comes from.
const ( // OpPidFile is creating or removing the supervisor pid file. OpPidFile Op = iota + 1 // OpNext is getting a valid Spec from the Source. OpNext // OpStart is starting the child. OpStart // OpChildPidFile is writing or removing the child pid file. OpChildPidFile // OpRestart is asking the Source about a restart. OpRestart // OpCheck is the periodic check. OpCheck )
type Options ¶
type Options struct {
// PidFile and ChildPidFile are the pid files of the supervisor and of the
// child; the package documentation states when each is written.
PidFile string
ChildPidFile string
// StopSignals are the received signals that stop the child and end Run.
StopSignals []syscall.Signal
// IgnoreSignals are the received signals that are dropped: never
// forwarded, never stopping anything.
IgnoreSignals []syscall.Signal
// ReloadSignal is the received signal that runs OnReload before the
// child is signalled as its Spec says (Spec.ReloadSignal). It is set
// together with OnReload.
ReloadSignal syscall.Signal
// OnReload runs on ReloadSignal, for example to reopen a log file.
OnReload func() error
// RestartDelay is the pause between an exit and the next start.
RestartDelay time.Duration
// CheckPeriod is how often Check runs while a child is running. It is set
// together with Check.
CheckPeriod time.Duration
// Check is the periodic check. It runs on a goroutine of its own; an
// error or a panic kills the child and ends Run, which never restarts
// it.
Check func(ctx context.Context) error
// Cleanup runs once when Run returns, before the supervisor pid file is
// removed, unless that file could not be created.
Cleanup func()
OnEvent func(event Event)
// Start starts every child; nil means StartCmd.
Start StartFunc
}
Options configure an Engine. A zero field means "none", except for Start.
type PidFileWritten ¶
PidFileWritten reports a pid file written: the supervisor's before the first start, the child's after every start.
type Restarting ¶
Restarting reports the restart delay beginning.
type SignalAction ¶
type SignalAction int
SignalAction is what the engine did with a received signal.
const ( // ActionStop stopped the child, or ended the restart delay. ActionStop SignalAction = iota + 1 // ActionReload ran the reload hook and signalled the child as its Spec // says (Spec.ReloadSignal), if there was one. ActionReload // ActionForward forwarded the signal to the child. ActionForward // ActionDrop dropped the signal: no child was running. ActionDrop // ActionIgnore dropped the signal: Options.IgnoreSignals has it. ActionIgnore )
func (SignalAction) String ¶
func (action SignalAction) String() string
type SignalReceived ¶
type SignalReceived struct {
Signal syscall.Signal
Action SignalAction
Err error
}
SignalReceived reports a received signal and what was done with it. Err holds a failure of the reload hook or of the delivery to the child.
type Source ¶
type Source interface {
// Next returns the Spec for the next start. It is called before every
// start, including the first. An error ends Run.
Next(ctx context.Context) (Spec, error)
// Restart reports whether to start the child again after it exited on
// its own. It is not called when the exit followed a stop or a failed
// check. An error ends Run.
Restart(ctx context.Context, exit Exit) (bool, error)
}
Source decides what the engine runs and whether it runs it again. Its methods are called from the goroutine running Engine.Run.
type Spec ¶
type Spec struct {
// Path is the executable. A name without a slash is looked up in PATH.
Path string
// Args are the arguments, without the program name.
Args []string
// Env is the environment; nil inherits the supervisor's environment.
Env []string
// Dir is the working directory; empty keeps the supervisor's one.
Dir string
// Stdin is written to the child's standard input, which is closed
// afterwards; nil connects the standard input to the null device.
Stdin []byte
// Stdout and Stderr receive the child's output; nil discards it.
Stdout io.Writer
Stderr io.Writer
// StopSignal is the signal that stops the child: it is sent on a stop
// signal unless ForwardStopSignal is set, and on context cancellation.
StopSignal syscall.Signal
// ForwardStopSignal sends the child the stop signal the supervisor
// received instead of StopSignal.
ForwardStopSignal bool
// StopTimeout is how long a stopping child may take before it is killed
// with SIGKILL. It also bounds how long the output of an exited child is
// drained.
StopTimeout time.Duration
// ReloadSignal is the signal the child gets after Options.OnReload has
// run on the reload signal, unless ForwardReloadSignal is set; zero
// sends the child nothing.
ReloadSignal syscall.Signal
// ForwardReloadSignal sends the child the reload signal the supervisor
// received instead of ReloadSignal.
ForwardReloadSignal bool
// ProcessGroup starts the child in a process group of its own and sends
// every signal to that whole group.
ProcessGroup bool
}
Spec describes one run of the child.
func (*Spec) Command ¶
Command prepares a command that runs the Spec once, for a caller that runs it without a supervisor. Cancelling ctx sends the process Spec.StopSignal, or its whole group with Spec.ProcessGroup; Spec.StopTimeout later the process is killed. The output of the process is drained for at most Spec.StopTimeout after it has exited.
type StartFunc ¶
StartFunc starts a fully prepared command, the way exec.Cmd.Start does: on success cmd.Process is the running process, and the caller waits for it with cmd.Wait. It may change how the process is started, as long as the process started is a child of the caller. It starts one process, through the command it is given: a process started through another command, or before the command is replaced, is not known to the engine and is neither stopped nor waited for.
Source Files