Documentation
¶
Overview ¶
Package workqueue is one durable runner for every long job (docs/plans/33-workqueue.md).
Long work used to be started from an HTTP handler five different ways, each with its own machinery. The config apply had none at all: it ran on the request context, so a browser that stopped waiting killed it wherever it had got to, and the error it tried to write used the same dead context and was never recorded. The webhook path was worst, GitHub hangs up after about ten seconds.
Two rules make the difference. A job runs on context.Background() with its kind's timeout, never on a request context. And the queue is a table, so a row left running by a restart is dealt with on boot instead of vanishing with the process.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Handler ¶
Handler runs one job. The context is the queue's own, with the kind's timeout on it and a cancel handle held for Cancel.
type Job ¶
Job is one item as a handler sees it, plus the two writes it may make while running.
func (*Job) SetProgress ¶
SetProgress records the fine-grained blob (bytes done and total, and friends). Whatever the UI for this kind reads.
type KindOpts ¶
type KindOpts struct {
// Timeout bounds one run. Zero means 30 minutes, which is what the deploy
// engine has always used.
Timeout time.Duration
// Concurrency is how many of this kind run at once. Zero means one.
// Ignored when Limit is set.
Concurrency int
// Limit reads the kind's concurrency from the server settings. Read on
// every drain, so a change applies within one poll. Lowering it lets
// running jobs finish; only new ones wait.
Limit func(settings.Resolved) int
// OnRestart is what boot recovery does with a row left running.
OnRestart Recovery
// RestartFail is the error recorded when OnRestart is Fail. A generic
// "the panel restarted" tells an operator nothing about what to do next.
RestartFail string
// Cleanup runs on a job that OnRestart failed, before the row is closed:
// deleting a partial artifact, putting a service back up. Best effort.
// A non-empty return replaces RestartFail, for a kind whose message
// depends on how far it got.
Cleanup func(ctx context.Context, j *Job) string
// OnSuperseded runs for each queued item a newer Enqueue with the same
// dedupe key pushed aside, so the domain row it carried can be closed.
// Without it the deploy or plan row stays open with nothing to run it.
OnSuperseded func(ctx context.Context, j *Job)
}
KindOpts is the per-kind configuration.
type Queue ¶
type Queue struct {
// contains filtered or unexported fields
}
Queue is the runner. One per process.
func (*Queue) Cancel ¶
Cancel stops a running job, or drops a queued one. A job that has already finished is not an error: the caller is racing the work, which is the normal case for a Stop button.
func (*Queue) Enqueue ¶
func (q *Queue) Enqueue(ctx context.Context, kindName, dedupeKey string, payload any) (string, error)
Enqueue adds an item and supersedes the older queued ones sharing its dedupe key. Returns the new item's id.
The key is a scope, not an identity: for a config apply it is the stack, so two pushes a second apart do not both apply; for a deploy it is the tile, which is what the engine already did by hand. An empty key never dedupes.
type Recovery ¶
type Recovery int
Recovery says what happens to a row left running by a restart.
const ( // Requeue runs it again from the top. Only for work that is convergent: // re-running it on a half-finished state finishes the job. Requeue Recovery = iota // Fail gives up and records why. For work that cannot be resumed blind, // like a two-pass rsync with the service scaled to zero, or a half-written // backup artifact. Fail )