workqueue

package
v0.1.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 16, 2026 License: Apache-2.0 Imports: 9 Imported by: 0

Documentation

Overview

Package workqueue is one durable runner for every long job (docs/plans/33-workqueue.md).

Long work used to be started from an HTTP handler five different ways, each with its own machinery. The config apply had none at all: it ran on the request context, so a browser that stopped waiting killed it wherever it had got to, and the error it tried to write used the same dead context and was never recorded. The webhook path was worst, GitHub hangs up after about ten seconds.

Two rules make the difference. A job runs on context.Background() with its kind's timeout, never on a request context. And the queue is a table, so a row left running by a restart is dealt with on boot instead of vanishing with the process.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Handler

type Handler func(ctx context.Context, j *Job) error

Handler runs one job. The context is the queue's own, with the kind's timeout on it and a cancel handle held for Cancel.

type Job

type Job struct {
	Item *repo.WorkItem
	// contains filtered or unexported fields
}

Job is one item as a handler sees it, plus the two writes it may make while running.

func (*Job) Payload

func (j *Job) Payload(v any) error

Payload decodes the item's payload into v.

func (*Job) SetProgress

func (j *Job) SetProgress(ctx context.Context, v any)

SetProgress records the fine-grained blob (bytes done and total, and friends). Whatever the UI for this kind reads.

func (*Job) SetStep

func (j *Job) SetStep(ctx context.Context, step string)

SetStep records coarse progress. It is also the resume point for a kind whose recovery is "carry on from here", which is why it is a column and not a log line.

type KindOpts

type KindOpts struct {
	// Timeout bounds one run. Zero means 30 minutes, which is what the deploy
	// engine has always used.
	Timeout time.Duration
	// Concurrency is how many of this kind run at once. Zero means one.
	Concurrency int
	// OnRestart is what boot recovery does with a row left running.
	OnRestart Recovery
	// RestartFail is the error recorded when OnRestart is Fail. A generic
	// "the panel restarted" tells an operator nothing about what to do next.
	RestartFail string
	// Cleanup runs on a job that OnRestart failed, before the row is closed:
	// deleting a partial artifact, putting a service back up. Best effort.
	Cleanup func(ctx context.Context, j *Job)
	// OnSuperseded runs for each queued item a newer Enqueue with the same
	// dedupe key pushed aside, so the domain row it carried can be closed.
	// Without it the deploy or plan row stays open with nothing to run it.
	OnSuperseded func(ctx context.Context, j *Job)
}

KindOpts is the per-kind configuration.

type Queue

type Queue struct {
	// contains filtered or unexported fields
}

Queue is the runner. One per process.

func New

func New(store repo.Store) *Queue

New builds the queue. Nothing runs until Start.

func (*Queue) Cancel

func (q *Queue) Cancel(ctx context.Context, id string) error

Cancel stops a running job, or drops a queued one. A job that has already finished is not an error: the caller is racing the work, which is the normal case for a Stop button.

func (*Queue) Enqueue

func (q *Queue) Enqueue(ctx context.Context, kindName, dedupeKey string, payload any) (string, error)

Enqueue adds an item and supersedes the older queued ones sharing its dedupe key. Returns the new item's id.

The key is a scope, not an identity: for a config apply it is the stack, so two pushes a second apart do not both apply; for a deploy it is the tile, which is what the engine already did by hand. An empty key never dedupes.

func (*Queue) Register

func (q *Queue) Register(name string, h Handler, opts KindOpts)

Register wires a handler for one kind. Call before Start.

func (*Queue) Start

func (q *Queue) Start(ctx context.Context)

Start recovers what the last process left behind and then runs the loop until ctx is done. Non-blocking.

type Recovery

type Recovery int

Recovery says what happens to a row left running by a restart.

const (
	// Requeue runs it again from the top. Only for work that is convergent:
	// re-running it on a half-finished state finishes the job.
	Requeue Recovery = iota
	// Fail gives up and records why. For work that cannot be resumed blind,
	// like a two-pass rsync with the service scaled to zero, or a half-written
	// backup artifact.
	Fail
)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL