Documentation
¶
Overview ¶
Package queue runs the server's tasks: one database row per piece of work (match a work, check it for updates, download a chapter). Tasks are claimed with SKIP LOCKED, hold a lease while they run, retry with back-off, and are capped per kind and per limit key (a source), across all users' work. Identical live tasks are stored once (dedupe keys).
Index ¶
- Constants
- Variables
- func DefaultBackoff(attempts int32) time.Duration
- func Permanent(err error) error
- type Event
- type Handler
- type Kind
- type KindState
- type NewTask
- type Progress
- type Queue
- func (q *Queue) Cancel(ctx context.Context, id uuid.UUID) (db.Task, error)
- func (q *Queue) Enqueue(ctx context.Context, n NewTask) (db.Task, bool, error)
- func (q *Queue) EnqueueWith(ctx context.Context, dbq *db.Queries, n NewTask) (db.Task, bool, error)
- func (q *Queue) Get(ctx context.Context, id uuid.UUID) (db.Task, error)
- func (q *Queue) Kinds() []string
- func (q *Queue) Register(kind string, k Kind)
- func (q *Queue) Retry(ctx context.Context, id uuid.UUID) (db.Task, error)
- func (q *Queue) Run(ctx context.Context) error
- func (q *Queue) SetPaused(kind string, paused bool) error
- func (q *Queue) State() []KindState
- type RetryAfterError
- type Task
Constants ¶
const ( StateQueued = "queued" StateRunning = "running" StateSucceeded = "succeeded" StateFailed = "failed" StateCancelled = "cancelled" )
Task states.
Variables ¶
var ( ErrNotFound = errors.New("queue: task not found") ErrNotRetrying = errors.New("queue: only failed or cancelled tasks can be retried") ErrFinished = errors.New("queue: the task has finished") ErrUnknownKind = errors.New("queue: unknown task kind") )
Errors callers map to responses.
Functions ¶
func DefaultBackoff ¶
DefaultBackoff waits 30s, 1m, 2m, … up to an hour, with jitter.
Types ¶
type Event ¶
type Event struct {
ID uuid.UUID `json:"id"`
Kind string `json:"kind"`
State string `json:"state"`
Attempts int32 `json:"attempts"`
LastError string `json:"last_error,omitempty"`
Progress *Progress `json:"progress,omitempty"`
}
Event is the data of task events.
type Kind ¶
type Kind struct {
Handler Handler
// Concurrency caps how many of this kind run at once. Default 2.
Concurrency int
// MaxAttempts before a task fails. Default 5.
MaxAttempts int
}
Kind configures one kind of task.
type NewTask ¶
type NewTask struct {
Kind string
Payload any
// DedupeKey keeps one live (queued or running) task per key.
DedupeKey string
// LimitKey caps concurrent tasks per key, e.g. "source:<id>".
LimitKey string
Priority int32
CreatedBy uuid.UUID
// RunAfter delays the task.
RunAfter time.Time
}
NewTask is a task to enqueue.
type Progress ¶
type Progress struct {
Phase string `json:"phase,omitempty"`
Done int64 `json:"done"`
Total int64 `json:"total,omitempty"`
Unit string `json:"unit,omitempty"`
}
Progress is how far a running task is.
type Queue ¶
type Queue struct {
// Workers caps all running tasks. Default 8.
Workers int
// Lease is how long a claimed task stays claimed without a heartbeat.
Lease time.Duration
// Poll is how often the queue looks for due tasks without a wake-up.
Poll time.Duration
// Backoff gives the delay before attempt n+1 after n failed attempts.
Backoff func(attempts int32) time.Duration
// contains filtered or unexported fields
}
Queue runs tasks.
func New ¶
func New(st *store.Store, hub *events.Hub, lim *limits.Registry, m *metrics.Metrics, log *slog.Logger) *Queue
New returns a queue. hub, lim and m may be nil.
func (*Queue) Enqueue ¶
Enqueue adds a task, or returns the live task with the same dedupe key (created is then false).
func (*Queue) EnqueueWith ¶
EnqueueWith enqueues with dbq, so the task can join a transaction.
func (*Queue) Run ¶
Run claims and runs tasks until ctx ends, then waits for running tasks to stop (they see ctx cancelled and go back to the queue).
type RetryAfterError ¶
RetryAfterError asks for the next attempt no sooner than After (e.g. a site rate-limited the plugin).
func (*RetryAfterError) Error ¶
func (e *RetryAfterError) Error() string
func (*RetryAfterError) Unwrap ¶
func (e *RetryAfterError) Unwrap() error
type Task ¶
Task is a running task, handed to its handler.
func (*Task) SetProgress ¶
SetProgress reports how far the task is. Writes and events are throttled.