taskqueue

Overview ¶

Package taskqueue provides a Redis-backed distributed task queue with at-least-once delivery semantics.

The queue maintains an in-flight processing set to track tasks currently being executed. A configurable recovery mechanism periodically moves stale tasks from the processing set back to the main queue, ensuring no task is lost if a worker crashes. The queue supports multiple concurrent workers.

Basic usage:

executor := &MyExecutor{}
queue, err := taskqueue.New(
    redisClient,
    "tasks:myapp",
    executor,
    taskqueue.WithWorkerCount(5),
)
if err != nil {
    return err
}

queue.Push(ctx, taskqueue.Task{ID: "task-1", Payload: []byte(`{"key":"value"}`)})
queue.Start(ctx)

Tasks are stored in Redis as a list (main queue) and a sorted set (processing set with timestamps). Worker failures are detected via a configurable timeout on the processing set entries.