Documentation
¶
Overview ¶
Package queue runs typed background jobs. A job is a struct with a Kind; its handler is a plain function registered explicitly:
type SendInvoice struct{ OrderID int64 }
func (SendInvoice) Kind() string { return "send_invoice" }
q.Register(queue.Handle(func(ctx context.Context, j SendInvoice) error { ... },
queue.Retries(5), queue.Backoff(queue.Exponential(time.Second)), queue.Timeout(time.Minute)))
err := q.Dispatch(ctx, SendInvoice{OrderID: 1}, queue.OnQueue("billing"), queue.Delay(5*time.Minute))
Payloads are JSON. Drivers store messages: Sync runs jobs inline, Memory keeps them in process, the database driver (and Redis) persist them.
Index ¶
- Variables
- func Constant(d time.Duration) func(int) time.Duration
- func Dispatched[J Job](f *Fake) []J
- func Exponential(base time.Duration) func(int) time.Duration
- type Database
- func (d *Database) Ack(ctx context.Context, m *Message) error
- func (d *Database) CreateTables(ctx context.Context) error
- func (d *Database) Fail(ctx context.Context, m *Message) error
- func (d *Database) Failed(ctx context.Context) ([]FailedJob, error)
- func (d *Database) Forget(ctx context.Context, ids ...string) (int, error)
- func (d *Database) Pending(ctx context.Context) (int, error)
- func (d *Database) Pop(ctx context.Context, queues []string) (*Message, error)
- func (d *Database) Push(ctx context.Context, m *Message) error
- func (d *Database) PushTx(ctx context.Context, ex database.Executor, m *Message) error
- func (d *Database) Release(ctx context.Context, m *Message) error
- func (d *Database) Retry(ctx context.Context, ids ...string) (int, error)
- type DatabaseOptions
- type Driver
- type FailedJob
- type FailedStore
- type Fake
- type Inline
- type Job
- type Memory
- func (d *Memory) Ack(_ context.Context, m *Message) error
- func (d *Memory) Fail(_ context.Context, m *Message) error
- func (d *Memory) Failed() []Message
- func (d *Memory) Pending() int
- func (d *Memory) Pop(_ context.Context, queues []string) (*Message, error)
- func (d *Memory) Push(_ context.Context, m *Message) error
- func (d *Memory) Release(_ context.Context, m *Message) error
- type Message
- type Option
- type Queue
- func (q *Queue) Dispatch(ctx context.Context, j Job, opts ...Option) error
- func (q *Queue) DispatchKind(ctx context.Context, kind string, payload any, opts ...Option) error
- func (q *Queue) DispatchTx(ctx context.Context, ex database.Executor, j Job, opts ...Option) error
- func (q *Queue) Driver() Driver
- func (q *Queue) Message(j Job, opts ...Option) (*Message, error)
- func (q *Queue) Process(ctx context.Context, m *Message) error
- func (q *Queue) Register(regs ...Registration) error
- func (q *Queue) Work(ctx context.Context, o WorkOptions) error
- type Registration
- type Sync
- type TxPusher
- type WorkOptions
Constants ¶
This section is empty.
Variables ¶
var ErrDuplicateKind = errors.New("queue: job kind already registered")
ErrDuplicateKind is returned by Register for a kind registered twice.
var ErrNoTx = errors.New("queue: driver does not support DispatchTx")
ErrNoTx is returned by DispatchTx when the driver cannot enqueue inside a transaction.
var ErrUnknownKind = errors.New("queue: no handler for job kind")
ErrUnknownKind is returned by Dispatch for a kind without a handler.
Functions ¶
func Dispatched ¶
Dispatched returns the dispatched jobs of type J, decoded, in order.
Types ¶
type Database ¶
type Database struct {
// contains filtered or unexported fields
}
Database stores jobs in SQL tables (SQLite or Postgres). Workers reserve jobs atomically (Postgres uses FOR UPDATE SKIP LOCKED), so any number of worker processes can share the tables. A job whose worker died is reserved again once its timeout plus a grace period has passed: delivery is at least once, so handlers should be idempotent.
func NewDatabase ¶
func NewDatabase(db *database.DB, o DatabaseOptions) (*Database, error)
NewDatabase returns a database driver; call CreateTables (or a migration with the same schema) before use.
func (*Database) CreateTables ¶
CreateTables creates the jobs and failed jobs tables if missing.
func (*Database) Forget ¶
Forget implements FailedStore: it deletes failed jobs (all when ids is empty).
func (*Database) PushTx ¶
PushTx inserts m with ex, so it is enqueued only if ex's transaction commits (Queue.DispatchTx).
type DatabaseOptions ¶
type DatabaseOptions struct {
// Table holds pending jobs (default "jobs").
Table string
// FailedTable holds failed jobs (default "failed_jobs").
FailedTable string
// Grace is added to a job's timeout before an abandoned reservation is
// taken over (default 1 minute).
Grace time.Duration
// Now is the clock (default time.Now); tests set it.
Now func() time.Time
}
DatabaseOptions name the tables.
type Driver ¶
type Driver interface {
// Push stores m; it becomes available at m.AvailableAt.
Push(ctx context.Context, m *Message) error
// Pop reserves the next available message on one of queues (in
// order), or returns nil when there is none.
Pop(ctx context.Context, queues []string) (*Message, error)
// Ack deletes a processed message.
Ack(ctx context.Context, m *Message) error
// Release makes m available again at m.AvailableAt (a retry).
Release(ctx context.Context, m *Message) error
// Fail moves m to the failed jobs.
Fail(ctx context.Context, m *Message) error
}
Driver stores messages and hands them to workers.
type FailedStore ¶
type FailedStore interface {
// Failed lists failed jobs, newest first.
Failed(ctx context.Context) ([]FailedJob, error)
// Retry re-queues failed jobs (all when ids is empty).
Retry(ctx context.Context, ids ...string) (int, error)
// Forget deletes failed jobs (all when ids is empty).
Forget(ctx context.Context, ids ...string) (int, error)
}
FailedStore is implemented by drivers that keep failed jobs (Database).
type Fake ¶
type Fake struct {
*Memory
// contains filtered or unexported fields
}
Fake records dispatched jobs for tests; it stores them like Memory, so a test can also run them with Queue.Work.
fake := queue.NewFake()
app := vukemtest.New(t, bootstrap.New, vukemtest.Options{AppOptions: []vukem.Option{vukem.WithQueueDriver(fake)}})
... exercise the app ...
invoices := queue.Dispatched[jobs.SendInvoice](fake)
type Inline ¶
type Inline interface {
Inline() bool
}
Inline is implemented by drivers that run jobs during Dispatch (Sync).
type Job ¶
type Job interface {
Kind() string
}
Job is a unit of background work. Kind names its handler and must be stable across deploys (it is stored with the payload).
type Memory ¶
type Memory struct {
// contains filtered or unexported fields
}
Memory keeps messages in process: jobs are lost on restart. Use it for tests and single-process apps that can afford that.
func (*Memory) Pop ¶
Pop implements Driver: the earliest available message of the first queue in queues that has one.
type Message ¶
type Message struct {
ID string `json:"id"`
Kind string `json:"kind"`
Queue string `json:"queue"`
Payload json.RawMessage `json:"payload"`
Attempts int `json:"attempts"`
MaxRetries int `json:"max_retries"`
Timeout time.Duration `json:"timeout"`
AvailableAt time.Time `json:"available_at"`
CreatedAt time.Time `json:"created_at"`
// LastError is set when the job failed.
LastError string `json:"last_error,omitempty"`
}
Message is a stored job.
type Option ¶
type Option func(*options)
Option configures a handler (defaults) or a single dispatch.
type Queue ¶
type Queue struct {
// contains filtered or unexported fields
}
Queue dispatches jobs to a driver and runs them in workers.
func (*Queue) Dispatch ¶
Dispatch enqueues j. Options override the handler's defaults for this dispatch. With an Inline driver the job runs now and its error is returned.
func (*Queue) DispatchKind ¶
DispatchKind enqueues payload (encoded as JSON) for the handler of kind.
func (*Queue) DispatchTx ¶
DispatchTx enqueues j inside the transaction of ex: the job exists only if the transaction commits, so it never runs for rolled-back work and is never lost for committed work.
func (*Queue) Message ¶
Message builds the stored message for j (drivers with their own transactional push use it).
func (*Queue) Process ¶
Process runs one reserved message and acks, releases (retry with backoff) or fails it.
func (*Queue) Register ¶
func (q *Queue) Register(regs ...Registration) error
Register adds handlers; a kind may be registered once.
type Registration ¶
type Registration struct {
// contains filtered or unexported fields
}
Registration binds a handler to its job kind; create it with Handle.
func HandleKind ¶
func HandleKind[T any](kind string, fn func(ctx context.Context, v T) error, opts ...Option) Registration
HandleKind registers fn for jobs named kind whose payload decodes into T (framework jobs such as queued mail and listeners; DispatchKind sends them).
func (Registration) Kind ¶
func (r Registration) Kind() string
Kind returns the job kind of the registration.
type Sync ¶
type Sync struct{}
Sync runs jobs inline during Dispatch (development, tests, simple apps). Delays and retries do not apply: the handler's error goes to the caller.
type TxPusher ¶
TxPusher is implemented by drivers that can enqueue inside a database transaction (Database).
type WorkOptions ¶
type WorkOptions struct {
// Queues are polled in priority order (default ["default"]).
Queues []string
// Concurrency is the number of parallel workers (default 1).
Concurrency int
// Sleep is the wait when no job is available (default 1s).
Sleep time.Duration
// MaxJobs stops after processing this many jobs (0 = no limit), e.g.
// for "queue:work --once".
MaxJobs int
// StopWhenEmpty returns when no job is available.
StopWhenEmpty bool
}
WorkOptions configure Work.