worker

package
v0.23.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 3, 2026 License: MIT Imports: 15 Imported by: 0

Documentation

Overview

Package worker provides background job processing for vogel-based applications, built on River (github.com/riverqueue/river) backed by PostgreSQL.

The Queue interface abstracts the job queue for the application layer, and RiverQueue (see river.go) is the River-backed implementation this package ships. Both live in the same package on purpose: a consumer defining River jobs already imports river directly (river.WorkerDefaults[T], river.Job[T]), so hiding River behind a separate port would be ceremony without benefit. This package depends on River openly, and that's fine.

Usage from the application layer:

// Enqueue a job within the same transaction as a business mutation:
txManager.WithTx(ctx, func(txCtx context.Context) error {
    repo.Create(txCtx, entity)
    tx := repository.TxFromContext(txCtx)
    _, err := queue.EnqueueTx(txCtx, tx, MyJobArgs{ID: entity.ID}, nil)
    return err
})

// Enqueue a job without a transaction:
queue.Enqueue(ctx, MyJobArgs{ID: "123"}, nil)

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func EnsureSchema

func EnsureSchema(ctx context.Context, pool *pgxpool.Pool, schema string) error

EnsureSchema creates the PostgreSQL schema if it does not exist. Idempotent and safe to call on every worker startup. River creates its tables INSIDE the configured schema but does not create the schema itself, so this must run before RiverQueue.Migrate on a freshly provisioned database.

Types

type Config

type Config struct {
	// Schema is the PostgreSQL schema River's tables live in, and that
	// Migrate creates tables inside of. Must be a valid, unquoted PostgreSQL
	// identifier (see isValidSchemaName in schema.go). Typical default:
	// "river".
	Schema string

	// DefaultMaxWorkers is the max concurrent workers for the default River
	// queue, used when Queues is empty. Typical default: 100.
	DefaultMaxWorkers int

	// Queues configures named queues and their max concurrent workers. If
	// empty, a single default queue (river.QueueDefault) is used with
	// DefaultMaxWorkers.
	Queues map[string]int

	// PollOnly makes a worker-mode client find new jobs by polling instead
	// of waiting on PostgreSQL LISTEN/NOTIFY; it is passed to River's
	// river.Config.PollOnly. LISTEN needs a session that stays on one server
	// connection, so set it when the worker connects through PgBouncer in
	// transaction pooling mode. The cost is latency: a new job may wait up
	// to River's FetchPollInterval (1s by default) before it is locked for
	// work, and a new leader takes up to five seconds to elect itself after
	// the previous one resigns. Ignored in insert-only mode, which never
	// listens. Default false: LISTEN/NOTIFY, as before.
	PollOnly bool
}

Config configures a RiverQueue.

Applying defaults is the caller's responsibility: this package does not read environment variables or fill in zero values itself. Both applications this package was extracted from (go-crucible, go-licencias) parse an equivalent config with Schema defaulting to "river" and DefaultMaxWorkers defaulting to 100 — mirror those defaults in your own application config if you want the same behavior.

type Option

type Option func(*options)

Option configures optional NewRiverQueue behavior.

func WithMetrics added in v0.11.0

func WithMetrics(reg prometheus.Registerer) Option

WithMetrics records the duration of every job attempt the RiverQueue works as the Prometheus histogram river_job_duration_seconds{kind,outcome}, registered on reg (prometheus.DefaultRegisterer if nil). outcome is success, error, cancel (river.JobCancel), snooze (river.JobSnooze) or panic; kind is the job's Kind(). The histogram's _count doubles as the attempt counter by outcome. Registering twice on the same Registerer reuses the collector.

The metrics are a worker middleware, installed after the request ID one, so they only exist in worker-binary mode: with a nil *river.Workers the option is ignored and nothing is registered, as with WithPeriodicJobs. Without this option no metrics middleware is installed.

func WithMiddleware added in v0.12.0

func WithMiddleware(mw ...rivertype.Middleware) Option

WithMiddleware adds River middlewares (job insert middlewares, worker middlewares, or both) to the RiverQueue, after the request ID middleware and before the WithMetrics one, in the order given; calls accumulate. River runs the first middleware outermost, so these see the job's request ID in the context, and a span they open is in the context the metrics middleware observes. Unlike WithMetrics they are installed in insert-only mode too: an insert middleware is how the enqueuing process stamps its context (a trace, say) into the job.

func WithPeriodicJobs

func WithPeriodicJobs(jobs []*river.PeriodicJob) Option

WithPeriodicJobs registers River periodic jobs on the RiverQueue created by NewRiverQueue.

Periodic jobs are only ever scheduled in worker-binary mode (workers != nil passed to NewRiverQueue): an insert-only client processes nothing, so periodic jobs passed alongside a nil *river.Workers are silently not registered. Omitting this option entirely — the go-licencias call shape — is equivalent to passing WithPeriodicJobs(nil): no periodic jobs are registered, same as before this option existed.

type Queue

type Queue interface {
	// Enqueue adds a job to the queue outside of a transaction.
	Enqueue(ctx context.Context, args river.JobArgs, opts *river.InsertOpts) (*rivertype.JobInsertResult, error)

	// EnqueueTx adds a job within an existing database transaction. The job
	// is only visible after the transaction commits (Transactional Outbox).
	EnqueueTx(ctx context.Context, tx pgx.Tx, args river.JobArgs, opts *river.InsertOpts) (*rivertype.JobInsertResult, error)

	// Start begins processing jobs. Blocks until Stop is called or the
	// context is canceled. Only the worker binary should call Start — the
	// API process never does.
	Start(ctx context.Context) error

	// Stop initiates graceful shutdown. Running jobs are allowed to
	// complete.
	Stop(ctx context.Context) error
}

Queue abstracts the job queue system for background processing.

Two modes of operation:

  • Insert-only: the API process calls Enqueue/EnqueueTx but never Start.
  • Full worker: the worker binary calls Start to begin processing jobs.

type RiverQueue

type RiverQueue struct {
	// contains filtered or unexported fields
}

RiverQueue implements Queue using River over PostgreSQL. River uses PostgreSQL as its job store via SELECT FOR UPDATE SKIP LOCKED, providing transactional job enqueue (Outbox Pattern) with zero extra infrastructure.

func NewRiverQueue

func NewRiverQueue(pool *pgxpool.Pool, workers *river.Workers, cfg Config, logger *slog.Logger, opts ...Option) (*RiverQueue, error)

NewRiverQueue creates a River-backed queue.

If workers is nil, the client operates in insert-only mode (API process). If workers is provided, the client can be started to process jobs (worker binary). Pass WithPeriodicJobs to register periodic jobs and WithMetrics to export job metrics — see their doc comments for how they interact with insert-only mode.

func (*RiverQueue) Enqueue

Enqueue adds a job to the queue outside of a transaction.

func (*RiverQueue) EnqueueTx

func (q *RiverQueue) EnqueueTx(ctx context.Context, tx pgx.Tx, args river.JobArgs, opts *river.InsertOpts) (*rivertype.JobInsertResult, error)

EnqueueTx adds a job within an existing database transaction. The job is only visible after the transaction commits.

func (*RiverQueue) Migrate

func (q *RiverQueue) Migrate(ctx context.Context) error

Migrate runs River's schema migrations. Call this before Start in the worker binary. The target schema must already exist — see EnsureSchema.

func (*RiverQueue) Start

func (q *RiverQueue) Start(ctx context.Context) error

Start begins processing jobs. Blocks by waiting on the River client.

func (*RiverQueue) Stop

func (q *RiverQueue) Stop(ctx context.Context) error

Stop initiates graceful shutdown.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL