resque

package module
v0.0.0-...-20993ee Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 17, 2026 License: BSD-3-Clause Imports: 9 Imported by: 0

README

go-ruby-resque/resque

resque — go-ruby-resque

Docs License Go Coverage

A pure-Go (no cgo) reimplementation of the queue and job model of the Ruby resque background-job library, backed by Redis through the go-redis client. It writes and reads the exact keys and JSON payloads that a real Resque — and a Ruby MRI worker — uses, so a job enqueued from Go can be reserved by a Ruby Resque worker and vice-versa.

It is the Resque backend for go-embedded-ruby, but is a standalone, reusable module — a sibling of go-ruby-redis and go-ruby-set.

What it is — and isn't. The wire format, key layout and worker bookkeeping are fully deterministic and live here as pure Go. The two things that are genuinely Ruby — the body of a job (Job#perform) and the class→queue mapping a job class derives — are injected seams, mirroring the go-ruby-* pattern. The host (a Ruby VM, or your Go code) owns those; everything else is byte-for-byte Resque.

Byte-compatible with Resque

A job is the JSON object Resque RPUSHes, with no HTML escaping and no trailing newline:

{"class":"SendEmail","args":[42,"a<b>&c",[1,2]]}
Concern Redis key Value
queue registry resque:queues (set) queue names
a queue resque:queue:<name> (list) job payloads
failures resque:failed (list) Resque failure hash
worker registry resque:workers (set) worker ids <host>:<pid>:<queues>
a busy worker resque:worker:<id> {"queue":…,"run_at":…,"payload":…}
worker start time resque:worker:<id>:started Time#to_s
counters resque:stat:processed / :failed integers

The failure record uses the Resque::Failure field order verbatim:

{"failed_at":"2026/07/06 12:00:00 UTC","payload":{"class":"C","args":[]},
 "exception":"RuntimeError","error":"kaboom","backtrace":[],"worker":"","queue":"q"}

Features

  • Enqueue / dequeueEnqueue(class, args…) (queue resolved from the class via the injected resolver), EnqueueTo(queue, class, args…), Dequeue and DestroyFrom (remove by class, or by exact class+args payload).
  • Queue introspectionSize, Peek(queue, start, count), Pop, Queues, Workers.
  • JobsReserve(queues…) (LPOP the first non-empty queue + decode), Job.Perform() via the injectable PerformFunc seam, and Job.Fail(err, worker) which writes the Resque failure hash. A *JobError carries the true Ruby exception class, message and backtrace; a plain error is recorded as RuntimeError.
  • Worker — a synchronous, goroutine-free Worker: Register / Unregister, WorkingOn, DoneWorking, WorkOne, and a Work loop that drains its queues one job at a time (interval-zero) and keeps every Resque counter and registry key faithful. Failed jobs still count as processed, just as in Resque's ensure-block done_working.
  • Deterministic — injectable clock (WithClock) and worker identity; no sleeps and no goroutines, so behaviour is fully reproducible.
  • CGO-free and validated on all six supported 64-bit targets (amd64, arm64, riscv64, loong64, ppc64le, s390x) across Linux, macOS and Windows.

Usage

package main

import (
	"fmt"

	"github.com/go-ruby-resque/resque"
	"github.com/redis/go-redis/v9"
)

func main() {
	rdb := redis.NewClient(&redis.Options{Addr: "localhost:6379"})
	defer rdb.Close()

	r := resque.New(rdb,
		// The class→queue mapping is Ruby's job — inject it.
		resque.WithQueueResolver(func(class string) (string, error) {
			return "default", nil
		}),
		// The job body is Ruby — inject it.
		resque.WithPerform(func(class string, args []any) error {
			fmt.Printf("running %s%v\n", class, args)
			return nil
		}),
	)

	// Enqueue a job, byte-compatible with a Ruby Resque producer.
	_ = r.Enqueue("SendEmail", 42, "hello")

	// A worker drains the queue.
	w := r.NewWorker(resque.WorkerConfig{
		Hostname: "host", PID: 1234, Queues: []string{"default"},
	})
	processed, _ := w.Work()
	fmt.Println("processed:", processed)
}

Tests & coverage

The tests run against an in-process miniredis started per test (torn down via t.Cleanup), so no external Redis server is needed and nothing leaks between tests. miniredis is a test-only dependency: the runtime import graph pulls in the redis client only.

go test -race ./...            # race-clean
# 100% line coverage, enforced in CI:
COVERPKG=$(go list ./... | paste -sd, -)
go test -race -coverpkg="$COVERPKG" -coverprofile=cover.out ./...
go tool cover -func=cover.out | tail -1   # total: 100.0%

License

BSD-3-Clause — see LICENSE. Copyright the go-ruby-resque/resque authors.

WebAssembly

Being pure Go (CGO=0), this library also compiles to WebAssembly — both GOOS=js GOARCH=wasm (browser / Node.js) and GOOS=wasip1 GOARCH=wasm (WASI). CI builds both targets on every push, alongside the six 64-bit native/qemu arches.

GOOS=js     GOARCH=wasm go build ./...   # browser / Node
GOOS=wasip1 GOARCH=wasm go build ./...   # WASI (wasmtime, wasmer, wasmedge, …)

Documentation

Overview

Package resque is a pure-Go (CGO=0) reimplementation of the queue and job model of the Ruby Resque background-job library, backed by Redis through the github.com/redis/go-redis/v9 client.

It writes and reads exactly the keys and JSON payloads that a real Resque (and a Ruby MRI worker) uses, so a job enqueued here can be reserved by a Ruby Resque worker and vice-versa:

  • a job is the JSON object {"class":"<Name>","args":[...]} RPUSH'd to the list resque:queue:<name>, and the queue name is SADD'd to resque:queues;
  • a worker registers under resque:workers, records the job it is running under resque:worker:<id>, tracks resque:stat:processed / resque:stat:failed, and writes the Resque failure hash to the resque:failed list.

The actual body of a job — Resque's Job#perform — is Ruby, so it is an injected seam (WithPerform); likewise the class→queue mapping that Ruby derives from a job class (WithQueueResolver) and the wall clock (WithClock). Everything else — the wire format, key layout and worker bookkeeping — is deterministic Go with no goroutines and no sleeps.

It is the Resque backend for go-embedded-ruby, a sibling of go-ruby-redis and go-ruby-set, but is a standalone, reusable module.

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrNoQueueResolver is returned by [Resque.Enqueue] / [Resque.Dequeue]
	// when no [WithQueueResolver] seam has been configured, mirroring the fact
	// that Ruby derives the queue from the job class.
	ErrNoQueueResolver = errors.New("resque: no queue resolver configured")
	// ErrNoPerform is returned by [Job.Perform] when no [WithPerform] seam has
	// been configured (the job body is Ruby).
	ErrNoPerform = errors.New("resque: no perform seam configured")
)

Functions

This section is empty.

Types

type Clock

type Clock func() time.Time

Clock returns the current time; it is injectable so worker and failure bookkeeping is deterministic in tests.

type Job

type Job struct {
	// Queue is the queue the job was taken from.
	Queue string
	// Class is the Ruby job class name.
	Class string
	// Args are the decoded arguments (JSON numbers stay as json.Number).
	Args []any
	// contains filtered or unexported fields
}

Job is a reserved unit of work: the decoded {"class":...,"args":[...]} payload plus the queue it came from.

func (*Job) Fail

func (j *Job) Fail(cause error, w *Worker) error

Fail records a job failure in the resque:failed list using the Resque failure hash format. The worker may be nil (its id is recorded as the empty string).

func (*Job) Perform

func (j *Job) Perform() error

Perform runs the job body via the configured WithPerform seam. It returns ErrNoPerform if no seam is set, otherwise the error the body raised.

type JobError

type JobError struct {
	// Exception is the Ruby exception class name (e.g. "ArgumentError").
	Exception string
	// Message is the exception message (exception.to_s).
	Message string
	// Backtrace is the Ruby backtrace, innermost frame first.
	Backtrace []string
}

JobError describes a Ruby exception raised by a job body. A PerformFunc may return one so that Job.Fail records the true Ruby exception class, message and backtrace; a plain error is recorded with the [defaultException] class.

func (*JobError) Error

func (e *JobError) Error() string

Error implements error.

type Option

type Option func(*Resque)

Option configures a Resque in New.

func WithClock

func WithClock(c Clock) Option

WithClock injects the wall clock used for worker and failure timestamps.

func WithContext

func WithContext(ctx context.Context) Option

WithContext sets the context passed to every Redis command.

func WithNamespace

func WithNamespace(ns string) Option

WithNamespace overrides the Redis key prefix (default "resque").

func WithPerform

func WithPerform(f PerformFunc) Option

WithPerform injects the job-body seam (Resque's Job#perform).

func WithQueueResolver

func WithQueueResolver(q QueueResolver) Option

WithQueueResolver injects the class→queue mapping used by Resque.Enqueue.

type PerformFunc

type PerformFunc func(class string, args []any) error

PerformFunc is the injectable seam for a job body (Resque's Job#perform). It receives the decoded class name and arguments and returns the error the Ruby body raised, if any.

type QueueResolver

type QueueResolver func(class string) (string, error)

QueueResolver maps a job class to its queue, mirroring the Ruby idiom where a job class responds to `queue`. It is consulted by Resque.Enqueue and Resque.Dequeue.

type Resque

type Resque struct {
	// contains filtered or unexported fields
}

Resque is a handle onto a Redis-backed Resque instance. It is safe to share a single value; it holds no mutable state of its own.

func New

func New(rdb *redis.Client, opts ...Option) *Resque

New wraps a go-redis client in a Resque handle. The client is owned by the caller (including Close).

func (*Resque) Dequeue

func (r *Resque) Dequeue(class string, args ...any) (int64, error)

Dequeue removes matching jobs from the queue derived from class, mirroring Resque.dequeue(klass, *args). It returns the number of jobs removed.

func (*Resque) DestroyFrom

func (r *Resque) DestroyFrom(queue, class string, args ...any) (int64, error)

DestroyFrom removes matching jobs from the named queue, mirroring Resque::Job.destroy(queue, klass, *args). With no args every job of the class is removed; with args only jobs whose payload equals the encoded class+args are removed. It returns the number of jobs removed.

func (*Resque) Enqueue

func (r *Resque) Enqueue(class string, args ...any) error

Enqueue pushes a job onto the queue derived from class via the configured WithQueueResolver, mirroring Resque.enqueue(klass, *args).

func (*Resque) EnqueueTo

func (r *Resque) EnqueueTo(queue, class string, args ...any) error

EnqueueTo pushes a job onto the named queue, mirroring Resque.enqueue_to(queue, klass, *args). It SADDs the queue to resque:queues and RPUSHes the {"class":...,"args":[...]} payload to resque:queue:<queue>.

func (*Resque) FailedCount

func (r *Resque) FailedCount() (int64, error)

FailedCount returns the length of the resque:failed list (Resque::Failure.count).

func (*Resque) FailedStat

func (r *Resque) FailedStat() (int64, error)

FailedStat returns the failed-jobs counter (Resque.info[:failed]).

func (*Resque) NewWorker

func (r *Resque) NewWorker(cfg WorkerConfig) *Worker

NewWorker builds a worker bound to this handle.

func (*Resque) Peek

func (r *Resque) Peek(queue string, start, count int64) ([]*Job, error)

Peek returns up to count jobs starting at index start without removing them (Resque.peek).

func (*Resque) Pop

func (r *Resque) Pop(queue string) (*Job, error)

Pop removes and returns the next job on a queue, or nil if it is empty (Resque.pop).

func (*Resque) Processed

func (r *Resque) Processed() (int64, error)

Processed returns the number of jobs processed (Resque.info[:processed]).

func (*Resque) Queues

func (r *Resque) Queues() ([]string, error)

Queues returns the registered queue names, sorted for determinism (Resque.queues).

func (*Resque) Reserve

func (r *Resque) Reserve(queues ...string) (*Job, error)

Reserve LPOPs the next job from the first non-empty queue, decodes it, and returns it ready to perform. A nil job with a nil error means every queue was empty (Resque::Job.reserve over a worker's queue list).

func (*Resque) Size

func (r *Resque) Size(queue string) (int64, error)

Size returns the number of jobs on a queue (Resque.size).

func (*Resque) Stat

func (r *Resque) Stat(name string) (int64, error)

Stat returns a Resque counter (resque:stat:<name>), 0 when unset.

func (*Resque) Workers

func (r *Resque) Workers() ([]string, error)

Workers returns the registered worker ids, sorted for determinism (Resque.workers).

type Worker

type Worker struct {
	Hostname string
	PID      int
	Queues   []string
	// contains filtered or unexported fields
}

Worker models a Resque worker: it reserves jobs from an ordered list of queues, performs them and keeps the Resque registry and stats up to date. It runs synchronously — no goroutines, no sleeps — so its behaviour is fully deterministic.

func (*Worker) DoneWorking

func (w *Worker) DoneWorking() error

DoneWorking clears the busy marker and bumps processed counters (Resque::Worker#done_working + #processed!).

func (*Worker) ID

func (w *Worker) ID() string

ID returns the Resque worker id "<hostname>:<pid>:<queues>".

func (*Worker) Register

func (w *Worker) Register() error

Register adds the worker to resque:workers and records its start time (Resque::Worker#register_worker + #started!).

func (*Worker) Unregister

func (w *Worker) Unregister() error

Unregister removes the worker and its per-worker bookkeeping (Resque::Worker#unregister_worker).

func (*Worker) Work

func (w *Worker) Work() (int, error)

Work registers the worker, drains its queues one job at a time until they are empty (Resque's interval-zero work loop), then unregisters. It returns the number of jobs processed. Failed jobs still count as processed, matching Resque, where done_working runs in an ensure block.

func (*Worker) WorkOne

func (w *Worker) WorkOne() (bool, error)

WorkOne reserves and processes a single job. It reports whether a job was found; a false with a nil error means every queue was empty.

func (*Worker) WorkingOn

func (w *Worker) WorkingOn(j *Job) error

WorkingOn marks the worker as busy on job (Resque::Worker#working_on).

type WorkerConfig

type WorkerConfig struct {
	Hostname string
	PID      int
	Queues   []string
}

WorkerConfig identifies a worker. Hostname and PID are seams (Ruby reads them from the process) so worker ids are deterministic in tests.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL