queue

package
v0.0.0-...-e3c07e6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 27, 2026 License: Apache-2.0 Imports: 7 Imported by: 0

Documentation

Overview

Package queue provides a multi-backend message queue abstraction: a Consumer interface, a Publisher interface, and per-backend implementations registered at runtime.

Phase 1 ships:

  • Postgres backend (used by the embedded-queue dev path and as the prod fallback when SQS is unavailable)
  • SQLite backend (fcdev embedded mode)
  • SQS backend (production)

Future backends (NATS, AMQP) plug in via the same Register pattern.

Index

Constants

This section is empty.

Variables

View Source
var ErrNotImplemented = errors.New("queue: not implemented for this backend")

ErrNotImplemented is returned when a backend doesn't support an optional method.

View Source
var ErrQueueMissing = errors.New("queue: does not exist")

ErrQueueMissing means the broker has no such queue (yet). It is not a failure: the platform creates a dispatch queue lazily, on the first message published to it, so a queue the router is configured to consume can legitimately not exist. A consumer factory returns it instead of a consumer; a Poll returns it when the queue disappears under a running consumer. The router starts no consumer for the queue, raises no warning, and builds one as soon as the queue appears (Manager.RecheckMissingQueues).

View Source
var ErrStopped = errors.New("queue: consumer stopped")

ErrStopped is returned by a Consumer's Poll once the consumer has been stopped (Stop called). It is terminal for that consumer object — a stopped consumer never resumes — so a poll loop that receives it should exit and let the router's stall watchdog respawn a fresh consumer rather than spin on the dead one. Backends may return it wrapped; callers match with errors.Is.

Functions

func IsUrgentAck

func IsUrgentAck(ctx context.Context) bool

IsUrgentAck reports whether ctx was marked by WithUrgentAck.

func RegisterConsumer

func RegisterConsumer(scheme string, f ConsumerFactory)

RegisterConsumer registers a backend-specific consumer factory. Backends call this in init() so that NewConsumer can route by scheme.

func RegisterPublisher

func RegisterPublisher(scheme string, f PublisherFactory)

RegisterPublisher registers a backend-specific publisher factory.

func WithUrgentAck

func WithUrgentAck(ctx context.Context) context.Context

WithUrgentAck marks an Ack call as urgent: the caller (the router, for an ORDERED message whose group cannot proceed until the ack lands) must not be held back by a backend's ack batching. Backends that do not batch acks ignore it.

Types

type Consumer

type Consumer interface {
	// Identifier returns a stable name for this consumer (typically the queue URL).
	Identifier() string
	// Poll fetches up to maxMessages messages.
	//
	// Every returned QueuedMessage.BrokerMessageID MUST be STABLE across
	// redeliveries of the same message: the router's duplicate filter takes a
	// changed broker id under a known app message id to mean a *different* copy
	// — an external process requeued work the pipeline still owns — and ACKs
	// (deletes) the arrival. A per-delivery id therefore makes the broker
	// destroy its own copy on every redelivery, leaving the in-memory copy as
	// the only one and turning at-least-once into at-most-once. Use the
	// broker's per-message identity (SQS MessageId, the queue row id, the
	// JetStream *stream* sequence), never a per-delivery counter.
	Poll(ctx context.Context, maxMessages uint32) ([]common.QueuedMessage, error)
	// Ack deletes the message from the queue. brokerMessageID is the broker's
	// own id for the message (common.QueuedMessage.BrokerMessageID), passed
	// explicitly because a receipt handle alone can't identify the message
	// across a redelivery: SQS needs the broker id to remember that it deleted
	// this message, and deriving it from the receipt requires state that may
	// have been evicted by then. Empty when the caller genuinely has no id.
	Ack(ctx context.Context, receipt string, brokerMessageID string) error
	// Nack marks the message visible again after delay. Counts as a failure.
	Nack(ctx context.Context, receipt string, delaySeconds *uint32) error
	// Defer marks the message visible again without counting as a failure.
	// Use for backpressure / rate-limiting; default falls back to Nack.
	Defer(ctx context.Context, receipt string, delaySeconds *uint32) error
	// HonoursDelayedReturn reports whether a Nack's delay is actually
	// enforced by this backend before it redelivers — R5 (owner ruling
	// 2026-09-17, docs/spec/router-deferral-handback.md, second unit).
	//
	// SQS and Postgres answer true: a Nack genuinely holds the message
	// invisible for the requested delay (Postgres additionally blocks a
	// nacked-with-delay group head's successors from claiming ahead of it —
	// see its Poll doc comment). NATS answers false: its stream is one
	// durable WorkQueue consumer with no per-group subject, so a Nack never
	// blocks a group's successors the way Postgres's claim does, and each
	// redelivery spends one of the stream's limited MaxDeliver attempts —
	// treating a deferral as a hand-back there would turn a bounded number
	// of in-memory retries into the same number of deliveries and then
	// silent, permanent redelivery loss.
	//
	// Deliberately no default: every backend answers explicitly (Go has no
	// default interface methods to omit it from), which is what the router
	// relies on to decide, per delay-bearing deferral, whether to hand the
	// message back to the broker (R1) or keep retrying it in memory (R5).
	HonoursDelayedReturn() bool
	// Healthy reports liveness.
	Healthy() bool
	// Stop signals the consumer to wind down.
	Stop()
	// Metrics returns the broker-side metrics. May call the broker; nil if unavailable.
	Metrics(ctx context.Context) (*Metrics, error)
	// Counters returns process-local counters (atomic reads). Nil if unsupported.
	Counters() *Metrics
}

Consumer is the trait every queue backend implements for the consume side.

There is deliberately no visibility-extension method. The router does not hold a message's visibility open while it works: it lets the window lapse and recognises the redelivery instead (the in-flight tracker adopts the fresher receipt handle), which is the same path a crashed router relies on. One mechanism, exercised constantly, beats two where the rare one rots — an ExtendVisibility existed on all three backends for some time with no caller.

func NewConsumer

func NewConsumer(ctx context.Context, cfg common.QueueConfig) (Consumer, error)

NewConsumer builds a Consumer for the supplied URI. The URI scheme (e.g. "sqs", "postgres", "sqlite") determines the backend.

type ConsumerFactory

type ConsumerFactory func(ctx context.Context, cfg common.QueueConfig) (Consumer, error)

ConsumerFactory builds a Consumer from a backend-specific URI.

type Embedded

type Embedded interface {
	Consumer
	Publisher
	// InitSchema creates the broker's required tables / topics.
	InitSchema(ctx context.Context) error
}

Embedded combines Consumer + Publisher for in-process queue backends.

type Metrics

type Metrics struct {
	QueueIdentifier  string
	PendingMessages  uint64
	InFlightMessages uint64
	TotalPolled      uint64
	TotalAcked       uint64
	TotalNacked      uint64
	TotalDeferred    uint64
}

Metrics captures queue health snapshot.

type Publisher

type Publisher interface {
	Identifier() string
	Publish(ctx context.Context, m common.Message) (string, error)
	PublishBatch(ctx context.Context, msgs []common.Message) ([]string, error)
}

Publisher is the produce side.

func NewPublisher

func NewPublisher(ctx context.Context, cfg common.QueueConfig) (Publisher, error)

NewPublisher builds a Publisher for the supplied URI.

type PublisherFactory

type PublisherFactory func(ctx context.Context, cfg common.QueueConfig) (Publisher, error)

PublisherFactory builds a Publisher from a backend-specific URI.

Directories

Path Synopsis
Package nats is the NATS JetStream queue backend.
Package nats is the NATS JetStream queue backend.
Package postgres is the Postgres-backed queue backend.
Package postgres is the Postgres-backed queue backend.
Package sqs is the AWS SQS-backed queue backend.
Package sqs is the AWS SQS-backed queue backend.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL