Documentation
¶
Overview ¶
Package outbox implements a transactional outbox for PostgreSQL.
Messages are written to the outbox table inside the caller's business transaction; a background relay later claims them with FOR UPDATE SKIP LOCKED (leased to an externally supplied instance id) and hands them to a transport-agnostic Publisher. Delivery is at-least-once: consumers must deduplicate.
This file is the package's single public surface: it re-exports the types and constructors from the implementation subpackages (message, port, config, backoff, engine, migrations) and provides the Outbox facade. The database surface is the Executor interface (Exec, Query, QueryRow); plug your own. The logger is the standard library's *slog.Logger, injected via WithLogger.
Index ¶
- Constants
- Variables
- func Migrations() []string
- func MigrationsFS() embed.FS
- type AbortError
- type Backoff
- type Codec
- type Config
- type DiscardError
- type DiscardHooks
- type Executor
- type Hooks
- type Message
- type NoopHooks
- type Option
- type Outbox
- func (o *Outbox) Enqueue(ctx context.Context, m Message) error
- func (o *Outbox) EnqueueBatch(ctx context.Context, ms []Message) error
- func (o *Outbox) EnqueueValue(ctx context.Context, topic, partitionKey, msgType string, v any) error
- func (o *Outbox) Requeue(ctx context.Context, id string) (bool, error)
- func (o *Outbox) Run(ctx context.Context) error
- type PanicError
- type PermanentError
- type Publisher
- type RetryAfterError
- type Settings
- type Status
Constants ¶
const ( StatusPending = message.StatusPending StatusProcessing = message.StatusProcessing StatusPublished = message.StatusPublished StatusDead = message.StatusDead StatusDiscarded = message.StatusDiscarded )
Status values.
const TableName = message.TableName
TableName is the fixed outbox table name (only the schema is configurable).
Variables ¶
var ( WithConfig = config.WithConfig WithInstanceID = config.WithInstanceID WithSchema = config.WithSchema WithPollInterval = config.WithPollInterval WithBatchSize = config.WithBatchSize WithLeaseDuration = config.WithLeaseDuration WithConcurrency = config.WithConcurrency WithMaxAttempts = config.WithMaxAttempts WithRetryBackoff = config.WithRetryBackoff WithRetention = config.WithRetention WithDeadRetention = config.WithDeadRetention WithDiscardRetention = config.WithDiscardRetention WithClearPayloadOnComplete = config.WithClearPayloadOnComplete WithPublishTimeout = config.WithPublishTimeout WithCleanupInterval = config.WithCleanupInterval WithOrdered = config.WithOrdered WithoutNotify = config.WithoutNotify WithListenPool = config.WithListenPool WithFatal = config.WithFatal WithHooks = config.WithHooks WithCodec = config.WithCodec WithLogger = config.WithLogger )
var ( Permanent = port.Permanent RetryAfter = port.RetryAfter Discard = port.Discard Abort = port.Abort ErrLeaseLost = port.ErrLeaseLost )
Publisher outcome controls apply to the whole batch and preserve wrapped errors.
var ( // ErrEmptyTopic is returned when a message has no topic. ErrEmptyTopic = message.ErrEmptyTopic // ErrNilPayload is returned when a message payload is nil. ErrNilPayload = message.ErrNilPayload // ErrInvalidSchema is returned by New when the schema is not a valid identifier. ErrInvalidSchema = config.ErrInvalidSchema // ErrNoCodec is returned by EnqueueValue when no codec was configured. ErrNoCodec = errors.New("outbox: no codec configured (use WithCodec)") // ErrNoExecutor is returned by New when no relay Executor was supplied. ErrNoExecutor = errors.New("outbox: a relay executor is required") )
var DefaultBackoff = backoff.Default
DefaultBackoff is exponential 100ms -> 30s with full jitter.
var ExpBackoff = backoff.Exp
ExpBackoff returns an exponential backoff (base doubled per attempt, capped).
Functions ¶
func Migrations ¶
func Migrations() []string
Migrations returns the contents of every forward-only *.sql file, ordered by filename, ready to pass to Exec for callers applying migrations without a tool.
func MigrationsFS ¶
MigrationsFS returns the embedded migration files for golang-migrate (iofs source), goose, atlas, etc. The SQL is schema-unqualified: set search_path on the migration connection to install into a non-default schema.
Types ¶
type AbortError ¶ added in v1.1.1
type AbortError = port.AbortError
type DiscardError ¶ added in v1.1.0
type DiscardError = port.DiscardError
type DiscardHooks ¶ added in v1.1.0
type DiscardHooks = port.DiscardHooks
type Outbox ¶
type Outbox struct {
// contains filtered or unexported fields
}
Outbox is the public facade: enqueue messages (transactionally) and run the background relay + cleaner.
func New ¶
New builds an Outbox.
- exec is the relay/cleaner Executor: it runs the background claim, mark and cleanup queries and must NOT be bound to a business transaction (pass a *pgxpool.Pool, or your own Executor implementation).
- enq is the enqueue Executor: pass one that resolves the caller's transaction from the context (e.g. pgtx's tx.DB) so inserts join the business transaction.
- pub is the transport publisher.
LISTEN/NOTIFY wake-ups require a *pgxpool.Pool: supply one with WithListenPool, or pass a *pgxpool.Pool as exec and it is detected automatically. Without one the relay falls back to polling.
func (*Outbox) EnqueueBatch ¶
EnqueueBatch writes multiple messages inside the caller's current transaction.
func (*Outbox) EnqueueValue ¶
func (o *Outbox) EnqueueValue(ctx context.Context, topic, partitionKey, msgType string, v any) error
EnqueueValue marshals v with the configured codec and enqueues it. Requires WithCodec.
func (*Outbox) Requeue ¶ added in v1.1.0
Requeue atomically returns a dead row to pending and resets its attempts. It uses the enqueue executor, so authorization and application checks can be performed in the same caller transaction. False means no dead row matched. IDs and creation time are preserved; the next claim gets a fresh lease token.
func (*Outbox) Run ¶
Run starts the relay and cleaner and blocks until ctx is cancelled, then returns ctx.Err(). A fatal error (an Abort outcome, a recovered panic, or an error the WithFatal classifier accepts) stops both and is returned instead; the batch it hit stays claimed and is re-claimed after its lease expires. The lease owner id was resolved at New time (WithInstanceID, or auto-generated).
type PanicError ¶ added in v1.1.1
type PanicError = port.PanicError
type PermanentError ¶ added in v1.1.0
type PermanentError = port.PermanentError
type RetryAfterError ¶ added in v1.1.0
type RetryAfterError = port.RetryAfterError
Directories
¶
| Path | Synopsis |
|---|---|
|
Package backoff provides retry backoff strategies for the outbox relay.
|
Package backoff provides retry backoff strategies for the outbox relay. |
|
Package config holds the outbox runtime settings, the functional options that build them, and the declarative Config struct (mapstructure/validate/default tags) for loading from files or environment.
|
Package config holds the outbox runtime settings, the functional options that build them, and the declarative Config struct (mapstructure/validate/default tags) for loading from files or environment. |
|
contrib
|
|
|
libs/xtrace
module
|
|
|
publishers/kafka
module
|
|
|
publishers/nats
module
|
|
|
publishers/valkey
module
|
|
|
Package engine implements the outbox background machinery: the store (enqueue/claim/mark/cleanup queries) and the relay and cleaner workers.
|
Package engine implements the outbox background machinery: the store (enqueue/claim/mark/cleanup queries) and the relay and cleaner workers. |
|
Package message defines the outbox data model.
|
Package message defines the outbox data model. |
|
Package migrations exposes the embedded outbox SQL migrations.
|
Package migrations exposes the embedded outbox SQL migrations. |
|
Package port defines the pluggable integration interfaces of the outbox: the transport Publisher, the value Codec, observability Hooks, and the database Executor.
|
Package port defines the pluggable integration interfaces of the outbox: the transport Publisher, the value Codec, observability Hooks, and the database Executor. |