outbox

package
v0.62.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 2, 2026 License: MIT Imports: 24 Imported by: 0

Documentation

Overview

Package outbox provides a transactional outbox pattern for reliable event publishing.

The transactional outbox solves the dual-write problem in microservices: events are written to an outbox table in the SAME database transaction as business data, then reliably delivered to the message broker by a background relay.

This guarantees at-least-once delivery: events are never lost even if the broker is temporarily unavailable. Consumers MUST be idempotent — use the outbox.HeaderEventID ("x-outbox-event-id") header for deduplication, via outbox.EventIDFromHeaders.

Usage:

func (m *Module) Init(deps *app.ModuleDeps) error {
    m.outbox = deps.Outbox
    return nil
}

func (s *Service) CreateOrder(ctx context.Context, order Order) error {
    tx, err := db.Begin(ctx)
    if err != nil { return err }
    defer tx.Rollback(ctx)

    tx.Exec(ctx, "INSERT INTO orders ...", args...)
    s.outbox.Publish(ctx, tx, &app.OutboxEvent{
        EventType:   "order.created",
        AggregateID: "order-123",
        Payload:     payload,
        Exchange:    "order.events",
    })
    return tx.Commit(ctx)
}

Index

Constants

View Source
const (
	// HeaderEventID carries the unique outbox event id (a UUID) used for
	// consumer-side deduplication.
	HeaderEventID = "x-outbox-event-id"

	// HeaderEventType carries the event type of the published outbox record.
	HeaderEventType = "x-outbox-event-type"
)

AMQP delivery header names stamped by the relay for consumer idempotency. These constants are the single source of truth; the relay references them when publishing, so consumers can dedupe without re-declaring the literals.

View Source
const (
	LaneAMQP   = "amqp"
	LaneStream = "stream"
)

Lane constants name the transport a row is dispatched on.

View Source
const (
	StatusPending   = "pending"
	StatusPublished = "published"
	StatusFailed    = "failed"
)

Event status constants.

View Source
const DefaultTableName = "gobricks_outbox"

DefaultTableName is the default outbox table name.

Variables

View Source
var (
	// ErrStreamTargetRequiresTenant is returned when an event targets a super stream
	// but the context carries no tenant to take the partition key from.
	ErrStreamTargetRequiresTenant = errors.New("outbox: a stream target takes its partition key from the context tenant, and the context carries none")

	// ErrConflictingTargets is returned when an event names both a stream and an
	// exchange or routing key.
	ErrConflictingTargets = errors.New("outbox: an event targets either an exchange or a stream; a stream target takes no exchange or routing key")

	// ErrStreamNotAnOutboxTarget is returned when an event names a stream the relay
	// was not configured to publish to.
	ErrStreamNotAnOutboxTarget = errors.New("outbox: stream is not listed in outbox.superstreams")

	// ErrNotLeader is returned by Store.Lead when another relay instance holds the
	// ledger's leader row.
	ErrNotLeader = errors.New("outbox: another relay instance leads this ledger")
)

Functions

func EventIDFromHeaders added in v0.40.0

func EventIDFromHeaders(h amqp.Table) (string, bool)

EventIDFromHeaders extracts the outbox event id from AMQP delivery headers, returning ok=false when the header is absent, empty, or not a string/[]byte. AMQP header values can arrive as either string or []byte depending on the broker and client, so both are normalized.

Types

type Cleanup

type Cleanup struct {
	// contains filtered or unexported fields
}

Cleanup is a scheduler.Executor that removes published events older than the configured retention period.

Runs daily at 04:00 by default (registered via scheduler.DailyAt). Like the relay, it resolves the database through the tenant-aware getDB resolver and fans out across the configured tenants in multi-tenant mode (shared with the inbox cleanup via multitenant.FanOutRetentionCleanup).

func (*Cleanup) Execute

func (c *Cleanup) Execute(jobCtx scheduler.JobContext) error

type Leadership added in v0.61.0

type Leadership interface {
	// Probe fails once the claim is gone (statement timeout, recycled connection,
	// partition). The caller must stop draining on the first failed probe.
	Probe(ctx context.Context) error

	// Release gives up the claim. It is safe to defer.
	Release(ctx context.Context) error
}

Leadership is a held claim on a ledger's leader row. Probe reports whether the claim still stands; Release gives it up.

type Module

type Module struct {
	// contains filtered or unexported fields
}

Module implements the GoBricks Module interface for transactional outbox. It provides reliable event publishing by writing events to a database table within the caller's transaction, then publishing them to the message broker via a background relay job.

The module is registered like any other GoBricks module:

for _, m := range []app.Module{
    scheduler.NewModule(), // Required: relay runs as a scheduled job
    outbox.NewModule(),    // Outbox module
    &myapp.OrderModule{},
} {
    if err := fw.RegisterModule(m); err != nil {
        log.Fatal(err)
    }
}

func NewModule

func NewModule() *Module

NewModule creates a new Module instance.

func (*Module) DeclareStreams added in v0.61.0

func (m *Module) DeclareStreams(decls *streams.Declarations)

DeclareStreams registers one publisher per configured super stream, satisfying streams.StreamDeclarer. It is a no-op for a disabled outbox or one with no targets, so a deployment that never mentions superstreams declares nothing.

func (*Module) Init

func (m *Module) Init(deps *app.ModuleDeps) error

Init implements app.Module. Stores dependencies and initializes the publisher; the vendor-specific store is created lazily on first use (see ensureStoreInitialized).

func (*Module) Name

func (m *Module) Name() string

Name implements app.Module.

func (*Module) OutboxPublisher

func (m *Module) OutboxPublisher() app.OutboxPublisher

OutboxPublisher implements app.OutboxProvider — returns the Publisher for ModuleDeps wiring.

func (*Module) RegisterJobs

func (m *Module) RegisterJobs(registrar app.JobRegistrar) error

RegisterJobs implements app.JobProvider. Registers the relay and cleanup jobs with the scheduler.

func (*Module) SetSharedResolvers added in v0.54.0

func (m *Module) SetSharedResolvers(
	db func(context.Context) (dbtypes.Interface, error),
	msg func(context.Context) (messaging.AMQPClient, error),
)

SetSharedResolvers injects the control-plane ("" key) resolvers. Called by app.RegisterModule; used only when outbox.tenancy=shared.

func (*Module) Shutdown

func (m *Module) Shutdown() error

Shutdown implements app.Module.

type Record

type Record struct {
	ID           string     // UUID, generated on insert
	EventType    string     // Event type for routing
	AggregateID  string     // Aggregate identifier for correlation
	Payload      []byte     // Event payload; JSON-encoded unless the caller supplied []byte, which is stored as-is
	Headers      []byte     // JSON-encoded AMQP headers (nullable)
	Exchange     string     // Target AMQP exchange
	RoutingKey   string     // AMQP routing key
	Lane         string     // LaneAMQP or LaneStream; the store fills an empty lane with LaneAMQP
	Stream       string     // Stream-lane target super stream (empty on the AMQP lane)
	PartitionKey string     // Stream-lane partition key: the row's tenant stamp
	Seq          int64      // Per-ledger sequence assigned by the database at insert; zero before insert, never written by Insert
	Status       string     // "pending", "published", or "failed"
	RetryCount   int        // Number of failed publish attempts so far (not incremented on eventual success)
	Error        string     // Last recorded failure message; NOT cleared on a later successful publish
	CreatedAt    time.Time  // When the event was created
	PublishedAt  *time.Time // When the event was successfully published (nil if pending)
}

Record represents a single row in the outbox table. Records are created by Publisher.Publish() and consumed by the relay job.

type Relay

type Relay struct {
	// contains filtered or unexported fields
}

Relay is a scheduler.Executor that polls for pending outbox events and publishes them to the message broker via existing AMQP infrastructure.

The relay runs as a scheduled job (registered via scheduler.FixedRate), getting overlapping prevention, panic recovery, and OTel metrics for free.

Resources are resolved through the tenant-aware getDB/getMessaging resolvers (the module's deps.DB/deps.Messaging) rather than the scheduler JobContext, because the scheduler builds the JobContext from a tenant-less context — so in multi-tenant mode the relay must inject each tenant into the context itself before resolving that tenant's database and broker.

func (*Relay) Execute

func (r *Relay) Execute(jobCtx scheduler.JobContext) error

Execute runs one relay cycle per configured tenant. In single-tenant mode this is a single pass with no tenant in context; in static multi-tenant mode it fans out across the configured tenants, resolving each tenant's database and broker independently. Per-tenant failures are collected so one unhealthy tenant does not block the others.

type Store

type Store interface {
	// Insert writes an event row to the outbox table within the given transaction.
	Insert(ctx context.Context, tx dbtypes.Tx, record *Record) error

	// FetchPending retrieves up to batchSize pending events in ledger sequence order.
	// Selection is status-gated only: parking is driven by the "failed" status
	// (set by MarkDeadLettered), NOT by retry_count, so an outage-inflated count can
	// never freeze a healthy pending event.
	FetchPending(ctx context.Context, db dbtypes.Interface, batchSize int) ([]Record, error)

	// MarkPublished updates the event status to published with a timestamp.
	MarkPublished(ctx context.Context, db dbtypes.Interface, eventID string) error

	// MarkFailed increments retry count and records the error, leaving the event
	// "pending" so the relay retries it on a later cycle.
	MarkFailed(ctx context.Context, db dbtypes.Interface, eventID, errMsg string) error

	// MarkDeadLettered increments retry count, records the error, and sets the event
	// status to "failed" — a terminal state the relay stops retrying. Used ONLY for
	// poison events (undecodable headers, or a destination the AMQP frame can never carry)
	// that exhaust MaxRetries. Connectivity failures
	// (broker down, NACK, confirmation timeout) must never call this — they advance
	// retry_count via MarkFailed and keep retrying indefinitely.
	MarkDeadLettered(ctx context.Context, db dbtypes.Interface, eventID, errMsg string) error

	// DeletePublished removes events that were published before the given time.
	// Returns the number of rows deleted.
	DeletePublished(ctx context.Context, db dbtypes.Interface, before time.Time) (int64, error)

	// Lead takes the ledger's leader row FOR UPDATE NOWAIT in a transaction it holds
	// until Release. ErrNotLeader when another instance holds it. Probe fails once the
	// transaction is gone (timeout, recycled connection, partition), and the caller
	// must stop draining on the first failed probe.
	Lead(ctx context.Context, db dbtypes.Interface) (Leadership, error)

	// CreateTable creates the outbox table, its indexes, and the companion leader
	// table with its single row, if they do not exist.
	// Used for auto-migration when outbox.autocreatetable is true.
	CreateTable(ctx context.Context, db dbtypes.Interface) error
}

Store abstracts outbox table operations for vendor-agnostic SQL. Implementations exist for PostgreSQL and Oracle with vendor-specific placeholder styles and DDL.

func NewOracleStore

func NewOracleStore(tableName string) (Store, error)

NewOracleStore creates a new Oracle outbox store. Returns an error if the table name contains invalid identifier characters.

func NewPostgresStore

func NewPostgresStore(tableName string) (Store, error)

NewPostgresStore creates a new PostgreSQL outbox store. Returns an error if the table name contains invalid identifier characters.

Directories

Path Synopsis
Package testing provides test utilities for the transactional outbox pattern.
Package testing provides test utilities for the transactional outbox pattern.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL