relay

Open Outbox Relay

OpenOutbox Relay is the official reference implementation of the Open Outbox Specification.

It is a high-performance daemon designed for the Transactional Outbox Pattern. By bridging your database and message brokers like Kafka or NATS, it ensures at-least-once delivery, solving distributed consistency challenges without the complexity of 2PC or distributed transactions.

✨ Features

• Guaranteed Delivery: at-least-once semantics ensure events are never lost, even during broker outages or relay crashes.
• Database-Native Scaling: Leverages database-native locking (like Postgres SKIP LOCKED) where available to support horizontal scaling without double-processing.
• Self-Healing: Built-in Lease Reaper automatically identifies and recovers "stuck" events from crashed worker instances.
• Smart Retries: Exponential backoff with jitter to prevent "thundering herd" issues during downstream outages.
• Observability: Native OpenTelemetry integration for distributed tracing and Prometheus metrics.
• Provider Agnostic: Pluggable storage (Postgres) and publishers (Kafka, NATS JetStream, Redis).

📊 Performance at Scale

• Non-Blocking I/O: Written in Go with a focus on zero-allocation paths in the event loop.
• Drain Mode: Automatically shifts from interval-based polling to high-speed "drain mode" during traffic bursts.
• Resource Efficient: Designed to run as a sidecar or a standalone microservice with a minimal memory footprint.

🚀 Quick Start

1. Database Schema

The Relay requires an outbox_events table. Run the standard DDL found in: schema/postgres/openoutbox.sql

2. Run with Docker

docker run -d \
  --name openoutbox-relay \
  -e STORAGE_URL="postgres://user:pass@localhost:5432/db" \
  -e PUBLISHER_TYPE="kafka" \
  -e KAFKA_BROKERS="localhost:9092" \
  openoutbox/relay:latest

Or with docker-compose:

---

services:
  relay:
    image: openoutbox/relay:latest
    container_name: openoutbox-relay
    environment:
      - STORAGE_TYPE=postgres
      - STORAGE_URL=postgres://postgres:postgres@db:5432/postgres
      - PUBLISHER_TYPE=kafka
      - KAFKA_BROKERS=kafka:9092
    restart: always

Operational Commands

The Relay includes a built-in CLI for maintenance, cleanup, and manual intervention.

Pruning Historical Data

To keep the outbox table performant, you can periodically prune successfully delivered or exhausted (dead) events.

relay-cli prune --delivered-age 7d --dead-age 30d --dry-run

relay-cli prune --delivered-age 7d --dead-age 30d

Using docker:

docker run --rm openoutbox/relay:latest ./cli prune --delivered-age 7d --dead-age 30d

🛡️ Reliability Guarantees

The Relay is designed for At-Least-Once Delivery.

• No Data Loss: Events are only marked as DELIVERED after the broker acknowledges receipt.
• Idempotency: In rare edge cases (e.g., a crash immediately after publishing but before the DB update), the same event may be published twice. Consumers must be idempotent.

⚙️ Configuration

The OpenOutbox Relay is configured entirely via environment variables.

1. Core Infrastructure & Connectivity

2. Engine Tuning (Outbox Polling)

3. Reliability & Backoff

4. Kafka Specific Tuning

5. Observability (OpenTelemetry)

📚 Documentation & Community

Stay connected and help us improve the OpenOutbox ecosystem:

• Contribution Guide: Ready to help? Check out our guide on setting up your local environment, running tests with the Makefile, and our pull request process.
• Changelog: View a detailed list of changes, improvements, and bug fixes for every release.
• OpenOutbox Specification: Learn more about the standard this relay implements.
• Architecture Deep Dive: A technical look at the internal locking mechanisms and event lifecycle management.

License

MIT License — Created by the OpenOutbox Team.