README
¶
BabelQueue idempotency store — Redis (Go)
A Redis-backed idempotency.Store
for the BabelQueue Go runtime: a shared,
persistent record of processed message ids so a fleet of consumers dedupes on the
envelope's meta.id (ADR-0022) — instead of every worker keeping
its own in-memory set.
The framework-agnostic core stays zero-dependency (GR-7): the dep-free Store interface
(and the reference InMemoryStore) live in the core; this module is a separate submodule
with its own go.mod, the only place a Redis driver is pulled in. It reuses the same
go-redis client as the …/redis transport submodule,
so a consumer importing both shares one driver version.
go get github.com/babelqueue/babelqueue-go/idempotency-redis
import (
babelqueue "github.com/babelqueue/babelqueue-go"
"github.com/babelqueue/babelqueue-go/idempotency"
redis "github.com/babelqueue/babelqueue-go/idempotency-redis"
)
store, err := redis.New("redis://localhost:6379/0", redis.WithTTL(24*time.Hour))
if err != nil { panic(err) }
defer store.Close()
app.Handle("urn:babel:orders:created", idempotency.Wrap(store, handler))
Bring your own client with redis.NewWithClient(client) (custom pool / TLS / auth).
WithPrefix("…") isolates keys per app on a shared Redis (default bq:idemp:); WithTTL(d)
expires ids after d so they may be re-processed once the window lapses (default: keys never
expire — evict with Forget).
Atomic claim (parking)
The race-safety comes from SET key value NX [PX ttl]:
SET bq:idemp:<message_id> 1 NX PX <ttl-ms>
Redis applies NX ("set only if Not eXists") on its single command thread, so under any number
of concurrent consumers exactly one SET creates the key (reply OK → that caller won
the claim); every duplicate sees the key present (reply nil → a duplicate / parked
delivery). The TTL rides the same atomic command (PX), so the key appears and gets its expiry
in one round trip, and Redis expires it natively — after which Seen is false and the id may be
re-claimed. This is the same "exactly one wins, the rest park" contract the in-memory store
models — now durable across processes.
Claim(ctx, id) (bool, error) exposes that race result directly (true = first delivery,
false = duplicate). Remember (the idempotency.Store write side) delegates to Claim and
discards the bool, so idempotency.Wrap accepts this store unchanged. Seen is an EXISTS
check; Forget is a DEL.
Tests
Unit tests run with no broker (a fake go-redis client seam that models SET NX / EXISTS /
DEL), covering the claim / duplicate / concurrent-serialize / TTL / Seen / Forget logic.
Integration tests skip cleanly unless a live Redis is provided:
go test ./... # unit only (no broker) — what CI runs on every push
BABELQUEUE_TEST_REDIS='redis://localhost:6379/0' \
go test ./... # + integration (concurrent claims serialize, duplicate rejected, TTL)
Full spec: babelqueue.com · MIT © Muhammet Şafak
Documentation
¶
Overview ¶
Package redis is a Redis-backed idempotency.Store for the BabelQueue Go runtime: a shared, persistent record of processed message ids so a fleet of consumers dedupes on the envelope's meta.id (ADR-0022) instead of every worker keeping its own in-memory set.
store, _ := redis.New("redis://localhost:6379/0", redis.WithTTL(24*time.Hour))
defer store.Close()
app.Handle(urn, idempotency.Wrap(store, handler))
It satisfies the dependency-free idempotency.Store interface from the core (Seen / Remember / Forget) with the SAME contract the reference in-memory store guarantees, and adds atomic-claim semantics so concurrent consumers serialize:
- Remember is an ATOMIC claim — SET key value NX [PX ttl]. Redis evaluates NX ("set only if Not eXists") on a single thread, so exactly one of N concurrent callers sets the key (and is the "winner"); a duplicate delivery sees the key already present and is a no-op. This is race-safe across processes — the key's existence is the lock.
- Claim exposes that race result directly: true iff THIS caller set the key (first delivery), false when the key already existed (duplicate / in-flight "parked" delivery). Remember delegates to Claim and discards the bool, so the idempotency.Store contract is preserved exactly.
- Seen reports whether the key currently exists.
- TTL/expiry is honored by Redis natively via PX: a key auto-expires after the TTL, after which Seen is false and the id may be re-claimed (the documented at-least-once behaviour). With no TTL the key persists until Forget.
The GR-7 zero-dependency core stays dependency-free: this driver lives in its own submodule (its own go.mod) and is the only place go-redis is pulled in — the same client the …/redis *transport* submodule already uses, so a consumer importing both shares one driver version.
Index ¶
- Constants
- type Option
- type Store
- func (s *Store) Claim(ctx context.Context, messageID string) (bool, error)
- func (s *Store) Close() error
- func (s *Store) Forget(ctx context.Context, messageID string) error
- func (s *Store) Remember(ctx context.Context, messageID string) error
- func (s *Store) Seen(ctx context.Context, messageID string) (bool, error)
Constants ¶
const DefaultPrefix = "bq:idemp:"
DefaultPrefix is the key namespace used when WithPrefix is not given. The stored key is prefix + message id.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Option ¶
type Option func(*Store)
Option customizes a Store.
func WithPrefix ¶
WithPrefix overrides the key namespace (default DefaultPrefix). Use distinct prefixes to isolate idempotency keys per app on a shared Redis.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store is a Redis-backed idempotency.Store. Construct it with New (owns the client) or NewWithClient (bring your own go-redis client).
func New ¶
New connects to Redis from a redis:// or rediss:// URL and returns a Store. The caller owns the lifetime: call Store.Close to release the client.
func NewWithClient ¶
NewWithClient wraps an existing go-redis client (bring your own pool / TLS / auth). The caller keeps ownership: Store.Close is a no-op for a borrowed client.
func (*Store) Claim ¶
Claim atomically records messageID and reports whether THIS call won the race — true on the first delivery (the key was set), false when the key already existed (a duplicate / in-flight "parked" delivery).
The race-safety comes from SET key value NX [PX ttl]: Redis applies NX on its single command thread, so under any number of concurrent consumers exactly one SET creates the key and reports a claim; the rest see it present and report a duplicate. The TTL (when set) rides the same atomic command, so the key both appears and gets its expiry in one round trip.
This is the atomic primitive [Remember] is built on; the idempotency.Store interface does not expose it, so it is available only on a *Store.
func (*Store) Close ¶
Close releases the underlying client when this Store opened it (New). For a borrowed client (NewWithClient) it is a no-op, leaving the caller's client intact.
func (*Store) Forget ¶
Forget drops messageID from the store (manual eviction; the idempotency.Store contract). Removing an absent id is a no-op.
func (*Store) Remember ¶
Remember records messageID as processed (atomic claim; the idempotency.Store write side). It delegates to Store.Claim and discards whether this caller won the race — the seen-set contract only needs the key to exist afterwards.
Source Files