timebox

package module
v0.0.0-...-ec5cb51 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 3, 2026 License: MIT Imports: 14 Imported by: 2

README

Timebox

Build Status Code Coverage Maintainability GitHub

Timebox is a small, opinionated event sourcing library for Go with pluggable persistence backends including memory, Redis/Valkey, PostgreSQL, and Raft. It provides an append-only event log, optimistic concurrency, snapshotting, and append-time indexing so multiple instances can coordinate through the same store.

Backends

Timebox currently ships with:

  • memory for tests and single-process use
  • redis for Redis or Valkey deployments
  • postgres for PostgreSQL-backed persistence
  • raft for multi-node consensus

Core Concepts

  • Store: event-store semantics over a Persistence
  • Executor: loads aggregate state, runs a command, persists raised events, and retries on optimistic conflicts
  • Aggregator: accumulates events and exposes the current aggregate view during a command
  • Indexer: optional append-time hook that derives status and label updates from an appended event batch
  • Snapshot: cached aggregate state plus the sequence it represents

Store Behavior

timebox.Config controls store behavior regardless of backend:

  • TrimEvents: whether saving a snapshot trims older stored events
  • SnapshotRatio: when an Executor should opportunistically refresh a snapshot while loading state
  • MaxRetries: optimistic concurrency retry limit
  • CacheSize: executor projection cache size
  • Indexer: optional function that derives status and label updates from an appended event batch

Create a store by opening backend persistence and then binding a store to it:

p, err := postgres.NewPersistence(postgres.Config{...})
store, err := p.NewStore(timebox.Config{...})

You can also call timebox.NewStore(p, cfg) directly when you already have a backend value that satisfies timebox.Backend.

Snapshotting is available in two ways:

  • explicit saves through Executor.SaveSnapshot(id) or Store.PutSnapshot(id, value, sequence)
  • opportunistic executor saves while loading aggregates when no snapshot exists yet or when trailing event data grows past SnapshotRatio

Backend Config

Postgres

postgres.Config adds:

  • URL: connection URL
  • Prefix: logical store namespace
  • MaxConns: pgx pool size cap

The Postgres backend stores:

  • aggregate status and labels in timebox_index
  • snapshots in timebox_snapshot
  • events in timebox_events
Redis

redis.Config adds:

  • Addr: Redis or Valkey host:port
  • Password: optional password
  • Prefix: logical store namespace
  • Shard: optional hash-tag value for cluster slot affinity
  • DB: logical database index
Raft

raft.Config fields:

  • LocalID: stable local Raft node ID
  • Address: node address used for Raft traffic
  • DataDir: durable local state directory
  • LogTailSize: hot retained WAL suffix cache size, default 20480
  • Servers: bootstrap voter set
  • Publisher: optional callback for committed events after they are durably applied

Indexing

Config.Indexer lets you derive indexed metadata from an appended event batch. Index currently supports:

  • Status: aggregate status plus the time it entered that status
  • Labels: current aggregate label values

Read paths exposed by the store:

  • Store.GetAggregateStatus(id)
  • Store.ListAggregatesByStatus(status)
  • Store.ListLabelValues(label)
  • Store.ListAggregatesByLabel(label, value)

Archiving

Archiving moves an aggregate's snapshot and event history into backend-specific archive storage and clears the live records. It is a one-way operation. The memory, redis, and raft backends support archiving, while postgres does not.

Call Store.Archive(id).

To consume archived records, call Store.ConsumeArchive(ctx, handler). It blocks until one record is processed or the context is done. Use context.WithTimeout to poll with a deadline.

Handlers must be idempotent because processing is at-least-once.

Examples

  • examples/order.go shows a simple order lifecycle over Timebox

Status

Work in progress. Not ready for production use.

Documentation

Overview

Package timebox implements an event sourcing toolkit with pluggable persistence backends such as memory, Redis/Valkey, PostgreSQL, and Raft. It couples an append-only event log, snapshots, optimistic concurrency, and append-time indexing into a library that can be embedded into services

Typical usage looks like:

  • Open backend persistence and create a Store from it
  • Define Appliers that fold events into your aggregate state
  • Optionally define an Indexer to project current status or label indexes
  • Use an Executor to run Commands that raise events on an Aggregator
  • Save snapshots explicitly or let the Executor refresh them while loading
  • Query the Store directly for events and aggregate state

The examples/ directory contains a runnable order workflow that exercises the API in a small domain

Index

Constants

View Source
const (
	// DefaultTrimEvents determines whether snapshots trim stored events
	DefaultTrimEvents = false

	// DefaultSnapshotRatio is the default EventsSize/SnapshotSize threshold
	DefaultSnapshotRatio = 1.0

	// DefaultMaxRetries is the default optimistic concurrency retry count
	DefaultMaxRetries = 16

	// DefaultCacheSize controls the projection LRU size
	DefaultCacheSize = 128
)

Variables

View Source
var (
	// JSONEvent encodes complete events as JSON
	JSONEvent EventCodec[[]byte] = jsonEventCodec{}

	// BinEvent encodes complete events using the internal binary format
	BinEvent interface {
		EventCodec[[]byte]
		AppendAll(buf []byte, evs []*Event) ([]byte, error)
		ReadAll(data []byte) ([]*Event, []byte, error)
	} = binEventCodec{}

	// EncodeJSONEvents encodes complete events as JSON bytes
	EncodeJSONEvents = MakeEncodeAll(JSONEvent)

	// DecodeJSONEvents decodes complete events from JSON bytes
	DecodeJSONEvents = MakeDecodeAll(JSONEvent)

	// EncodeBinEvents encodes complete events to the internal binary format
	EncodeBinEvents = MakeEncodeAll(BinEvent)

	// DecodeBinEvents decodes complete events from the internal binary format
	DecodeBinEvents = MakeDecodeAll(BinEvent)
)
View Source
var (
	// ErrInvalidMaxRetries indicates MaxRetries is below the allowed range
	ErrInvalidMaxRetries = errors.New("max retries must be > 0")

	// ErrInvalidCacheSize indicates CacheSize is below the allowed range
	ErrInvalidCacheSize = errors.New("cache size must be > 0")

	// ErrInvalidSnapshotRatio indicates SnapshotRatio is below allowed range
	ErrInvalidSnapshotRatio = errors.New("snapshot ratio must be >= 0")
)
View Source
var (
	// ErrUnexpectedResult indicates data returned in an unexpected shape
	ErrUnexpectedResult = errors.New("unexpected result")

	// ErrArchivingDisabled indicates archiving is not enabled
	ErrArchivingDisabled = errors.New("archiving not enabled for this store")

	// ErrArchiveRecordMalformed indicates an archive record was malformed
	ErrArchiveRecordMalformed = errors.New("archive record malformed")

	// ErrArchiveHandlerMissing indicates a consume call is missing a handler
	ErrArchiveHandlerMissing = errors.New("archive handler is required")
)
View Source
var (
	// ErrMaxRetriesExceeded indicates optimistic concurrency retries were
	// exhausted while attempting to persist events
	ErrMaxRetriesExceeded = errors.New("max retries exceeded")
)

Functions

func Configure

func Configure[T With[T]](defaults T, others ...T) T

Configure overlays each supplied value on top of defaults in order

func GetEventValue

func GetEventValue[T any](e *Event) (T, error)

GetEventValue unmarshals the event data into the requested type. It reuses a cached value when the requested type matches, and otherwise unmarshals without replacing the cached type. This is safe for concurrent access

func MakeDecodeAll

func MakeDecodeAll[Value, Data any](
	codec Codec[Value, Data],
) func([]Data) ([]Value, error)

MakeDecodeAll returns a batch decoder for the provided codec

func MakeEncodeAll

func MakeEncodeAll[Value, Data any](
	codec Codec[Value, Data],
) func([]Value) ([]Data, error)

MakeEncodeAll returns a batch encoder for the provided codec

func Raise

func Raise[T, V any](ag *Aggregator[T], typ EventType, value V) error

Raise marshals the value and enqueues a new event on the Aggregator

Types

type AggregateID

type AggregateID []ID

AggregateID identifies an aggregate as a set of parts ("order", "123")

func NewAggregateID

func NewAggregateID(parts ...ID) AggregateID

NewAggregateID builds an AggregateID from its parts

func (AggregateID) Equal

func (id AggregateID) Equal(other AggregateID) bool

Equal compares two AggregateIDs for equality

func (AggregateID) HasPrefix

func (id AggregateID) HasPrefix(prefix AggregateID) bool

HasPrefix checks if the AggregateID starts with the provided prefix

func (AggregateID) String

func (id AggregateID) String() string

String returns a human-readable AggregateID representation

type Aggregator

type Aggregator[T any] struct {
	// contains filtered or unexported fields
}

Aggregator maintains aggregate state for a command and tracks events raised through Raise. It is not safe for concurrent use

func (*Aggregator[_]) ID

func (a *Aggregator[_]) ID() AggregateID

ID returns the aggregate's identifier components

func (*Aggregator[_]) NextSequence

func (a *Aggregator[_]) NextSequence() int64

NextSequence returns the next sequence number that will be assigned to a new event

func (*Aggregator[T]) OnSuccess

func (a *Aggregator[T]) OnSuccess(fn SuccessAction[T])

OnSuccess registers an action to run after Executor.Exec persists the raised events successfully

func (*Aggregator[T]) Value

func (a *Aggregator[T]) Value() T

Value returns the aggregate's current state

type AlwaysReady

type AlwaysReady struct{}

AlwaysReady can be embedded in a Persistence to provide a Ready() implementation for backends that are immediately available

func (AlwaysReady) Ready

func (AlwaysReady) Ready() <-chan struct{}

Ready returns a pre-closed channel, indicating immediate readiness

type AppendRequest

type AppendRequest struct {
	StatusAt time.Time
	*Store
	Status           *string
	Labels           map[string]string
	ID               AggregateID
	Events           []*Event
	ExpectedSequence int64
}

AppendRequest contains primitive inputs required for an atomic append

type Applier

type Applier[T any] func(T, *Event) T

Applier applies an event to an aggregate state, returning the new state

func MakeApplier

func MakeApplier[T, Data any](fn func(T, *Event, Data) T) Applier[T]

MakeApplier wraps a strongly typed applier that receives the event payload value and returns an Applier that works with Event

type Appliers

type Appliers[T any] map[EventType]Applier[T]

Appliers is a map of EventType to Applier for a given aggregate

type ArchiveHandler

type ArchiveHandler func(context.Context, *ArchiveRecord) error

ArchiveHandler handles a single archive record

type ArchiveRecord

type ArchiveRecord struct {
	StreamID         string
	AggregateID      AggregateID
	SnapshotData     json.RawMessage
	Events           []*Event
	SnapshotSequence int64
}

ArchiveRecord stores stream metadata and aggregate artifacts

type Archiver

type Archiver interface {
	// Archive moves an aggregate's persisted artifacts into archive storage
	Archive(id AggregateID) error

	// ConsumeArchive blocks until one archive record is available or ctx
	// is done
	ConsumeArchive(ctx context.Context, handler ArchiveHandler) error
}

Archiver provides optional archive lifecycle support for Store

type Backend

type Backend interface {
	Persistence
	Queries
}

Backend is used by Store to provide Persistence and Queries

type Codec

type Codec[Value, Data any] interface {
	// Encode encodes a value to the codec data format
	Encode(Value) (Data, error)

	// Decode decodes codec data to a value
	Decode(Data) (Value, error)

	// Append appends an encoded value to a buffer
	Append(Data, Value) (Data, error)

	// Read reads a value from a buffer and returns the remainder
	Read(Data) (Value, Data, error)
}

Codec encodes and decodes values

type Command

type Command[T any] func(T, *Aggregator[T]) error

Command is user code that inspects state and raises events on an Aggregator. Returning an error aborts the operation

type Config

type Config struct {
	Indexer       Indexer
	SnapshotRatio float64
	MaxRetries    int
	CacheSize     int
	TrimEvents    bool
}

Config configures Store behavior

func DefaultConfig

func DefaultConfig() Config

DefaultConfig returns a Config populated with sensible defaults

func (Config) Validate

func (cfg Config) Validate() error

Validate reports whether the configuration contains invalid values

func (Config) With

func (cfg Config) With(other Config) Config

With overlays the non-zero values from other onto cfg

type Event

type Event struct {
	Timestamp   time.Time       `json:"timestamp"`
	Type        EventType       `json:"type"`
	AggregateID AggregateID     `json:"aggregate_id"`
	Data        json.RawMessage `json:"data"`
	Sequence    int64           `json:"sequence"`
	Raised      bool            `json:"-"`
	// contains filtered or unexported fields
}

Event represents a single immutable event in the log, including its sequence, timestamp, and serialized payload

type EventCodec

type EventCodec[Data any] interface {
	Codec[*Event, Data]
}

EventCodec encodes and decodes complete events

type EventType

type EventType string

EventType is the string identifier associated with an Event

type EventsResult

type EventsResult struct {
	Events        []*Event
	StartSequence int64
}

EventsResult contains raw persisted events and the sequence to assign to the first event in the slice

type Executor

type Executor[T any] struct {
	// contains filtered or unexported fields
}

Executor orchestrates loading aggregate state, executing commands, and persisting resulting events with optimistic retries

func NewExecutor

func NewExecutor[T any](
	store *Store, cons constructor[T], apps Appliers[T],
	onSuccess ...SuccessAction[T],
) *Executor[T]

NewExecutor constructs an Executor bound to a Store with the given appliers and state constructor

func (*Executor[T]) AppliesEvent

func (e *Executor[T]) AppliesEvent(ev *Event) bool

AppliesEvent reports whether the executor has an applier for the event type

func (*Executor[T]) Exec

func (e *Executor[T]) Exec(id AggregateID, cmd Command[T]) (T, error)

Exec loads the aggregate state, executes the command, and persists raised events. It retries on version conflicts up to MaxRetries

func (*Executor[T]) Get

func (e *Executor[T]) Get(id AggregateID) (T, error)

Get returns the current aggregate state

func (*Executor[T]) GetStore

func (e *Executor[T]) GetStore() *Store

GetStore exposes the Store used by the Executor

func (*Executor[T]) SaveSnapshot

func (e *Executor[T]) SaveSnapshot(id AggregateID) error

SaveSnapshot forces an immediate snapshot save for the given Aggregate

type Flusher

type Flusher func(int64, []*Event) error

Flusher persists enqueued events and returns an error if the write fails

type Handler

type Handler func(*Event) error

Handler processes a single Event

func MakeDispatcher

func MakeDispatcher(handlers map[EventType]Handler) Handler

MakeDispatcher routes events to handlers keyed by EventType, ignoring unmatched event types

func MakeHandler

func MakeHandler[T any](fn func(ev *Event, data T) error) Handler

MakeHandler decodes event data into the provided type before invoking fn. It reuses the Event's cached value when available

type ID

type ID string

ID is a single component of an AggregateID

type Index

type Index struct {
	// Status represents the resultant aggregate status. nil means no
	// status change, and "" clears any prior status
	Status *string `json:"status,omitempty"`

	// Labels updates current label values for the aggregate. nil means no
	// label changes, and empty values remove the label
	Labels map[string]string `json:"labels,omitempty"`
}

Index stores optional projection metadata derived from an event

type Indexer

type Indexer func([]*Event) []*Index

Indexer derives projection metadata for an event batch

type LoadEventsRequest

type LoadEventsRequest struct {
	*Store
	ID      AggregateID
	FromSeq int64
}

LoadEventsRequest contains primitive inputs required for an event load

type LoadSnapshotRequest

type LoadSnapshotRequest struct {
	*Store
	ID AggregateID
}

LoadSnapshotRequest contains primitive inputs required for a snapshot load

type Persistence

type Persistence interface {
	io.Closer

	// NewStore creates a Store using this Persistence instance
	NewStore(Config) (*Store, error)

	// Ready reports when the persistence backend can serve requests
	Ready() <-chan struct{}

	// Append atomically appends events if the expected sequence still
	// matches. It returns a VersionConflictError when the sequence check
	// fails
	Append(AppendRequest) error

	// LoadEvents loads raw persisted events starting at fromSeq
	LoadEvents(LoadEventsRequest) (*EventsResult, error)

	// LoadSnapshot loads the raw snapshot and any trailing raw events
	LoadSnapshot(LoadSnapshotRequest) (*SnapshotRecord, error)

	// SaveSnapshot stores raw snapshot data using the supplied semantics
	SaveSnapshot(SnapshotRequest) error
}

Persistence provides the low-level primitives Store uses to implement Store semantics

type Queries

type Queries interface {
	// ListAggregates lists aggregate IDs that match the provided prefix
	ListAggregates(id AggregateID) ([]AggregateID, error)

	// GetAggregateStatus loads the current indexed status for an aggregate
	GetAggregateStatus(id AggregateID) (string, error)

	// ListAggregatesByStatus lists aggregates currently indexed by status
	ListAggregatesByStatus(status string) ([]StatusEntry, error)

	// ListAggregatesByLabel lists aggregates currently indexed by
	// label/value
	ListAggregatesByLabel(label, value string) ([]AggregateID, error)

	// ListLabelValues lists the distinct indexed values for a label
	ListLabelValues(label string) ([]string, error)
}

Queries provides aggregate and index query operations

type SnapshotRecord

type SnapshotRecord struct {
	Data     json.RawMessage
	Events   []*Event
	Sequence int64
}

SnapshotRecord contains raw snapshot data and any raw trailing events

type SnapshotRequest

type SnapshotRequest struct {
	*Store
	ID       AggregateID
	Data     []byte
	Sequence int64
}

SnapshotRequest contains primitive inputs required for a snapshot save

type SnapshotResult

type SnapshotResult struct {
	AdditionalEvents []*Event
	NextSequence     int64
	SnapshotSize     int
	EventsSize       int
}

SnapshotResult holds the loaded snapshot, the sequence at which it was taken, and any events that need to be applied after it

type StatusEntry

type StatusEntry struct {
	Timestamp time.Time
	ID        AggregateID
}

StatusEntry holds an aggregate ID and the time it entered a status

type Store

type Store struct {
	Queries
	// contains filtered or unexported fields
}

Store persists, queries, and snapshots aggregate events

func NewStore

func NewStore(b Backend, cfg Config) (*Store, error)

NewStore creates a Store backed by the supplied Backend

func (*Store) AppendEvents

func (s *Store) AppendEvents(id AggregateID, atSeq int64, evs []*Event) error

AppendEvents atomically appends events for an aggregate if the expected sequence matches the current log sequence

func (*Store) Archive

func (s *Store) Archive(id AggregateID) error

Archive moves aggregate artifacts to persistent archive storage

func (*Store) Close

func (s *Store) Close() error

Close closes the underlying persistence

func (*Store) Config

func (s *Store) Config() *Config

Config returns the Store configuration

func (*Store) ConsumeArchive

func (s *Store) ConsumeArchive(
	ctx context.Context, handler ArchiveHandler,
) error

ConsumeArchive reads one archive record and invokes handler

func (*Store) GetEvents

func (s *Store) GetEvents(id AggregateID, fromSeq int64) ([]*Event, error)

GetEvents returns all events for an aggregate starting at fromSeq

func (*Store) GetSnapshot

func (s *Store) GetSnapshot(
	id AggregateID, target any,
) (*SnapshotResult, error)

GetSnapshot loads the latest snapshot into target and returns any events stored after the snapshot sequence

func (*Store) PutSnapshot

func (s *Store) PutSnapshot(id AggregateID, value any, sequence int64) error

PutSnapshot saves a snapshot value and sequence if the provided sequence is newer than any stored snapshot

func (*Store) Ready

func (s *Store) Ready() <-chan struct{}

Ready reports when the underlying persistence can serve requests

func (*Store) WaitReady

func (s *Store) WaitReady(ctx context.Context) error

WaitReady blocks until the underlying persistence can serve requests

type SuccessAction

type SuccessAction[T any] func(T, []*Event)

SuccessAction receives the Aggregator's final value after Executor.Exec succeeds, as well as the Events persisted by that execution

type VersionConflictError

type VersionConflictError struct {
	NewEvents        []*Event
	ExpectedSequence int64
	ActualSequence   int64
}

VersionConflictError is returned when AppendEvents encounters a sequence mismatch. NewEvents contains the conflicting events

func (*VersionConflictError) Error

func (e *VersionConflictError) Error() string

type With

type With[T any] interface {
	With(T) T
}

With is a generic overlay contract used by Configure

Directories

Path Synopsis
internal
id
Package raft implements Timebox persistence semantics using Raft for the replicated commit path, a durable write-ahead log, and a local materialized read store
Package raft implements Timebox persistence semantics using Raft for the replicated commit path, a durable write-ahead log, and a local materialized read store

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL