Documentation
¶
Overview ¶
Package consumer runs Consumers: named, Cursor-tracked readers of the event log.
A Consumer reads the events matching its event-type filter in Cursor order and hands each one to its Handler, usually a Router built from typed handlers:
r := consumer.NewRouter(codec)
consumer.On(r, func(ctx context.Context, env eventstore.Envelope, e TaskOpened) error {
return q.InsertTask(ctx, postgres.Conn(ctx, pool), env.StreamID, e.Title)
})
tasks, err := consumer.New(rt, consumer.Config{
Name: "task_list",
Mode: consumer.Transactional,
Handler: r,
})
In Transactional mode the handler's writes and the Cursor commit in one Cursor transaction, so every event takes effect exactly once. In OwnTransaction mode the Cursor advances after each event, so delivery is at least once.
A failing event is retried with backoff and then the Consumer goes Stopped; it never skips an event. Permanent errors, panics and decode errors stop it at once. A Stopped Consumer stays at its Cursor until Runtime.Resume.
The filter is stored on the Cursor and checked when a release changes it (ADR-0006). Removed types are always accepted. An OwnTransaction Consumer handles added types from its Cursor onwards. A Transactional Consumer accepts added types only when no event of them lies at or below its Cursor; otherwise it goes Stopped, and the change needs a new Projection version. Resume does not skip this check.
The Runtime keeps the Cursors. NewPgRuntime runs on an eventstore.PgStore with Cursors in the consumer_cursors table; handlers write through postgres.Conn, which joins the Cursor transaction. Each Consumer runs under a lease, so only one process at a time processes it, and wakes up on NOTIFY when a Listener is given. NewInMemoryRuntime runs on an eventstore.InMemoryStore; its handlers register writes with postgres.Stage so they roll back with the Cursor transaction.
Consumers only see events below pg_snapshot_xmin (or the in-memory equivalent), so an event of a transaction that commits late is never skipped. Any open transaction that holds a transaction ID, in any database of the cluster, delays visibility until it ends.
Run all Consumers of a process with a Group:
err := consumer.Group{tasks, mailer}.Run(ctx)
Tests and one-shot jobs call Drain, which processes synchronously up to the head of the log at the time of the call.
Observability ¶
Each batch runs in a CONSUMER span "process {consumer}". It is a new root, linked to the send spans that appended up to 128 of its events. WithEventSpans adds an INTERNAL span "handle {event type}" per event. The histograms eventsourcing.consumer.batch.duration and eventsourcing.consumer.process.duration carry only the Consumer's name. While Run holds the lease, the gauges eventsourcing.consumer.pending, eventsourcing.consumer.lag and eventsourcing.consumer.stopped are polled from Status at each metric collection, so only the lease owner reports them; Drain does not. A PgRuntime reports eventsourcing.xmin.age.
Index ¶
- Constants
- Variables
- func On[E aggregate.Event](r *Router, fn func(ctx context.Context, env eventstore.Envelope, e E) error)
- func Permanent(err error) error
- type Config
- type Consumer
- type Freshness
- type Group
- type Handler
- type HandlerFunc
- type InMemoryRuntime
- type InMemoryRuntimeConfig
- type Mode
- type Option
- type PgRuntime
- type PgRuntimeConfig
- type Retry
- type Router
- type Runtime
- type Status
- type Stop
Constants ¶
const PendingCap = 10_000
PendingCap caps Status.Pending: a Consumer with more pending events reports PendingCap+1.
Variables ¶
var ( // ErrInvalidConfig reports a Config or Option that New rejects. ErrInvalidConfig = errors.New("consumer: invalid config") // ErrDuplicate reports a second Consumer with the same name on one // Runtime. ErrDuplicate = errors.New("consumer: duplicate name") // ErrRunning reports Run on a Consumer that already runs, or Drain while // the Consumer runs or drains. ErrRunning = errors.New("consumer: already running") // ErrStopped reports a Consumer that is Stopped. It stays Stopped at its // Cursor until Runtime.Resume clears it. ErrStopped = errors.New("consumer: stopped") // ErrNotFound reports a Consumer name the Runtime does not know. ErrNotFound = errors.New("consumer: not found") )
Functions ¶
func On ¶
func On[E aggregate.Event](r *Router, fn func(ctx context.Context, env eventstore.Envelope, e E) error)
On routes events of type E to fn, which receives the Envelope and the decoded, upcast event. It panics on a nil fn, when E is not registered on the Router's codec, or when E is already routed. It is meant to be called at startup.
Types ¶
type Config ¶
type Config struct {
// Name identifies the Consumer and its Cursor; unique per Runtime.
Name string
// EventTypes is the filter. When empty, a Handler with an EventTypes
// method (a Router) provides it; otherwise every event is delivered.
EventTypes []string
Mode Mode
Handler Handler
}
Config configures a Consumer.
type Consumer ¶
type Consumer struct {
// contains filtered or unexported fields
}
Consumer is a named, Cursor-tracked reader of the event log. It reads the events matching its filter in Cursor order and hands each to its Handler. A failing event is retried and then stops the Consumer; it is never skipped.
func New ¶
New creates a Consumer on rt. It fails with ErrInvalidConfig for an empty Name, a nil Handler, an unknown Mode or invalid options, and with ErrDuplicate when rt already has a Consumer with that Name.
func (*Consumer) Await ¶
Await waits until the Consumer has caught up to pos, typically a command's CommandResult.Position, so an API can serve read-your-writes or mark its response stale. The budget is the ctx deadline, else 2s.
The Consumer has caught up when its Cursor is at or after pos, or when no matching event lies between its Cursor and pos and every transaction up to pos has ended. So a filtered Consumer whose Cursor sits at its last matching event is Fresh. A zero pos is Fresh at once. Await returns StaleStopped as soon as it finds the Consumer Stopped short of pos, and Stale when the budget runs out. All Awaits on a Consumer share one poller.
func (*Consumer) Drain ¶
Drain processes events synchronously up to the head of the log at the time of the call. It fails with ErrRunning while the Consumer runs or drains, and with a wrapped ErrStopped when the Consumer is or goes Stopped.
type Freshness ¶
type Freshness int
Freshness tells whether a Consumer has caught up to a Cursor.
const ( // Fresh means the Consumer has processed every matching event up to the // Cursor. Fresh Freshness = iota + 1 // Stale means the Consumer did not catch up within the budget. Stale // StaleStopped means the Consumer has not caught up and is Stopped, so // it will not catch up without Resume. StaleStopped )
type Handler ¶
type Handler interface {
Handle(ctx context.Context, env eventstore.Envelope) error
}
Handler handles one event. It runs with the Envelope's Tenant and Correlation in ctx. A returned error is retried, unless it is Permanent.
type HandlerFunc ¶
type HandlerFunc func(ctx context.Context, env eventstore.Envelope) error
HandlerFunc adapts a function to a Handler.
func (HandlerFunc) Handle ¶
func (f HandlerFunc) Handle(ctx context.Context, env eventstore.Envelope) error
Handle calls f.
type InMemoryRuntime ¶
type InMemoryRuntime struct {
// contains filtered or unexported fields
}
InMemoryRuntime runs Consumers on an eventstore.InMemoryStore, for prototypes and tests. Cursors live in memory. A Cursor transaction is a postgres.InMemoryUnitOfWork transaction, so handler writes registered with postgres.Stage commit or roll back together with the Cursor.
func NewInMemoryRuntime ¶
func NewInMemoryRuntime(cfg InMemoryRuntimeConfig) *InMemoryRuntime
NewInMemoryRuntime creates an InMemoryRuntime over cfg.Store. It panics on a nil Store.
type InMemoryRuntimeConfig ¶
type InMemoryRuntimeConfig struct {
// Store is the event log the Consumers read.
Store *eventstore.InMemoryStore
}
InMemoryRuntimeConfig configures an InMemoryRuntime.
type Mode ¶
type Mode int
Mode selects the delivery guarantee of a Consumer.
const ( // Transactional runs the handlers of a batch and the Cursor update in one // Cursor transaction, so every event takes effect exactly once. The // handler writes through the transaction in ctx (postgres.Conn, or // postgres.Stage in memory). Transactional Mode = iota + 1 // OwnTransaction runs each handler on its own and advances the Cursor // after each event, so delivery is at least once. OwnTransaction )
type Option ¶
type Option func(*options)
Option configures a Consumer.
func WithBatchSize ¶
WithBatchSize sets how many events one batch reads. The default is 100.
func WithEventSpans ¶
WithEventSpans turns the per-event "handle {event type}" spans on or off. They are off by default.
func WithLogger ¶
WithLogger sets the logger. The default is slog.Default().
func WithMeterProvider ¶
func WithMeterProvider(mp metric.MeterProvider) Option
WithMeterProvider sets the meter provider of the histograms and gauges. The default is the OTel global.
func WithPollInterval ¶
WithPollInterval sets how long a running Consumer waits for new events before it polls again. The wait is jittered down by up to a fifth. The default is 1s.
func WithRetry ¶
WithRetry sets the retry schedule. The default is 5 attempts with delays from 1s up to 30s.
func WithTracerProvider ¶
func WithTracerProvider(tp trace.TracerProvider) Option
WithTracerProvider sets the tracer provider. The default is the OTel global.
type PgRuntime ¶
type PgRuntime struct {
// contains filtered or unexported fields
}
PgRuntime runs Consumers on an eventstore.PgStore. Cursors live in the consumer_cursors table of docs/postgres/schema.sql. A Consumer reads only the events below pg_snapshot_xmin, so it never skips an event of a transaction that commits later with a smaller Cursor. A filtered Consumer reads one index range per event type.
Each Consumer runs under a lease from Claims, also while Stopped. A Cursor transaction is a Postgres transaction in ctx: handler writes made through postgres.Conn commit or roll back together with the Cursor.
func NewPgRuntime ¶
func NewPgRuntime(cfg PgRuntimeConfig) (*PgRuntime, error)
NewPgRuntime creates a PgRuntime. It fails with ErrInvalidConfig when Pool, Store or Claims is missing.
type PgRuntimeConfig ¶
type PgRuntimeConfig struct {
// Pool runs the reads and Cursor transactions. Required.
Pool *pgxpool.Pool
// Store is the event log the Consumers read; its Schema qualifies the
// consumer tables, the wake-up channel <schema>.events and the lease
// namespace <schema>.consumer. Required.
Store *eventstore.PgStore
// Listener wakes Consumers on new events. Optional: without it the
// Consumers only poll.
Listener *notify.Listener
// Claims grants each Consumer its lease. Required, and Claims.Run must
// be running.
Claims *lease.Claims
// MeterProvider records the eventsourcing.xmin.age gauge, one series
// per MeterProvider until Close. Optional: the default is the OTel
// global.
MeterProvider metric.MeterProvider
// SetLocalTenant makes a Cursor transaction set app.tenant_id to the
// Tenant of each event before its handler runs, like
// postgres.SetLocalTenant does for a UnitOfWork, so row-level security
// on read models sees the Tenant. Off by default.
SetLocalTenant bool
}
PgRuntimeConfig configures a PgRuntime.
type Retry ¶
Retry is the retry schedule of a failing event: up to MaxAttempts attempts in total, with jittered exponential delays from Base up to Max.
type Router ¶
type Router struct {
// contains filtered or unexported fields
}
Router is a Handler built from typed handlers registered with On. Its EventTypes are the Consumer's filter unless Config.EventTypes is set. Events of unrouted types are ignored.
func NewRouter ¶
func NewRouter(codec *eventstore.Codec) *Router
NewRouter returns an empty Router for events registered on codec. It panics on a nil codec.
func (*Router) EventTypes ¶
EventTypes returns the routed Event types, sorted.
type Runtime ¶
type Runtime interface {
// Resume clears the Stopped state of the named Consumer, which then
// continues from its Cursor. It fails with ErrNotFound for an unknown
// name and is a no-op when the Consumer is not Stopped.
Resume(ctx context.Context, name string) error
// contains filtered or unexported methods
}
Runtime runs Consumers on one event log and keeps their Cursors. It is implemented by PgRuntime and InMemoryRuntime.
type Status ¶
type Status struct {
// Cursor is the last processed event.
Cursor eventstore.Cursor
// Pending counts the visible matching events after Cursor, up to
// PendingCap+1.
Pending int
// Age is how long the oldest pending event has been in the log; zero
// when nothing is pending.
Age time.Duration
// Stopped is set while the Consumer is Stopped.
Stopped *Stop
}
Status is a Consumer's progress.