dispatch

package
v0.1.0-alpha.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 11, 2026 License: MIT Imports: 8 Imported by: 0

Documentation

Overview

Package dispatch provides KeyedPool, a worker pool that keeps per-key order: work items that share a key are processed one at a time, in the order they were submitted, while items with different keys run concurrently.

How it works

KeyedPool has a fixed number of lanes. Each lane is one goroutine reading its own bounded queue. SubmitBlocking takes the item's key beside the item, and the item goes to lane fnv1a(key) % Lanes, so every item with the same key lands on the same lane and is processed in submit order. Process receives the lane index, so a caller can keep per-lane state (for example an applied-sequence guard) without locking: only that lane's goroutine touches it. Process returns nothing: it disposes of its own failures. A panic in Process is recovered; the lane keeps running and the optional OnPanic callback is called with the item, so the caller can dispose of it (for example Nak the message it came from).

Use it when two items with the same key must never be processed out of order or at the same time, as graph-ingest needs for one entity's updates (ADR-072, in SemStreams).

pool, err := dispatch.NewKeyedPool(ctx, dispatch.KeyedConfig[ingestWork]{
    Lanes:           lanes,
    QueueDepth:      256,
    Name:            "graph_ingest",
    Process:         process, // func(ctx, lane, work)
    OnPanic:         func(w ingestWork, _ any) { _ = w.msg.Nak() },
    MetricsRegistry: registry, // optional
    Logger:          logger,   // optional
})
err = pool.SubmitBlocking(submitCtx, work.entityID, work)

SubmitBlocking applies backpressure: when the item's lane queue is full, the call waits until the lane has room or its context ends.

Contexts and stopping

The context given to NewKeyedPool reaches only Process. Ending it stops nothing: the lanes keep running, and items queued or accepted afterwards still reach Process, which sees the ended context. Ending it is the owner's way to make the remaining Process calls finish fast.

Shutdown(ctx) is the only stop. A SubmitBlocking call that begins after the first Shutdown returns ErrStopped; one already waiting on a full lane may still be accepted, and is then processed. Every lane finishes its queue and exits, and Shutdown returns nil once they have, even if ctx has ended by then. If ctx ends first, Shutdown returns ctx.Err() and the lanes keep draining; call Shutdown again to wait for them. Shutdown has no timeout of its own: a context with no deadline waits as long as Process takes. A nil context is refused with an error. A pool nobody shuts down keeps its lanes until the process ends.

Wait() returns once every lane has exited, and takes no context. Lanes exit only after Shutdown has begun and every accepted item has been processed, so a Wait called before Shutdown blocks until then, and a Wait called after the lanes exited returns at once. It is the join after a Shutdown whose context had already ended: Shutdown begins the drain and returns ctx.Err(), and Wait waits for it. Wait has no bound: if nobody calls Shutdown it blocks forever.

A producer parked in SubmitBlocking on a full lane holds Shutdown up until the lane takes its item. On shutdown, cancel the context passed to SubmitBlocking first, then call Shutdown.

The pool runs one goroutine per lane, plus one that the first Shutdown starts to close the queues and join the lanes. Shutdown returning nil means all of them have exited.

Metrics

With KeyedConfig.MetricsRegistry set, the pool registers six collectors labelled pool=<Name> through metric.RegisterOrGet: dispatch_queue_wait_seconds, dispatch_processing_duration_seconds, dispatch_queue_depth, dispatch_inflight, dispatch_submitted_total and dispatch_completed_total. Two pools with the same Name on one registry share these collectors. dispatch_queue_depth counts the items accepted and not yet passed to Process: it rises when SubmitBlocking accepts an item and falls when a lane picks it up, so it sums the waiting items of every pool of that Name. A registration the registry refuses makes NewKeyedPool return the error. With no registry the pool keeps no metrics.

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrInvalidConfig is returned by NewKeyedPool for a nil context or
	// a KeyedConfig that is incomplete or out of range (Lanes <= 0,
	// QueueDepth <= 0, a missing Process). Catches wiring bugs at
	// construction time.
	ErrInvalidConfig = errors.New("dispatch: invalid config")

	// ErrStopped is returned by a KeyedPool.SubmitBlocking call that
	// begins after Shutdown has begun.
	ErrStopped = errors.New("dispatch: pool stopped")
)

Package error sentinels. Callers compare with errors.Is.

Functions

This section is empty.

Types

type KeyedConfig

type KeyedConfig[W any] struct {
	// Lanes is the number of parallel serial lanes. Same-key work is
	// serialized within a lane; different keys spread across lanes.
	// Must be > 0. Fixed for the pool's lifetime (lane assignment is a
	// pure function of the key over this count).
	Lanes int

	// QueueDepth bounds each lane's queue. SubmitBlocking waits while
	// the target lane is at capacity. Must be > 0.
	QueueDepth int

	// Process handles one work item. It is called by the assigned
	// lane's single goroutine, with that lane's index — a composer can
	// use the index to shard per-lane state without locking (the pool
	// guarantees at most one goroutine per lane). Must be non-nil. The
	// pool reports nothing about the outcome: Process disposes of its own
	// failures (logs them, settles the message it came from).
	Process func(ctx context.Context, lane int, work W)

	// OnPanic — optional. When Process panics, the pool recovers it,
	// keeps the lane goroutine alive, and (if set) calls OnPanic with
	// the work item and the recovered value so the composer can
	// dispose of it (e.g. Nak the underlying message). Called from the
	// lane goroutine. If nil, a recovered panic is logged and dropped.
	OnPanic func(work W, recovered any)

	// Name labels this pool's metrics (the `pool` label). Optional;
	// the pool has metrics only when MetricsRegistry is set.
	Name string

	// MetricsRegistry — optional. When set, the pool registers its
	// metrics on it; when nil, the pool keeps no metrics.
	MetricsRegistry *metric.MetricsRegistry

	// Logger — optional; nil means slog.Default.
	Logger *slog.Logger
}

KeyedConfig parameterizes KeyedPool construction. The zero value is NOT valid — NewKeyedPool rejects Lanes <= 0, QueueDepth <= 0, and a missing Process.

type KeyedPool

type KeyedPool[W any] struct {
	// contains filtered or unexported fields
}

KeyedPool is the framework's keyed-ordered bounded-concurrency primitive: it partitions work into N lanes by the key each SubmitBlocking call names. All items sharing a key route to the same lane and are processed serially in submit order, while items with different keys spread across lanes and run concurrently.

It exists for at-least-once consumers that need per-key ordered concurrency — e.g. graph-ingest, whose arrival-order (full-set- replace) merge requires that all messages for one entity apply in order, but which is otherwise latency-bound on a serial Get+CAS chain (ADR-072).

Structure: N separate bounded lane channels, one goroutine per lane. A shared channel read by N workers cannot preserve per-key order, which is why each lane has its own channel and goroutine.

Concurrency: SubmitBlocking and Shutdown are safe to call from concurrent goroutines. The Process function for a given lane is only ever invoked by that lane's single goroutine, so a composer maintaining per-lane state (indexed by the lane number passed into Process) needs no locking on it.

func NewKeyedPool

func NewKeyedPool[W any](ctx context.Context, cfg KeyedConfig[W]) (*KeyedPool[W], error)

NewKeyedPool constructs and starts a KeyedPool. Lane goroutines are running and ready to receive SubmitBlocking calls when it returns — callers don't call a separate Start.

The ctx is handed to every Process call and read nowhere else: its end stops no lane, refuses no submit and drops no queued item, so items processed after it ended see the ended ctx. Shutdown is the only stop; the lanes run until it is called. The ctx must be non-nil.

Returns ErrInvalidConfig for a nil ctx or a missing or out-of-range required field, and the registry's refusal when one of the pool's metrics cannot be registered; either way no goroutine is started.

func (*KeyedPool[W]) Shutdown

func (p *KeyedPool[W]) Shutdown(ctx context.Context) error

Shutdown is the pool's only stop: it stops accepting new work (a SubmitBlocking call that begins after the first Shutdown returns ErrStopped; one already waiting on a full lane may still be accepted, and is processed), lets each lane process its queued items, and returns nil once every lane has exited, also when ctx has ended by then. If ctx ends first it returns ctx.Err(); the lanes keep processing and exit once their queues are empty, and a later Shutdown returns nil. Given a ctx with no deadline, Shutdown waits for the join however long it takes: it has no wait of its own. A nil ctx is refused with an error and begins no shutdown.

Shutdown does not end the context NewKeyedPool was given; ending it is the owner's, and makes the remaining Process calls see an ended context.

func (*KeyedPool[W]) SubmitBlocking

func (p *KeyedPool[W]) SubmitBlocking(ctx context.Context, key string, work W) error

SubmitBlocking routes work to the lane its key maps to, blocking until the lane has capacity or the ctx ends (returns ctx.Err()). Items submitted with equal keys are processed one at a time, in submit order. A call that begins after Shutdown has begun returns ErrStopped; a call already waiting on a full lane when Shutdown begins may still be accepted, and is then processed before Shutdown returns nil. This is the backpressure form — a full lane blocks the caller rather than dropping.

Composer note (ADR-072 M3): Shutdown waits for a call already parked on a full lane, and that call returns only when the lane takes its item or its ctx ends. On shutdown, cancel the ctx passed here BEFORE calling Shutdown, so a synchronous producer (e.g. a NATS consume callback) parked here unblocks and can dispose of its message.

func (*KeyedPool[W]) Wait

func (p *KeyedPool[W]) Wait()

Wait returns once every lane has exited. Lanes exit only after Shutdown has begun and every accepted item has been processed, so a Wait called before Shutdown blocks until a later Shutdown drains the pool, and a Wait called after the lanes exited returns at once. It takes no context and has no bound: like sync.WaitGroup.Wait, it blocks forever if nobody calls Shutdown. It is the join for a caller whose Shutdown(ctx) returned ctx.Err() because ctx had already ended.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL