projectors

package
v0.8.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 13, 2026 License: Apache-2.0 Imports: 12 Imported by: 0

Documentation

Overview

Package projectors provides an asynchronous runtime for committed-event read models, with idempotency, retry classification, and telemetry.

Projectors normally consume publisher SNS fan-out through raw-delivery SNS-to-SQS. NewSQSHandler and NewSNSHandler decode CloudWatchEvent envelopes into StreamRecords. NewLambdaHandler supports direct DynamoDB Streams consumption.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewLambdaHandler

func NewLambdaHandler(runtime *Runtime) func(ctx context.Context, event events.DynamoDBEvent) (DynamoDBStreamResponse, error)

NewLambdaHandler returns a function suitable for lambda.Start() that processes DynamoDB Streams events directly through the provided Runtime.

This wires a projector directly to the event-log stream, bypassing SNS fan-out. Use it when the projector must consume DynamoDB Streams directly.

func NewSNSHandler added in v0.7.0

func NewSNSHandler(runtime *Runtime) func(ctx context.Context, event events.SNSEvent) error

NewSNSHandler returns a lambda.Start-compatible handler for a direct SNS->Lambda subscription to the events topic. Prefer NewSQSHandler for most projectors; reach for a direct subscription only for latency-sensitive consumers that can tolerate SNS's at-least-once, whole-invocation retries.

SNS->Lambda has no partial-batch-failure protocol, so returning an error retries (and ultimately dead-letters) the whole invocation. Configure a redrive policy on the subscription for failures.

func NewSQSHandler added in v0.7.0

func NewSQSHandler(runtime *Runtime) func(ctx context.Context, event events.SQSEvent) (events.SQSEventResponse, error)

NewSQSHandler returns a lambda.Start-compatible handler for raw-delivery SNS-to-SQS messages from the events topic. It processes the messages through the Runtime and reports partial batch failures.

It reports partial batch failures, so only messages whose events fail to project are redelivered (enable ReportBatchItemFailures on the event source mapping). A message whose body cannot be decoded is dropped and logged rather than retried — a malformed envelope will never succeed.

func ToSerializedEvent

func ToSerializedEvent(rec StreamRecord) evt.SerializedEvent

ToSerializedEvent converts a DynamoDB stream record shape into an evt SerializedEvent. Invalid metadata is treated as empty metadata so projection code can still process the event payload.

Types

type BatchItemFailure

type BatchItemFailure struct {
	ItemIdentifier string `json:"itemIdentifier"`
}

BatchItemFailure identifies a single record that Lambda should retry. The ItemIdentifier corresponds to the DynamoDB stream record's EventID.

type DynamoDBStreamResponse

type DynamoDBStreamResponse struct {
	BatchItemFailures []BatchItemFailure `json:"batchItemFailures"`
}

DynamoDBStreamResponse is the Lambda response for DynamoDB Streams event sources with partial batch failure reporting enabled.

type IdempotencyGuard

type IdempotencyGuard interface {
	// IsProcessed returns true if the event was already successfully processed
	// by the named projector.
	IsProcessed(ctx context.Context, projectorName string, eventID string) (bool, error)

	// MarkProcessed records that the named projector successfully processed the event.
	MarkProcessed(ctx context.Context, projectorName string, eventID string) error
}

IdempotencyGuard tracks which events have been successfully processed by a given projector, preventing duplicate application during retries or replays.

type InMemoryIdempotencyGuard

type InMemoryIdempotencyGuard struct {
	// contains filtered or unexported fields
}

InMemoryIdempotencyGuard is a thread-safe in-memory implementation of IdempotencyGuard, suitable for testing and single-invocation Lambda contexts.

func NewInMemoryIdempotencyGuard

func NewInMemoryIdempotencyGuard() *InMemoryIdempotencyGuard

NewInMemoryIdempotencyGuard creates a new empty in-memory guard.

func (*InMemoryIdempotencyGuard) IsProcessed

func (g *InMemoryIdempotencyGuard) IsProcessed(_ context.Context, projectorName string, eventID string) (bool, error)

IsProcessed checks if an event was already processed.

func (*InMemoryIdempotencyGuard) MarkProcessed

func (g *InMemoryIdempotencyGuard) MarkProcessed(_ context.Context, projectorName string, eventID string) error

MarkProcessed records successful processing of an event.

func (*InMemoryIdempotencyGuard) Reset

func (g *InMemoryIdempotencyGuard) Reset()

Reset clears all recorded processing state. Useful in tests and replay scenarios.

type PermanentError

type PermanentError struct {
	Err error
}

PermanentError wraps an error to indicate it should never be retried. Projectors can return this to force DLQ routing.

func NewPermanentError

func NewPermanentError(err error) *PermanentError

NewPermanentError wraps err so that ClassifyError returns RetryPermanent.

func (*PermanentError) Error

func (e *PermanentError) Error() string

func (*PermanentError) Unwrap

func (e *PermanentError) Unwrap() error

type Projector

type Projector interface {
	// Process handles a batch of stream records, returning identifiers of records
	// that should be retried (partial batch failure).
	Process(ctx context.Context, records []StreamRecord) ([]BatchItemFailure, error)

	// Name returns a stable identifier for this projector, used in idempotency
	// keys and telemetry labels.
	Name() string
}

Projector processes committed events and maintains read models. Records are normally delivered via the SNS fan-out (see the package doc), but a Projector is transport-agnostic — it just receives StreamRecord batches.

type RetryClassification

type RetryClassification string

RetryClassification categorizes an error for retry vs DLQ routing.

const (
	// RetryTransient indicates a temporary failure that should be retried.
	RetryTransient RetryClassification = "transient"

	// RetryPermanent indicates a non-recoverable failure that should be sent to the DLQ.
	RetryPermanent RetryClassification = "permanent"

	// RetryUnknown indicates an unclassified error that should be retried with backoff.
	RetryUnknown RetryClassification = "unknown"
)

func ClassifyError

func ClassifyError(err error) RetryClassification

ClassifyError inspects err and returns a retry classification. Transient errors (network, context deadline, temporary I/O) are retried. PermanentErrors are routed to the DLQ. Everything else is classified as unknown (retry with backoff).

type Runtime

type Runtime struct {
	// contains filtered or unexported fields
}

Runtime wraps a Projector with idempotency checking, retry classification, and structured logging/telemetry.

func NewRuntime

func NewRuntime(projector Projector, idempotency IdempotencyGuard, logger *slog.Logger) *Runtime

NewRuntime creates a Runtime that decorates the given projector with the provided idempotency guard and logger.

func (*Runtime) Logger

func (r *Runtime) Logger() *slog.Logger

Logger returns the runtime's logger, falling back to slog.Default() when unset.

func (*Runtime) Process

func (r *Runtime) Process(ctx context.Context, records []StreamRecord) ([]BatchItemFailure, error)

Process runs the projector over the given records, skipping already-processed events (idempotency) and classifying errors for retry vs DLQ routing.

On a batch-level projector error, the error is classified:

  • Permanent: the batch's records are reported as partial-batch failures ([]BatchItemFailure). Lambda retries them up to the event source mapping's configured limit and then routes them to its on-failure destination (DLQ); there is no handler-initiated "send straight to DLQ" in Lambda.
  • Transient or unknown: the error is returned so Lambda fails and retries the whole invocation.

Per-record failures returned by the projector itself (with a nil error) are passed through as partial-batch failures and the successful records are marked processed for idempotency.

type StreamRecord

type StreamRecord struct {
	EventID                     string
	EntityID                    string
	EntityType                  string
	EventType                   string
	Version                     int
	Sequence                    int
	Payload                     []byte
	Metadata                    json.RawMessage
	ApproximateCreationDateTime time.Time
}

StreamRecord is one event in a domain-friendly shape, ready for a Projector. It is decoded from whichever transport delivered it — the SNS/SQS envelope via StreamRecordFromEnvelope, or a raw DynamoDB stream record.

func FromSerializedEvent added in v0.7.0

func FromSerializedEvent(event evt.SerializedEvent) StreamRecord

FromSerializedEvent converts a decoded domain event into a StreamRecord. It is the consumer-side inverse of ToSerializedEvent and underpins the SNS/SQS handlers below.

func StreamRecordFromEnvelope added in v0.7.0

func StreamRecordFromEnvelope(body []byte) (StreamRecord, error)

StreamRecordFromEnvelope decodes one CloudWatchEvent envelope — the message shape stream.SNSPublisher writes to the events topic — into a StreamRecord. The envelope's Detail field carries the serialized domain event.

Pass the raw message body: the SQS message body when the SNS->SQS subscription uses raw message delivery, or record.SNS.Message for a direct SNS->Lambda subscription. Both handlers below call this for you.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL