Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type DLQConfig ¶
type DLQConfig struct {
// Enabled enables dead letter queue.
Enabled bool
// TopicSuffix is appended to the original topic name to create the DLQ topic.
// For example, if original topic is "orders" and suffix is "_dlq", DLQ topic will be "orders_dlq".
TopicSuffix string
}
DLQConfig configures dead letter queue behavior.
type Delivery ¶
type Delivery interface {
// Message returns the delivered message.
Message() entityqueue.Message
// Ack acknowledges successful processing of the message.
// The message will be removed from the queue and not redelivered.
Ack(ctx context.Context) error
// Nack negatively acknowledges the message, indicating processing failure.
// The message is requeued for redelivery immediately; the visibility
// timeout is what spaces retries (a crash or missed ack redelivers on the
// same schedule). The redelivery counts toward the failure budget.
//
// f describes the failure. It is carried so that the redelivery which
// finally exhausts the budget can dead-letter with the reason that caused
// it, rather than with a generic one — a nack whose reason is dropped
// leaves the eventual dead letter unable to say what went wrong.
Nack(ctx context.Context, f failure.Failure) error
// Postpone finishes this delivery as "processed successfully, redeliver
// later": the message becomes invisible for delayMs and acts as a barrier —
// its partition is not consumed past it until it redelivers, in order.
// Unlike Nack, the redelivery does not count against the failure budget
// (retry limit / DLQ); postponing resets the failure streak.
// Postpone is terminal for this delivery, like Ack/Nack/Reject.
Postpone(ctx context.Context, delayMs int64) error
// Reject moves the message to the dead letter entityqueue.
// Use for poison pill messages that should never be retried.
// f is recorded with the dead-lettered message for diagnosis and is what
// Failure returns when it is redelivered from the DLQ.
// If DLQ is not configured, the message is acked (removed from queue).
Reject(ctx context.Context, f failure.Failure) error
// ExtendVisibilityTimeout extends the time before this message becomes
// visible to other consumers. Use when processing takes longer than expected.
ExtendVisibilityTimeout(ctx context.Context, durationMillis int64) error
// DeliveryID returns a backend-specific identifier for this delivery.
DeliveryID() string
// Attempt returns how many times this message has been delivered.
// Starts at 1 for first delivery.
Attempt() int
// ReceivedAt returns when this delivery was received (Unix milliseconds).
ReceivedAt() int64
// Metadata returns backend-specific delivery metadata.
Metadata() map[string]string
// Failure returns why this message was dead-lettered, and whether it was
// dead-lettered at all. It reports false for a message delivered from its
// original topic, so a DLQ consumer can distinguish "no failure recorded"
// from a failure that recorded nothing.
Failure() (failure.Failure, bool)
}
Delivery represents a message delivered by a Subscriber. Provides access to the message and methods to acknowledge or reject it.
Implementations must be safe for concurrent Message() calls. Ack/Nack/ExtendVisibilityTimeout should not be called concurrently on the same instance.
type Publisher ¶
type Publisher interface {
// Publish sends a message to the specified topic.
Publish(ctx context.Context, topic string, message entityqueue.Message) error
// Close gracefully shuts down the publisher, flushing pending messages.
Close() error
}
Publisher publishes messages to topics. Implementations must be thread-safe.
type Queue ¶
type Queue interface {
// Publisher returns a Publisher instance.
// May return a singleton or new instance depending on implementation.
Publisher() Publisher
// Subscriber returns a Subscriber instance.
// May return a singleton or new instance depending on implementation.
Subscriber() Subscriber
// Close shuts down the queue and all associated resources.
Close() error
}
Queue creates and manages queue publishers and subscribers. Implementations handle connection pooling, consumer group configuration, and resource lifecycle.
type RetryConfig ¶
type RetryConfig struct {
// MaxAttempts is the maximum number of processing attempts.
// After this many attempts, the message is moved to DLQ (if enabled).
MaxAttempts int
// InitialBackoffMs is the initial backoff duration for retries (in milliseconds).
InitialBackoffMs int64
// MaxBackoffMs is the maximum backoff duration (in milliseconds).
MaxBackoffMs int64
// BackoffMultiplier is the multiplier for exponential backoff.
BackoffMultiplier float64
}
RetryConfig configures message retry behavior.
type Subscriber ¶
type Subscriber interface {
// Subscribe starts consuming messages from the specified topic with the given config.
// Returns a channel of Delivery instances and an error if subscription fails.
//
// Each subscription can have its own configuration for polling, batching,
// retries, and dead letter queue behavior.
//
// The channel is closed when the subscriber is closed or context is cancelled.
// Implementations should handle infrastructure errors internally (e.g., reconnect).
//
// Each Delivery provides the message and methods to acknowledge or reject it.
// Consumers should call delivery.Ack() or delivery.Nack() for each delivery.
Subscribe(ctx context.Context, topic string, config SubscriptionConfig) (<-chan Delivery, error)
// Close gracefully shuts down the subscriber.
// All delivery channels will be closed.
// Idempotent - safe to call multiple times.
Close() error
}
Subscriber consumes messages from topics. Implementations must be thread-safe.
type SubscriptionConfig ¶
type SubscriptionConfig struct {
// SubscriberName uniquely identifies this subscriber instance for partition leases.
// Different workers should use different names (e.g., hostname, pod name, UUID).
// Combined with ConsumerGroup, this determines which worker owns a partition lease.
SubscriberName string
// ConsumerGroup identifies this consumer for offset tracking.
// Different consumer groups maintain independent offsets.
ConsumerGroup string
// PollIntervalMs is how often to poll for new messages (in milliseconds).
PollIntervalMs int64
// PartitionDiscoveryIntervalMs is how often to discover partitions,
// attempt lease acquisition, and reconcile partition workers (in
// milliseconds). Separate from PollIntervalMs: message polling needs low
// latency, while discovery drives topic-wide queries whose volume
// multiplies with subscribers and topics and whose outcome only changes
// on membership or partition changes.
PartitionDiscoveryIntervalMs int64
// BatchSize is the maximum number of messages to fetch per poll.
BatchSize int
// VisibilityTimeoutMs is how long a message is invisible after being fetched (in milliseconds).
// If the worker crashes or doesn't ack/nack in time, the message becomes
// visible again after this duration.
VisibilityTimeoutMs int64
// LeaseRenewalIntervalMs is how often to renew partition leases (in milliseconds).
LeaseRenewalIntervalMs int64
// LeaseDurationMs is how long a lease is valid without renewal (in milliseconds).
// Stale leases (not renewed within this duration) can be stolen by other workers.
LeaseDurationMs int64
// Retry configures message retry behavior.
Retry RetryConfig
// DLQ configures dead letter queue behavior.
DLQ DLQConfig
}
SubscriptionConfig holds per-subscription configuration. Each subscription (topic) can have its own settings for polling, batching, retries, and dead letter queue behavior.
func DLQSubscriptionConfig ¶
func DLQSubscriptionConfig(subscriberName, consumerGroup string) SubscriptionConfig
DLQSubscriptionConfig returns a SubscriptionConfig for consuming a dead-letter topic (DLQ reconciliation). It starts from DefaultSubscriptionConfig and applies the two overrides every DLQ consumer needs:
- DLQ.Enabled is false, so a reconciliation failure retries in place instead of cascading to a second-level "_dlq_dlq" topic that nobody consumes.
- Retry.MaxAttempts is a very high backstop so the per-message retry budget effectively never runs out. This pairs with errs.AlwaysRetryableProcessor wired into the DLQ consumer: reconciliation converges eventually instead of being silently dropped after the default retry count.
func DefaultSubscriptionConfig ¶
func DefaultSubscriptionConfig(subscriberName, consumerGroup string) SubscriptionConfig
DefaultSubscriptionConfig returns a SubscriptionConfig with sensible defaults.