idempotency

package
v3.0.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 19, 2026 License: Apache-2.0 Imports: 19 Imported by: 0

Documentation

Overview

Package idempotency provides stable HTTP idempotency middleware. The middleware buffers responses before replay/storage and is therefore not suitable for streaming, hijacking, HTTP/2 push, or other handlers that rely on optional http.ResponseWriter interfaces. If a completed response cannot be persisted for replay, the middleware fails closed with 503 and stores an ambiguous state for that key instead of reopening it for another execution. Buffered responses that exceed Options.MaxResponseBytes follow the same ambiguous-outcome path. The default request hash includes authenticated actor and tenant scope when earlier middleware has populated them in request context. For multi-tenant APIs backed by shared idempotency storage, set Options.StorageKeyFunc to TenantScopedStorageKeyFunc() after auth and tenant middleware so the backing store receives a stable hashed key scoped by tenant and actor while replay responses still return the original client key. Set Options.RequireKey when unsafe write route contracts require an Idempotency-Key; when enabled, handled requests without a key fail with a Problem Details 400 instead of falling through to the handler.

Token-aware release helpers:

  • Store implementations must implement ports.IdempotencyReservationReleaser so middleware releases only the current tokened in-flight reservation. Maintained stores pass the token-aware adapter contract tests for token mismatch, missing-token legacy cleanup, completed record preservation, and ambiguous record preservation.
  • OnLegacyInFlightCompatibility or LegacyInFlightCompatibilitySink emits additive mixed-version recovery telemetry with method/path/store correlation for both fallback attempts and outcomes.
  • By default, OnLegacyInFlightCompatibility is a logger sink when unset, so fallback telemetry is always emitted with low-cardinality fields and stable key redaction behavior.
  • Legacy compatibility key values are hashed by default. Set LegacyInFlightCompatibilityRawKey=true to opt in to raw key emission.
  • LegacyInFlightCompatibilityAsync disables request-path coupling to callback execution in high-volume telemetry windows by using one bounded queue and four workers per middleware instance. The queue holds 1024 events, drops new events when full, emits a warning with dropped_events/queue_size, and drains best-effort while the process is running, and emits queued events with cancellation stripped from the first enqueue context so request cancellation does not suppress telemetry delivery. There is no request-path flush or shutdown wait; use a synchronous sink if every telemetry event must be durably observed.
  • LegacyInFlightCompatibilitySampleEvery emits one event per N emitted events and is the preferred low-cost throttle for high-volume mixed-version windows.
  • Logger and KnownInFlightTTLs can be used to run startup checks for mixed- version InFlightTTL alignment. Set FailOnInFlightTTLMismatch to fail-fast when rollout rules require strict enforcement.
  • FailOnInFlightClockSkewPreflight enables strict startup governance for clock-skew sensitive startup preflights while preserving advisory mode by default.
  • Use LegacyInFlightCompatibilityMetricSink for metrics-first consumers; event labels are exposed through MetricLabels() and use the stable bounded schema: method, store_class, and outcome.
  • Use OnOutcome for normal request-path idempotency telemetry. OutcomeEvent intentionally omits paths, keys, tenants, request IDs, body data, and raw error strings; MetricLabels() exposes method, store_class, outcome, and status_class.

Recommended event contract:

- `legacy_in_flight_fallback_entered` - `legacy_in_flight_fallback_recovered` - `legacy_in_flight_fallback_rejected` - `legacy_in_flight_fallback_unknown`

Each event should include method/path/store_type/outcome plus an optional key and error payload when failures occur.

Practical guidance:

  • Keep event key hashed by default to limit cardinality.
  • Set LegacyInFlightCompatibilitySampleEvery when compatibility traffic becomes noisy during mixed-version windows to lower event volume deterministically before the bounded async queue sees the event.
  • Prefer an explicit metric sink for dashboard counters and keep logger sink in place during transition.
  • If you rely on request latency, avoid heavy synchronous callback work unless LegacyInFlightCompatibilityAsync is enabled.
  • Treat async compatibility telemetry as lossy operational evidence. It must never be the only source of correctness for idempotency recovery decisions, and raw keys should stay disabled in production unless a short, access- controlled incident review requires them.

Index

Constants

This section is empty.

Variables

View Source
var (
	ErrLegacyInFlightTTLMismatch            = errors.New("idempotency in-flight ttl mismatch in rollout contract")
	ErrLegacyInFlightClockSkewPreflightRisk = errors.New("idempotency legacy in-flight clock preflight risk")
)

Functions

func DefaultHash

func DefaultHash(r *http.Request, body []byte) (string, error)

DefaultHash returns a stable SHA-256 hash of caller scope, method, path, query, content type, and body. Authenticated actor and tenant context are included when present.

Types

type HashFunc

type HashFunc func(*http.Request, []byte) (string, error)

HashFunc computes a request hash used to detect key reuse with different payloads.

type KeyFunc

type KeyFunc func(*http.Request) string

KeyFunc extracts an idempotency key from the request.

type LegacyInFlightCompatibilityEvent

type LegacyInFlightCompatibilityEvent struct {
	Method    string
	Path      string
	Key       string
	StoreType string
	Outcome   LegacyInFlightCompatibilityEventName
	Error     string
}

LegacyInFlightCompatibilityEvent contains structured fields for idempotency compatibility telemetry.

func (LegacyInFlightCompatibilityEvent) MetricLabels

func (event LegacyInFlightCompatibilityEvent) MetricLabels() map[string]string

MetricLabels returns the canonical metric label set for compatibility telemetry. Metrics intentionally expose only bounded labels. High-cardinality event details such as raw paths, key hashes, raw keys, and error strings remain on LegacyInFlightCompatibilityEvent for structured logs or traces.

type LegacyInFlightCompatibilityEventName

type LegacyInFlightCompatibilityEventName string

LegacyInFlightCompatibilityEventName identifies structured compatibility events for mixed-version in-flight migrations.

const (
	// LegacyInFlightCompatibilityEntered reports that legacy fallback was considered.
	LegacyInFlightCompatibilityEntered LegacyInFlightCompatibilityEventName = "legacy_in_flight_fallback_entered"
	// LegacyInFlightCompatibilityRecovered reports legacy fallback was successful.
	LegacyInFlightCompatibilityRecovered LegacyInFlightCompatibilityEventName = "legacy_in_flight_fallback_recovered"
	// LegacyInFlightCompatibilityRejected reports fallback was explicitly rejected.
	LegacyInFlightCompatibilityRejected LegacyInFlightCompatibilityEventName = "legacy_in_flight_fallback_rejected"
	// LegacyInFlightCompatibilityUnknown reports fallback ended with an unexpected error.
	LegacyInFlightCompatibilityUnknown LegacyInFlightCompatibilityEventName = "legacy_in_flight_fallback_unknown"
)

type LegacyInFlightCompatibilityEventSink

type LegacyInFlightCompatibilityEventSink interface {
	Emit(context.Context, LegacyInFlightCompatibilityEvent)
}

LegacyInFlightCompatibilityEventSink emits telemetry from legacy compatibility events.

Implementations should avoid blocking and avoid panicking, since compatibility telemetry must not alter request behavior.

type LegacyInFlightCompatibilityHandler

type LegacyInFlightCompatibilityHandler func(context.Context, LegacyInFlightCompatibilityEvent)

LegacyInFlightCompatibilityHandler receives telemetry from legacy recovery paths.

type LegacyInFlightCompatibilityMetricLabels

type LegacyInFlightCompatibilityMetricLabels map[string]string

LegacyInFlightCompatibilityMetricLabels is the canonical compatibility label set for metric adapters.

type LegacyInFlightCompatibilityMetricSink

type LegacyInFlightCompatibilityMetricSink interface {
	Emit(context.Context, LegacyInFlightCompatibilityMetricLabels)
}

LegacyInFlightCompatibilityMetricSink emits structured compatibility metrics.

Implementations should avoid blocking and avoid panicking, since compatibility telemetry must not alter request behavior.

type LegacyInFlightCompatibilityMetricSinkFunc

type LegacyInFlightCompatibilityMetricSinkFunc func(context.Context, LegacyInFlightCompatibilityMetricLabels)

LegacyInFlightCompatibilityMetricSinkFunc adapts a function to LegacyInFlightCompatibilityMetricSink.

func (LegacyInFlightCompatibilityMetricSinkFunc) Emit

type LegacyInFlightCompatibilitySinkFunc

type LegacyInFlightCompatibilitySinkFunc func(context.Context, LegacyInFlightCompatibilityEvent)

LegacyInFlightCompatibilitySinkFunc adapts a function to LegacyInFlightCompatibilityEventSink.

func (LegacyInFlightCompatibilitySinkFunc) Emit

type Middleware

type Middleware struct {
	// contains filtered or unexported fields
}

Middleware enforces Idempotency-Key semantics.

func New

func New(opts Options) (*Middleware, error)

New constructs an idempotency middleware.

func (*Middleware) Handler

func (m *Middleware) Handler(next http.Handler) http.Handler

Handler wraps the next handler with idempotency logic.

func (*Middleware) Middleware

func (m *Middleware) Middleware() func(http.Handler) http.Handler

Middleware implements ports.Middleware via Handler adapter.

type Options

type Options struct {
	Store            ports.IdempotencyStore
	HeaderName       string
	KeyFunc          KeyFunc
	StorageKeyFunc   StorageKeyFunc
	HashFunc         HashFunc
	TTL              time.Duration
	InFlightTTL      time.Duration
	MaxBodyBytes     int64
	MaxResponseBytes int64
	// RequireKey rejects handled unsafe requests that omit the idempotency key.
	// The default false preserves v2 pass-through behavior for existing callers.
	RequireKey          bool
	Clock               ports.Clock
	ShouldHandle        func(*http.Request) bool
	ShouldStore         func(status int) bool
	ResponseHeaderAllow []string
	ResponseHeaderDeny  []string
	ReplayHeaderName    string
	FailOpen            bool
	OnError             func(error)
	OnOutcome           OutcomeHandler
	Logger              ports.Logger
	// KnownInFlightTTLs maps discovered peers to their observed in-flight TTL.
	KnownInFlightTTLs map[string]time.Duration
	// FailOnInFlightTTLMismatch enables hard-fail on TTL mismatch during startup.
	FailOnInFlightTTLMismatch bool
	// FailOnInFlightClockSkewPreflight promotes startup clock-skew risk checks into hard-fail.
	FailOnInFlightClockSkewPreflight bool
	// LegacyInFlightCompatibilityRawKey exposes the raw request key in compatibility events.
	// Defaults to false (hashed key output). Keep this disabled in production
	// unless a short, access-controlled incident review needs exact keys.
	LegacyInFlightCompatibilityRawKey bool
	// LegacyInFlightCompatibilitySink emits compatibility telemetry independently of the
	// legacy callback contract. Use this for logging adapters or custom integrations.
	LegacyInFlightCompatibilitySink LegacyInFlightCompatibilityEventSink
	// LegacyInFlightCompatibilityMetricSink emits metric label sets for compatibility events.
	// Use this for Prometheus/observability pipelines that prefer canonical label contracts.
	LegacyInFlightCompatibilityMetricSink LegacyInFlightCompatibilityMetricSink
	// LegacyInFlightCompatibilityAsync avoids blocking request execution for telemetry
	// work by enqueueing events to a bounded worker. When the queue is full,
	// events are dropped and a warning is emitted; request execution is not
	// backpressured.
	LegacyInFlightCompatibilityAsync bool
	// LegacyInFlightCompatibilitySampleEvery emits one event per N emitted events for
	// high-volume environments. Values <= 1 preserve full event output. Use this
	// to reduce async queue pressure and metric/log volume. Cardinality remains
	// bounded by key hashing defaults unless RawKey is enabled.
	LegacyInFlightCompatibilitySampleEvery int
	// OnLegacyInFlightCompatibility receives additive mixed-version telemetry.
	OnLegacyInFlightCompatibility LegacyInFlightCompatibilityHandler
}

Options configures the idempotency middleware.

type OutcomeEvent

type OutcomeEvent struct {
	Method    string
	Status    int
	StoreType string
	Outcome   OutcomeEventName
	FailOpen  bool
}

OutcomeEvent contains bounded idempotency outcome fields suitable for logs or metric labels. It intentionally omits paths, keys, tenants, request IDs, body data, and raw error strings.

func (OutcomeEvent) MetricLabels

func (event OutcomeEvent) MetricLabels() map[string]string

MetricLabels returns the canonical low-cardinality label set for idempotency outcome metrics.

type OutcomeEventName

type OutcomeEventName string

OutcomeEventName identifies the low-cardinality outcome of an idempotency decision.

const (
	IdempotencyOutcomeMissingKey             OutcomeEventName = "missing_key"
	IdempotencyOutcomeInvalidRequest         OutcomeEventName = "invalid_request"
	IdempotencyOutcomeLookupFailed           OutcomeEventName = "lookup_failed"
	IdempotencyOutcomeFailOpen               OutcomeEventName = "fail_open"
	IdempotencyOutcomeConflict               OutcomeEventName = "conflict"
	IdempotencyOutcomeReplayed               OutcomeEventName = "replayed"
	IdempotencyOutcomeInFlight               OutcomeEventName = "in_flight"
	IdempotencyOutcomeAmbiguous              OutcomeEventName = "ambiguous"
	IdempotencyOutcomeReservationFailed      OutcomeEventName = "reservation_failed"
	IdempotencyOutcomeReservationUnavailable OutcomeEventName = "reservation_unavailable"
	IdempotencyOutcomeCompletedStored        OutcomeEventName = "completed_stored"
	IdempotencyOutcomeCompletedReleased      OutcomeEventName = "completed_released"
	IdempotencyOutcomeResponseTooLarge       OutcomeEventName = "response_too_large"
	IdempotencyOutcomePersistenceFailed      OutcomeEventName = "persistence_failed"
)

type OutcomeHandler

type OutcomeHandler func(context.Context, OutcomeEvent)

OutcomeHandler receives bounded idempotency outcome events.

type StorageKeyFunc

type StorageKeyFunc func(*http.Request, string) string

StorageKeyFunc maps a client-supplied idempotency key to the key used by the backing store. Implementations should keep the result stable for replay and avoid embedding raw tenant IDs, user IDs, or client keys.

func TenantScopedStorageKeyFunc

func TenantScopedStorageKeyFunc() StorageKeyFunc

TenantScopedStorageKeyFunc returns a StorageKeyFunc that hashes the client idempotency key with authenticated tenant and actor scope before it reaches the backing store. Use it after auth and tenant middleware have populated the request context. The returned store key is stable and intentionally does not include raw tenant IDs, user IDs, or client-supplied keys.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL