Documentation
¶
Overview ¶
Package idempotency provides HTTP-layer idempotency primitives for GoCell.
It is a minimal specialization of kernel/idempotency tailored to the HTTP request/response lifecycle: ClaimDone carries a replayed response blob (RecordedResponse) rather than a bare "already processed" signal, and the HTTP-specific Receipt.Record method commits the response alongside the done state.
Relation to kernel/idempotency:
- ClaimState (ClaimAcquired / ClaimDone / ClaimBusy) is REUSED from kernel/idempotency. Do NOT clone the enum.
- DefaultTTL (24h) and DefaultLeaseTTL (5min) constants come from kernel/idempotency.
- This is a specialization, NOT a parallel abstraction.
Index ¶
- Variables
- func FrameworkStatuses() []int
- func MarshalRecordedResponse(r RecordedResponse) ([]byte, error)
- func Middleware(clk clock.Clock, store Store, opts ...Option) func(http.Handler) http.Handler
- type FingerprintMismatchError
- type IdempotencyKey
- type MemStore
- type MetricsObserver
- type Option
- type Receipt
- type RecordedResponse
- type RequestState
- type Store
Constants ¶
This section is empty.
Variables ¶
var ErrFingerprintMismatch = errcode.New( errcode.KindUnprocessable, errcode.ErrIdempotencyKeyReused, "idempotency key reused with a different request body", )
ErrFingerprintMismatch is returned by Store.Claim when the same Idempotency-Key is presented again with a different request body fingerprint. The middleware converts this sentinel into a 422 (KindUnprocessable) response with code ErrIdempotencyKeyReused, optionally enriched with the names of the top-level request fields that differ (per-field diff).
Implementations MUST return a *FingerprintMismatchError (which wraps this sentinel) so callers can use errors.Is for detection AND errors.As to recover the stored fingerprint blob for diffing.
Functions ¶
func FrameworkStatuses ¶
func FrameworkStatuses() []int
FrameworkStatuses returns the client-facing HTTP status codes the idempotency middleware injects on a mutating, non-exempt route: 409 (ClaimBusy, in-flight key) and 422 (key reused with a different request body). Both are derived from the actual sentinels the middleware emits, so this set cannot drift from runtime behavior.
It is the single source the kernel/metadata oracle HTTPTransportMeta.IdempotencyFrameworkStatuses() is bound to by archtest IDEMPOTENCY-FRAMEWORK-STATUS-ORACLE-ALIGN-01. The kernel oracle re-declares the set as an integer literal because kernel/ must not import runtime/ (layering); the archtest forbids the two from diverging.
func MarshalRecordedResponse ¶
func MarshalRecordedResponse(r RecordedResponse) ([]byte, error)
MarshalRecordedResponse serializes r to JSON for storage by a Store implementation. The inverse is UnmarshalRecordedResponse.
func Middleware ¶
Middleware returns an HTTP middleware that enforces per-(tenant,user,key) idempotency for mutating methods (POST, PUT, PATCH, DELETE).
The middleware reads the "Idempotency-Key" request header and the authenticated Principal from the request context (set by runtime/auth). If either is absent, or if the Principal is not a user principal, the request passes through without idempotency tracking.
Key composition (namespace, key) is derived by DeriveKey (key.go) from (tenantID, subject, method, path, idemKey); see it for the exact byte layout, NUL-separator rationale, and node-agnostic invariant. Including method+path in the key means the same client-supplied header value is independent per endpoint — a key for POST /orders does NOT collide with POST /payments. (Stripe / IETF idempotency-key draft §3 aligned.)
Body fingerprinting: the request body is read and reduced to a canonical fingerprint blob (whole-body hash + per-field hashes; see computeFingerprint). BodyLimit middleware MUST run before this middleware so r.Body is already size-bounded. Fingerprint mismatch (same key, different body) returns 422 (KindUnprocessable) ErrIdempotencyKeyReused, with the differing top-level field names in the error details (per-field diff).
clk must be non-nil; clock.MustHaveClock panics on nil. store must be non-nil; a nil store causes a panic with panicregister.Approved (B-class programmer-error; callers must supply a valid store at wiring time).
Types ¶
type FingerprintMismatchError ¶
type FingerprintMismatchError struct {
// Stored is the canonical fingerprint blob (see computeFingerprint) recorded
// for the original Claim. Opaque to the Store; parsed only by the middleware.
Stored string
}
FingerprintMismatchError wraps ErrFingerprintMismatch and carries the stored canonical fingerprint blob recorded for the original request, so the middleware can compute a per-field diff naming the top-level fields that changed. Stored holds ONLY per-field hashes (hex sha256), never raw request values — echoing a stored request value is structurally impossible because raw values are never persisted.
func (*FingerprintMismatchError) Error ¶
func (e *FingerprintMismatchError) Error() string
func (*FingerprintMismatchError) Unwrap ¶
func (e *FingerprintMismatchError) Unwrap() error
Unwrap returns the ErrFingerprintMismatch sentinel so errors.Is keeps working.
type IdempotencyKey ¶
type IdempotencyKey struct {
// contains filtered or unexported fields
}
IdempotencyKey is the sealed (namespace, key) pair that scopes one idempotency record. Its fields are unexported, so a populated IdempotencyKey{...} literal cannot be constructed outside this package — the only producers are the sanctioned constructors DeriveKey (HTTP records) and DeriveCommandKey (commands). Stores read it through Namespace()/Key().
Why sealed (the node-agnostic invariant, #1449/#1610) ¶
The framework HTTP idempotency replay store is assembly-wide: every pod in an assembly sharing one Redis deduplicates the same logical request. That rests on the (ns,key) carrying NO per-pod / per-listener / per-cell dimension — it is derived ONLY from request + principal data. Sealing the type makes that structural, on two axes (AI-robust grading, frozen by archtest HTTP-IDEMPOTENCY-KEY-NODE-AGNOSTIC-01; governance真值源 = ADR docs/architecture/202606051000-1449-adr-http-idempotency-assembly-scope-namespace.md):
- node-agnostic / ban-external-source — Hard, both directions: upstream-external Hard (unexported fields ⇒ no other package can mint a key and splice in node-local state — "sealed construction") + downstream Hard (Store.Claim takes IdempotencyKey, so a raw (ns,key) string pair is inexpressible at the store boundary). NOTE: upstream is Hard only against OTHER packages; an in-package populated literal or a second in-package producer is NOT a compile error — the archtest sole-producer freeze is the Medium backstop there (same split as metrics.CellLabel / holder-seal #893).
- require-isolation-tuple (each sanctioned constructor's body must FLOW all its dimensions into the key) — Medium, genuine Go ceiling: Go cannot express "a body consumes all its inputs", and the same-type string params could be transposed at the (single, reviewed) callsite. The β archtest's AST taint walk (with a flat-composition guard that rejects value laundering) is the Medium backstop. Won't-do ceiling tracked at gh #1650 (same family as #851/#893/#1282/#1552).
Two sanctioned constructors (HTTP record + cross-cell command, #1669) ¶
DeriveKey mints the HTTP record key. DeriveCommandKey (#1669) mints the command dedup key from (tenant, subject, command_id) — the bridge primitive the cross-cell same-slot half of #1610 consumes (routing one logical command to a single dedup slot across cells). Both produce this SAME sealed IdempotencyKey and flow through the SAME Store.Claim sink — the sealed type + funnel are the durable infra and need no change. The archtest sole-producer allowlist + the per-constructor signature/taint gate are single-sourced from one table (idempotencyKeyConstructors), so both constructors are locked identically and a third producer cannot be added to one gate while skipping the other.
func DeriveCommandKey ¶
func DeriveCommandKey(tenantID, subject, commandID string) IdempotencyKey
DeriveCommandKey is the SECOND sanctioned constructor of an IdempotencyKey (sibling of DeriveKey, pre-blessed in this file's type godoc). It encodes the cross-cell command isolation tuple (tenantID, subject, commandID) into the (namespace, key) pair stores expect — the #1610 cross-cell same-slot mapping primitive (ADR docs/architecture/202606040550-1044-adr-command-bus-dispatch-funnel.md §5 ⑤):
ns = tenantID, or noTenantSentinel when empty. key = subject + "\x00" + commandID
commandID is the per-instance idempotency identity of a dispatched command — an OPAQUE string that uniquely names one logical command instance. It is NOT the contract-level command identifier (the kind:command contract id / runtime/command.CommandID used to route a command to its handler): two distinct invocations of the same contract command carry DIFFERENT commandID values and MUST occupy different dedup slots. Sourcing a stable per-instance commandID is the caller's job (the deferred relay-side Claimer wrap, ⑤ PR-B). Because runtime/command.CommandID is the contract-level routing id (a different concept), callers pass the per-instance identity as a plain string here, e.g. DeriveCommandKey(string(tenantID), string(subject), instanceID).
Caller obligations (same store-side contract as DeriveKey, deliberately NOT folded into this store-agnostic constructor): commandID is opaque and may be untrusted; the Redis-backed store rejects a key whose body contains the Redis-Cluster hash-tag braces "{"/"}" or is empty (returns KindInternal on Claim → permanent error → MarkDead). The caller MUST therefore validate commandID is brace-free and non-empty before deriving (DeriveKey's HTTP path does this in middleware on the Idempotency-Key header). When subject is empty (a service principal with no subject), the slot is distinguished within its tenant namespace by commandID alone, so the caller MUST keep commandID unique in that scope.
Why no method/path (unlike DeriveKey): DeriveKey scopes an HTTP record per endpoint so the same Idempotency-Key header on POST /orders and POST /payments stays independent. A command is ALREADY one logical operation named by commandID; routing it to ONE dedup slot across cells is the whole point of #1610, so folding method/path back in would re-partition the slot per listener/endpoint and defeat cross-cell same-slot. commandID IS the per-instance identity that method+path+idemKey jointly play for HTTP. The NUL (\x00) separator is unambiguous: command ids are opaque tokens free of NUL, so subject="alic",commandID="e:x" never collides with subject="alice",commandID="x" (a colon separator would) — same rationale as DeriveKey.
Assembly-scope (node-agnostic) invariant — identical to DeriveKey: the result is derived ONLY from its three parameters; there is no pod/listener/cell input and no call reading node-local state. Do NOT add a node/listener/cell parameter (a 4th param is the node-id injection vector the β archtest signature freeze rejects). All three params MUST flow into the result; dropping one collapses cross-tenant / cross-subject / cross-command isolation (the Medium taint-walk backstop, gh #1650).
func DeriveKey ¶
func DeriveKey(tenantID, subject, method, path, idemKey string) IdempotencyKey
DeriveKey is the HTTP-record constructor of an IdempotencyKey (DeriveCommandKey is the sibling command constructor; the two are the sanctioned producers). It encodes the isolation tuple (tenantID, subject, method, path, idemKey) into the (namespace, key) pair stores expect:
ns = tenantID, or noTenantSentinel when empty. key = subject + "\x00" + method + "\x00" + path + "\x00" + idemKey
Including method+path means the same Idempotency-Key header value is independent per endpoint — POST /orders and POST /payments with the same header are separate records (aligned with Stripe's idempotency design and the IETF idempotency-key draft §3). The NUL (\x00) separator cannot appear in HTTP header values (RFC 7230 §3.2.6 limits field-value to VCHAR/obs-text, excluding NUL), so subject="alic",rest="e:x" never collides with subject="alice",rest="x" — a colon separator would.
Assembly-scope (node-agnostic) invariant: the result is derived ONLY from its parameters — there is no pod/listener/cell input and no call that could read node-local state. That is what makes the replay domain assembly-wide. Do NOT add a node/listener/cell parameter (a 6th param is the node-id injection vector the β archtest signature freeze rejects). All five params MUST flow into the result; dropping one silently collapses cross-tenant / cross-user / cross-endpoint isolation (the Medium taint-walk backstop, gh #1650).
DeriveKey is error-free: brace/empty rejection for Redis-cluster hash-tags is a store-specific concern handled by the Redis adapter on Claim, NOT folded here (MemStore has no such constraint — over-coupling the store-agnostic key to Redis would be wrong).
func (IdempotencyKey) Key ¶
func (k IdempotencyKey) Key() string
Key returns the per-request key within the namespace.
func (IdempotencyKey) Namespace ¶
func (k IdempotencyKey) Namespace() string
Namespace returns the idempotency namespace that scopes keys to a tenant (or noTenantSentinel when the principal carries no tenant). Redis-backed stores use it as the tenant prefix segment outside the hash-tag; the per-request Key() value is the hash-tag payload that colocates lease/response/fingerprint keys on one cluster slot.
type MemStore ¶
type MemStore struct {
// contains filtered or unexported fields
}
MemStore is an in-process Store implementation backed by a mutex-guarded map. It is safe for use in unit tests and single-pod demo deployments; it does NOT coordinate idempotency across replicas.
Use a Redis-backed Store for multi-replica deployments.
func NewMemStore ¶
NewMemStore creates a new MemStore using the given clock. clock.MustHaveClock panics on nil clock (programmer-error guard).
func (*MemStore) Claim ¶
func (ms *MemStore) Claim(ctx context.Context, k IdempotencyKey, fingerprint string, leaseTTL time.Duration) (idempotency.ClaimState, *RecordedResponse, Receipt, error)
Claim implements Store.
type MetricsObserver ¶
type MetricsObserver interface {
ObserveRequest(ctx context.Context, state RequestState)
}
MetricsObserver is the best-effort sink for per-request idempotency metrics. The HTTP idempotency Middleware calls ObserveRequest once per idempotency state event with the corresponding RequestState. Most requests emit a single event; an acquired request whose response is oversize emits two — StateAcquired at claim time, then StateOversize when the body overflows (see RequestState for the acquired ⊇ oversize relationship). The other five states are mutually exclusive and emit once each. The interface lives in this producer package (not in runtime/observability/metrics) so the middleware never imports the metrics package — mirroring the executor.Observer placement that lets runtime/observability/metrics.SagaCollector import the producer without an import cycle.
Implementations MUST be non-blocking and tolerate canceled contexts: they run on the request hot path and must never affect idempotency correctness. The canonical production implementation is runtime/observability/metrics.IdempotencyCollector, which records idempotency_requests_total{cell,state} and derives the {cell} label from the request context. ObserveRequest deliberately takes no cellID argument: the owner cell is read from ctx by the collector (kernel/ctxkeys.CellID, set by the router root CellAttribution middleware) so the {cell} sentinel single source stays in the metrics package and the middleware stays metrics-agnostic.
type Option ¶
type Option func(*middlewareConfig)
Option is a functional option for Middleware.
func WithDoneTTL ¶
WithDoneTTL sets how long a successfully recorded response is retained for future replay.
Non-positive values are clamped to the default (kernel/idempotency.DefaultTTL = 24 h).
Default: kernel/idempotency.DefaultTTL (24 h).
func WithExemptMatcher ¶
WithExemptMatcher installs a predicate that opts individual routes out of idempotency recording. When fn(r) returns true the middleware passes the request directly to the next handler — no Claim is issued, no response is recorded, and any Idempotency-Key header sent by the client is silently ignored. The check runs BEFORE reading the request body so exempt routes pay no body-buffering cost.
A nil fn is a no-op (all routes remain subject to idempotency tracking). Router.buildMux wires this via a lazy closure so FinalizeAuth can compile the exempt set after the middleware is constructed.
func WithLeaseTTL ¶
WithLeaseTTL sets the in-flight processing-lease TTL. If a handler does not respond before the TTL expires, the lease is released and another request may re-claim the key.
Non-positive values are clamped to the default (kernel/idempotency.DefaultLeaseTTL = 5 min).
Default: kernel/idempotency.DefaultLeaseTTL (5 min).
func WithMaxBodyBytes ¶
WithMaxBodyBytes sets the maximum number of response body bytes to buffer for future replay. Responses whose body exceeds this limit are served to the client normally but are not stored; a subsequent identical request will re-invoke the handler.
Non-positive values are clamped to the default (256 KiB) to prevent accidentally disabling body buffering.
Default: 256 KiB.
func WithMetrics ¶
func WithMetrics(obs MetricsObserver) Option
WithMetrics installs an optional MetricsObserver that records one idempotency_requests_total{cell,state} increment per terminal idempotency decision. Metrics are best-effort: a nil/typed-nil observer is not stored and emission is skipped — idempotency correctness never depends on it.
type Receipt ¶
type Receipt interface {
// Record persists resp as the canonical response for this idempotency key
// and commits the lease, making the key's state ClaimDone for doneTTL.
// doneTTL controls how long future requests replay the stored response
// (typically kernel/idempotency.DefaultTTL = 24h).
Record(ctx context.Context, resp *RecordedResponse, doneTTL time.Duration) error
// Release abandons the in-progress lease without recording a response,
// allowing the key to be re-claimed by a subsequent request.
Release(ctx context.Context) error
}
Receipt is the HTTP-specific lifecycle handle for a single acquired idempotency lease. It mirrors kernel/idempotency.Receipt but Commit carries the response blob — the HTTP layer must persist the response for future replay before committing.
Usage for ClaimAcquired:
- Run the HTTP handler to produce a response.
- Call Record(ctx, resp, doneTTL) to persist the response and commit the lease.
- On handler error (before or after partial writes), call Release(ctx) so the lease expires and another request may retry.
Record and Release are idempotent with respect to the lease state: calling them on a non-acquired claim state returns kernel/idempotency.ErrNoClaimLease.
Store implementors: both Record and Release are typically called from a defer after the request context may already be canceled (timeout/client disconnect). Use context.WithoutCancel(ctx) when issuing the underlying store operations so the commit/release reaches the backend even when the request context is canceled. See runtime/http/idempotency.Middleware for the reference implementation.
type RecordedResponse ¶
type RecordedResponse struct {
// contains filtered or unexported fields
}
RecordedResponse is the sealed blob stored by the idempotency middleware for future replay on ClaimDone. All fields are unexported; consumers read state through the value-receiver getters below.
Sealed construction: the only ways to obtain a RecordedResponse are newRecordedResponse (package-internal producer constructor used by the HTTP middleware after handler execution) and UnmarshalRecordedResponse (the wire-decode funnel used by Store implementations when loading a stored blob). Composite literals RecordedResponse{...} outside this package are a compile error (all fields unexported).
func UnmarshalRecordedResponse ¶
func UnmarshalRecordedResponse(raw []byte) (RecordedResponse, error)
UnmarshalRecordedResponse deserializes raw (produced by MarshalRecordedResponse) back into a RecordedResponse. It validates that status is in [100, 599] and that recordedAt is non-zero; any violation returns an errcode error.
func (RecordedResponse) Body ¶
func (r RecordedResponse) Body() []byte
Body returns a defensive copy of the recorded response body. Mutations to the returned slice do not affect the stored copy.
func (RecordedResponse) Header ¶
func (r RecordedResponse) Header() http.Header
Header returns a clone of the recorded response headers. Mutations to the returned map do not affect the stored copy.
func (RecordedResponse) RecordedAt ¶
func (r RecordedResponse) RecordedAt() time.Time
RecordedAt returns the wall-clock time at which this response was recorded, derived from the injected clock at construction time.
func (RecordedResponse) Status ¶
func (r RecordedResponse) Status() int
Status returns the HTTP status code of the recorded response.
type RequestState ¶
type RequestState string
RequestState is the typed, wire-stable label value identifying the outcome of a single idempotency decision. Its value set is frozen as the metric label values of idempotency_requests_total{state} and is enforced by archtest IDEMPOTENCY-REQUESTS-STATE-LABEL-VALUES-FROZEN-01 (A1 freezes the value set; A2 forbids inline literals / RequestState("x") conversions from reaching the label, requiring every value to flow from one of the declared constants below). The same string-typed-enum funnel is used by the saga executor (executor.TickResult / DriveResult / …) and reconcile (resultLabel).
Cardinality: the value set is a closed, registration-time enumeration (6 values); combined with the bounded {cell} dimension the worst-case series count is tiny. There is no per-request / per-key dimension — keys are SHA-256 hashed for logs only and never become metric labels.
const ( // StateAcquired is recorded once per fresh claim (ClaimAcquired): the lease // was acquired and the business handler runs. It is the denominator of // processed mutating requests. It is emitted at claim time, BEFORE the // handler completes, so it counts every fresh claim regardless of the // handler's terminal outcome (successful record, 4xx/5xx, oversize, or // panic) — acquired is not the sum of the other states. StateOversize is a // sub-event of an acquired request (an acquired request whose response was // too large to record increments both acquired and oversize), so // oversize/acquired is the uncacheable-response rate. StateAcquired RequestState = "acquired" // StateReplayed is recorded when a stored response is replayed (ClaimDone): // the idempotency key hit and the cached response was served without // re-invoking the handler. StateReplayed RequestState = "replayed" // StateBusy is recorded when an in-flight lease exists for the key // (ClaimBusy → 409 with Retry-After). A sustained busy rate signals a // retry storm or a stuck in-flight request. StateBusy RequestState = "busy" // StateStoreError is recorded when Store.Claim returns a non-fingerprint // error (→ 500). It is the Claim-path store failure rate. Record/Release // failures (after the response is already served) are intentionally not a // state — they are logged at slog.Error and are out of this counter's scope. StateStoreError RequestState = "store_error" // StateOversize is recorded when an acquired request's response body exceeds // the buffer limit and is therefore not recorded for future replay // (the response is still served to the client normally). See StateAcquired // for the acquired ⊇ oversize relationship. StateOversize RequestState = "oversize" // StateKeyReused is recorded when the same Idempotency-Key is presented with // a different request body (fingerprint mismatch → 422 ErrIdempotencyKeyReused). // It is security-relevant: a sustained rate can indicate a client bug or a // replay attempt. StateKeyReused RequestState = "key_reused" )
type Store ¶
type Store interface {
Claim(ctx context.Context, k IdempotencyKey, fingerprint string, leaseTTL time.Duration) (idempotency.ClaimState, *RecordedResponse, Receipt, error) //nolint:lll // full Claim contract signature cannot be wrapped in an interface decl
}
Store is the HTTP-layer idempotency backend. Implementations live in adapters/ (e.g., adapters/redis).
Claim semantics — discriminate on the returned ClaimState:
switch state {
case idempotency.ClaimAcquired:
// rec is nil; receipt is the acquired lease.
// Run the handler, call receipt.Record on success, receipt.Release on error.
case idempotency.ClaimDone:
// rec is non-nil; replay it. MUST NOT run the handler again.
case idempotency.ClaimBusy:
// rec is nil; another request is in-flight. Return 409.
}
On error (err != nil), fail closed: do not run the handler. If err wraps ErrFingerprintMismatch (errors.Is), the same key was previously claimed with a different fingerprint — the middleware returns 422 (KindUnprocessable) with ErrIdempotencyKeyReused. Do NOT add a 4th ClaimState for this case; use the sentinel error path instead. Implementations MUST return a *FingerprintMismatchError (which wraps the sentinel) so the middleware can recover the stored fingerprint blob for the per-field diff.
k is the sealed (namespace, key) pair produced by a sanctioned constructor (DeriveKey from the HTTP request + principal isolation tuple, or DeriveCommandKey from the command isolation tuple); the Store reads it via k.Namespace() (the tenant scope, or "_notenant" when absent) and k.Key() (the per-request key — see those constructors for the byte layout). A raw (ns,key) string pair cannot reach Claim — that typed boundary is the downstream Hard gate of the node-agnostic invariant (#1449/#1610). fingerprint is the opaque canonical blob produced by computeFingerprint (JSON whose Body field is hex(sha256(rawBody)) and whose Fields field maps top-level field names to per-field hashes for the diff); the Store treats it as an opaque string and only compares / round-trips it.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package idempotencytest provides a reusable conformance suite for implementations of idemhttp.Store.
|
Package idempotencytest provides a reusable conformance suite for implementations of idemhttp.Store. |