idempotency

package module
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 26, 2026 License: MIT Imports: 8 Imported by: 0

README

idempotency

CI CodeQL Coverage Mutation Documentation Go Reference Release Go License

idempotency provides durable ownership, fencing, and bounded result replay for retried Go operations. It is being built for HTTP, JSON-RPC, webhook, queue, import, and command workloads.

The package deliberately does not claim exactly-once execution. A lease can expire while an old process is still performing a side effect. Correct callers must use the returned fencing token in a database transaction, conditional write, or another application invariant that rejects stale owners.

Status

The public contract, deterministic memory adapter, bounded JSON canonicalization, PostgreSQL and Valkey adapters, buffered HTTP middleware, method-aware JSON-RPC middleware, queue and webhook deduplication, named command and import helpers, transactional outbox coordination, and bounded logging and telemetry observers are implemented. The API is stable at v1 and is released as stable.

Core acquisition outcomes

  • acquired: this caller owns the first or a deliberately released attempt.
  • stale_owner_takeover: this caller owns a new fenced attempt after expiry.
  • in_progress: another unexpired owner is current.
  • replayed: the same fingerprint has a completed bounded result.
  • terminal_failure: the same fingerprint has a recorded terminal failure.
  • conflict: the key already identifies a different fingerprint.
  • unavailable: ownership could not be established; execution must fail closed unless an integration exposes and the caller selects an explicit duplicate- tolerant policy.

Start in five minutes

go get github.com/faustbrian/go-idempotency

The quickstart demonstrates acquisition, completion, and replay with the deterministic API, then routes production deployments to the PostgreSQL or Valkey 9 adapter. Read the comparison with locks, unique constraints, retries, and exactly-once claims before choosing the application's side-effect invariant.

See the state machine and the crash semantics before using the package. The HTTP middleware guide covers bounded handler response replay, and the JSON-RPC guide covers result and protocol-error replay. The queue guide explains broker settlement behavior. The command and import guide covers stable source record identities. The PostgreSQL guide covers transactional locking, cleanup, permissions, and persisted-record privacy. The transaction and outbox guide shows atomic business, outbox envelope, and completion commits with idempotencyoutbox. The webhook guide covers signature ordering, provider delivery identities, and response mapping. The operations guide covers health, observability, retention, cleanup, capacity, and incident recovery. The threat model, hardening report, and resource budgets define the verified security and operational envelope. See bounded logging and telemetry integration, troubleshooting, migration and compatibility policy, and the FAQ for production adoption.

Licensed under the MIT License. Release history is maintained in the changelog, and the complete guide index is in docs/README.md.

Ecosystem

Use the Golib documentation portal to choose companion packages, supported stacks, recipes, and operations guidance.

Documentation

Overview

Package idempotency defines durable ownership and replay semantics for operations that may be retried.

The package does not guarantee exactly-once execution. A lease proves only that an owner may act during a bounded interval; it cannot prove that an expired owner stopped. Applications must carry the fencing token into the transaction or conditional write that protects the business side effect.

Index

Examples

Constants

View Source
const (
	// MaxKeyPartBytes bounds each individual logical key component.
	MaxKeyPartBytes = 256
	// MaxFingerprintVersionBytes bounds the canonicalization policy identifier.
	MaxFingerprintVersionBytes = 128
	// MaxOwnerTokenBytes bounds opaque ownership proofs stored by adapters.
	MaxOwnerTokenBytes = 256
	// MaxResultBytes bounds a result stored for terminal replay.
	MaxResultBytes = 1 << 20
	// MaxMetadataEntries bounds the number of stored metadata pairs.
	MaxMetadataEntries = 32
	// MaxMetadataKeyBytes bounds each metadata key.
	MaxMetadataKeyBytes = 128
	// MaxMetadataValueBytes bounds each metadata value.
	MaxMetadataValueBytes = 1024
	// MaxLease is the longest ownership lease accepted by the semantic core.
	MaxLease = 24 * time.Hour
)

Variables

This section is empty.

Functions

func WithOwnership

func WithOwnership(ctx context.Context, ownership Ownership) context.Context

WithOwnership adds the elected owner and fencing token to a handler context.

Types

type AcquireRequest

type AcquireRequest struct {
	Key         Key
	Fingerprint Fingerprint
	Lease       time.Duration
}

AcquireRequest supplies the stable identity, fingerprint, and requested lease.

type AcquireResult

type AcquireResult struct {
	Outcome Outcome
	Record  Record
}

AcquireResult contains the acquisition outcome and authoritative record.

type AvailabilityPolicy

type AvailabilityPolicy uint8

AvailabilityPolicy controls whether work may run after acquisition storage fails.

const (
	// AvailabilityFailClosed rejects execution when ownership is unavailable.
	AvailabilityFailClosed AvailabilityPolicy = iota
	// AvailabilityAllowUntracked permits explicitly duplicate-tolerant execution.
	AvailabilityAllowUntracked
)

type BeginRequest

type BeginRequest struct {
	Acquire      AcquireRequest
	Availability AvailabilityPolicy
}

BeginRequest combines durable acquisition with an availability policy.

type BeginResult

type BeginResult struct {
	Outcome Outcome
	Record  Record
	Execute bool
	Durable bool
	Failure error
}

BeginResult describes whether work should execute and whether it is tracked.

type CompleteRequest

type CompleteRequest struct {
	Ownership Ownership
	Result    []byte
	Metadata  map[string]string
}

CompleteRequest records a bounded successful terminal result.

type Error

type Error struct {
	// Reason classifies the failure for programmatic handling.
	Reason Reason
	// Field identifies the input, transition, or backend property involved.
	Field string
	// Cause retains the underlying failure without changing the stable reason.
	Cause error
}

Error reports a stable reason and field while retaining an optional cause.

func (*Error) Error

func (e *Error) Error() string

func (*Error) Unwrap

func (e *Error) Unwrap() error

type FailRequest

type FailRequest struct {
	Ownership Ownership
	Result    []byte
	Metadata  map[string]string
}

FailRequest records a bounded terminal failure result.

type Fingerprint

type Fingerprint struct {
	// contains filtered or unexported fields
}

Fingerprint is a versioned SHA-256 digest of canonical business input.

func NewFingerprint

func NewFingerprint(version string, canonical []byte) (Fingerprint, error)

NewFingerprint hashes canonical business input under a stable policy version.

func NewFingerprintFromSum

func NewFingerprintFromSum(version string, sum []byte) (Fingerprint, error)

NewFingerprintFromSum reconstructs a fingerprint from a persisted SHA-256 sum.

func (Fingerprint) Equal

func (f Fingerprint) Equal(other Fingerprint) bool

Equal reports whether both the policy version and digest match.

func (Fingerprint) Sum

func (f Fingerprint) Sum() []byte

Sum returns a copy of the SHA-256 digest bytes.

func (Fingerprint) Version

func (f Fingerprint) Version() string

Version returns the canonicalization policy version.

type HeartbeatRequest

type HeartbeatRequest struct {
	Ownership Ownership
	Lease     time.Duration
}

HeartbeatRequest extends a live current owner's lease.

type Key

type Key struct {
	// contains filtered or unexported fields
}

Key is the fully scoped logical identity of one repeatable operation. Construct keys with NewKey so every component is present and bounded.

func NewKey

func NewKey(namespace, tenant, operation, caller, value string) (Key, error)

NewKey validates and constructs a namespaced operation identity.

func (Key) Caller

func (k Key) Caller() string

Caller returns the authenticated caller identity for the key.

func (Key) Namespace

func (k Key) Namespace() string

Namespace returns the broad collision domain for the key.

func (Key) Operation

func (k Key) Operation() string

Operation returns the stable business operation name for the key.

func (Key) Tenant

func (k Key) Tenant() string

Tenant returns the authenticated tenant identity for the key.

func (Key) Value

func (k Key) Value() string

Value returns the caller-supplied idempotency value for the key.

type KeyHasher

type KeyHasher func(Key) string

KeyHasher returns a non-reversible correlation value for a logical key.

func NewHMACKeyHasher

func NewHMACKeyHasher(secret []byte) (KeyHasher, error)

NewHMACKeyHasher constructs a deterministic SHA-256 HMAC key hasher. The secret is copied and must contain at least 32 bytes.

type Observation

type Observation struct {
	// Transition identifies the semantic operation.
	Transition Transition
	// Outcome is populated for acquisition attempts.
	Outcome Outcome
	// Reason classifies a failed transition and is empty on success.
	Reason Reason
	// Durable reports whether the requested state or result is durably established.
	Durable bool
	// Correlation is a keyed digest when ServiceOptions provides a KeyHasher.
	Correlation string
}

Observation is a bounded service transition signal safe for instrumentation. Correlation is suitable for restricted logs, never metric labels.

type Observer

type Observer interface {
	Observe(context.Context, Observation)
}

Observer receives bounded semantic service transition signals. Service isolates observer and key-hasher panics so instrumentation cannot change a semantic result. Implementations should still return quickly and honor ctx.

type ObserverFunc

type ObserverFunc func(context.Context, Observation)

ObserverFunc adapts a function to Observer.

func (ObserverFunc) Observe

func (f ObserverFunc) Observe(ctx context.Context, observation Observation)

Observe calls f with the bounded transition signal.

type Outcome

type Outcome string

Outcome describes the semantic result of attempting acquisition.

const (
	// OutcomeAcquired means the caller owns a new executable attempt.
	OutcomeAcquired Outcome = "acquired"
	// OutcomeReplayed means the same fingerprint has a completed result.
	OutcomeReplayed Outcome = "replayed"
	// OutcomeInProgress means another unexpired owner is current.
	OutcomeInProgress Outcome = "in_progress"
	// OutcomeConflict means the retained key has a different fingerprint.
	OutcomeConflict Outcome = "conflict"
	// OutcomeUnavailable means durable ownership could not be established.
	OutcomeUnavailable Outcome = "unavailable"
	// OutcomeStaleOwnerTakeover means the caller replaced an elapsed active owner.
	OutcomeStaleOwnerTakeover Outcome = "stale_owner_takeover"
	// OutcomeTerminalFailure means a terminal failure is available for replay.
	OutcomeTerminalFailure Outcome = "terminal_failure"
)

type Ownership

type Ownership struct {
	Key          Key
	OwnerToken   string
	FencingToken uint64
}

Ownership identifies one leased attempt with an opaque token and fence.

func OwnershipFromContext

func OwnershipFromContext(ctx context.Context) (Ownership, bool)

OwnershipFromContext returns the elected handler ownership, when present.

type Reason

type Reason string

Reason is a stable machine-readable classification for a semantic error.

const (
	// ReasonInvalidKey identifies a missing or invalid logical key component.
	ReasonInvalidKey Reason = "invalid_key"
	// ReasonInvalidFingerprint identifies an invalid fingerprint or policy version.
	ReasonInvalidFingerprint Reason = "invalid_fingerprint"
	// ReasonLimitExceeded identifies input that crosses a documented resource bound.
	ReasonLimitExceeded Reason = "limit_exceeded"
	// ReasonStaleOwner identifies an ownership proof from a superseded attempt.
	ReasonStaleOwner Reason = "stale_owner"
	// ReasonLeaseExpired identifies a current proof used after its lease boundary.
	ReasonLeaseExpired Reason = "lease_expired"
	// ReasonNotFound identifies an operation targeting a missing record.
	ReasonNotFound Reason = "not_found"
	// ReasonInvalidTransition identifies an operation illegal for the current state.
	ReasonInvalidTransition Reason = "invalid_transition"
	// ReasonUnavailable identifies failure to establish or mutate durable state.
	ReasonUnavailable Reason = "unavailable"
	// ReasonInvalidConfiguration identifies invalid constructor or policy options.
	ReasonInvalidConfiguration Reason = "invalid_configuration"
	// ReasonInvalidLease identifies a nonpositive lease duration.
	ReasonInvalidLease Reason = "invalid_lease"
	// ReasonInvalidPayload identifies malformed or unsupported persisted data.
	ReasonInvalidPayload Reason = "invalid_payload"
	// ReasonUnsafeBackend identifies a backend configuration that breaks correctness.
	ReasonUnsafeBackend Reason = "unsafe_backend"
)

type Record

type Record struct {
	Key            Key
	Fingerprint    Fingerprint
	State          State
	OwnerToken     string
	FencingToken   uint64
	LeaseExpiresAt time.Time
	HeartbeatAt    time.Time
	Attempt        uint64
	CreatedAt      time.Time
	UpdatedAt      time.Time
	CompletedAt    time.Time
	FailedAt       time.Time
	AbandonedAt    time.Time
	ExpiredAt      time.Time
	Result         []byte
	Metadata       map[string]string
}

Record is a snapshot of one retained idempotency state machine.

func (Record) Ownership

func (r Record) Ownership() Ownership

Ownership returns the proof required for current-owner mutations.

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service normalizes store failures and applies availability policy to acquisition.

func NewService

func NewService(store Store) (*Service, error)

NewService constructs the semantic service over a non-nil store.

func NewServiceWithOptions

func NewServiceWithOptions(store Store, options ServiceOptions) (*Service, error)

NewServiceWithOptions constructs a service with optional bounded observation.

func (*Service) Begin

func (s *Service) Begin(
	ctx context.Context,
	request BeginRequest,
) (result BeginResult, err error)

Begin attempts ownership and decides whether the caller may execute.

Example
package main

import (
	"context"
	"fmt"
	"time"

	"github.com/faustbrian/go-idempotency"
	"github.com/faustbrian/go-idempotency/memory"
)

type exampleClock struct {
	now time.Time
}

func (c exampleClock) Now() time.Time { return c.now }

func main() {
	clock := exampleClock{now: time.Date(2026, 7, 15, 12, 0, 0, 0, time.UTC)}
	token := 0
	store, err := memory.New(memory.Options{
		Clock: clock,
		OwnerTokens: func() (string, error) {
			token++
			return fmt.Sprintf("owner-%d", token), nil
		},
	})
	if err != nil {
		panic(err)
	}
	service, err := idempotency.NewService(store)
	if err != nil {
		panic(err)
	}
	key, err := idempotency.NewKey(
		"billing", "tenant-42", "create-invoice", "api-client-7", "request-123",
	)
	if err != nil {
		panic(err)
	}
	fingerprint, err := idempotency.NewFingerprint(
		"invoice-v1", []byte("invoice:9001:EUR"),
	)
	if err != nil {
		panic(err)
	}
	request := idempotency.BeginRequest{Acquire: idempotency.AcquireRequest{
		Key: key, Fingerprint: fingerprint, Lease: 30 * time.Second,
	}}

	first, err := service.Begin(context.Background(), request)
	if err != nil {
		panic(err)
	}
	fmt.Println(first.Outcome, first.Execute, first.Durable)
	_, err = service.Complete(context.Background(), idempotency.CompleteRequest{
		Ownership: first.Record.Ownership(),
		Result:    []byte(`{"invoice_id":"inv-9001"}`),
	})
	if err != nil {
		panic(err)
	}
	retry, err := service.Begin(context.Background(), request)
	if err != nil {
		panic(err)
	}
	fmt.Printf("%s: %s\n", retry.Outcome, retry.Record.Result)

}
Output:
acquired true true
replayed: {"invoice_id":"inv-9001"}

func (*Service) Complete

func (s *Service) Complete(
	ctx context.Context,
	request CompleteRequest,
) (record Record, err error)

Complete conditionally records a successful terminal result.

func (*Service) Expire

func (s *Service) Expire(ctx context.Context, key Key) (record Record, err error)

Expire records that an active lease elapsed without granting new ownership.

func (*Service) Fail

func (s *Service) Fail(
	ctx context.Context,
	request FailRequest,
) (record Record, err error)

Fail conditionally records a terminal failure result.

func (*Service) Heartbeat

func (s *Service) Heartbeat(
	ctx context.Context,
	request HeartbeatRequest,
) (record Record, err error)

Heartbeat extends a live current owner's lease.

func (*Service) Inspect

func (s *Service) Inspect(ctx context.Context, key Key) (record Record, err error)

Inspect returns the authoritative record for key.

func (*Service) Release

func (s *Service) Release(
	ctx context.Context,
	ownership Ownership,
) (record Record, err error)

Release conditionally abandons a live attempt without a terminal result.

type ServiceOptions

type ServiceOptions struct {
	// Observer receives one signal after each instrumented semantic transition.
	Observer Observer
	// KeyHasher produces restricted-log correlation without exposing logical keys.
	KeyHasher KeyHasher
}

ServiceOptions configures optional bounded service instrumentation.

type State

type State string

State is the durable lifecycle state of an idempotency record.

const (
	// StateAcquired means an owner holds a fresh lease but has not heartbeated.
	StateAcquired State = "acquired"
	// StateRunning means the current owner has extended its lease.
	StateRunning State = "running"
	// StateCompleted means a successful bounded result is terminal and replayable.
	StateCompleted State = "completed"
	// StateFailed means a bounded terminal failure is replayable.
	StateFailed State = "failed"
	// StateExpired means an elapsed active lease was explicitly recorded.
	StateExpired State = "expired"
	// StateAbandoned means the owner deliberately released without a result.
	StateAbandoned State = "abandoned"
)

type Store

Store atomically persists every semantic state-machine transition. Implementations must satisfy the ownership and fencing contract described by the package, including backend-authoritative time when durable.

type Transition

type Transition string

Transition identifies one fixed-cardinality semantic service operation.

const (
	// TransitionAcquire identifies a Service.Begin acquisition attempt.
	TransitionAcquire Transition = "acquire"
	// TransitionInspect identifies a Service.Inspect read.
	TransitionInspect Transition = "inspect"
	// TransitionHeartbeat identifies a Service.Heartbeat lease extension.
	TransitionHeartbeat Transition = "heartbeat"
	// TransitionComplete identifies a Service.Complete terminal transition.
	TransitionComplete Transition = "complete"
	// TransitionFail identifies a Service.Fail terminal transition.
	TransitionFail Transition = "fail"
	// TransitionRelease identifies a Service.Release abandonment transition.
	TransitionRelease Transition = "release"
	// TransitionExpire identifies a Service.Expire audit transition.
	TransitionExpire Transition = "expire"
)

Directories

Path Synopsis
Package canonical provides bounded, explicit request fingerprint policies.
Package canonical provides bounded, explicit request fingerprint policies.
Package idempotencycommand provides durable named command and source-record import execution with bounded result replay.
Package idempotencycommand provides durable named command and source-record import execution with bounded result replay.
Package idempotencyhttp provides buffered net/http middleware backed by an idempotency.Service.
Package idempotencyhttp provides buffered net/http middleware backed by an idempotency.Service.
Package idempotencylog adapts bounded idempotency observations to log/slog.
Package idempotencylog adapts bounded idempotency observations to log/slog.
Package idempotencyoutbox coordinates a transactional outbox insert with PostgreSQL idempotency completion in one caller-owned transaction.
Package idempotencyoutbox coordinates a transactional outbox insert with PostgreSQL idempotency completion in one caller-owned transaction.
Package idempotencyqueue provides durable consumer ownership and redelivery deduplication for messages exposing a Payload method.
Package idempotencyqueue provides durable consumer ownership and redelivery deduplication for messages exposing a Payload method.
Package idempotencyrpc provides method-aware durable JSON-RPC invocation ownership and bounded response or protocol-error replay.
Package idempotencyrpc provides method-aware durable JSON-RPC invocation ownership and bounded response or protocol-error replay.
Package idempotencytelemetry adapts bounded observations to OpenTelemetry metrics.
Package idempotencytelemetry adapts bounded observations to OpenTelemetry metrics.
Package idempotencytest provides reusable adapter conformance and fixtures.
Package idempotencytest provides reusable adapter conformance and fixtures.
Package idempotencywebhook provides provider-delivery deduplication for webhook messages with bounded payload fingerprints and durable ownership.
Package idempotencywebhook provides provider-delivery deduplication for webhook messages with bounded payload fingerprints and durable ownership.
Package memory implements deterministic, process-local idempotency storage.
Package memory implements deterministic, process-local idempotency storage.
Package postgres implements durable idempotency storage on PostgreSQL using pgx, transaction-scoped advisory locks, row locks, server time, and bounded retention cleanup.
Package postgres implements durable idempotency storage on PostgreSQL using pgx, transaction-scoped advisory locks, row locks, server time, and bounded retention cleanup.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL