storage

package module
v0.4.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 24, 2026 License: Apache-2.0 Imports: 6 Imported by: 0

README

storage

Neutral, stdlib-only storage contracts for durable session/workspace persistence.

storage defines four small primitives — Ledger (append-only, CAS-sequenced record log), Leaser (single-writer epoch lease), KV (revision-CAS key/value), and Blobs (content-addressed immutable bytes) — plus a typed error taxonomy, the AppendDefinite ambiguity resolver, an in-memory reference backend (memstore), and a backend conformance suite (storetest). Consumers depend only on these interfaces; concrete backends (fsstore, natsstore, …) live in their own modules.

This module has zero third-party dependencies and will keep it that way.

Documentation

Overview

Package storage defines four neutral storage primitives — Ledger (an append-only, CAS-sequenced record log), Leaser (a single-writer epoch lease), KV (revision-CAS metadata), and Blobs (content-addressed immutable bytes) — plus a typed error taxonomy, ValidateName, and the AppendDefinite ambiguity resolver.

Names and keys are canonical by construction (see ValidateName), so no two valid names alias one backend location. Every backend must accept ledger payloads and KV values up to 1 MiB; larger payloads are the engine's responsibility to offload to Blobs.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AppendDefinite

func AppendDefinite(ctx context.Context, l Ledger, name string, expected uint64, payload []byte) error

AppendDefinite turns any Append into a definite outcome. On AmbiguousError it retries the identical append once; on conflict (from either attempt) it reads the record at expected+1 and byte-compares: equal payload means the original landed (success); a foreign payload means this writer has been fenced (ConflictError). A second ambiguous outcome surfaces AmbiguousError unresolved.

func ValidateName

func ValidateName(name string) error

ValidateName returns a non-nil *InvalidNameError if name violates the storage name grammar, or nil if it is valid. The grammar is canonical by construction: empty, ".", and ".." segments are unrepresentable, so no two valid names alias one backend location.

A valid name is 1..512 bytes of one or more segments joined by single '/', with no leading, trailing, or doubled '/'. Each segment starts with a byte in [a-z0-9] and continues with bytes in [a-z0-9_.-].

Types

type AmbiguousError

type AmbiguousError struct {
	Name     string
	Expected uint64
	Cause    error
}

AmbiguousError reports a ledger append whose acknowledgement was lost or timed out: the record may or may not have been committed at Expected. Cause carries the underlying transport/timeout error and may be nil.

func (*AmbiguousError) Error

func (e *AmbiguousError) Error() string

func (*AmbiguousError) Unwrap

func (e *AmbiguousError) Unwrap() error

Unwrap returns the underlying cause (possibly nil). AmbiguousError is the only error in the taxonomy that carries and exposes a cause.

type AppendVerifyError

type AppendVerifyError struct {
	Name  string
	Seq   uint64
	Cause error
}

AppendVerifyError reports that AppendDefinite could not read the record at Seq to resolve an ambiguous/conflicting append. Fail closed: the caller must treat the append outcome as unknown. Cause carries the underlying read failure (the Read error or the cursor's non-EOF Next error) and is exposed via Unwrap.

func (*AppendVerifyError) Error

func (e *AppendVerifyError) Error() string

func (*AppendVerifyError) Unwrap

func (e *AppendVerifyError) Unwrap() error

Unwrap returns the underlying read failure so callers can errors.Is/As it.

type BlobConflictError

type BlobConflictError struct {
	Key string
}

BlobConflictError reports a blob Put where Key already exists with different content (blob writes are content-addressed and immutable per key).

func (*BlobConflictError) Error

func (e *BlobConflictError) Error() string

type BlobNotFoundError

type BlobNotFoundError struct {
	Key string
}

BlobNotFoundError reports that a blob is absent at Key.

func (*BlobNotFoundError) Error

func (e *BlobNotFoundError) Error() string

type Blobs

type Blobs interface {
	Put(ctx context.Context, key string, r io.Reader) error
	Get(ctx context.Context, key string) (io.ReadCloser, error)
	Delete(ctx context.Context, key string) error
	List(ctx context.Context, prefix string) ([]string, error)
}

Blobs holds bulk immutable bytes (large-record offload; workspace snapshots). Put streams; keys are content-addressed by callers. Existing byte-identical content is a success/no-op; existing different content returns *BlobConflictError and leaves the original object unchanged. Delete is idempotent: deleting an absent key succeeds.

type Composite

type Composite struct {
	Ledger
	Leaser
	KV
	Blobs
}

Composite satisfies Ledger+Leaser+KV+Blobs by embedding one provider per primitive. Assembled where dependencies are wired, never inside engines.

func NewComposite

func NewComposite(l Ledger, le Leaser, kv KV, bl Blobs) (*Composite, error)

NewComposite assembles a Composite, rejecting any nil primitive up front so a partially-wired backend fails at the composition root rather than at first use.

type ConflictError

type ConflictError struct {
	Name     string
	Expected uint64
}

ConflictError reports a ledger compare-and-swap that failed because the caller appended at the wrong expected sequence (the head had moved).

func (*ConflictError) Error

func (e *ConflictError) Error() string

type Cursor

type Cursor interface {
	Next(ctx context.Context) (Record, error) // io.EOF when drained
	Close() error
}

type IncompleteCompositeError

type IncompleteCompositeError struct {
	Missing []string
}

IncompleteCompositeError reports that NewComposite was handed one or more nil primitives. Missing names them in field order (Ledger, Leaser, KV, Blobs) so the assembly site knows exactly which providers were not wired.

func (*IncompleteCompositeError) Error

func (e *IncompleteCompositeError) Error() string

type InvalidNameError

type InvalidNameError struct {
	Name string
	Rule string
}

InvalidNameError reports a name that violates the storage grammar: one or more segments joined by single '/', each segment matching [a-z0-9][a-z0-9_.-]*, no leading/trailing '/', at most 512 bytes total.

func (*InvalidNameError) Error

func (e *InvalidNameError) Error() string

type KV

type KV interface {
	Get(ctx context.Context, key string) (val []byte, rev uint64, err error)
	Put(ctx context.Context, key string, expectedRev uint64, val []byte) (rev uint64, err error)
	Keys(ctx context.Context, prefix string) ([]string, error)
	Delete(ctx context.Context, key string) error
}

KV holds small CAS'd metadata (the session catalog). Revisions are per-key, strictly increasing; Put with expectedRev 0 requires the key to be absent.

type KeyNotFoundError

type KeyNotFoundError struct {
	Key string
}

KeyNotFoundError reports that a KV key is absent.

func (*KeyNotFoundError) Error

func (e *KeyNotFoundError) Error() string

type Lease

type Lease interface {
	Epoch() uint64
	Lost() <-chan struct{}             // closed when ownership is lost (expiry, takeover)
	Release(ctx context.Context) error // releasing may cross the network; ctx bounds it
}

type LeaseHeldError

type LeaseHeldError struct {
	Name        string
	HolderEpoch uint64
}

LeaseHeldError reports an Acquire that was refused because another holder owns the lease at HolderEpoch.

func (*LeaseHeldError) Error

func (e *LeaseHeldError) Error() string

type LeaseLostError

type LeaseLostError struct {
	Name  string
	Epoch uint64
}

LeaseLostError reports a write attempted after the caller's lease at Epoch was lost or expired (fenced by a newer holder).

func (*LeaseLostError) Error

func (e *LeaseLostError) Error() string

type Leaser

type Leaser interface {
	Acquire(ctx context.Context, name string) (Lease, error)
}

Leaser grants exclusive, epoch-fenced ownership of a name. Acquire fails with *LeaseHeldError while a live holder exists; a dead holder's lease is reclaimed by the backend's native mechanism (flock released by the OS, KV TTL expiry, PG advisory-lock session end). Epochs are strictly increasing across grants of the same name.

type Ledger

type Ledger interface {
	Append(ctx context.Context, name string, expected uint64, payload []byte) error
	Read(ctx context.Context, name string, from uint64) (Cursor, error)
	Tip(ctx context.Context, name string) (uint64, error)
	Delete(ctx context.Context, name string) error
}

Ledger addresses many ledgers by name. Append commits payload as the record immediately after sequence `expected` (CAS on the tip; expected == 0 means the ledger must be empty). The committed record's seq is expected+1 by definition, so Append returns no sequence. Sequences are 1-based, contiguous, immutable.

Append outcomes are a tri-state:

  • nil — committed, definitely.
  • *ConflictError — something already occupies expected+1. Definite: the record did not land.
  • *AmbiguousError — the outcome is unknown (lost ack / lost COMMIT response). Only networked backends may return this; fs and memory never do.

Any other error is a definite failure (fail closed, tip unadvanced).

Edge semantics:

  • Absent == empty. A never-written (or deleted) ledger behaves as empty: Tip returns 0; Read yields an immediately-drained cursor; Append with expected 0 creates it implicitly; Delete of an absent ledger is a no-op success (idempotent).
  • Reads beyond the tip are empty, not errors. Any from > tip (including tip+1) yields a drained cursor. Cursors are bounded: they observe the tip as of Read and never tail later appends.
  • Payloads are caller-owned. A backend must not reuse or mutate Record.Payload after Next returns. Zero-length payloads are legal.
  • Listings are canonical: KV.Keys and Blobs.List return lexicographically ascending, duplicate-free results.

type PathReporter

type PathReporter interface {
	StoragePaths() []string
}

PathReporter is an optional capability implemented by providers that persist data on the local filesystem. StoragePaths returns the provider's canonical local roots in a caller-owned slice. Remote and in-memory providers need not implement it.

type Record

type Record struct {
	Seq     uint64
	Payload []byte
}

type RecordNotFoundError

type RecordNotFoundError struct {
	Name string
	Seq  uint64
}

RecordNotFoundError reports that a ledger has no record at the requested Seq.

func (*RecordNotFoundError) Error

func (e *RecordNotFoundError) Error() string

Directories

Path Synopsis
Package memstore is the in-memory reference backend for storage's four primitives.
Package memstore is the in-memory reference backend for storage's four primitives.
Package storetest provides backend-conformance suites for the four storage primitives — Ledger, Leaser, KV, and Blobs.
Package storetest provides backend-conformance suites for the four storage primitives — Ledger, Leaser, KV, and Blobs.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL