fsstore

package module
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 14, 2026 License: Apache-2.0 Imports: 15 Imported by: 0

README

fsstore

fsstore implements storage's four storage primitives — Ledger, Leaser, KV, and Blobs — over the local filesystem, giving a durable, single-host backend for session and workspace persistence. The concrete on-disk formats live here, behind storage's neutral contracts: the ledger is an append-only frame log (length-prefixed, CRC-32C-checked frames) with automatic torn-tail recovery on reopen; leases are flock-fenced lock files carrying a durable, strictly-increasing epoch counter; KV is one revision-CAS'd file per key (atomic temp-file + rename writes); and Blobs are content-addressed immutable byte objects. It depends only on the Go standard library and github.com/looprig/storage.

The backend assumes single-writer-per-name: reads take no advisory lock and trust that no one rewrites committed bytes concurrently. Enforcing that discipline is the caller's responsibility — acquire the name's lease via the Leaser before writing its ledger or KV entries. Path containment is lexical (validated names, filepath.Clean, a root-prefix check); it does not resolve symlinks, so owning the store root directory is the deployment's responsibility.

Usage

Open wires the four backends under one root directory and returns them bundled as a *storage.Composite (the primitives collide on method names, so they are composed as a field-bundle, not a single all-four type — reach each as store.Ledger, store.Leaser, store.KV, store.Blobs). Hand the bundle to a consumer such as sessionstore.Open:

store, err := fsstore.Open(fsstore.Options{Root: "/var/lib/looprig"})
if err != nil {
    return err
}
defer store.Close() // releases in-process state; idempotent

sess, err := sessionstore.Open(store.Backend()) // or store.Composite

Root is required (an empty Root is rejected with an *OptionsError) and is created at 0700 if absent. Close releases the ledger backend's in-process cache and is idempotent; after Close the Store must not be reused.

Documentation

Index

Constants

View Source
const (

	// MaxFramePayload caps a single frame's payload at 16 MiB. storage only
	// requires backends to accept 1 MiB payloads; the 16 MiB ceiling is
	// deliberate headroom. It also bounds the allocation a decoder performs for
	// any declared length: a larger declared length is rejected as corruption
	// before any buffer is sized to it, so a bogus header cannot force a huge
	// allocation.
	MaxFramePayload = 16 << 20
)

Variables

This section is empty.

Functions

This section is empty.

Types

type BlobIOError

type BlobIOError struct {
	Op    string
	Path  string
	Cause error
}

BlobIOError wraps an underlying filesystem failure (mkdir, open, read, write, fsync, rename, remove, walk) with the operation and path for diagnosis. Cause is the os/syscall error and is exposed via Unwrap.

func (*BlobIOError) Error

func (e *BlobIOError) Error() string

func (*BlobIOError) Unwrap

func (e *BlobIOError) Unwrap() error

Unwrap exposes the underlying filesystem error.

type BlobPathError

type BlobPathError struct {
	Key  string
	Path string
}

BlobPathError reports that a blob key mapped to a filesystem location outside the store's blobs directory. It is defense in depth: ValidateName already forbids the '..' and empty segments that could escape, so a valid key never triggers it — a triggered BlobPathError means an unvalidated key reached path mapping.

func (*BlobPathError) Error

func (e *BlobPathError) Error() string

type BlobRootError

type BlobRootError struct {
	Root   string
	Reason string
}

BlobRootError reports that newBlobStore was given an unusable root (for example, an empty path).

func (*BlobRootError) Error

func (e *BlobRootError) Error() string

type FlockError

type FlockError struct {
	Op    string
	Path  string
	Cause error
}

FlockError reports that acquiring or releasing a cross-process advisory lock failed. Op is "lock" or "unlock"; Cause carries the underlying syscall error (or errFlockUnsupported on a platform without advisory locking). Callers classify with errors.As(&FlockError), never by string.

func (*FlockError) Error

func (e *FlockError) Error() string

func (*FlockError) Unwrap

func (e *FlockError) Unwrap() error

Unwrap exposes the underlying syscall error (or errFlockUnsupported).

type FrameError

type FrameError struct {
	Fault FrameFault
	// Have is the number of bytes available (Torn faults).
	Have int
	// Need is the number of bytes the frame required (Torn faults).
	Need int
	// Length is the declared or requested payload length (Oversize fault).
	Length uint64
}

FrameError is the single typed error the frame codec returns. Its Fault names the exact cause; IsTorn / IsCorrupt classify it into the two recovery modes the ledger acts on:

  • Torn — a clean truncation at a byte boundary (an interrupted final write). Recoverable: the ledger truncates back to the last good frame.
  • Corrupt — every declared byte is present but the frame is internally inconsistent (bad CRC, or a length beyond the ceiling). Not recoverable by truncation; the ledger must refuse to open and fail closed.

Callers classify with errors.As(&FrameError) and the predicates, never by string. The Error() text carries only integers, never payload bytes.

func (*FrameError) Error

func (e *FrameError) Error() string

func (*FrameError) IsCorrupt

func (e *FrameError) IsCorrupt() bool

IsCorrupt reports a present-but-invalid frame: a CRC mismatch or a length beyond MaxFramePayload. The ledger cannot recover by truncation and fails closed.

func (*FrameError) IsTorn

func (e *FrameError) IsTorn() bool

IsTorn reports a clean truncation: the header or payload is incomplete. The ledger recovers by truncating the file to the previous whole frame.

type FrameFault

type FrameFault uint8

FrameFault identifies the precise reason a frame failed to encode or decode. It is the machine-classifiable cause; callers fold faults into the two recovery modes with FrameError.IsTorn / FrameError.IsCorrupt.

const (
	// FaultShortHeader: fewer than frameHeaderSize bytes are present, so the
	// header itself is incomplete. Torn.
	FaultShortHeader FrameFault = iota + 1
	// FaultShortPayload: the header is complete and declares a payload length,
	// but fewer than that many payload bytes are present. Torn.
	FaultShortPayload
	// FaultOversize: a declared or requested payload length exceeds
	// MaxFramePayload. Corrupt — rejected before any buffer is sized to it.
	FaultOversize
	// FaultCRCMismatch: every declared byte is present but the payload CRC does
	// not match the header. Corrupt.
	FaultCRCMismatch
)

type KVCorruptError

type KVCorruptError struct {
	Path  string
	Cause error
}

KVCorruptError reports that a key's file could not be trusted: it is present but too short to hold the revision header. Both Get and Put fail closed rather than treat the file as absent (rev 0), which would let a torn file silently reset the revision counter. Cause is errShortRevHeader.

func (*KVCorruptError) Error

func (e *KVCorruptError) Error() string

func (*KVCorruptError) Unwrap

func (e *KVCorruptError) Unwrap() error

Unwrap exposes the underlying cause (errShortRevHeader).

type KVIOError

type KVIOError struct {
	Op    string
	Path  string
	Cause error
}

KVIOError wraps an underlying filesystem failure (mkdir, open, read, write, fsync, rename, remove, walk) with the operation and path for diagnosis. Cause is the os/syscall error and is exposed via Unwrap.

func (*KVIOError) Error

func (e *KVIOError) Error() string

func (*KVIOError) Unwrap

func (e *KVIOError) Unwrap() error

Unwrap exposes the underlying filesystem error.

type KVPathError

type KVPathError struct {
	Key  string
	Path string
}

KVPathError reports that a KV key mapped to a filesystem location outside the store's kv directory. It is defense in depth: ValidateName already forbids the '..' and empty segments that could escape, so a valid key never triggers it — a triggered KVPathError means an unvalidated key reached path mapping.

func (*KVPathError) Error

func (e *KVPathError) Error() string

type KVRootError

type KVRootError struct {
	Root   string
	Reason string
}

KVRootError reports that newKVStore was given an unusable root (for example, an empty path).

func (*KVRootError) Error

func (e *KVRootError) Error() string

type LeaseCorruptError

type LeaseCorruptError struct {
	Path  string
	Cause error
}

LeaseCorruptError reports that a lock file's persisted epoch counter could not be trusted: it is present but not a valid decimal uint64. Acquire fails closed rather than reset the counter to 0 (which would let a stale holder's epoch be reissued, breaking the strictly-increasing guarantee). Cause is errBadEpoch.

func (*LeaseCorruptError) Error

func (e *LeaseCorruptError) Error() string

func (*LeaseCorruptError) Unwrap

func (e *LeaseCorruptError) Unwrap() error

Unwrap exposes the underlying cause (errBadEpoch).

type LeaseIOError

type LeaseIOError struct {
	Op    string
	Path  string
	Cause error
}

LeaseIOError wraps an underlying filesystem failure (mkdir, open, read, write, truncate, fsync, close) with the operation and path for diagnosis. Cause is the os/syscall error and is exposed via Unwrap.

func (*LeaseIOError) Error

func (e *LeaseIOError) Error() string

func (*LeaseIOError) Unwrap

func (e *LeaseIOError) Unwrap() error

Unwrap exposes the underlying filesystem error.

type LeasePathError

type LeasePathError struct {
	Name string
	Path string
}

LeasePathError reports that a lease Name mapped to a filesystem location outside the store's leases directory. It is defense in depth: ValidateName already forbids the '..' and empty segments that could escape, so a valid name never triggers it — a triggered LeasePathError means an unvalidated name reached path mapping.

func (*LeasePathError) Error

func (e *LeasePathError) Error() string

type LeaseRootError

type LeaseRootError struct {
	Root   string
	Reason string
}

LeaseRootError reports that newLeaserStore was given an unusable root (for example, an empty path).

func (*LeaseRootError) Error

func (e *LeaseRootError) Error() string

type LedgerCorruptError

type LedgerCorruptError struct {
	Path  string
	Seq   uint64
	Cause error
}

LedgerCorruptError reports that a ledger file could not be trusted: a frame at Seq is present-but-invalid (CRC mismatch, over-ceiling length) or the sequence is non-contiguous. Unlike a torn tail, this is not recoverable by truncation, so every operation fails closed rather than silently drop data. Cause carries the underlying frame fault or errNonContiguousSeq.

func (*LedgerCorruptError) Error

func (e *LedgerCorruptError) Error() string

func (*LedgerCorruptError) Unwrap

func (e *LedgerCorruptError) Unwrap() error

Unwrap exposes the underlying frame fault (a *FrameError) or errNonContiguousSeq.

type LedgerIOError

type LedgerIOError struct {
	Op    string
	Path  string
	Cause error
}

LedgerIOError wraps an underlying filesystem failure (open, read, write, fsync, truncate, mkdir, stat) with the operation and path for diagnosis. Cause is the os/syscall error and is exposed via Unwrap.

func (*LedgerIOError) Error

func (e *LedgerIOError) Error() string

func (*LedgerIOError) Unwrap

func (e *LedgerIOError) Unwrap() error

Unwrap exposes the underlying filesystem error.

type LedgerPathError

type LedgerPathError struct {
	Name string
	Path string
}

LedgerPathError reports that a ledger Name mapped to a filesystem location outside the store root. It is defense in depth: ValidateName already forbids the '..' and empty segments that could escape, so a valid name never triggers it — a triggered LedgerPathError means an unvalidated name reached path mapping.

func (*LedgerPathError) Error

func (e *LedgerPathError) Error() string

type LedgerRootError

type LedgerRootError struct {
	Root   string
	Reason string
}

LedgerRootError reports that newLedgerStore was given an unusable root (for example, an empty path).

func (*LedgerRootError) Error

func (e *LedgerRootError) Error() string

type Options

type Options struct {
	// Root is the store root directory. It is created at 0700 if absent. Required:
	// an empty Root is rejected with an *OptionsError.
	Root string
}

Options configures Open. Root is the single directory under which the store lays out its four backends (streams/, leases/, kv/, blobs/); it is required.

type OptionsError

type OptionsError struct {
	Field  string
	Reason string
	Cause  error
}

OptionsError reports an invalid or unusable Open option. Field names the option (currently always "Root") and Reason explains the fault; Cause carries the underlying filesystem error (or errRootNotDir) when the fault was not a plain validation failure, and is exposed via Unwrap for errors.Is/As.

func (*OptionsError) Error

func (e *OptionsError) Error() string

func (*OptionsError) Unwrap

func (e *OptionsError) Unwrap() error

Unwrap exposes the underlying cause (a filesystem error or errRootNotDir), or nil for a pure validation failure such as an empty Root.

type Store

type Store struct {
	*storage.Composite
	// contains filtered or unexported fields
}

Store is an open fsstore rooted at a single directory on the local filesystem. It embeds *storage.Composite, so a caller reaches each primitive as a promoted field — store.Ledger, store.Leaser, store.KV, store.Blobs — and hands the whole bundle to a consumer as store.Composite or via Backend. The four primitives collide on method names, so no single type can implement all four (see the file-level comment); embedding the field-bundle is how Store sidesteps that.

func Open

func Open(opts Options) (*Store, error)

Open assembles a filesystem-backed storage bundle under opts.Root. It requires a non-empty Root (empty -> *OptionsError), creates it at 0700, and verifies it resolves to a directory. It then wires the four backends under that root — <root>/streams, <root>/leases, <root>/kv, <root>/blobs, each created at 0700 by its constructor — and bundles them with storage.NewComposite.

On any backend-construction failure Open returns that backend's typed error and no Store; because every fs backend only creates directories (none holds an open file descriptor until first use), a partial failure leaks no handles and needs no unwind.

func (*Store) Backend

func (s *Store) Backend() *storage.Composite

Backend returns the assembled four-primitive bundle to hand to a consumer such as sessionstore.Open. It is the embedded *storage.Composite; callers may read store.Composite directly instead.

func (*Store) Close

func (s *Store) Close() error

Close releases the store's in-process state. Concretely it drops the ledger backend's cached name->file registry (the tip/size metadata retained after the last write). The fs backends hold no long-lived file descriptors of their own — the ledger opens and closes its fd within each Append/Delete (the per-append advisory-lock model), the KV and Blobs writers do the same, and a Leaser grant's fd is owned and released by the caller via Lease.Release — so there are no open handles for Close to reclaim beyond that in-memory registry.

Close is idempotent: a second call is a no-op returning nil. After Close the Store must not be reused; open a fresh Store on the same root to use it again.

func (*Store) StoragePaths

func (s *Store) StoragePaths() []string

StoragePaths reports the canonical local root that contains every primitive. The result is newly allocated so callers cannot mutate subsequent reports.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL