backend

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 26, 2026 License: Apache-2.0 Imports: 5 Imported by: 0

Documentation

Overview

Package backend defines the L1 storage seam (DESIGN.md §3, §5): a common Backend interface (Read/Write/List/Delete over whole-object keys) with interchangeable implementations. The in-memory backend (Memory, in this package) is first-class — the reference and test default — so the whole engine runs with no disk or object store. The file backend lives in backend/file. The s3 backend and compare-and-swap (CAS) are M5.

Index

Constants

This section is empty.

Variables

View Source
var ErrNotExist = errors.New("backend: key does not exist")

ErrNotExist is the sentinel returned (wrapped) by Backend.Read and Backend.Delete when a key is absent. Test for it with errors.Is.

Functions

This section is empty.

Types

type Backend

type Backend interface {
	// IsEphemeral reports whether the backend stores data only in RAM (dropped on
	// process exit). [Memory] is ephemeral; file and s3 are not.
	IsEphemeral() bool

	// PutIfAbsent stores data under key only if the key does not already exist. It
	// returns true if the write happened, false if the key was already present (no
	// change). Like [Backend.Write] it is atomic per object. It is the compare-and-swap
	// primitive for manifest commits.
	PutIfAbsent(ctx context.Context, key string, data []byte) (bool, error)

	// Write stores data under key, overwriting any existing value. The write is
	// atomic per object: a reader never observes a partially written value. The
	// implementation takes ownership semantics by copying data as needed; callers may
	// reuse the buffer after Write returns.
	Write(ctx context.Context, key string, data []byte) error

	// Read returns the value stored under key. It returns an error satisfying
	// errors.Is(err, [ErrNotExist]) if the key is absent. The returned slice is owned
	// by the caller (implementations return a fresh copy, never aliased state).
	Read(ctx context.Context, key string) ([]byte, error)

	// List returns, sorted ascending, every key with the given prefix (empty prefix
	// lists all keys).
	List(ctx context.Context, prefix string) ([]string, error)

	// Delete removes key. It returns an error satisfying errors.Is(err, [ErrNotExist])
	// if the key is absent.
	Delete(ctx context.Context, key string) error
}

Backend is the L1 storage seam (DESIGN.md §3, §5): a common interface over interchangeable implementations — memory (ephemeral, the reference), file, and (later) s3. The same engine code path runs over all three.

Data is addressed by an opaque, slash-delimited string key (e.g. a time-bucketed object path or a file-relative path). Values are whole objects: the part format (`block`) maps one part to a key prefix and one object per column/marks/manifest, so whole-object Read/Write is sufficient and gives per-object write atomicity. All methods are safe for concurrent use.

Ranged/streaming reads are deliberately not part of this interface: a part column is read whole, and the multi-key layout already gives projection pushdown (read only the referenced column objects) without ranged reads.

Backend.PutIfAbsent is the conditional-write primitive (added in M5) on which atomic manifest / block-list commits build: a versioned manifest key is written only if no writer has claimed that version, so single-writer-wins coordination needs no Raft (it maps to S3 If-None-Match, a filesystem exclusive create, and a guarded map insert).

func Memory

func Memory() Backend

Memory returns an ephemeral in-memory Backend (DESIGN.md §5): the whole engine runs over it with no disk or object store; objects live in RAM and are dropped when the process exits. It is the reference implementation and the default in tests.

Directories

Path Synopsis
Package backendtest provides a shared conformance suite that every backend.Backend implementation must pass, proving the implementations are interchangeable (DESIGN.md §2: "backends are interchangeable behind backend.Backend").
Package backendtest provides a shared conformance suite that every backend.Backend implementation must pass, proving the implementations are interchangeable (DESIGN.md §2: "backends are interchangeable behind backend.Backend").
Package bucketindex maintains a compact, incremental index of the immutable parts under a key prefix, so a stateless reader enumerates a tenant's parts (and prunes them by time) from a single object instead of a full, expensive bucket LIST (DESIGN.md §11, the object-store-native read path).
Package bucketindex maintains a compact, incremental index of the immutable parts under a key prefix, so a stateless reader enumerates a tenant's parts (and prunes them by time) from a single object instead of a full, expensive bucket LIST (DESIGN.md §11, the object-store-native read path).
Package file implements a backend.Backend over a local directory tree.
Package file implements a backend.Backend over a local directory tree.
Package s3 implements a backend.Backend over an S3-compatible object store.
Package s3 implements a backend.Backend over an S3-compatible object store.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL