backend

package
v0.12.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 12, 2026 License: Apache-2.0 Imports: 6 Imported by: 0

Documentation

Overview

Package backend defines where ciphertext lives. The Backend interface is a flat object store keyed by base-relative path: it only moves opaque bytes. The append-only segment/snapshot layout a namespace is assembled from lives a layer up, in internal/secrets. RcloneStorage is the only implementation.

Index

Constants

This section is empty.

Variables

View Source
var ErrHeaderChanged = errors.New("the header changed since this operation started")

ErrHeaderChanged is returned by SwapHeader when the stored header does not match the bytes the caller's operation started from: another writer landed first. The caller re-reads, re-applies its change, and retries.

View Source
var ErrNotFound = errors.New("object not found")

ErrNotFound is returned by Get and Delete when no object exists at the key.

View Source
var ErrRcloneMissing = errors.New("rclone not found in PATH")

ErrRcloneMissing is returned when no rclone binary is on PATH.

Functions

func CreateRemote

func CreateRemote(ctx context.Context, name, kind string, params map[string]string) error

CreateRemote drives `rclone config create` into the user's global rclone config. Params pass via argv: briefly visible in /proc to same-user processes, but with no shell nothing lands in history. Acceptable for bucket credentials, which guard only ciphertext; weigh it for SFTP/WebDAV passwords, which may guard a whole server (prefer key-based SFTP auth). Revisit if rclone grows a stdin-based config path.

func ListRemotes

func ListRemotes(ctx context.Context) ([]string, error)

ListRemotes returns the names of the user's configured rclone remotes.

func RcloneInstalled

func RcloneInstalled() bool

RcloneInstalled reports whether an rclone binary is available.

func RemoteType

func RemoteType(ctx context.Context, name string) (string, error)

RemoteType returns a remote's backend type (for example "b2" or "s3"). Reads the local rclone config only, no network.

Types

type Backend

type Backend interface {
	// Get returns the object stored at key (or ErrNotFound).
	Get(ctx context.Context, key string) ([]byte, error)
	// Put stores data at key, overwriting any existing object.
	Put(ctx context.Context, key string, data []byte) error
	// List returns the keys of every object under prefix, base-relative and
	// recursive. An absent prefix yields no keys, not an error.
	List(ctx context.Context, prefix string) ([]string, error)
	// Delete removes the object at key. Removing an absent key is not an error.
	Delete(ctx context.Context, key string) error
}

Backend is a flat object store. Keys are base-relative paths (for example "myapp/snapshot.age"); the store prepends its own base and moves bytes, nothing more.

type HeaderStore

type HeaderStore interface {
	// GetHeader returns the raw header object (or ErrNotFound).
	GetHeader(ctx context.Context) ([]byte, error)
	// PutHeader stores the raw header object unconditionally. Mutations of an
	// existing header should go through SwapHeader; PutHeader remains for
	// recovery paths that must overwrite no matter what.
	PutHeader(ctx context.Context, raw []byte) error
	// SwapHeader stores updated iff the current header bytes equal base (nil
	// base: no header may exist yet), and returns ErrHeaderChanged otherwise:
	// the compare-and-swap every concurrent header mutation serializes on.
	// Implementations make this as atomic as their storage allows; see each
	// implementation for the guarantee it actually provides.
	SwapHeader(ctx context.Context, base, updated []byte) error
	// BackupHeader preserves the current header before an overwrite, so a
	// clobbered header doesn't lock the user out of every blob. It is a no-op
	// when the remote keeps native object versions (the versions ARE the
	// backup) or when no header exists yet; otherwise it copies the header to
	// a sibling backup object. The safe-write protocol calls it before
	// PutHeader and refuses to proceed if it errors.
	BackupHeader(ctx context.Context) error
	// RestoreHeaderBackup copies the sibling backup object back over the
	// header, the recovery counterpart to BackupHeader. It returns
	// ErrNotFound when no backup exists. On versioned remotes there is no
	// ".prev" backup (restore a prior object version with rclone instead), so
	// implementations there return ErrNotFound.
	RestoreHeaderBackup(ctx context.Context) error
}

HeaderStore is implemented by client-side-crypto backends, which keep the key-slot header next to the ciphertext objects (see internal/crypto: LUKS2-style wrapped master key). Backends where the provider holds plaintext have no key material and won't implement it.

type RcloneStorage

type RcloneStorage struct {
	Remote string // rclone remote name, e.g. "b2"
	Base   string // path within the remote, e.g. "my-bucket/notenv"
	// Versioned: the remote retains old versions on overwrite (B2 does
	// natively), so the header's ".prev" backup copy (~3s server-side on B2)
	// is redundant and skipped (see BackupHeader).
	Versioned bool
}

RcloneStorage implements Backend by shelling out to a system rclone. This keeps the binary small and the dependency explicit; embedding the library is a possible later optimization.

func (*RcloneStorage) BackupHeader added in v0.2.0

func (s *RcloneStorage) BackupHeader(ctx context.Context) error

BackupHeader copies the current header to its ".prev" sibling so a bad overwrite is recoverable. It is a no-op when the remote keeps native object versions (those versions are the backup) and when no header exists yet (nothing to preserve). Any other copy failure is returned so the caller can refuse to overwrite a header it couldn't back up.

func (*RcloneStorage) Delete added in v0.3.0

func (s *RcloneStorage) Delete(ctx context.Context, key string) error

func (*RcloneStorage) Get

func (s *RcloneStorage) Get(ctx context.Context, key string) ([]byte, error)

func (*RcloneStorage) GetHeader

func (s *RcloneStorage) GetHeader(ctx context.Context) ([]byte, error)

func (*RcloneStorage) List

func (s *RcloneStorage) List(ctx context.Context, prefix string) ([]string, error)

List returns base-relative keys of every object under prefix, recursively.

func (*RcloneStorage) Preflight

func (s *RcloneStorage) Preflight(ctx context.Context) error

Preflight verifies rclone is installed and the remote exists.

func (*RcloneStorage) Probe

func (s *RcloneStorage) Probe(ctx context.Context) error

Probe round-trips a marker object through the configured base path so a bad credential or bucket fails here, with context, not at the first real `set` days later.

func (*RcloneStorage) Put

func (s *RcloneStorage) Put(ctx context.Context, key string, data []byte) error

func (*RcloneStorage) PutHeader

func (s *RcloneStorage) PutHeader(ctx context.Context, raw []byte) error

PutHeader writes the header object. It does NOT back up first: the safe-write protocol (internal/keymgmt) calls BackupHeader before this, because a clobbered header locks the user out of every blob under it.

func (*RcloneStorage) RestoreHeaderBackup added in v0.2.0

func (s *RcloneStorage) RestoreHeaderBackup(ctx context.Context) error

RestoreHeaderBackup copies the ".prev" backup back over the header. Returns ErrNotFound when there is no backup to restore, including on versioned remotes, which keep no ".prev" (use rclone's version listing to recover a prior object version there).

func (*RcloneStorage) SwapHeader added in v0.8.0

func (s *RcloneStorage) SwapHeader(ctx context.Context, base, updated []byte) error

SwapHeader implements the compare-and-swap as read-compare-put-readback, which is the strongest rclone offers: object stores expose no conditional write through it. Two writers that both pass the compare inside the same sub-second window still last-write-wins; the read-back converts the loss into ErrHeaderChanged whenever the winner's bytes have already landed, and the one ordering it cannot see (our read-back completes before the winner's put) is recovered by the manifest's adoption path, never lost silently. A backend with native conditional writes can implement this atomically.

Directories

Path Synopsis
Package backendtest holds the shared conformance suites for backend implementations.
Package backendtest holds the shared conformance suites for backend implementations.
Package chaos wraps a backend.Backend and injects deterministic faults from a seed, for torture-testing code that must stay correct when storage misbehaves.
Package chaos wraps a backend.Backend and injects deterministic faults from a seed, for torture-testing code that must stay correct when storage misbehaves.
Package local is a pure-Go backend over a directory: the zero-account, zero-dependency vault.
Package local is a pure-Go backend over a directory: the zero-account, zero-dependency vault.
Package memstore is an in-memory backend.HeaderStore (and backend.Backend) for tests.
Package memstore is an in-memory backend.HeaderStore (and backend.Backend) for tests.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL