metadatadb

package
v0.0.0-...-e308f56 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 10, 2026 License: Apache-2.0 Imports: 15 Imported by: 0

Documentation

Overview

Package metadatadb provides an eventually consistent metadata store for coordinating state across cachew replicas. Mutations are applied to local state immediately and synced periodically to a shared backend. Last flush wins — the lock serialises all writes.

Index

Constants

This section is empty.

Variables

View Source
var ErrNotFound = errors.New("metadata backend not found")

ErrNotFound is returned when a metadata backend is not found.

Functions

func Register

func Register[Config any, B Backend](r *Registry, id, description string, factory Factory[Config, B])

Register a metadata backend factory function.

func RegisterMemory

func RegisterMemory(r *Registry)

RegisterMemory registers the in-memory metadata backend.

func RegisterS3

func RegisterS3(r *Registry, clientProvider s3client.ClientProvider)

RegisterS3 registers the S3 metadata backend. The clientProvider supplies the shared minio client constructed from the global s3 config block.

Types

type Backend

type Backend interface {
	// Apply applies one or more write operations to the given namespace.
	Apply(ctx context.Context, namespace string, ops ...Op) error
	// Query executes a read query and unmarshals the result into target.
	// Target must be a pointer to the expected result type.
	Query(ctx context.Context, namespace string, q ReadOp, target any) error
	// Flush forces pending state to be persisted for the given namespace.
	Flush(ctx context.Context, namespace string) error
	// Close shuts down the backend, flushing any pending state.
	Close(ctx context.Context) error
}

Backend is the pluggable storage layer for the metadata store. Implementations handle write operations (Apply), read queries (Query), and persistence (Flush).

type Factory

type Factory[Config any, B Backend] func(ctx context.Context, config Config) (B, error)

Factory is a function that creates a new Backend from the given hcl-tagged configuration struct.

type Int

type Int struct {
	// contains filtered or unexported fields
}

Int is an integer with arithmetic operations.

func NewInt

func NewInt(ns *Namespace, name string) *Int

NewInt creates or retrieves a named integer within a namespace.

func (*Int) Add

func (i *Int) Add(delta int64)

func (*Int) Div

func (i *Int) Div(divisor int64)

func (*Int) Get

func (i *Int) Get() int64

func (*Int) Mul

func (i *Int) Mul(factor int64)

func (*Int) Set

func (i *Int) Set(value int64)

type IntAdd

type IntAdd struct {
	Key   string
	Delta int64
}

IntAdd adds a delta to an integer.

type IntDiv

type IntDiv struct {
	Key     string
	Divisor int64
}

IntDiv divides an integer by a divisor. Zero divisor is a no-op.

type IntGet

type IntGet struct{ Key string }

IntGet reads an integer value. Returns int64 (zero if absent).

type IntMap

type IntMap[K comparable] struct {
	// contains filtered or unexported fields
}

IntMap is a keyed collection of integer values supporting atomic increment.

func NewIntMap

func NewIntMap[K comparable](ns *Namespace, name string) *IntMap[K]

NewIntMap creates or retrieves a named integer map within a namespace.

func (*IntMap[K]) Add

func (m *IntMap[K]) Add(key K, delta int64)

func (*IntMap[K]) Delete

func (m *IntMap[K]) Delete(key K)

func (*IntMap[K]) Div

func (m *IntMap[K]) Div(key K, divisor int64)

func (*IntMap[K]) Entries

func (m *IntMap[K]) Entries() map[K]int64

func (*IntMap[K]) Get

func (m *IntMap[K]) Get(key K) int64

func (*IntMap[K]) Keys

func (m *IntMap[K]) Keys() []K

func (*IntMap[K]) Mul

func (m *IntMap[K]) Mul(key K, factor int64)

func (*IntMap[K]) Set

func (m *IntMap[K]) Set(key K, value int64)

type IntMapAdd

type IntMapAdd struct {
	Key    string
	MapKey any
	Delta  int64
}

IntMapAdd adds a delta to a keyed integer.

type IntMapDelete

type IntMapDelete struct {
	Key    string
	MapKey any
}

IntMapDelete removes a key from an integer map.

type IntMapDiv

type IntMapDiv struct {
	Key     string
	MapKey  any
	Divisor int64
}

IntMapDiv divides a keyed integer by a divisor. Zero divisor is a no-op.

type IntMapEntries

type IntMapEntries struct{ Key string }

IntMapEntries returns all entries in an integer map.

type IntMapGet

type IntMapGet struct {
	Key    string
	MapKey any
}

IntMapGet reads a keyed integer value. Returns int64 (zero if absent).

type IntMapKeys

type IntMapKeys struct{ Key string }

IntMapKeys returns all keys in an integer map.

type IntMapMul

type IntMapMul struct {
	Key    string
	MapKey any
	Factor int64
}

IntMapMul multiplies a keyed integer by a factor.

type IntMapSet

type IntMapSet struct {
	Key    string
	MapKey any
	Value  int64
}

IntMapSet sets a keyed integer to an exact value.

type IntMul

type IntMul struct {
	Key    string
	Factor int64
}

IntMul multiplies an integer by a factor.

type IntSet

type IntSet struct {
	Key   string
	Value int64
}

IntSet sets an integer to an exact value.

type List

type List[V any] struct {
	// contains filtered or unexported fields
}

List is an append-only ordered collection.

func NewList

func NewList[V any](ns *Namespace, name string) *List[V]

NewList creates or retrieves a named append-only list within a namespace.

func (*List[V]) Append

func (l *List[V]) Append(value V)

func (*List[V]) Entries

func (l *List[V]) Entries() []V

func (*List[V]) Len

func (l *List[V]) Len() int

type ListAppend

type ListAppend struct {
	Key   string
	Value any
}

ListAppend appends a value to a list.

type ListEntries

type ListEntries struct{ Key string }

ListEntries returns all list entries as []any.

type ListLen

type ListLen struct{ Key string }

ListLen returns the length of a list as int.

type Map

type Map[K comparable, V any] struct {
	// contains filtered or unexported fields
}

Map is a keyed collection of values. Last write per key wins.

func NewMap

func NewMap[K comparable, V any](ns *Namespace, name string) *Map[K, V]

NewMap creates or retrieves a named map within a namespace.

func (*Map[K, V]) Delete

func (m *Map[K, V]) Delete(key K)

func (*Map[K, V]) Entries

func (m *Map[K, V]) Entries() map[K]V

func (*Map[K, V]) Get

func (m *Map[K, V]) Get(key K) (V, bool)

func (*Map[K, V]) Keys

func (m *Map[K, V]) Keys() []K

func (*Map[K, V]) Set

func (m *Map[K, V]) Set(key K, value V)

type MapDelete

type MapDelete struct {
	Key    string
	MapKey any
}

MapDelete removes a key from a map.

type MapEntries

type MapEntries struct{ Key string }

MapEntries returns all entries in a map.

type MapGet

type MapGet struct {
	Key    string
	MapKey any
}

MapGet reads a keyed value from a map. Returns (value, true) or (nil, false).

type MapKeys

type MapKeys struct{ Key string }

MapKeys returns all keys in a map.

type MapSet

type MapSet struct {
	Key    string
	MapKey any
	Value  any
}

MapSet sets a keyed value in a map. Last write per key wins.

type MemoryBackend

type MemoryBackend struct {
	// contains filtered or unexported fields
}

MemoryBackend is an in-memory Backend for testing and single-instance deployments. Ops are applied directly — there is no sync or persistence.

func NewMemoryBackend

func NewMemoryBackend() *MemoryBackend

func (*MemoryBackend) Apply

func (m *MemoryBackend) Apply(_ context.Context, namespace string, ops ...Op) error

func (*MemoryBackend) Close

func (m *MemoryBackend) Close(_ context.Context) error

func (*MemoryBackend) Flush

func (m *MemoryBackend) Flush(_ context.Context, _ string) error

func (*MemoryBackend) Query

func (m *MemoryBackend) Query(_ context.Context, namespace string, q ReadOp, target any) error

type MemoryConfig

type MemoryConfig struct{}

MemoryConfig is the configuration for the in-memory metadata backend.

type Namespace

type Namespace struct {
	// contains filtered or unexported fields
}

Namespace is a scoped collection of named data structures.

func (*Namespace) Flush

func (n *Namespace) Flush(ctx context.Context) error

Flush forces an immediate sync with the backend.

type Op

type Op interface {
	// contains filtered or unexported methods
}

Op is a write operation applied to the metadata store. Concrete types form a closed set (sum type) — backends handle each variant via exhaustive type switch.

type ReadOp

type ReadOp interface {
	// contains filtered or unexported methods
}

ReadOp is a read query against the metadata store. Like Op, concrete types form a closed set dispatched by the backend.

type Registry

type Registry struct {
	// contains filtered or unexported fields
}

Registry holds registered metadata backend factories.

func NewRegistry

func NewRegistry() *Registry

NewRegistry creates a new metadata backend registry.

func (*Registry) Create

func (r *Registry) Create(ctx context.Context, name string, config *hcl.Block, vars map[string]string) (Backend, error)

Create a new Backend from the given name and configuration.

Returns ErrNotFound if the backend is not found.

func (*Registry) Exists

func (r *Registry) Exists(name string) bool

Exists returns true if a backend with the given name is registered.

func (*Registry) Schema

func (r *Registry) Schema() *hcl.AST

Schema returns the schema for all registered metadata backends.

type S3Backend

type S3Backend struct {
	// contains filtered or unexported fields
}

S3Backend stores metadata state as JSON objects in S3 with periodic sync. Writes are applied to local state immediately and queued for the next flush. Locking uses a separate lock object with TTL-based expiry for stale lock recovery. The idempotence token maps to the S3 object ETag.

func NewS3Backend

func NewS3Backend(ctx context.Context, clientProvider s3client.ClientProvider, config S3BackendConfig) (*S3Backend, error)

func (*S3Backend) Apply

func (s *S3Backend) Apply(_ context.Context, namespace string, ops ...Op) error

func (*S3Backend) Close

func (s *S3Backend) Close(_ context.Context) error

func (*S3Backend) Flush

func (s *S3Backend) Flush(ctx context.Context, namespace string) error

func (*S3Backend) Query

func (s *S3Backend) Query(_ context.Context, namespace string, q ReadOp, target any) error

type S3BackendConfig

type S3BackendConfig struct {
	Bucket       string        `hcl:"bucket" help:"S3 bucket name."`
	LockTTL      time.Duration `hcl:"lock-ttl,optional" help:"TTL for namespace locks." default:"30s"`
	SyncInterval time.Duration `hcl:"sync-interval,optional" help:"Interval between periodic syncs." default:"30s"`
}

S3BackendConfig configures the S3 metadata backend.

type Scalar

type Scalar[V any] struct {
	// contains filtered or unexported fields
}

Scalar is a single value. Last write wins.

func NewScalar

func NewScalar[V any](ns *Namespace, name string) *Scalar[V]

NewScalar creates or retrieves a named scalar within a namespace.

func (*Scalar[V]) Delete

func (s *Scalar[V]) Delete()

func (*Scalar[V]) Get

func (s *Scalar[V]) Get() (V, bool)

func (*Scalar[V]) Set

func (s *Scalar[V]) Set(value V)

type ScalarDelete

type ScalarDelete struct{ Key string }

ScalarDelete removes a scalar value.

type ScalarGet

type ScalarGet struct{ Key string }

ScalarGet reads a scalar value. Returns (value, true) or (nil, false).

type ScalarSet

type ScalarSet struct {
	Key   string
	Value any
}

ScalarSet sets a scalar value. Last write wins.

type Set

type Set[V comparable] struct {
	// contains filtered or unexported fields
}

Set is an unordered collection of unique members.

func NewSet

func NewSet[V comparable](ns *Namespace, name string) *Set[V]

NewSet creates or retrieves a named set within a namespace.

func (*Set[V]) Add

func (s *Set[V]) Add(member V)

func (*Set[V]) Contains

func (s *Set[V]) Contains(member V) bool

func (*Set[V]) Members

func (s *Set[V]) Members() []V

func (*Set[V]) Remove

func (s *Set[V]) Remove(member V)

type SetAdd

type SetAdd struct {
	Key    string
	Member any
}

SetAdd adds a member to a set. Idempotent.

type SetContains

type SetContains struct {
	Key    string
	Member any
}

SetContains checks whether a member exists in a set. Returns bool.

type SetMembers

type SetMembers struct{ Key string }

SetMembers returns all set members.

type SetRemove

type SetRemove struct {
	Key    string
	Member any
}

SetRemove removes a member from a set. No-op if absent.

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store is the top-level metadata store.

func New

func New(ctx context.Context, backend Backend) *Store

New creates a new metadata store backed by the given Backend.

func (*Store) Close

func (s *Store) Close(ctx context.Context) error

Close shuts down the store and its backend.

func (*Store) Namespace

func (s *Store) Namespace(name string) *Namespace

Namespace returns the namespace with the given name, creating it if needed.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL