store

package
v0.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 5, 2026 License: Apache-2.0 Imports: 4 Imported by: 0

Documentation

Overview

Package store defines the storage of the server's databases: blocks, versions and leases. Blocks are ciphertext and are stored as they arrive. Every implementation must pass storetest.Run.

Index

Constants

View Source
const (
	MinPageSize = 512
	MaxPageSize = 65536
)

Page size limits of SQLite.

View Source
const ChangeWindow = 1024

ChangeWindow is the number of recent commits for which a store keeps the indexes of the changed blocks. A client whose local copy is older than the window must reload the database.

At about ten blocks per commit (measured), the change log takes less than 100 KB per database. The window is larger than a client needs: once more than about a third of the database has changed, the client reloads the whole database anyway.

Variables

View Source
var (
	// ErrNotFound means the database does not exist or was deleted.
	ErrNotFound = errors.New("database not found")
	// ErrFenced means the lease epoch is not the current one. Another instance took the lease, or it was released.
	ErrFenced = errors.New("lease epoch is stale")
	// ErrSlotTaken means another key holds the owner's slot.
	ErrSlotTaken = errors.New("another key holds the owner's slot")
)

Functions

func BadRequest

func BadRequest(format string, args ...any) error

BadRequest builds a BadRequestError.

func CheckRange

func CheckRange(first, count, pageCount uint64) error

CheckRange checks that the count blocks starting at first lie within pageCount.

func ValidPageSize

func ValidPageSize(size uint32) bool

ValidPageSize reports whether SQLite allows the page size.

func ValidateCommit

func ValidateCommit(pageSize uint32, req CommitRequest) error

ValidateCommit checks the commit ID and the blocks of a commit against the database's page size and the new page count.

Types

type BadRequestError

type BadRequestError struct {
	Reason string
}

BadRequestError means the request is invalid. Repeating it cannot succeed.

func (*BadRequestError) Error

func (e *BadRequestError) Error() string

type Block

type Block struct {
	Index uint64
	Data  []byte
}

Block is one page-sized block of the database file.

type ChangeSet

type ChangeSet struct {
	FromVersion uint64
	ToVersion   uint64
	// Blocks holds the index of every block changed by a commit in between, sorted and without duplicates. It is
	// only valid if Complete is true.
	Blocks []uint64
	// Complete is false if the change log no longer reaches back to FromVersion.
	Complete bool
}

ChangeSet lists the blocks changed between two versions.

type ClaimRequest added in v0.3.0

type ClaimRequest struct {
	Owner   string
	Subject string
	Now     time.Time
}

ClaimRequest passes an owner's slot to a key.

type ClaimResult added in v0.3.0

type ClaimResult struct {
	// Slot is the owner's slot after the claim.
	Slot Slot
	// Replaced is the slot before the claim if another key held it, otherwise nil.
	Replaced *Slot
	// Deleted lists the databases of the replaced key. The claim deleted them completely.
	Deleted []Key
}

ClaimResult describes a claim.

type CommitRequest

type CommitRequest struct {
	Key         Key
	CommitID    []byte
	LeaseEpoch  uint64
	BaseVersion uint64
	// PageCount is the file length in blocks after the commit. Blocks at or beyond it are deleted.
	PageCount uint64
	Blocks    []Block
	Now       time.Time
	TTL       time.Duration
}

CommitRequest applies the blocks changed by one or more SQLite transactions in one atomic step.

type DeleteRequest

type DeleteRequest struct {
	Key Key
	// Takeover deletes even if another instance holds an unexpired lease.
	Takeover bool
	Now      time.Time
}

DeleteRequest deletes a database.

type DeleteResult

type DeleteResult struct {
	// Epoch is the lease epoch after the deletion. Every lease with a lower epoch is fenced. 0 if nothing was deleted
	// because the database did not exist or was already deleted.
	Epoch uint64
	// Revoked is true if the deletion took an unexpired lease from its holder.
	Revoked bool
}

DeleteResult describes a deletion.

type Key

type Key struct {
	Subject string
	DBID    string
}

Key identifies a database by the subject that owns it and its ID within that subject.

type Lease

type Lease struct {
	ID []byte
	// Epoch increases with every new lease. Requests with an older epoch are rejected (fencing).
	Epoch uint64
}

Lease is the exclusive right of one client instance to commit to a database.

type LeaseHeldError

type LeaseHeldError struct {
	Since time.Time
}

LeaseHeldError means another instance holds a lease that has not expired.

func (*LeaseHeldError) Error

func (e *LeaseHeldError) Error() string

type OpenRequest

type OpenRequest struct {
	Key Key
	// Owner is the owner of the client's key, the `sub` of its access token. Empty without access tokens. If another
	// key holds the owner's slot, Open returns ErrSlotTaken, also for a Resume. See Store.ClaimSlot.
	Owner      string
	InstanceID []byte
	// PageSize is required to create a database. For an existing database it must be 0 or match.
	PageSize uint32
	Create   bool
	// Takeover takes the lease even if another instance holds an unexpired lease. The instance that holds the lease
	// (the same InstanceID) gets a new lease without it: a client that restarted has lost its lease ID and cannot
	// resume. Its earlier lease is fenced, and Revoked is set.
	Takeover bool
	// Resume, if set, continues the given lease instead of granting a new one.
	Resume *Resume
	Now    time.Time
	TTL    time.Duration
}

OpenRequest opens a database and takes its lease.

type OpenResult

type OpenResult struct {
	Lease Lease
	State State
	// Revoked is true if the open took an unexpired lease, from another instance or from an earlier lease of the same
	// instance.
	Revoked bool
}

OpenResult holds the lease and the database state after a successful open.

type Resume

type Resume struct {
	LeaseID    []byte
	LeaseEpoch uint64
}

Resume identifies a lease to continue after a reconnect.

type Slot added in v0.3.0

type Slot struct {
	Subject   string
	Label     string
	ClaimedAt time.Time
}

Slot is an owner's slot: the key that holds it, identified by its subject, and a label, e.g. the id of the client's device. An owner has at most one slot.

type State

type State struct {
	PageSize     uint32
	PageCount    uint64
	Version      uint64
	LastCommitID []byte
}

State describes a database at one version.

type Store

type Store interface {
	Open(ctx context.Context, req OpenRequest) (OpenResult, error)
	// Fetch returns count blocks starting at index first. version must be the current version.
	Fetch(ctx context.Context, key Key, version, first, count uint64) ([][]byte, error)
	// Commit applies a commit and returns the new version. If the request repeats the last commit (same commit
	// ID), Commit applies nothing and returns that commit's version.
	Commit(ctx context.Context, req CommitRequest) (uint64, error)
	// Changes returns the blocks changed since fromVersion. Clients use it to update a stale local copy. A
	// fromVersion newer than the current version is a bad request. If fromVersion is older than the change window,
	// the result has Complete set to false.
	Changes(ctx context.Context, key Key, fromVersion uint64) (ChangeSet, error)
	// Renew extends the lease with the given epoch.
	Renew(ctx context.Context, key Key, epoch uint64, now time.Time, ttl time.Duration) error
	// Release gives up the lease with the given epoch.
	Release(ctx context.Context, key Key, epoch uint64) error
	// Delete deletes a database: its blocks and its change log. The store keeps a record of the database and
	// increases its lease epoch and version, so that no earlier lease can commit again, also after the database was
	// created anew. Deleting a missing or deleted database succeeds and changes nothing. A deleted database behaves
	// as missing, except that Open with Resume returns ErrFenced and Open with Create continues epoch and version.
	Delete(ctx context.Context, req DeleteRequest) (DeleteResult, error)
	// DeleteUnused deletes, as Delete does, up to limit databases whose lease expired before cutoff, oldest first,
	// and returns their keys. Every open, commit and lease renewal extends the lease, so its expiry marks the last
	// use of the database. A database whose lease expires at or after cutoff is kept. Several callers may run at
	// once; each database is deleted by one of them.
	//
	// The databases of a key that holds a slot are deleted completely, without a record. A slot that was claimed
	// before cutoff and whose key has no databases left is released.
	DeleteUnused(ctx context.Context, cutoff time.Time, limit int) ([]Key, error)
	// GetSlot returns the owner's slot, and false if the owner has none.
	GetSlot(ctx context.Context, owner string) (Slot, bool, error)
	// ClaimSlot passes the owner's slot to req.Subject, with an empty label. If another key held it, all databases of
	// that key are deleted completely: unlike Delete, no record stays. Claiming a slot that req.Subject already holds
	// changes nothing.
	ClaimSlot(ctx context.Context, req ClaimRequest) (ClaimResult, error)
	// SetSlotLabel sets the label of the owner's slot if subject holds it. It returns ErrSlotTaken if another key holds
	// the slot and ErrNotFound if the owner has none.
	SetSlotLabel(ctx context.Context, owner, subject, label string) (Slot, error)
	// DeleteSlot releases the owner's slot and deletes all databases of subject completely. If another key holds the
	// slot, it returns ErrSlotTaken and deletes nothing. It returns the deleted databases.
	DeleteSlot(ctx context.Context, owner, subject string) ([]Key, error)
	// Ping checks that the store is reachable. The readiness endpoint uses it.
	Ping(ctx context.Context) error
}

Store keeps the databases. Implementations must be safe for concurrent use.

A lease stays valid until another instance takes it or its holder releases it. Expiry only decides whether another instance may take the lease without Takeover. An expired lease that no other instance has taken can still commit and renew.

type VersionConflictError

type VersionConflictError struct {
	Current uint64
}

VersionConflictError means the request was based on a version other than the current one.

func (*VersionConflictError) Error

func (e *VersionConflictError) Error() string

Directories

Path Synopsis
Package memory implements store.Store in memory, for tests and local runs.
Package memory implements store.Store in memory, for tests and local runs.
Package postgres implements store.Store in PostgreSQL.
Package postgres implements store.Store in PostgreSQL.
db
Package storetest contains the contract tests that every store.Store implementation must pass.
Package storetest contains the contract tests that every store.Store implementation must pass.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL