Documentation
¶
Overview ¶
Package replica is the ONE shared HA-SQLite substrate every Hanzo service uses (HIP-0107): per-org SQLite on the local disk for speed, its durable copy in hanzoai/vfs (SeaweedFS-backed object store), a deterministic single-writer election (Rendezvous/HRW over IAM membership — no coordinator, no lock service, no Postgres, no Redis), and per-org at-rest encryption via KMS.
It was proven inside hanzoai/cloud (internal/org) and is promoted here so EVERY service — visor, iam, commerce, base, and every Base adopter — imports the SAME code instead of re-inventing durability. The adoption contract at each service's per-org store seam:
hydrate-on-open : Pull() the latest snapshot before first use of an org DB.
single-writer : ha.IsOwner(key, self, members) gates writes (github.com/hanzoai/ha,
the election primitive); a non-owner serves stale-tolerant reads
(Pull-then-read) and forwards writes.
post-commit ship: the owner runs PushLoop (or Push after a write burst); the
durable copy in vfs means a lost pod loses no committed data.
Index ¶
- Variables
- func DBPath(orgID, scope, service string) string
- func RestoreFile(path string, data []byte) error
- func SnapshotFile(ctx context.Context, path string) ([]byte, error)
- func Version(data []byte) string
- type BackendStore
- type Cipher
- type ConditionalStore
- type DB
- type FencedStore
- func (f *FencedStore) CarryForward(ctx context.Context, key string, round uint64, ...) error
- func (f *FencedStore) Get(ctx context.Context, key string) ([]byte, uint64, error)
- func (f *FencedStore) Put(ctx context.Context, key string, payload []byte, round uint64) error
- func (f *FencedStore) Round(ctx context.Context, key string) (uint64, error)
- type Option
- type Replicator
- type SQLiteDB
- type Store
Constants ¶
This section is empty.
Variables ¶
var ( // ErrNotFound is returned by ConditionalStore.Get for a key never written. // FencedStore treats an absent object as recorded round 0, so any real lease // round (>= 1) is admissible against an empty slot. ErrNotFound = errors.New("replica: fenced object not found") // ErrStaleRound is returned when a Put's round is below the round already // recorded for the object — the deposed/partitioned-writer rejection. The // object is left byte-for-byte untouched. This is the invariant that closes // the split-brain double-write: once a successor advances the recorded round, // the predecessor can never write again. ErrStaleRound = errors.New("replica: write round is below the recorded round (writer fenced)") // ErrConflict is returned when the atomic conditional write lost a race to a // concurrent writer (the object changed between our read and our write). // Put retries internally; ErrConflict escaping Put means the bounded retries // were exhausted under sustained contention, not that a caller should blindly // retry with the same round. ErrConflict = errors.New("replica: conditional write lost a concurrent race") )
Functions ¶
func DBPath ¶
DBPath is the canonical object-store location of one of an org's SQLite DBs (HIP-0302 layout). Every replica computes the same path for the same inputs.
org root: DBPath("acme", "", "iam") -> orgs/acme/iam.db
project: DBPath("acme", "projects/site", "base") -> orgs/acme/projects/site/base.db
user: DBPath("acme", "users/dave", "kv") -> orgs/acme/users/dave/kv.db
func RestoreFile ¶
RestoreFile atomically writes snapshot bytes over the SQLite database at path — the handle-less adoption primitive. The caller MUST have no open handle to path (close the service's engine first; reopen after). Drops the -wal/-shm sidecars so a reopen can't replay stale WAL over the restored file. Used at hydrate-on-open, BEFORE the service opens its own engine.
func SnapshotFile ¶
SnapshotFile returns a consistent snapshot of the SQLite database at path WITHOUT holding a persistent handle — the adoption primitive for a service that manages the DB with its OWN library (xorm, GORM, …): open a transient read handle, VACUUM INTO a temp, return the bytes. In WAL mode the transient handle sees all committed data, so this is consistent even while the service writes.
func Version ¶ added in v0.6.2
Version is the canonical content version for a snapshot — SHA-256 of the bytes, matching hanzoai/vfs's content-addressable model. Exported so any Store impl (cloud's deps.VFS-backed store, the BackendStore) computes the SAME version the Replicator's skip-redundant-pull logic expects — one hash function, one place.
Types ¶
type BackendStore ¶
type BackendStore struct {
// contains filtered or unexported fields
}
BackendStore is a Store backed by a vfs backend.Backend.
func NewBackendStore ¶
func NewBackendStore(be backend.Backend) *BackendStore
NewBackendStore wraps be as a Replicator Store.
func (*BackendStore) Get ¶
Get returns the snapshot bytes and their authoritative content version (recomputed from the bytes, so it always matches what a writer stored).
type Cipher ¶
type Cipher interface {
Seal(org string, plaintext []byte) ([]byte, error)
Open(org string, ciphertext []byte) ([]byte, error)
}
Cipher seals/opens a snapshot with an org-bound key (envelope encryption via the KMS master). Optional — omit it for local dev (plaintext object).
type ConditionalStore ¶ added in v0.6.4
type ConditionalStore interface {
// Get returns the object bytes and the store's authoritative version token
// (ETag/generation) for key, or a wrapped ErrNotFound when key is absent.
Get(ctx context.Context, key string) (data []byte, version string, err error)
// PutIfVersion writes data at key iff the store's CURRENT version for key
// still equals expectVersion ("" means the object must not exist yet — a
// create-only put, i.e. If-None-Match: *) and returns the new version. A
// failed precondition is reported as a wrapped ErrConflict.
PutIfVersion(ctx context.Context, key string, data []byte, expectVersion string) (newVersion string, err error)
}
ConditionalStore is the atomic compare-and-set object surface the fence needs: the S3 If-Match / GCS generation-match primitive, nothing more. PutIfVersion MUST evaluate its precondition atomically, server-side; a client-side "Get, compare in Go, then Put" is NOT a valid implementation — that non-atomicity is the exact race this fence exists to close. The concrete implementation lives with the caller that owns an S3/SeaweedFS client (e.g. cloud's minio-go If-Match store); this package stays dependency-free over the seam, and the in-memory fake in fence_test.go models the same server-side CAS.
type DB ¶
type DB interface {
Snapshot(ctx context.Context) ([]byte, error)
Restore(ctx context.Context, data []byte) error
}
DB is the local per-org SQLite the owner writes and readers restore into. Snapshot returns a consistent (WAL-checkpointed) copy; Restore replaces the local bytes atomically. Satisfied by *SQLiteDB (sqlite.go).
type FencedStore ¶ added in v0.6.4
type FencedStore struct {
// contains filtered or unexported fields
}
FencedStore is a round-fenced writer over a ConditionalStore: one object per key, a monotone round admitted only when at least the recorded one.
func NewFencedStore ¶ added in v0.6.4
func NewFencedStore(store ConditionalStore) *FencedStore
NewFencedStore builds a FencedStore over an atomic-CAS store.
func (*FencedStore) CarryForward ¶ added in v0.6.4
func (f *FencedStore) CarryForward(ctx context.Context, key string, round uint64, hydrate func(payload []byte) error) error
CarryForward is the safe TAKEOVER seal: it reads the latest durably-stored payload for key and re-seals it UNCHANGED at round, invoking hydrate on that payload first. It exists because a successor must never seal a snapshot it read BEFORE a predecessor's last landed ship, or that acknowledged write would be clobbered. The read and the seal are one compare-and-set on the same version: if a predecessor write lands in between, the CAS conflicts and CarryForward retries — re-reading the newer payload, re-hydrating, and re-sealing IT — so the successor always advances past exactly the state it carried forward, never a stale one. After CarryForward returns, every predecessor write at a round below `round` is fenced.
round is the successor's lease round (issued monotone by the Fencer); it must be >= the recorded round or CarryForward returns ErrStaleRound (a newer owner already passed us — we lost the race and must step aside). hydrate is the caller's restore-into-local-DB step; it may run more than once across retries, so it must be idempotent (a whole-snapshot restore is).
func (*FencedStore) Get ¶ added in v0.6.4
Get returns the fenced payload and the round it was written at, or ErrNotFound when key is absent. A reader restores the payload; a successor reads the round to learn what to advance past on takeover.
func (*FencedStore) Put ¶ added in v0.6.4
Put admits and (atomically with admission) writes payload for key at round. It rejects with ErrStaleRound when round is below the recorded round — the fence — before any write is attempted, then advances the header and appends the payload in a single PutIfVersion conditioned on the version read THIS attempt (never a cached one). If a concurrent writer's write lands first, our CAS fails (ErrConflict) and we retry: re-read the now-authoritative round, re-run the strict check (a lower-round racer now loses as ErrStaleRound; an equal-or-higher one retries the CAS against the new version and, absent further contention, wins). The store is the sole arbiter of "what round currently owns this key."
type Option ¶
type Option func(*Replicator)
Option configures a Replicator.
func WithEncryption ¶
WithEncryption seals every pushed snapshot with the org's per-org key so the object is ciphertext; orgID is bound as AAD, so a blob cannot be replayed under another org. Omit it and the DB is stored in the clear (local dev only).
type Replicator ¶
type Replicator struct {
// contains filtered or unexported fields
}
Replicator binds ONE per-org SQLite (at a DBPath key) to its object-store slot.
owner replica (IsOwner true): Push() on a timer / after a write burst reader replica (IsOwner false): Pull() before a stale-tolerant read
It does NOT decide ownership — that is Owner()/IsOwner(). Push/Get/Put are whole-object (a WAL-checkpointed snapshot), the proven HIP-0107 mechanism.
func NewReplicator ¶
func NewReplicator(key string, store Store, db DB, opts ...Option) *Replicator
NewReplicator binds the DB at key (from DBPath) to its store slot.
func (*Replicator) Key ¶
func (r *Replicator) Key() string
Key is this replicator's object-store location.
func (*Replicator) Pull ¶
func (r *Replicator) Pull(ctx context.Context) (changed bool, err error)
Pull downloads the latest snapshot and restores it into the local DB — called by a READER before serving stale-tolerant reads, or by a NEW OWNER on handover to load the last committed state. A no-op when the remote is unchanged since our last push/pull.
func (*Replicator) Push ¶
func (r *Replicator) Push(ctx context.Context) error
Push snapshots the local DB (WAL-checkpointed) and writes it to the object store. Called by the OWNER — the single writer. The durable copy lives in vfs, so a lost pod loses no committed data.
func (*Replicator) PushLoop ¶
func (r *Replicator) PushLoop(ctx context.Context, every time.Duration)
PushLoop runs Push on an interval until ctx cancel — the owner's background WAL-shipper. On error it keeps the last good remote copy and retries next tick (fire-and-forget durability; never blocks writes).
type SQLiteDB ¶
type SQLiteDB struct {
// contains filtered or unexported fields
}
SQLiteDB is a Replicator DB backed by an on-disk SQLite file. It owns the sole *sql.DB handle to that path; Restore swaps the file underneath it. Safe for concurrent Snapshot / query use; Restore is exclusive.
func OpenSQLite ¶
OpenSQLite opens (creating parent dirs + the file) the SQLite DB at path with the standard durability pragmas. The returned *SQLiteDB satisfies the Replicator DB interface and exposes DB() for the service's own queries.
func (*SQLiteDB) DB ¶
DB returns the live *sql.DB for the service's own reads/writes. It stays valid across Restore (Restore reopens under the same *SQLiteDB), so hold the *SQLiteDB, call DB() per query, and never cache the *sql.DB across a Restore.
func (*SQLiteDB) Restore ¶
Restore atomically replaces the local database with data and reopens the handle. Exclusive: it closes the live handle, renames the new file over the old, and reopens. On any failure the previous DB is left intact and the handle is reopened on the original file, so the service never ends up with no DB.
type Store ¶
type Store interface {
Put(ctx context.Context, key string, data []byte) error
Get(ctx context.Context, key string) (data []byte, version string, err error)
Version(ctx context.Context, key string) (version string, err error)
}
Store is the object-store surface the Replicator needs — satisfied natively by hanzoai/vfs (SeaweedFS-backed, in-process). NO minio, NO external S3 SDK. version is a content hash so readers skip redundant pulls.
Source Files