store

package
v1.5.9 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 28, 2026 License: MIT Imports: 23 Imported by: 0

Documentation

Overview

Package store implements the canonical per-tenant SQLite storage model described in hanzo/ARCHITECTURE.md §5: a composable org / app / project / user isolation hierarchy (see the tenant-data-hierarchy HIP).

There is exactly one way to fetch the SQLite handle for the current request: 

db, err := store.ForCtx(r.Context())

Which goes through this package's MultiTenantStore. Behind the scenes the store hydrates the right DB from object storage, caches it in an LRU, and checkpoints dirty DBs back on eviction / shutdown.

No handler ever opens SQLite directly. No handler ever reads object storage directly. No handler ever knows whether its SQLite file is in memory or in a bucket.

Consistency model (v1)

A single tenant is served by exactly one pod at a time via gateway-side sticky-session affinity (consistent hash on X-Org-Id). The store does NOT perform object-storage CAS (ETag If-Match) on upload — see CAS. During an HPA rebalance a (short) single-writer window may overlap across pods; ops MUST drain before scaling. This is the "at-most-once delivery under rebalance" model. The ARCHITECTURE.md §5.5 document describes this contract verbatim; consumers that require generation-checked writes must wait for the CAS follow-up slice.

Index

Constants

View Source 
const CAS = false

CAS reports whether this package performs object-storage compare-and-swap on upload (ETag If-Match). In v1 CAS is false — single-writer is provided by gateway sticky-session affinity on X-Org-Id. Consumers that require generation-checked writes must feature-gate on this constant and refuse to boot until it flips true.

View Source 
const MaxSlugLen = 128

MaxSlugLen caps every tenant slug length. 128 bytes is generous: IAM ULIDs are 26 chars, UUIDs 36. A slug longer than this is either a mistake or an attempt to blow up filesystem error messages with attacker-controlled bytes — reject before any FS call.

Variables

View Source 
var (
	// ErrCorruptDB is returned from hydrate when the downloaded object is
	// non-empty and does not begin with the SQLite magic header. Callers
	// must NOT retry blindly — the bucket contents are hostile or
	// corrupted; an operator has to triage.
	ErrCorruptDB = errors.New("store: downloaded object is not a SQLite database (header mismatch)")

	// ErrUploadFailed wraps any object-storage upload failure surfaced
	// from Evict / Close / Checkpoint. The handle is retained in the
	// cache so the next reap cycle can retry.
	ErrUploadFailed = errors.New("store: upload failed")

	// ErrClosed is returned when Get is called after Close.
	ErrClosed = errors.New("store: closed")

	// ErrCrossTenant is returned when an authenticated caller asks for a Key
	// that does not belong to the caller's org. The claims->OrgID binding is
	// the first-line cross-tenant boundary; per-org KMS key separation is the
	// second. errors.Is-comparable.
	ErrCrossTenant = errors.New("store: cross-tenant access denied")
)

Sentinel errors surfaced to callers. They are errors.Is-comparable and MUST NOT be wrapped out of recognition.

View Source 
var ErrNoKeyMaterial = errors.New("store: no wrapped key material for tenant")

ErrNoKeyMaterial is returned when a DB object exists but its wrapped-key sidecar cannot be produced — never silently fall back to plaintext.

Functions

func AppFromContext 

func AppFromContext(ctx context.Context) string

AppFromContext returns the request's app scope, or "" when absent (composable fallback to the org tier).

func ProjectFromContext 

func ProjectFromContext(ctx context.Context) string

ProjectFromContext returns the request's project scope, or "" when absent (composable fallback to the app tier).

func WithApp 

func WithApp(ctx context.Context, app string) context.Context

WithApp returns a context carrying the request's app scope. Routers set it once the app slug is resolved (typically one app per service).

func WithProject 

func WithProject(ctx context.Context, project string) context.Context

WithProject returns a context carrying the request's project scope.

Types

type DBConnect 

type DBConnect func(path string) (*dbx.DB, error)

DBConnect opens a SQLite file and returns the dbx builder. The default matches core.DefaultDBConnect: WAL, busy_timeout, NORMAL sync, FK on.

type Key 

type Key struct {
	OrgID   string
	UserID  string // set when Scope == ScopeUser
	App     string // set when Scope == ScopeApp or ScopeProject
	Project string // set when Scope == ScopeProject
	Scope   Scope
}

Key is the tuple that identifies a per-tenant SQLite DB.

The tenant space is a composable hierarchy under one org: an org-wide DB, a per-app DB, a per-app-per-project DB, and a per-user DB. Splitting the space this way lets us keep a single cache and a single object-storage layout for every shape. Scope selects which fields are significant: 

ScopeOrg      OrgID
ScopeApp      OrgID, App
ScopeProject  OrgID, App, Project
ScopeUser     OrgID, UserID

func (Key) LocalPath 

func (k Key) LocalPath(cacheRoot string) string

LocalPath returns the in-pod filesystem path for the DB. It mirrors ObjectKey under cacheRoot so the on-disk cache and the bucket share one layout — the path structure lives in exactly one place.

func (Key) ObjectKey 

func (k Key) ObjectKey() string

ObjectKey returns the object-storage path for the DB. 

org-scoped:      {org}/org.db
app-scoped:      {org}/apps/{app}.db
project-scoped:  {org}/apps/{app}/projects/{project}.db
user-scoped:     {org}/users/{user}.db

func (Key) String 

func (k Key) String() string

String implements fmt.Stringer for log lines and metric labels.

func (Key) Valid 

func (k Key) Valid() error

Valid reports whether the key is well-formed. Callers MUST validate keys built from HTTP headers before passing them to the store — otherwise a crafted org / app / project slug can reach the filesystem with `..`. Every slug significant to the Scope passes the SAME validateSlug guard, so no tier can escape cacheRoot.

type KeyProvider 

type KeyProvider interface {
	Resolve(ctx context.Context, k Key) (*TenantKey, error)
}

KeyProvider yields the per-tenant at-rest key for a Key. Implementations MUST return a key unique to k, rooted in KMS, and MUST NOT log key material. Resolve is safe for concurrent use.

type Keyring 

type Keyring struct {
	// contains filtered or unexported fields
}

Keyring is the canonical KeyProvider: KMS-rooted envelope encryption with per-DB random age identities. Safe for concurrent use.

func NewKeyring 

func NewKeyring(root RootSource, sidecar *filesystem.System) (*Keyring, error)

NewKeyring builds a Keyring over a RootSource (KMS) and an object store for the wrapped-key sidecars (the SAME durable store as the DBs in production).

func (*Keyring) Resolve 

func (kr *Keyring) Resolve(ctx context.Context, k Key) (*TenantKey, error)

Resolve returns the per-tenant key for k, loading-and-unwrapping the existing sidecar or generating-and-wrapping a fresh identity on first use. In normal operation the sidecar is written at first hydrate (before any DB object exists), so a DB object without a sidecar can only be hostile injection — which surfaces downstream as an age-decrypt failure (ErrCorruptDB).

func (*Keyring) Rotate 

func (kr *Keyring) Rotate(ctx context.Context, k Key, prevRoot, newRoot RootSource) error

Rotate re-wraps k's key material from prevRoot's KEK to newRoot's KEK, leaving the DB ciphertext AND the underlying age identity unchanged. This is how an org KEK rotation propagates: only the tiny sidecar is rewritten, so no DB is re-encrypted. After Rotate, the old KEK can no longer unwrap the sidecar. Idempotent per (k, newRoot).

type MultiTenantStore 

type MultiTenantStore struct {
	// contains filtered or unexported fields
}

MultiTenantStore owns the per-tenant SQLite universe for one pod. Safe for concurrent use.

func New 

func New(opts Options) (*MultiTenantStore, error)

New constructs a MultiTenantStore with defaults applied.

func (*MultiTenantStore) Checkpoint 

func (s *MultiTenantStore) Checkpoint(ctx context.Context, k Key) error

Checkpoint flushes the WAL and uploads the DB to object storage.

In v1 there is NO generation check (CAS=false). Sticky-session gateway affinity provides the single-writer guarantee; during an HPA rebalance a brief dual-writer window may produce a lost-write. Ops MUST drain before scaling out. Callers that need generation-checked writes must refuse to boot until CAS flips true.

The returned error wraps ErrUploadFailed on object-storage failure; the handle is retained in the cache so the next Checkpoint / reap tick can retry without losing local writes that are still durable in the WAL.

func (*MultiTenantStore) Close 

func (s *MultiTenantStore) Close(ctx context.Context) error

Close flushes every resident handle to object storage and then closes the store. Safe to call multiple times.

Lock discipline (P7-H2): snapshot keys under s.mu, release, then flush each key with a fresh per-key lock. This bounds the shutdown window to sum-of-per-handle-latencies instead of holding s.mu for the entire drain. Upload failures are AGGREGATED into the returned error so ops can see exactly which tenants did not make it to durable storage.

func (*MultiTenantStore) Evict 

func (s *MultiTenantStore) Evict(ctx context.Context, k Key) error

Evict flushes (if dirty) and closes the handle. Subsequent Gets for the same key will re-hydrate from object storage.

On upload failure: returns ErrUploadFailed and RETAINS the handle in the cache so that (a) the next reap cycle retries, (b) the caller can surface the failure instead of losing the WAL-durable write.

func (*MultiTenantStore) ForApp 

func (s *MultiTenantStore) ForApp(ctx context.Context) (*dbx.DB, error)

ForApp resolves the app-scoped SQLite for the caller's org (per-org app settings). Org comes from IAM (claims); the app slug from the request (WithApp). An absent app composably falls back to the org tier.

func (*MultiTenantStore) ForCtx 

func (s *MultiTenantStore) ForCtx(ctx context.Context) (*dbx.DB, error)

ForCtx resolves the SQLite handle for the caller's (org, user). The caller must have a Claims attached via claims.Inject + claims.RequireGateway.

Returns ErrGatewayBypass when identity is missing.

func (*MultiTenantStore) ForOrg 

func (s *MultiTenantStore) ForOrg(ctx context.Context) (*dbx.DB, error)

ForOrg resolves the org-scoped SQLite for the caller. Callers must already be inside the tenant via claims.RequireGateway. Used for org-wide state (org settings, member list) that isn't per-user.

func (*MultiTenantStore) ForProject 

func (s *MultiTenantStore) ForProject(ctx context.Context) (*dbx.DB, error)

ForProject resolves the project-scoped SQLite for the caller's org+app (operational data: fleets, bots, machines). Org comes from IAM (claims); app+project from the request (WithApp / WithProject). Composable fallback: absent project → app tier, absent app → org tier.

func (*MultiTenantStore) Get 

func (s *MultiTenantStore) Get(ctx context.Context, k Key) (*dbx.DB, error)

Get is the low-level path used by ForCtx / ForOrg / ForApp / ForProject. Exposed so that background jobs (migrations, reports) can resolve a specific key without a synthetic HTTP request.

func (*MultiTenantStore) MarkDirty 

func (s *MultiTenantStore) MarkDirty(k Key, n int)

MarkDirty is called by instrumentation / orm hooks when a write occurs. It increments the dirty counter; Checkpoint drains it. Apps that use the default orm integration don't need to call this directly.

type Options 

type Options struct {
	// ObjectStore is the durable, cross-pod blob store. Production uses
	// filesystem.NewS3(...) / NewGCS(...); tests can pass filesystem.NewLocal.
	ObjectStore *filesystem.System

	// Keys provides the per-tenant at-rest encryption key for every DB.
	// REQUIRED — the store never writes plaintext SQLite to durable storage.
	// Production wires the KMS-rooted Keyring (NewKeyring + kmskeyring.Source).
	Keys KeyProvider

	// CacheRoot is the in-pod cache directory for hot SQLite files. Defaults
	// to "/data/cache".
	CacheRoot string

	// LRUSize is the max number of open SQLite handles a single pod will
	// keep resident at any time. Hitting the cap triggers checkpoint+close
	// on the coldest handle. Defaults to 1000.
	LRUSize int

	// IdleTTL is the time a handle may sit without any Get before the
	// reaper evicts it. Defaults to 5 minutes. Zero disables the reaper.
	IdleTTL time.Duration

	// CheckpointWrites: after this many writes since the last checkpoint,
	// a handle is eligible for upload. Defaults to 100.
	CheckpointWrites int

	// CheckpointInterval: after this much wall-clock time since the last
	// checkpoint, a handle is eligible for upload. Defaults to 60s.
	CheckpointInterval time.Duration

	// Connect is the SQLite open function. Defaults to a WAL-mode open
	// equivalent to core.DefaultDBConnect.
	Connect DBConnect

	// Now returns the current time. Swap in tests to drive IdleTTL.
	Now func() time.Time

	// OnReapFailure is an optional observer for reap-cycle upload errors.
	// Services wire this to a structured logger + Prometheus counter. Nil
	// means silent — the error is still returned from Evict, but the
	// reaper runs asynchronously and there is nowhere to surface it
	// except through a hook.
	OnReapFailure func(Key, error)
}

Options configures the MultiTenantStore. All fields have defaults, and the only required one is ObjectStore.

type RootSource 

type RootSource interface {
	OrgRoot(ctx context.Context, orgID string) ([]byte, error)
}

RootSource yields the org-scoped root key (KEK) — the single KMS-rooted master per org. Implementations MUST return exactly one stable 32-byte KEK per orgID and MUST NOT return a shared/global key. The KEK is secret; the source is the only component that ever touches KMS.

type Scope 

type Scope int

Scope selects which tier of the org's tenant hierarchy a Key addresses. 

const (
	ScopeUser Scope = iota
	ScopeOrg
	ScopeApp
	ScopeProject
)

type TenantKey 

type TenantKey struct {
	Recipient age.Recipient
	Identity  age.Identity
}

TenantKey is the per-tenant at-rest key: a post-quantum hybrid age identity. Recipient encrypts, Identity decrypts. Unique per Key tuple.

Source Files

View all Source files

Directories

Path Synopsis
Package encreplica is a replicate.ReplicaClient that PQ-encrypts every LTX segment with a per-tenant age key BEFORE it touches durable storage, and decrypts on read — the SOLE at-rest boundary for the replica stream, using the SAME key as the whole-file path (store.TenantKey).
 Package encreplica is a replicate.ReplicaClient that PQ-encrypts every LTX segment with a per-tenant age key BEFORE it touches durable storage, and decrypts on read — the SOLE at-rest boundary for the replica stream, using the SAME key as the whole-file path (store.TenantKey).
Package kmskeyring binds store.RootSource to Hanzo KMS.
 Package kmskeyring binds store.RootSource to Hanzo KMS.
Package replicator gives Base's per-tenant SQLite substrate continuous streaming replication and point-in-time restore via hanzoai/replicate — the HA / resilience layer (Pillar 2): a pod dying or rescheduling restores each tenant DB from its replica (bounded RPO, no data loss).
 Package replicator gives Base's per-tenant SQLite substrate continuous streaming replication and point-in-time restore via hanzoai/replicate — the HA / resilience layer (Pillar 2): a pod dying or rescheduling restores each tenant DB from its replica (bounded RPO, no data loss).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.