Documentation
¶
Overview ¶
Package store implements the canonical per-(org, user) SQLite storage model described in hanzo/ARCHITECTURE.md §5.
There is exactly one way to fetch the SQLite handle for the current request:
db, err := store.ForCtx(r.Context())
Which goes through this package's MultiTenantStore. Behind the scenes the store hydrates the right DB from object storage, caches it in an LRU, and checkpoints dirty DBs back on eviction / shutdown.
No handler ever opens SQLite directly. No handler ever reads object storage directly. No handler ever knows whether its SQLite file is in memory or in a bucket.
Consistency model (v1) ¶
A single tenant is served by exactly one pod at a time via gateway-side sticky-session affinity (consistent hash on X-Org-Id). The store does NOT perform object-storage CAS (ETag If-Match) on upload — see CAS. During an HPA rebalance a (short) single-writer window may overlap across pods; ops MUST drain before scaling. This is the "at-most-once delivery under rebalance" model. The ARCHITECTURE.md §5.5 document describes this contract verbatim; consumers that require generation-checked writes must wait for the CAS follow-up slice.
Index ¶
- Constants
- Variables
- type DBConnect
- type Key
- type MultiTenantStore
- func (s *MultiTenantStore) Checkpoint(ctx context.Context, k Key) error
- func (s *MultiTenantStore) Close(ctx context.Context) error
- func (s *MultiTenantStore) Evict(ctx context.Context, k Key) error
- func (s *MultiTenantStore) ForCtx(ctx context.Context) (*dbx.DB, error)
- func (s *MultiTenantStore) ForOrg(ctx context.Context) (*dbx.DB, error)
- func (s *MultiTenantStore) Get(ctx context.Context, k Key) (*dbx.DB, error)
- func (s *MultiTenantStore) MarkDirty(k Key, n int)
- type Options
- type Scope
Constants ¶
const CAS = false
CAS reports whether this package performs object-storage compare-and-swap on upload (ETag If-Match). In v1 CAS is false — single-writer is provided by gateway sticky-session affinity on X-Org-Id. Consumers that require generation-checked writes must feature-gate on this constant and refuse to boot until it flips true.
const MaxSlugLen = 128
MaxSlugLen caps org / user slug length. 128 bytes is generous: IAM ULIDs are 26 chars, UUIDs 36. A slug longer than this is either a mistake or an attempt to blow up filesystem error messages with attacker-controlled bytes — reject before any FS call.
Variables ¶
var ( // ErrCorruptDB is returned from hydrate when the downloaded object is // non-empty and does not begin with the SQLite magic header. Callers // must NOT retry blindly — the bucket contents are hostile or // corrupted; an operator has to triage. ErrCorruptDB = errors.New("store: downloaded object is not a SQLite database (header mismatch)") // ErrUploadFailed wraps any object-storage upload failure surfaced // from Evict / Close / Checkpoint. The handle is retained in the // cache so the next reap cycle can retry. ErrUploadFailed = errors.New("store: upload failed") // ErrClosed is returned when Get is called after Close. ErrClosed = errors.New("store: closed") )
Sentinel errors surfaced to callers. They are errors.Is-comparable and MUST NOT be wrapped out of recognition.
Functions ¶
This section is empty.
Types ¶
type DBConnect ¶
DBConnect opens a SQLite file and returns the dbx builder. The default matches core.DefaultDBConnect: WAL, busy_timeout, NORMAL sync, FK on.
type Key ¶
Key is the tuple that identifies a per-tenant SQLite DB.
Scope = "users" for per-user DBs and "org" for the org-wide DB. Splitting the space this way lets us keep a single cache and a single object-storage layout for both shapes.
func (Key) ObjectKey ¶
ObjectKey returns the object-storage path for the DB.
user-scoped: {org}/users/{user}.db
org-scoped: {org}/org.db
type MultiTenantStore ¶
type MultiTenantStore struct {
// contains filtered or unexported fields
}
MultiTenantStore owns the per-(org, user) SQLite universe for one pod. Safe for concurrent use.
func New ¶
func New(opts Options) (*MultiTenantStore, error)
New constructs a MultiTenantStore with defaults applied.
func (*MultiTenantStore) Checkpoint ¶
func (s *MultiTenantStore) Checkpoint(ctx context.Context, k Key) error
Checkpoint flushes the WAL and uploads the DB to object storage.
In v1 there is NO generation check (CAS=false). Sticky-session gateway affinity provides the single-writer guarantee; during an HPA rebalance a brief dual-writer window may produce a lost-write. Ops MUST drain before scaling out. Callers that need generation-checked writes must refuse to boot until CAS flips true.
The returned error wraps ErrUploadFailed on object-storage failure; the handle is retained in the cache so the next Checkpoint / reap tick can retry without losing local writes that are still durable in the WAL.
func (*MultiTenantStore) Close ¶
func (s *MultiTenantStore) Close(ctx context.Context) error
Close flushes every resident handle to object storage and then closes the store. Safe to call multiple times.
Lock discipline (P7-H2): snapshot keys under s.mu, release, then flush each key with a fresh per-key lock. This bounds the shutdown window to sum-of-per-handle-latencies instead of holding s.mu for the entire drain. Upload failures are AGGREGATED into the returned error so ops can see exactly which tenants did not make it to durable storage.
func (*MultiTenantStore) Evict ¶
func (s *MultiTenantStore) Evict(ctx context.Context, k Key) error
Evict flushes (if dirty) and closes the handle. Subsequent Gets for the same key will re-hydrate from object storage.
On upload failure: returns ErrUploadFailed and RETAINS the handle in the cache so that (a) the next reap cycle retries, (b) the caller can surface the failure instead of losing the WAL-durable write.
func (*MultiTenantStore) ForCtx ¶
ForCtx resolves the SQLite handle for the caller's (org, user). The caller must have a Claims attached via claims.Inject + claims.RequireGateway.
Returns ErrGatewayBypass when identity is missing.
func (*MultiTenantStore) ForOrg ¶
ForOrg resolves the org-scoped SQLite for the caller. Callers must already be inside the tenant via claims.RequireGateway. Used for org-wide state (org settings, member list) that isn't per-user.
func (*MultiTenantStore) Get ¶
Get is the low-level path used by ForCtx / ForOrg. Exposed so that background jobs (migrations, reports) can resolve a specific key without a synthetic HTTP request.
func (*MultiTenantStore) MarkDirty ¶
func (s *MultiTenantStore) MarkDirty(k Key, n int)
MarkDirty is called by instrumentation / orm hooks when a write occurs. It increments the dirty counter; Checkpoint drains it. Apps that use the default orm integration don't need to call this directly.
type Options ¶
type Options struct {
// ObjectStore is the durable, cross-pod blob store. Production uses
// filesystem.NewS3(...) / NewGCS(...); tests can pass filesystem.NewLocal.
ObjectStore *filesystem.System
// CacheRoot is the in-pod cache directory for hot SQLite files. Defaults
// to "/data/cache".
CacheRoot string
// LRUSize is the max number of open SQLite handles a single pod will
// keep resident at any time. Hitting the cap triggers checkpoint+close
// on the coldest handle. Defaults to 1000.
LRUSize int
// IdleTTL is the time a handle may sit without any Get before the
// reaper evicts it. Defaults to 5 minutes. Zero disables the reaper.
IdleTTL time.Duration
// CheckpointWrites: after this many writes since the last checkpoint,
// a handle is eligible for upload. Defaults to 100.
CheckpointWrites int
// CheckpointInterval: after this much wall-clock time since the last
// checkpoint, a handle is eligible for upload. Defaults to 60s.
CheckpointInterval time.Duration
// Connect is the SQLite open function. Defaults to a WAL-mode open
// equivalent to core.DefaultDBConnect.
Connect DBConnect
// Now returns the current time. Swap in tests to drive IdleTTL.
Now func() time.Time
// OnReapFailure is an optional observer for reap-cycle upload errors.
// Services wire this to a structured logger + Prometheus counter. Nil
// means silent — the error is still returned from Evict, but the
// reaper runs asynchronously and there is nowhere to surface it
// except through a hook.
OnReapFailure func(Key, error)
}
Options configures the MultiTenantStore. All fields have defaults, and the only required one is ObjectStore.
Source Files