Documentation
¶
Index ¶
Constants ¶
const ( MirrorModeMemory = "memory" MirrorModeFile = "file" )
Mirror storage modes (--db-mirror-mode). See openMirror.
Variables ¶
This section is empty.
Functions ¶
func RegisterLoader ¶ added in v0.2.21
func RegisterLoader(name string, l MirrorLoader)
RegisterLoader records a mirror loader under name (e.g. "postgres"). Mirrors the feed/scheduled/artifact factory-registry pattern: the cloud overlay calls this from its init(), blank-imported by the cloud main, so open-core never references the overlay symbol. The built-in "sqlite" loader self-registers below.
Types ¶
type DbCache ¶
type DbCache struct {
Conf config.Config
Ctx context.Context
Db *sql.DB
Logger *zap.Logger
Mu sync.Mutex
// Source is the chassis's runtime *sql.DB handle — the live,
// configured connection pool that the rest of the chassis writes
// through. Reload() copies from this handle (via the SQLite online
// backup API over a borrowed Source connection) rather than opening
// its own connection to the file: in WAL mode a second uncoordinated
// connection races the main one's .db-shm state and fails with
// "database is locked" on first boot. Going through the same pool
// means there is no second connection to race.
Source *sql.DB
// OnReload, if set, runs inside every Reload against the freshly-built
// in-memory DB BEFORE it is published — the new mirror is still private,
// so no request ever observes the snapshot before the overlay is
// applied, and (deliberately) no reader is blocked behind the hook
// chain: only the pointer swap afterwards takes Mu. Used by
// chassis/sysops to re-apply the trusted system opstacks (Reload
// rebuilds :memory: from the runtime dump and would otherwise drop
// them), and by derived-cache rebuilds (redact registry, admission,
// static index, DNS zones). Hooks MUST read from the `*sql.DB` they are
// handed, never from Snapshot(): Snapshot() still returns the PREVIOUS
// mirror at that point. Hooks that rebuild global in-memory caches
// therefore publish new derived state a moment BEFORE the mirror swap —
// each hook already swaps atomically and keeps its prior state on
// error, and nothing reads cache+mirror as one consistent pair. A hook
// error is logged, not fatal: the freshly-built mirror is still
// published (same as it always was).
OnReload func(*sql.DB) error
// contains filtered or unexported fields
}
DbCache structure
func New ¶
func New(conf config.Config, logger *zap.Logger, ctx context.Context, source *sql.DB) (*DbCache, error)
New Create a new in-memory DB cache.
`source` is the chassis's runtime *sql.DB — the live connection pool that Reload() reads from. Required: passing nil here would fail at the first Reload. See DbCache.Source for the WAL rationale.
Critical: go-sqlite3 gives each *connection* in the pool its own `:memory:` database. So if connection #1 loads the schema and a later concurrent query opens connection #2, that second connection sees an empty DB and the query fails with "no such table: ops".
To avoid that, pin the in-memory cache to a single connection. Reads are fast and the cache is read-only on the hot path; serializing through one connection costs nothing visible but guarantees consistency under concurrent load.
func (*DbCache) BumpGen ¶ added in v0.2.19
func (dbc *DbCache) BumpGen()
BumpGen invalidates derived caches after mutating the LIVE snapshot in place (dev-only system-opstacks hot-reload). Not needed around Reload — the handle swap is the invalidation there.
func (*DbCache) Gen ¶ added in v0.2.19
Gen returns the in-place mutation generation of the live snapshot. Pair with the Db pointer: a derived cache is fresh only while BOTH the handle and the generation it captured are unchanged.
func (*DbCache) Reload ¶
Reload a db file into Memory. Sources from the runtime DB only — the auth DB (when present) is owned exclusively by the admin role and is never mirrored into the read cache.
Concurrency: the dump+replay+overlay runs under reloadMu (serializing reloads end-to-end) while ONLY the pointer swap touches Mu — so neither the expensive dump nor the OnReload chain ever blocks Snapshot() readers. Serialization is still required: two concurrent writers each calling Reload after their commits would otherwise dump in parallel (each capturing a snapshot before some of the OTHER writer's commits land), and the reload that finishes its dump LAST would publish a STALE snapshot, silently clobbering durably-committed rows from the mirror. Symptom: a row on disk but missing from the resolver until the next (unrelated) reload happens to dump after every commit settled. reloadMu held across dump+swap keeps the second reload's dump strictly after the first's swap. (This costs serial reloads under write bursts, but the dump was the dominant cost regardless — concurrent dumps were a parallelism mirage.)
func (*DbCache) ReloadAfterWrite ¶ added in v0.2.21
ReloadAfterWrite refreshes the read mirror after a write to the authoritative runtime store. On a shared (postgres) runtime the write is already durable in the shared store and the local mirror is only a read cache, so the reload runs in the background, coalesced — blocking the caller on a full mirror rebuild couples write latency to total fleet size (~60s on a large tenant, past every CLI deadline). On the local SQLite file runtime the file IS the source of truth and a reader immediately after the write must see it, so the reload stays synchronous and the error is returned for the caller to log.
func (*DbCache) ReloadDebounced ¶ added in v0.2.21
func (dbc *DbCache) ReloadDebounced()
ReloadDebounced schedules a background, coalesced Reload: it returns immediately, and one reload runs ~reloadDebounceQuiet after the most recent call. Trailing-edge only (bep/debounce), which fits the callers — a write burst or watcher-event burst ends. Unlike the control-feed applier, this path has no redelivery, so a failed background reload retries with backoff (reloadWithRetry) instead of silently leaving the mirror stale until the next unrelated write.
func (*DbCache) Snapshot ¶
Snapshot returns the current mirror handle under the lock. Callers that live longer than one reload (e.g. the ingress resolver) MUST call this per use rather than capturing dbc.Db once: Reload() swaps dbc.Db to a fresh *sql.DB, so a captured handle goes stale and never sees rows written after it was captured. The returned *sql.DB stays valid for the caller's immediate query (the old handle isn't closed on swap); at worst it's one reload-cycle stale, which is the same guarantee the rest of the read path has.
type MirrorLoader ¶ added in v0.2.21
MirrorLoader (re)builds the in-memory read mirror: given a freshly-opened, empty :memory: SQLite handle `dst` and the chassis's authoritative runtime store `src`, it populates `dst` with the rows the hot read path serves. `srcDSN` is the runtime DSN `src` was opened from: a loader whose dump would otherwise contend with request traffic on the shared `src` pool may open its OWN small pool from it (the overlay's Postgres loader does — a multi-second dump transaction on the shared handle can queue admin queries behind it). The built-in SQLite loader ignores it: there the dump MUST ride a borrowed `src` connection (see DbCache.Source for the WAL rationale).
The mirror is ALWAYS SQLite (every reader assumes a :memory: SQLite snapshot — see chassis/processor and chassis/server/ingress); only the SOURCE varies. For a file: SQLite runtime the built-in "sqlite" loader does a page-level online backup. For a postgres:// runtime the cloud overlay registers a "postgres" loader that applies the SQLite runtime schema to `dst` and copies the hot tables out of Postgres — keeping every Postgres line out of open-core (open-core compiles no Postgres driver).
Source Files