Documentation
¶
Overview ¶
Package sqlite is the SQLite-backed implementation of the observability-rollup `rollups.Store` interface — the durable, indexed sibling of the in-memory reference driver, built on `modernc.org/sqlite` (CGo-free; builds stay `CGO_ENABLED=0`).
Durable indexed parity ¶
The schema mirrors the reference driver's indexes 1:1 against the fixed-UTC MINUTE grid:
- `rollup_rows` carries one row per row key (minute bucket start + exactly the closed dimensions tenant / user / session / model). The bucket start is an exact INTEGER of unix nanoseconds, every measure column is INTEGER (exact int64), and there is NO REAL or DOUBLE column anywhere — cost is stored as integer micro-units of USD and nothing is ever accumulated or stored as float64.
- The secondary indexes serve the bounded read paths (bucket+tenant, bucket+tenant+user), the erasure-fence delete (the full identity triple), and one axis index per remaining closed dimension. A Query resolves its candidates through these indexes — the bounded window range plus exact IN filters per axis — and never scans the canonical event log (this driver holds no reference to it; the projection rows ARE the rollup store).
- `rollup_checkpoint` is the single-row durable watermark (the last applied local durable sequence); `rollup_fence` is the PERMANENT erasure fence. Both are plain tables, so they survive restarts.
Apply semantics ¶
ApplyBatch is ONE transaction: the deltas and the checkpoint move atomically, so a crash between applying deltas and checkpointing is impossible and re-applying a batch whose checkpoint does not advance the stored checkpoint is a no-op (idempotent replay). Every delta's merge is checked in Go against the exact int64 measure bounds BEFORE any row is written (a working copy per key, verified via `MeasureSet.Add`) — an overflowing or negative delta refuses the WHOLE batch and applies nothing. A delta for a fenced (erased) triple refuses the WHOLE batch with `rollups.ErrSessionFenced`.
Erasure fences are permanent ¶
FenceSession deletes the triple's rows AND writes the fence row in one transaction; Rebuild deletes rows + checkpoint only — the fence table is never touched — so reprojection can never resurrect an erased session, and a late event for a fenced triple is refused forever.
Quality persistence ¶
The durable components of the projector's quality surface live here: the watermark is `rollup_checkpoint.sequence`, and the retention horizon is the MIN/MAX `bucket_start` of `rollup_rows` (the row-level minute grid). The projector's live catch-up state (`current` / `catching_up` / `unavailable`) is projector-instance state — this driver persists the durable truth the projector re-derives it from, so a restart resumes honest quality from the durable watermark + source head on the first Advance.
Operating model ¶
- `New` accepts a bare file path or a `file:` URI. The special `:memory:` sentinel maps to a per-open uniquely named shared-cache memory database (each store is isolated; the pool shares one DB).
- `journal_mode=WAL` and `busy_timeout=5000` are pinned on every connection via DSN pragmas; transactions begin IMMEDIATE. The pool is pinned to a single connection so all access serialises at the Go layer — the driver matches SQLite's single-writer reality and never surfaces SQLITE_BUSY contention.
- The schema is applied from embedded `migrations/*.sql` via the shared `internal/persistence/sqlmigrate` runner.
The driver is constructed directly (no registry registration): the production driver-aggregator wiring is a runtime-assembly concern, not this package's.
Index ¶
- type Store
- func (s *Store) ApplyBatch(ctx context.Context, batch rollups.Batch) error
- func (s *Store) Checkpoint(ctx context.Context) (uint64, error)
- func (s *Store) Close(_ context.Context) error
- func (s *Store) FenceSession(ctx context.Context, id identity.Identity) error
- func (s *Store) IsFenced(ctx context.Context, id identity.Identity) (bool, error)
- func (s *Store) Query(ctx context.Context, q rollups.Query) (rollups.Result, error)
- func (s *Store) Rebuild(ctx context.Context) error
- func (s *Store) Retention(ctx context.Context) (time.Time, time.Time, error)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store is the SQLite-backed rollups.Store. It is a compiled artifact: the *sql.DB (internally synchronised by database/sql) and the atomic close flag are the only mutable state, so N goroutines can share one instance. The pool is pinned to a single connection, serialising all access at the Go layer (SQLite's single-writer reality).
func New ¶
New constructs a fresh, empty SQLite-backed rollups.Store against dsn.
DSN handling:
- Empty DSN → clear error (no silent default-fallback).
- `:memory:` → translated to a PER-OPEN uniquely named shared-cache memory URI (`file:harbor_rollups_mem_<entropy>?mode=memory&cache=shared`) so the pool's single connection sees one in-memory database while two `:memory:` stores opened by different callers stay isolated.
- Any other DSN is treated as a file path or URI form and passed through verbatim, with `journal_mode=WAL`, `busy_timeout=5000`, and `_txlock=immediate` appended as query parameters so modernc.org/sqlite applies them on every pooled connection.
Construction fails loudly on an empty DSN, an unparseable URI, a non-WAL journal mode (disk-backed), or a migration-apply failure.
func (*Store) ApplyBatch ¶
ApplyBatch implements rollups.Store. The batch's deltas and the checkpoint move are atomic (one transaction): a crash between applying deltas and checkpointing is impossible, and a batch whose Checkpoint does not advance the stored checkpoint is a no-op (idempotent replay — every event at or below the stored checkpoint is already applied). A delta for a fenced triple rejects the WHOLE batch with rollups.ErrSessionFenced (the checkpoint does not advance). Every delta's merge is verified against the exact int64 measure bounds on a working copy BEFORE any row is written, so a refused batch never leaves partial rows and the checkpoint does not advance.
func (*Store) Checkpoint ¶
Checkpoint implements rollups.Store: the durable watermark (the last applied local durable sequence), 0 when nothing has been applied.
func (*Store) Close ¶
Close implements rollups.Store. Setting the atomic flag BEFORE `db.Close()` ensures concurrent in-flight callers observe `rollups.ErrClosed` rather than racing into a half-closed pool. Close is idempotent — repeat calls are safe and return nil.
func (*Store) FenceSession ¶
FenceSession implements rollups.Store: it erases every row for the session triple and fences the triple PERMANENTLY so no future ApplyBatch can create rows for it (the erasure is never resurrected by a late event or by Rebuild). Both the delete and the fence insert happen in ONE transaction. Idempotent. There is no unfence operation.
func (*Store) Query ¶
Query implements rollups.Store. The query is re-validated (the wrapped ErrQueryInvalid / ErrQueryBudget / ErrBadCursor sentinels flow through), and the candidate rows are resolved through the bucket + dimension indexes — the bounded bucket_start window plus exact IN filters per closed axis — never a full-table scan of the projection rows and never the canonical event log (this driver holds no reference to it). Grouping (minute rows coarsened to the query's Bucket), the checked measure aggregation, the total sort, and the deterministic keyset pagination run in Go after the indexed candidate read. The response is deterministic for a stable store: same query + same cursor ⇒ same rows, and pages never skip or repeat a row. A group whose measure sums would overflow fails loudly with rollups.ErrMeasureOverflow.
func (*Store) Rebuild ¶
Rebuild implements rollups.Store: clears every projection row and the checkpoint (reset to 0) so the projector reprocesses the full log from the beginning. Erasure fences are PERMANENT and are deliberately NOT cleared — rebuilding rows or the checkpoint cannot authorize the resurrection of an erased session.
Source Files