Documentation
¶
Overview ¶
Package storeseed implements the declarative store-seed channel: reserved stack subtrees (VECTORS/, KV/) that `txco apply` reconciles into the runtime stores (the vector store, the KV store), so a stack's known data set travels WITH the deploy and the tree is the desired state (sync = re-apply).
This file (paths.go) is the leaf layer: just the reserved-path vocabulary, with no dependency on the stores. It is imported by the CLI (to collect the packs into the bundle), the admin producer (to CAS-back the pack bytes like FILES/), and the control-event applier (to keep pack bytes out of the in-memory runtime DB). The materializer registry that actually reconciles packs into stores lives alongside in this package (storeseed.go).
Index ¶
Constants ¶
const ( DirVectors = "VECTORS" DirKV = "KV" // Pack store kinds — the Materializer.Kind() each pack dir routes to. KindVector = "vector" KindKV = "kv" // PackExt is the required extension for a pack file. We pin it so the // pack channel stays unambiguous (a stray FILES-style asset under // VECTORS/ is a deploy error, not silently reconciled). PackExt = ".jsonl" )
Reserved top-level pack directories (siblings to OPS/ and FILES/). Each holds NDJSON packs: VECTORS/<collection>.jsonl, KV/<namespace>.jsonl.
Variables ¶
This section is empty.
Functions ¶
func IsPackPath ¶
IsPackPath reports whether a stack_files path is a store-seed pack — i.e. it lives under one of the reserved pack directories. Used by the producer / applier to give packs the same CAS-backed, out-of-runtime-DB treatment as FILES/ static assets (large pre-computed vectors must never inline into a control event or a data-plane node's in-memory DB).
func KindForPath ¶
KindForPath returns the store kind ("vector" | "kv") a pack path routes to, or "" when the path is not under a reserved pack directory.
func PackName ¶
PackName returns the collection/namespace name a pack path seeds: "VECTORS/books.jsonl" → "books". It returns "" when the path is not a pack, is nested below the pack dir (no slashes allowed in the name), or lacks the .jsonl extension — the same shape validateStackFilePath enforces at upload.
Types ¶
type Materializer ¶
type Materializer interface {
// Kind is the pack kind this materializer owns (KindVector | KindKV).
Kind() string
// (pgvector, redis) rather than per-node (sqlite-vec, boltdb). A shared
// store is reconciled exactly ONCE — on the activation origin (the control
// plane) — so concurrent data-plane appliers don't race the delete-missing
// pass. A per-node store is reconciled on every node from its CAS-resolved
// pack. The wiring layer that picks the backend declares this.
Shared() bool
// Reconcile makes the store match packs — the full set of this kind's packs
// in this version. Each pack owns its collection/namespace (managed scope):
// upsert the pack's items + delete managed items absent from it. A pack with
// zero items empties (but does not necessarily drop) its target. Whole-
// collection drop (a pack removed entirely between versions) is the
// Reconciler's job (P4 prior-version diff), not the Materializer's.
Reconcile(ctx context.Context, scope Scope, packs []RawPack) error
}
Materializer reconciles the store-seed packs of one Kind into its backing store. The concrete materializers live in subpackages (vecseed, kvseed) so this core package stays free of store dependencies — the CLI and the control-event applier import storeseed only for the path vocabulary.
type RawPack ¶
type RawPack struct {
Path string // e.g. "VECTORS/books.jsonl"
Kind string // KindVector | KindKV
Name string // collection / namespace the pack owns
Bytes []byte // raw NDJSON
}
RawPack is a store-seed pack as it travels through the deploy: the stack_files path it came from, the store kind + owned name derived from that path, and the resolved NDJSON bytes (one item per line). The control plane reads the bytes inline from stack_files; a data-plane node resolves them from the shared CAS by fingerprint. The materializer framework parses Bytes into typed items per kind.
func NewRawPack ¶
NewRawPack builds a RawPack from a pack path + its resolved bytes, deriving Kind and Name from the path. It returns (RawPack{}, false) when the path is not a well-formed pack path (caller should have validated, but this keeps the loader honest).
type Reconciler ¶
type Reconciler struct {
// contains filtered or unexported fields
}
Reconciler dispatches a version's packs to the registered materializers. It is built once at boot (server.go) from the live stores and injected into the activation path; reconcile is invoked best-effort AFTER the activation tx commits, so a slow or failing store never stalls or rolls back a deploy.
func NewReconciler ¶
func NewReconciler(ms ...Materializer) *Reconciler
NewReconciler builds a Reconciler from the given materializers. A nil or empty set yields a no-op Reconciler (Reconcile returns nil) — the open-core default when no seedable store is configured. A duplicate Kind is a programming error and panics at boot.
func (*Reconciler) Kinds ¶
func (r *Reconciler) Kinds() []string
Kinds returns the pack kinds this Reconciler can materialise (sorted), for diagnostics.
func (*Reconciler) Reconcile ¶
func (r *Reconciler) Reconcile(ctx context.Context, scope Scope, packs []RawPack, origin bool) error
Reconcile groups packs by kind and dispatches each group to its materializer. origin marks the activation origin (the control plane): shared-store materializers reconcile only when origin is true; per-node materializers always run. Best-effort and exhaustive — every materializer is attempted and the errors are aggregated, so one failing store doesn't skip the others. A pack kind with no registered materializer is reported (a deploy referencing a store this node can't seed should be visible, not silent).
Directories
¶
| Path | Synopsis |
|---|---|
|
Package kvseed is the KV store-seed Materializer: it reconciles KV/<namespace>.jsonl packs (chassis/storeseed) into the tenant KV store.
|
Package kvseed is the KV store-seed Materializer: it reconciles KV/<namespace>.jsonl packs (chassis/storeseed) into the tenant KV store. |
|
Package vecseed is the vector store-seed Materializer: it reconciles VECTORS/<collection>.jsonl packs (chassis/storeseed) into a vector.Store.
|
Package vecseed is the vector store-seed Materializer: it reconciles VECTORS/<collection>.jsonl packs (chassis/storeseed) into a vector.Store. |