Documentation
¶
Index ¶
Constants ¶
const ( DirVectors = "VECTORS" DirKV = "KV" DirBlobs = "BLOBS" KindVector = "vector" KindKV = "kv" KindBlob = "blob" PackExt = ".jsonl" )
Reserved store-seed trees under OPS/<stack>/. Each maps to a pack kind owned by one Materializer (vecseed, kvseed, blobseed).
Variables ¶
This section is empty.
Functions ¶
func IsBlobPath ¶ added in v0.2.24
IsBlobPath reports whether p is a BLOBS/ pack row (one blob per file).
func IsPackPath ¶
IsPackPath reports whether p lives in a store-seed tree.
func KindForPath ¶
KindForPath returns the pack kind for p ("" when p is not a pack path). Prefix match, exact-case — the SQL classification downstream is LIKE 'VECTORS/%' etc., and Postgres LIKE is case-sensitive.
func PackName ¶
PackName returns the name the pack owns, or "" when p is not a well-formed pack path. VECTORS/ and KV/ packs are a single "<name>.jsonl" segment (the collection / namespace is unambiguous; no nesting). A BLOBS/ row is the opposite shape: the tree IS the hierarchy, so the name is everything after "BLOBS/" — "BLOBS/faqs/house-01.doc" owns the blob name "faqs/house-01.doc". Whether that is a VALID blob name is the blob package's call (blob.ValidName), enforced at the write boundary and by the collector.
Types ¶
type Materializer ¶
type Materializer interface {
// Kind is the pack kind this materializer owns (KindVector | KindKV).
Kind() string
// (pgvector, redis) rather than per-node (sqlite-vec, boltdb). A shared
// store is reconciled exactly ONCE — on the activation origin (the control
// plane) — so concurrent data-plane appliers don't race the delete-missing
// pass. A per-node store is reconciled on every node from its CAS-resolved
// pack. The wiring layer that picks the backend declares this.
Shared() bool
// Reconcile makes the store match packs — the full set of this kind's packs
// in this version. Each pack owns its collection/namespace (managed scope):
// upsert the pack's items + delete managed items absent from it. A pack with
// zero items empties (but does not necessarily drop) its target. Whole-
// collection drop (a pack removed entirely between versions) is the
// Reconciler's job (P4 prior-version diff), not the Materializer's.
Reconcile(ctx context.Context, scope Scope, packs []RawPack) error
}
Materializer reconciles the store-seed packs of one Kind into its backing store. The concrete materializers live in subpackages (vecseed, kvseed) so this core package stays free of store dependencies — the CLI and the control-event applier import storeseed only for the path vocabulary.
type RawPack ¶
type RawPack struct {
Path string // e.g. "VECTORS/books.jsonl", "BLOBS/faqs/house-01.doc"
Kind string // KindVector | KindKV | KindBlob
Name string // collection / namespace / blob name the pack owns
Bytes []byte // raw NDJSON (nil for a blob row)
Hash string // sha256 hex of the row's bytes (blob rows: the CAS key)
}
RawPack is a store-seed pack as it travels through the deploy: the stack_files path it came from, the store kind + owned name derived from that path, and the resolved NDJSON bytes (one item per line). The control plane reads the bytes inline from stack_files; a data-plane node resolves them from the shared CAS by fingerprint. The materializer framework parses Bytes into typed items per kind.
A BLOBS/ row is the exception: its bytes are the blob itself and are NEVER loaded — the name index only needs the hash (Hash), which the loader sets from the stack_files row; Bytes stays nil.
func EmptyTree ¶ added in v0.2.24
EmptyTree is the marker pack a Reconciler passes to a kind whose tree was removed entirely between versions (Path == ""), so the materializer still runs its delete-missing pass. Only the blob kind consumes it today; the single-file kinds keep "pack removed = stop managing".
func NewRawPack ¶
NewRawPack builds a RawPack from a pack path + its resolved bytes, deriving Kind and Name from the path. It returns (RawPack{}, false) when the path is not a well-formed pack path (caller should have validated, but this keeps the loader honest).
type Reconciler ¶
type Reconciler struct {
// contains filtered or unexported fields
}
Reconciler dispatches a version's packs to the registered materializers. It is built once at boot (server.go) from the live stores and injected into the activation path; reconcile is invoked best-effort AFTER the activation tx commits, so a slow or failing store never stalls or rolls back a deploy.
func NewReconciler ¶
func NewReconciler(ms ...Materializer) *Reconciler
NewReconciler builds a Reconciler from the given materializers. A nil or empty set yields a no-op Reconciler (Reconcile returns nil) — the open-core default when no seedable store is configured. A duplicate Kind is a programming error and panics at boot.
func (*Reconciler) Kinds ¶
func (r *Reconciler) Kinds() []string
Kinds returns the pack kinds this Reconciler can materialise (sorted), for diagnostics.
func (*Reconciler) LocalKinds ¶ added in v0.2.24
func (r *Reconciler) LocalKinds() []string
LocalKinds returns the kinds whose materializer reconciles PER-NODE (Shared() == false), sorted. The activation origin uses it to warn when a fleet node seeds into a node-local store — the seed would be invisible to every other node.
func (*Reconciler) Reconcile ¶
func (r *Reconciler) Reconcile(ctx context.Context, scope Scope, packs []RawPack, origin bool) error
Reconcile groups packs by kind and dispatches each group to its materializer. origin marks the activation origin (the control plane): shared-store materializers reconcile only when origin is true; per-node materializers always run. Best-effort and exhaustive — every materializer is attempted and the errors are aggregated, so one failing store doesn't skip the others. A pack kind with no registered materializer is reported (a deploy referencing a store this node can't seed should be visible, not silent).
type Scope ¶
type Scope struct {
Tenant string
Stack string
Version int64
// Force is the `txco data apply --force` signal: the tree WINS over
// runtime drift — a materializer re-asserts every seeded item and drops
// the ones the tree no longer ships, even where it would otherwise leave
// a runtime edit alone (blobseed). The caller also reconciles every pack,
// not just the changed ones.
Force bool
}
Scope identifies the stack version a reconcile is materialising. Tenant scopes every store call; Stack/Version are carried for logging + future prior-version diffing (whole-collection drop, P4).
Directories
¶
| Path | Synopsis |
|---|---|
|
Package blobseed is the BLOBS/ store-seed Materializer: it reconciles a stack's BLOBS/** tree (chassis/storeseed) into the tenant's blob name index (chassis/blob).
|
Package blobseed is the BLOBS/ store-seed Materializer: it reconciles a stack's BLOBS/** tree (chassis/storeseed) into the tenant's blob name index (chassis/blob). |
|
Package kvseed is the KV store-seed Materializer: it reconciles KV/<namespace>.jsonl packs (chassis/storeseed) into the tenant KV store.
|
Package kvseed is the KV store-seed Materializer: it reconciles KV/<namespace>.jsonl packs (chassis/storeseed) into the tenant KV store. |
|
Package vecseed is the vector store-seed Materializer: it reconciles VECTORS/<collection>.jsonl packs (chassis/storeseed) into a vector.Store.
|
Package vecseed is the vector store-seed Materializer: it reconciles VECTORS/<collection>.jsonl packs (chassis/storeseed) into a vector.Store. |