sqlitevec

package
v0.7.29 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 13, 2026 License: AGPL-3.0 Imports: 15 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store is a sqlite-vec backed store.Store.

func Open

func Open(ctx context.Context, path string, dims int) (*Store, error)

Open opens (creating if needed) the sqlite database at path and ensures the schema exists for the given embedding dimensionality.

func (*Store) AppendEvents added in v0.6.8

func (s *Store) AppendEvents(ctx context.Context, events []store.Event) error

AppendEvents inserts one operation's rows in a single transaction, so they land contiguously and share a created_at — the adjacency ListEvents' ordering relies on to let the reader regroup flat rows back into whole events.

func (*Store) ChunkVectorSearch added in v0.7.3

func (s *Store) ChunkVectorSearch(ctx context.Context, namespace string, vec []float32, f store.Filter, k int) ([]store.Scored, error)

ChunkVectorSearch implements store.ChunkStore.

The vec0 KNN is wrapped in a subquery so its MATCH/k constraints stay isolated on its own plan and cannot be perturbed by the outer GROUP BY. MIN(distance) per memory is the max-pool: the same distance-to-score function then applies, so these scores land in the same space as VectorSearch's — which recall depends on, since its gates are absolute thresholds rather than ranks.

func (*Store) ClaimRepairs added in v0.7.16

func (s *Store) ClaimRepairs(ctx context.Context, state store.RepairState, now time.Time,
	lease time.Duration, limit int) ([]store.RepairRow, error)

ClaimRepairs implements store.RepairStore.

The single-statement UPDATE ... WHERE id IN (SELECT ... LIMIT n) RETURNING form is safe here: sqlite applies every database change during the first sqlite3_step and embargoes RETURNING output until they are all complete, and writes are serialized, so two claimants cannot take the same row. This is pinned by TestClaimRepairsIsExclusiveUnderConcurrency rather than trusted — memini builds on ncruces/go-sqlite3, not the drivers the upstream reports cover.

RETURNING row order is documented as arbitrary, so rows are re-sorted by due time in Go rather than relied on to arrive ordered. The rows must also be drained fully: the UPDATE has already applied by the time the first row is read, so abandoning the cursor early would strand claimed rows for a whole lease.

func (*Store) Close

func (s *Store) Close() error

Close closes the underlying database.

func (*Store) CountChunks added in v0.7.3

func (s *Store) CountChunks(ctx context.Context, namespace string) (int, error)

CountChunks implements store.ChunkStore. It counts mapping rows rather than vectors, attributing rows through the memories join; a row whose memory is gone (an orphan — exactly what this method exists to make visible) belongs to no namespace and is therefore included in every count.

func (*Store) CountUnchunked added in v0.7.3

func (s *Store) CountUnchunked(ctx context.Context, namespace string, minRunes int) (int, error)

CountUnchunked implements store.ChunkStore: ListUnchunked's queue in full, where the list shows one batch.

func (*Store) Delete

func (s *Store) Delete(ctx context.Context, namespace, id string) error

Delete removes a memory and its index entries.

func (*Store) DeleteAPIKey added in v0.6.7

func (s *Store) DeleteAPIKey(ctx context.Context, name string) (bool, error)

DeleteAPIKey removes the key by name. The bool reports whether a key existed to delete.

func (*Store) DeleteIfExpiredBefore

func (s *Store) DeleteIfExpiredBefore(ctx context.Context, namespace, id string, cutoff time.Time) error

DeleteIfExpiredBefore removes a memory only if its expiry is still at or before cutoff. Returns ErrNotFound when the memory is absent or its TTL was slid past cutoff by Reinforce since the last ListExpired call.

func (s *Store) DeleteLink(ctx context.Context, src, dst string) (bool, error)

DeleteLink removes the link from src to dst. The bool reports whether a link existed to delete.

func (*Store) DeleteNamespace added in v0.0.8

func (s *Store) DeleteNamespace(ctx context.Context, namespace string) (int64, error)

DeleteNamespace removes every memory in a namespace, including vector and FTS index entries, plus any namespace_links row that references the namespace on either side (gap G5: a deleted namespace must not leave a dangling link). Returns the number of memories deleted.

func (*Store) DeletePins added in v0.7.3

func (s *Store) DeletePins(ctx context.Context, keys []string) (int64, error)

DeletePins removes the entries with the given keys and returns the number of rows actually deleted.

func (*Store) EmbedModel added in v0.3.9

func (s *Store) EmbedModel(ctx context.Context) (string, error)

EmbedModel returns the recorded embedding model name, or "" if none was set.

func (*Store) FailRepair added in v0.7.16

func (s *Store) FailRepair(ctx context.Context, namespace, id, lastErr string, nextRunAt time.Time) error

FailRepair implements store.RepairStore.

func (*Store) Get

func (s *Store) Get(ctx context.Context, namespace, id string) (*memory.Memory, error)

Get returns a memory by ID.

func (*Store) GetAPIKeyByHash added in v0.6.7

func (s *Store) GetAPIKeyByHash(ctx context.Context, hash string) (*store.APIKey, error)

GetAPIKeyByHash returns the key whose hash matches, or nil, nil when none does.

func (*Store) GetByFingerprint added in v0.2.9

func (s *Store) GetByFingerprint(
	ctx context.Context, namespace string, tier memory.Tier, fingerprint string, now time.Time,
) (*memory.Memory, error)

GetByFingerprint returns the most recent live memory in namespace+tier whose content fingerprint matches. Superseded, expired, and validity-closed (contradicted) rows are excluded so a dead duplicate never absorbs a fresh write — re-asserting a contradicted fact must store a live row, not corroborate the invalidated one.

func (*Store) GetEmbedding added in v0.7.3

func (s *Store) GetEmbedding(ctx context.Context, namespace, id string) ([]float32, error)

GetEmbedding returns the stored vector for a memory, or nil when the row is vectorless. The lookup is a LEFT JOIN for the same reason Reassign's is: a vectorless memory has no vec_memories row at all, and an inner join would report it as ErrNotFound — indistinguishable from a memory that doesn't exist, which the caller must tell apart (one falls back to embedding, the other is an error).

func (*Store) GetPins added in v0.7.3

func (s *Store) GetPins(ctx context.Context, keys []string) ([]store.Pin, error)

GetPins returns the entries matching the given keys, in no particular order; a key with no matching row is simply absent.

func (*Store) GlobalClientSettings added in v0.7.0

func (s *Store) GlobalClientSettings(ctx context.Context) (store.ClientSettings, error)

GlobalClientSettings returns the stored global default ClientSettings, or the zero value (every field nil) if none has been set yet.

func (*Store) IDsByPrefix added in v0.7.7

func (s *Store) IDsByPrefix(ctx context.Context, namespace, prefix string, limit int) ([]string, error)

IDsByPrefix returns the IDs in the namespace beginning with prefix, ascending, bounded at limit rows. The LIKE with escaped metacharacters is the indexed prefix scan; the substr equality guard restores byte-literal matching (sqlite LIKE is ASCII case-insensitive, and the store contract is literal) and runs before LIMIT so a case-mismatched row never consumes a slot.

func (*Store) KeywordSearch

func (s *Store) KeywordSearch(ctx context.Context, namespace, query string, f store.Filter, k int) ([]store.Scored, error)

KeywordSearch returns the k best BM25 full-text matches in the namespace.

func (*Store) List

func (s *Store) List(ctx context.Context, namespace string, f store.Filter, limit int) ([]*memory.Memory, error)

List returns memories in a namespace matching f (without embeddings), ordered by f.Sort (newest-created first by default).

func (*Store) ListAPIKeys added in v0.6.7

func (s *Store) ListAPIKeys(ctx context.Context) ([]store.APIKey, error)

ListAPIKeys returns every key ordered by name.

func (s *Store) ListAllLinks(ctx context.Context) ([]store.NamespaceLink, error)

ListAllLinks returns every link in the store, ordered by Src then Dst.

func (*Store) ListEvents added in v0.6.8

func (s *Store) ListEvents(ctx context.Context, f store.EventFilter) ([]store.Event, error)

ListEvents returns rows matching f, newest first.

func (*Store) ListExpired

func (s *Store) ListExpired(ctx context.Context, now time.Time, limit int) ([]*memory.Memory, error)

ListExpired returns up to limit memories whose TTL has passed.

func (s *Store) ListLinks(ctx context.Context, src string) ([]store.NamespaceLink, error)

ListLinks returns the links whose Src is src, ordered by Dst.

func (*Store) ListNamespaces

func (s *Store) ListNamespaces(ctx context.Context) ([]string, error)

ListNamespaces returns the distinct namespaces holding memories.

func (*Store) ListPins added in v0.7.3

func (s *Store) ListPins(ctx context.Context) ([]store.Pin, error)

ListPins returns every entry ordered by Key.

func (*Store) ListUnchunked added in v0.7.3

func (s *Store) ListUnchunked(ctx context.Context, namespace string, minRunes int, afterID string, limit int) ([]*memory.Memory, error)

ListUnchunked implements store.ChunkStore. length() counts characters in SQLite (not bytes, unlike its BLOB overload), which is what internal/chunk bounds on, so the two agree.

func (*Store) MarkContradicted added in v0.5.6

func (s *Store) MarkContradicted(ctx context.Context, namespace, id, contradictedBy string, confidence float64, now time.Time) error

MarkContradicted invalidates a durable fact a newer write contradicts. The SET expressions read the pre-update confidence column (SQLite evaluates the right-hand side against the old row), snapshotting it into metadata for audit and reversal before overwriting it.

func (*Store) MarkRepairNeeded added in v0.7.16

func (s *Store) MarkRepairNeeded(ctx context.Context, namespace string, ids []string,
	state store.RepairState) (int64, error)

MarkRepairNeeded implements store.RepairStore.

func (*Store) NamespaceActivity added in v0.6.6

func (s *Store) NamespaceActivity(ctx context.Context, now time.Time) ([]store.NamespaceActivity, error)

NamespaceActivity implements store.ActivityStore: one aggregate query for per-namespace live count and most recent created_at. Liveness reuses filterClause with an empty Filter so it stays byte-identical to what a default List applies (not expired at now, not superseded, validity window not closed).

func (*Store) ParkRepair added in v0.7.16

func (s *Store) ParkRepair(ctx context.Context, namespace, id, lastErr string, now time.Time) error

ParkRepair implements store.RepairStore.

The park instant is recorded in embed_next_run_at. That column means "not before" for a claimable row, and a parked row is never claimable (the claim filters on state), so reusing it as "parked at" costs no extra column and keeps RearmRepairs on the same partial index. It must NOT be updated_at: repairs deliberately never bump that, precisely so a system re-embed cannot be mistaken for a content edit.

func (*Store) Ping

func (s *Store) Ping(ctx context.Context) error

Ping verifies the database is reachable.

func (*Store) PredecessorIDs added in v0.4.19

func (s *Store) PredecessorIDs(ctx context.Context, namespace, id string) ([]string, error)

PredecessorIDs returns the IDs of memories in the namespace superseded by id.

func (*Store) PruneEvents added in v0.6.8

func (s *Store) PruneEvents(ctx context.Context, olderThan time.Time, keepMax int) (int64, error)

PruneEvents trims the log by age and by row cap.

func (*Store) PutAPIKey added in v0.6.7

func (s *Store) PutAPIKey(ctx context.Context, k store.APIKey) error

PutAPIKey inserts or replaces the key keyed by k.Name.

Unlike PutLink (which deliberately overwrites created_at on every upsert, since links carry no recency semantics — see its doc above), this upsert preserves the existing row's created_at when k.CreatedAt is the zero value: API keys are long-lived identity, and rotating a key's hash or home namespace must not reset "when was this key first created". A non-zero k.CreatedAt (e.g. import restore replaying an original timestamp) still overwrites it. The lookup-then-upsert runs in a transaction so a concurrent PutAPIKey for the same name cannot race between the read of the existing created_at and the write.

func (*Store) PutChunks added in v0.7.3

func (s *Store) PutChunks(ctx context.Context, namespace, id string, updatedAt time.Time, chunks []memory.Chunk) (bool, error)

PutChunks implements store.ChunkStore. The updated_at guard and the chunk write share the transaction, so nothing can change the row between them — this is the write BackfillChunks uses precisely because a Get-then-Upsert round-trip could neither carry the document vector nor close that window.

func (s *Store) PutLink(ctx context.Context, l store.NamespaceLink) error

PutLink inserts or replaces the link keyed by (l.Src, l.Dst).

Unlike memory Put (created_at is immutable after insert, see the comment on its INSERT above), an upsert here overwrites created_at. This is intentional, not an oversight: links carry no recency semantics that a stable created_at would protect, and import restore relies on the overwrite being conditional on l.CreatedAt being non-zero (below) so it can replay a link's original creation time instead of stamping "now".

func (*Store) PutPins added in v0.7.3

func (s *Store) PutPins(ctx context.Context, entries []store.Pin) error

PutPins upserts entries in a single transaction, keyed by each entry's Key. An update preserves the existing row's created_at/created_by — looked up inside the same transaction so a concurrent Put for the same key cannot race between the read and the write, mirroring PutAPIKey's lookup-then-upsert pattern — while namespace/note/updated_at take the incoming values.

func (*Store) RearmRepairs added in v0.7.16

func (s *Store) RearmRepairs(ctx context.Context, failedBefore, now time.Time) (int64, error)

RearmRepairs implements store.RepairStore. Parked rows are identified by the park instant ParkRepair left in embed_next_run_at.

func (*Store) Reassign added in v0.0.11

func (s *Store) Reassign(ctx context.Context, fromNS string, ids []string, toNS string) (int64, error)

Reassign moves memories from fromNS to toNS, updating the namespace column and rewriting the vec0 partition row (the FTS row carries no namespace). IDs absent from fromNS are skipped; IDs are globally unique so a move never collides in toNS. The lookup is a LEFT JOIN (not an inner join) because a vectorless memory (see Upsert) has no vec_memories row at all — an inner join would silently skip it as "not found" instead of moving it.

func (*Store) Reinforce

func (s *Store) Reinforce(ctx context.Context, namespace string, ids []string, accessedAt time.Time, newExpiry *time.Time) error

Reinforce bumps access_count/last_accessed_at and optionally slides the TTL.

func (*Store) RenameAPIKeyNamespaces added in v0.6.7

func (s *Store) RenameAPIKeyNamespaces(ctx context.Context, from, to string) error

RenameAPIKeyNamespaces rewrites every key whose home_ns or default_ns equals from to to instead — both columns in one statement, so a namespace move (maintenance.Move, alongside RenameLinkEndpoints) leaves neither binding dangling. Unlike RenameLinkEndpoints there is no collision handling: neither column is part of a key's identity, so a plain UPDATE suffices. A no-op when from == to.

func (*Store) RenameLinkEndpoints added in v0.6.6

func (s *Store) RenameLinkEndpoints(ctx context.Context, from, to string) error

RenameLinkEndpoints rewrites every link whose src_ns or dst_ns equals from to to instead. When a rewritten link collides with a pre-existing row at its new key, the pre-existing row is kept and the renamed link dropped (ON CONFLICT DO NOTHING): the target namespace's own explicit grant wins over an inherited one, so a rename can never silently widen or narrow tier access the target had already configured. The SELECT is ordered by (src_ns, dst_ns) so which renamed link survives a multi-way collision (e.g. the reciprocal pair link(from,to)+link(to,from) collapsing onto (to,to)) is deterministic: the first row in key order wins. A no-op when from == to.

func (*Store) RenamePinNamespaces added in v0.7.3

func (s *Store) RenamePinNamespaces(ctx context.Context, from, to string) error

RenamePinNamespaces rewrites every entry whose namespace exactly equals from to to instead; a namespace that merely starts with from is untouched.

func (*Store) RepairStateOf added in v0.7.16

func (s *Store) RepairStateOf(ctx context.Context, namespace, id string) (store.RepairState, int, string, error)

RepairStateOf implements store.RepairStore.

func (*Store) RepairStats added in v0.7.16

func (s *Store) RepairStats(ctx context.Context) ([]store.RepairStat, error)

RepairStats implements store.RepairStore.

func (*Store) Restore added in v0.4.12

func (s *Store) Restore(ctx context.Context, namespace, id string) error

Restore clears superseded_by/valid_to so a tombstoned memory is live again.

func (*Store) Retier added in v0.0.11

func (s *Store) Retier(ctx context.Context, namespace, id string, tier memory.Tier, expiresAt *time.Time) error

Retier changes a memory's tier and expiry in place. Tier and expiry live only in the memories row, so no vector/FTS reindex is required.

func (*Store) ServedSnapshots added in v0.7.9

func (s *Store) ServedSnapshots(
	ctx context.Context, namespace string, ids []string, since time.Time,
) (map[string]store.MemorySnapshot, error)

ServedSnapshots returns the newest serve-row snapshot per memory ID. The inner MAX(id) picks one row per memory — ids are monotonic, so the greatest is the newest — and the outer select reads that row's columns; grouping and projecting in one statement would leave the non-aggregated columns ambiguous.

func (*Store) SetAssessedImportance added in v0.7.18

func (s *Store) SetAssessedImportance(ctx context.Context, namespace, id string, v float64, now time.Time) error

SetAssessedImportance stamps the LLM-assessed intrinsic importance in place. Deliberately does NOT bump updated_at: assessment is a system annotation, not a re-observation, and touching updated_at would reset confidence lazy-decay and demote eligibility. Validity-closed rows are skipped (ErrNotFound) for the same reason SetConfidence skips them: an invalidated fact must not be re-ranked back into recall.

func (*Store) SetConfidence added in v0.0.11

func (s *Store) SetConfidence(ctx context.Context, namespace, id string, confidence float64, now time.Time) error

SetConfidence updates a memory's confidence and bumps updated_at to now. Confidence lives only in the memories row, so no vector/FTS reindex is needed. Validity-closed rows are skipped (ErrNotFound): corroboration must never regrow an invalidated fact, even when MarkContradicted lands between the caller's read and this write.

func (*Store) SetEmbedModel added in v0.3.9

func (s *Store) SetEmbedModel(ctx context.Context, model string) error

SetEmbedModel records the embedding model the stored vectors were produced with.

func (*Store) SetEmbeddingIfUnchanged added in v0.7.16

func (s *Store) SetEmbeddingIfUnchanged(ctx context.Context, namespace, id, fingerprint string,
	vec []float32, next store.RepairState) (bool, error)

SetEmbeddingIfUnchanged implements store.RepairStore.

The fingerprint guard and both writes share one transaction, so a content edit landing mid-repair cannot slip between them — the check-then-act race a re-read outside the store can never close (the same reasoning as PutChunks). updated_at is deliberately left alone: a system re-embed is index maintenance, not a logical edit.

func (*Store) SetGlobalClientSettings added in v0.7.0

func (s *Store) SetGlobalClientSettings(ctx context.Context, cs store.ClientSettings) error

SetGlobalClientSettings replaces the stored global default ClientSettings wholesale (not a merge): only fields set on s are persisted, since nil pointer fields with `omitempty` marshal to nothing.

func (*Store) SetMetrics

func (s *Store) SetMetrics(m store.Metrics)

SetMetrics installs an observability sink. Passing nil disables metrics.

func (*Store) SetRepairState added in v0.7.16

func (s *Store) SetRepairState(ctx context.Context, namespace, id, fingerprint string,
	next store.RepairState) (bool, error)

SetRepairState implements store.RepairStore.

func (*Store) SetSuperseded

func (s *Store) SetSuperseded(ctx context.Context, namespace, id, supersededBy string) error

SetSuperseded records that a memory was replaced by supersededBy.

func (*Store) Upsert

func (s *Store) Upsert(ctx context.Context, m *memory.Memory) error

Upsert inserts or replaces a memory and its vector/keyword index entries. When m.Embedding is empty the row is stored with no vec_memories entry (keyword index still written) — the write path used when embedding generation is unavailable; any other length must equal the store's dims.

func (*Store) VectorSearch

func (s *Store) VectorSearch(ctx context.Context, namespace string, vec []float32, f store.Filter, k int) ([]store.Scored, error)

VectorSearch returns the k nearest live memories to vec in the namespace.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL