store

package
v0.6.6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 11, 2026 License: AGPL-3.0 Imports: 4 Imported by: 0

Documentation

Overview

Package store defines the storage abstraction memini retrieves memories through. Drivers (sqlite-vec, Postgres/VectorChord) implement Store; the hybrid-search and service layers depend only on this interface.

Index

Constants

This section is empty.

Variables

View Source
var ErrConflict = errors.New("id exists in a different namespace")

ErrConflict is returned by Upsert when the given ID already exists in a different namespace, preventing cross-tenant hijacking.

View Source
var ErrNotFound = errors.New("memory not found")

ErrNotFound is returned by Get/Delete when no memory matches.

Functions

func MemoryTypeLabel added in v0.2.5

func MemoryTypeLabel(m *memory.Memory) string

MemoryTypeLabel returns m's typed-extraction class for the memory_type metric label, or "" when absent or unrecognized (keeping cardinality bounded).

func OrEmptyMap added in v0.2.8

func OrEmptyMap(m map[string]any) map[string]any

OrEmptyMap returns m, or an empty map when m is nil, so drivers persist an empty JSON object rather than null for absent metadata.

func OrEmptySlice added in v0.2.8

func OrEmptySlice(s []string) []string

OrEmptySlice returns s, or an empty slice when s is nil, so drivers persist an empty JSON array rather than null for absent tags.

Types

type ActivityStore added in v0.6.6

type ActivityStore interface {
	// NamespaceActivity returns one row per namespace holding at least one
	// live memory, ordered by namespace. now is the expiry-evaluation instant
	// (mirroring Filter.Now); the zero value means the wall clock.
	NamespaceActivity(ctx context.Context, now time.Time) ([]NamespaceActivity, error)
}

ActivityStore is implemented by drivers that can compute per-namespace activity in a single aggregate query (SELECT namespace, COUNT(*), MAX(created_at) ... GROUP BY namespace). It is an optional capability interface — the EmbedModelStore/LinkStore precedent — so callers type-assert and degrade gracefully against a driver that predates it: the briefing child rollup skips itself entirely rather than falling back to per-namespace scans.

type EmbedModelStore added in v0.3.9

type EmbedModelStore interface {
	// EmbedModel returns the recorded embedding model name, or "" when none has
	// been recorded yet (a fresh store, or one created before this was tracked).
	EmbedModel(ctx context.Context) (string, error)
	// SetEmbedModel records model as the embedding model the stored vectors were
	// produced with, overwriting any previous value.
	SetEmbedModel(ctx context.Context, model string) error
}

EmbedModelStore is implemented by drivers that record which embedding model produced their stored vectors. It lets the bootstrap detect a silent model swap — a new MEMINI_EMBED_MODEL at the same dimensionality, which the dims guard cannot catch — where old and new vectors share a width but live in incomparable spaces, quietly degrading recall.

type Filter

type Filter struct {
	// Tiers restricts results to these tiers; empty means all tiers.
	Tiers []memory.Tier
	// Levels restricts results to memories whose derivation level (explicit or
	// deduced) matches one of the listed values; empty means no level constraint.
	Levels []memory.Level
	// Tags restricts results to memories carrying every listed tag (AND). Empty
	// means no tag constraint.
	Tags []string
	// Metadata restricts results to memories whose top-level metadata contains
	// each listed key with the given string value (AND). Empty means no metadata
	// constraint. Only top-level string-valued entries are matched.
	Metadata map[string]string
	// ExcludeMetadata drops memories whose top-level metadata carries any of the
	// listed key=value pairs (the inverse of Metadata). Empty means no exclusion.
	// Used to keep a caller from recalling its own just-written memories — e.g.
	// the OpenClaw plugin tags each captured turn with its session id and excludes
	// that session on the pre-turn auto-recall, so a turn already in the live
	// transcript is not echoed back as "long-term memory".
	ExcludeMetadata map[string]string
	// IncludeExpired includes memories past their TTL (default excludes them).
	IncludeExpired bool
	// IncludeSuperseded includes contradiction-tombstoned memories.
	IncludeSuperseded bool
	// Now is the instant expiry is evaluated at; the zero value means the wall
	// clock. Callers with an injected clock (service.WithClock) should set it so
	// store-level expiry filtering agrees with their notion of "now".
	Now time.Time
	// AsOf, when non-zero, switches to time-travel recall: results are the
	// memories whose validity window contained AsOf (valid_from <= AsOf < valid_to,
	// treating NULL bounds as open). It overrides the superseded exclusion, so a
	// fact that was true then but has since been replaced is still returned.
	AsOf time.Time
}

Filter narrows a search to a subset of memories. The zero value matches all live (non-expired, non-superseded) memories in the namespace.

type LinkStore added in v0.6.6

type LinkStore interface {
	// PutLink inserts or replaces the link keyed by (l.Src, l.Dst). An
	// upsert overwrites created_at — deliberately, unlike memory Put's
	// preserve-on-upsert: links have no recency semantics, and import
	// restore relies on this to replay a link's original CreatedAt rather
	// than stamping "now".
	PutLink(ctx context.Context, l NamespaceLink) error
	// DeleteLink removes the link from src to dst. The bool reports whether a
	// link existed to delete; a missing link is not an error.
	DeleteLink(ctx context.Context, src, dst string) (bool, error)
	// ListLinks returns the links whose Src is src, ordered by Dst. Empty (not
	// an error) when src has no outgoing links.
	ListLinks(ctx context.Context, src string) ([]NamespaceLink, error)
	// ListAllLinks returns every link in the store, for the CLI/UI.
	ListAllLinks(ctx context.Context) ([]NamespaceLink, error)
	// RenameLinkEndpoints rewrites every link whose Src or Dst equals from to
	// to instead, used when a namespace is moved (maintenance.Move). When a
	// rewritten link collides with a pre-existing row at its new key, the
	// pre-existing row wins and the renamed link is dropped: the target
	// namespace's own explicit grant must never be silently widened or
	// narrowed by an inherited one. A no-op when from == to.
	RenameLinkEndpoints(ctx context.Context, from, to string) error
}

LinkStore is implemented by drivers that persist namespace_links, the cross-namespace read-linking table (namespace-cascade design). It is an optional capability interface — the EmbedModelStore precedent above — so callers type-assert and degrade gracefully against a driver that predates it.

type Metrics

type Metrics interface {
	// Upsert records one Upsert outcome. op is "insert" or "update"; tier
	// is the memory's tier (working/episodic/semantic/procedural); memoryType
	// is the typed-extraction class (decision/preference/problem) or "".
	Upsert(op, tier, memoryType string)
	// Delete records a hard delete from the Forget API path.
	Delete()
	// SoftDelete records a tombstone written by consolidation.
	SoftDelete()
	// SweepExpired records one memory removed by the decay sweeper.
	SweepExpired(tier string)
	// ActiveByTier sets the current count of live (non-superseded,
	// non-expired) memories per tier. Called periodically after sweeps/fsck.
	ActiveByTier(tier string, n int)
	// DedupTombstoned records the number of memories tombstoned by one
	// dedup pass (the periodic vector-cluster job or a one-shot dedup call).
	// Called once per pass with the pass total.
	DedupTombstoned(n int)
}

Metrics receives store events for observability. Methods must be safe for concurrent use; a nil Metrics is replaced by a no-op. Implementations live alongside the Prometheus registry in cmd/memini.

func NopMetrics

func NopMetrics() Metrics

NopMetrics is exported for tests.

type NamespaceActivity added in v0.6.6

type NamespaceActivity struct {
	NS        string
	Total     int
	LastWrite time.Time
}

NamespaceActivity summarizes one namespace's live activity: the count of live memories (excluding expired, superseded, and closed-validity rows — the same liveness a default List filter applies) and the most recent created_at among them (the "last write" column Stats.LastWriteAt uses; unlike Stats, tombstoned rows do not advance it, since a single aggregate query shares one liveness WHERE for both figures).

type NamespaceLink struct {
	Src, Dst  string
	Tiers     []memory.Tier // nil = durable default applied by service
	Note      string
	CreatedAt time.Time
}

NamespaceLink is a directed cross-namespace read link: recall scoped to Src additionally reads durable memories from Dst. Tiers restricts which tiers cross the boundary; nil means the service layer applies its durable-tier default (only semantic/procedural cross namespace boundaries — episodic and working never do, in either direction). Note is a free-text annotation for operators explaining why the link exists.

type Scored

type Scored struct {
	Memory *memory.Memory
	Score  float64
}

Scored is a memory paired with a relevance score for the query that produced it. Results are always returned best-first; Score is higher-is-better and is only comparable within a single method's result set.

type Store

type Store interface {
	// Upsert inserts or replaces a memory (matched by ID within its namespace),
	// including its keyword-search index entry. When len(m.Embedding) == 0 the
	// row is stored with no vector-index entry — kept keyword-searchable only,
	// the write path used when embedding generation is unavailable; a stale
	// vector-index entry from a prior upsert of the same ID is removed. Any
	// other embedding length must equal the store's configured dims, or
	// Upsert errors. VectorSearch never returns a vectorless row. Returns
	// ErrConflict when the given ID already exists under a different namespace.
	Upsert(ctx context.Context, m *memory.Memory) error

	// Get returns a memory by ID, or ErrNotFound.
	Get(ctx context.Context, namespace, id string) (*memory.Memory, error)

	// PredecessorIDs returns the IDs of memories in the namespace whose
	// SupersededBy points at id — the versions id replaced. Empty when none.
	// Used to walk a memory's supersession lineage backwards.
	PredecessorIDs(ctx context.Context, namespace, id string) ([]string, error)

	// GetByFingerprint returns a live (non-superseded, non-expired) memory in the
	// namespace and tier whose content fingerprint (memory.Fingerprint) matches,
	// for exact-restatement dedup at write time. Returns ErrNotFound when none
	// matches. Now is the instant expiry is evaluated at (zero means wall clock).
	GetByFingerprint(ctx context.Context, namespace string, tier memory.Tier, fingerprint string, now time.Time) (*memory.Memory, error)

	// Delete removes a memory by ID, or returns ErrNotFound.
	Delete(ctx context.Context, namespace, id string) error

	// SetSuperseded tombstones a memory by recording the ID that replaced it,
	// excluding it from default search results. Returns ErrNotFound if missing.
	SetSuperseded(ctx context.Context, namespace, id, supersededBy string) error

	// Restore clears superseded_by/valid_to so a tombstoned memory is live
	// again. Returns ErrNotFound if missing.
	Restore(ctx context.Context, namespace, id string) error

	// VectorSearch returns the k memories nearest to vec, best-first.
	VectorSearch(ctx context.Context, namespace string, vec []float32, f Filter, k int) ([]Scored, error)

	// KeywordSearch returns the k best full-text matches for query, best-first.
	KeywordSearch(ctx context.Context, namespace, query string, f Filter, k int) ([]Scored, error)

	// Reinforce records that the given memories were just recalled: it bumps
	// access_count and last_accessed_at to accessedAt. When newExpiry is non-nil
	// it also slides the TTL forward for those that already expire (short-term
	// memories), so frequently-recalled memories don't go stale. Missing IDs are
	// ignored.
	Reinforce(ctx context.Context, namespace string, ids []string, accessedAt time.Time, newExpiry *time.Time) error

	// DeleteIfExpiredBefore removes a memory only when its expiry is still at or
	// before cutoff. Returns ErrNotFound when the memory was absent or when its
	// TTL was slid past cutoff by Reinforce, so the caller never over-deletes a
	// memory that was accessed between ListExpired and the delete call.
	DeleteIfExpiredBefore(ctx context.Context, namespace, id string, cutoff time.Time) error

	// ListExpired returns up to limit memories whose TTL has passed, for the
	// decay sweeper.
	ListExpired(ctx context.Context, now time.Time, limit int) ([]*memory.Memory, error)

	// List returns memories in a namespace matching f (without embeddings),
	// for maintenance tasks (short-term capacity, fsck). limit <= 0 means all.
	List(ctx context.Context, namespace string, f Filter, limit int) ([]*memory.Memory, error)

	// ListNamespaces returns the distinct namespaces that hold memories.
	ListNamespaces(ctx context.Context) ([]string, error)

	// DeleteNamespace removes every memory in a namespace (including embeddings
	// and keyword-search index entries). Returns the number of memories deleted.
	DeleteNamespace(ctx context.Context, namespace string) (int64, error)

	// Reassign moves the given memories from fromNS to toNS (recovery from a
	// botched import or a shared-pool migration). IDs absent from fromNS are
	// skipped; since IDs are globally unique a move never collides in toNS.
	// Returns the number of memories moved.
	Reassign(ctx context.Context, fromNS string, ids []string, toNS string) (int64, error)

	// Retier changes a memory's tier and expiry in place (used by retro-tiering
	// to demote stale durable memories). Tier lives only in the main row, so no
	// vector/keyword reindex is needed. Returns ErrNotFound if missing.
	Retier(ctx context.Context, namespace, id string, tier memory.Tier, expiresAt *time.Time) error

	// SetConfidence updates a memory's corroboration confidence in place and
	// bumps updated_at to now (resetting the lazy-decay baseline), used when a
	// durable fact is re-observed. Returns ErrNotFound if missing.
	SetConfidence(ctx context.Context, namespace, id string, confidence float64, now time.Time) error

	// MarkContradicted invalidates a durable fact a newer write contradicts: it
	// sets confidence, stamps valid_to=now (unless already set) so the fact
	// drops out of live recall while staying reachable via AsOf time-travel, and
	// records the contradicting id plus the pre-update confidence in metadata for
	// audit and reversal (Restore clears valid_to). Non-destructive: the row and
	// its history are kept. Bumps updated_at to now. Returns ErrNotFound if
	// missing.
	MarkContradicted(ctx context.Context, namespace, id, contradictedBy string, confidence float64, now time.Time) error

	// Ping verifies the backend is reachable, for readiness checks.
	Ping(ctx context.Context) error

	// Close releases backend resources.
	Close() error
}

Store is the persistence and retrieval contract for memories. Implementations must be safe for concurrent use.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL