storage

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 20, 2026 License: MIT Imports: 27 Imported by: 0

Documentation

Overview

Package storage holds LadyM's persistence layer: the SQLite store, pluggable embedding providers, and the vector index.

Index

Constants

View Source 
const Edition = "personal"

Edition marks the build flavour of this binary ("personal" here). The CLI surfaces it via --version; the enterprise variant lives in edition_enterprise.go behind the enterprise build tag.

Variables

View Source 
var ErrIndexLockHeld = errors.New("code indexing is already running")

ErrIndexLockHeld is returned by TryAcquireIndexLock when another process already holds the code-index lock for this store's database. The user-facing IndexInProgressError type lives in the code package (code imports storage, so the type cannot live here); callers translate ErrIndexLockHeld into it.

Functions

func AssertDimMatches 

func AssertDimMatches(stored, configured int) error

AssertDimMatches raises EmbeddingDimensionMismatch when dims differ.

func CosineSimilarity 

func CosineSimilarity(a, b []float32) float64

CosineSimilarity returns the cosine of two vectors (assumed or forced to be normalised). Returns 0 when dimensions differ or either vector is zero.

func RegisterCallable 

func RegisterCallable(name string, fn func(string) ([]float32, error))

RegisterCallable registers a Go function as a named embedding provider.

func Tokenize 

func Tokenize(text string) []string

Tokenize is the lightweight tokenizer: words + punctuation as separate tokens, with camelCase and snake_case splitting so getUserName and get_user_name tokenize similarly.

Types

type CachedEmbedding 

type CachedEmbedding struct {
	// contains filtered or unexported fields
}

CachedEmbedding wraps an inner EmbeddingProvider with an LRU cache for Embed() calls. EmbedBatch always delegates straight through.

func NewCachedEmbedding 

func NewCachedEmbedding(inner EmbeddingProvider, size int) *CachedEmbedding

NewCachedEmbedding wraps inner with an LRU cache of the given size.

func (*CachedEmbedding) Dim 

func (c *CachedEmbedding) Dim() int

func (*CachedEmbedding) Embed 

func (c *CachedEmbedding) Embed(text string) ([]float32, error)

func (*CachedEmbedding) EmbedBatch 

func (c *CachedEmbedding) EmbedBatch(texts []string) ([][]float32, error)

func (*CachedEmbedding) HealthCheck 

func (c *CachedEmbedding) HealthCheck() (bool, string)

type CallableEmbedding 

type CallableEmbedding struct {
	// contains filtered or unexported fields
}

CallableEmbedding wraps a Go function as an embedding provider.

func NewCallableEmbedding 

func NewCallableEmbedding(fn func(string) ([]float32, error), dim int) *CallableEmbedding

NewCallableEmbedding wraps fn as an embedding provider.

func (*CallableEmbedding) Dim 

func (c *CallableEmbedding) Dim() int

func (*CallableEmbedding) Embed 

func (c *CallableEmbedding) Embed(text string) ([]float32, error)

func (*CallableEmbedding) EmbedBatch 

func (c *CallableEmbedding) EmbedBatch(texts []string) ([][]float32, error)

func (*CallableEmbedding) HealthCheck 

func (c *CallableEmbedding) HealthCheck() (bool, string)

type EmbeddingDimensionMismatch 

type EmbeddingDimensionMismatch struct {
	Stored     int
	Configured int
}

EmbeddingDimensionMismatch is raised when a reopened DB holds vectors of a different dim than the live provider.

func (*EmbeddingDimensionMismatch) Error 

func (e *EmbeddingDimensionMismatch) Error() string

type EmbeddingProvider 

type EmbeddingProvider interface {
	// Embed returns a float32 vector for text.
	Embed(text string) ([]float32, error)
	// EmbedBatch embeds a batch of texts (defaults to looping Embed).
	EmbedBatch(texts []string) ([][]float32, error)
	// Dim returns the vector dimension, or 0 when unknown until the first
	// embed call (deferred-dim providers such as Ollama).
	Dim() int
	// HealthCheck performs a one-shot probe for the web UI "test embedding".
	HealthCheck() (bool, string)
}

EmbeddingProvider is the contract every provider implements.

func MakeProvider 

func MakeProvider(cfg *config.Config) (EmbeddingProvider, error)

MakeProvider resolves the configured embedding provider.

type EmbeddingProviderError 

type EmbeddingProviderError struct {
	Msg string
}

EmbeddingProviderError is the base error for provider-side failures.

func (*EmbeddingProviderError) Error 

func (e *EmbeddingProviderError) Error() string

type FakeHTTPPoster 

type FakeHTTPPoster struct {
	Responder    func(payload any) (any, error)
	ExpectedPath string
	LastPayload  any
	LastURL      string
}

FakeHTTPPoster is a test double. Responder maps payload → JSON-able result.

func (*FakeHTTPPoster) Post 

func (f *FakeHTTPPoster) Post(url string, payload any, headers map[string]string) (any, error)

type HTTPEmbedding 

type HTTPEmbedding struct {
	// contains filtered or unexported fields
}

HTTPEmbedding is a generic, template-driven embedding provider.

func NewHTTPEmbedding 

func NewHTTPEmbedding(opts HTTPEmbeddingOptions) *HTTPEmbedding

NewHTTPEmbedding builds a generic HTTP embedding provider.

func (*HTTPEmbedding) Dim 

func (h *HTTPEmbedding) Dim() int

func (*HTTPEmbedding) Embed 

func (h *HTTPEmbedding) Embed(text string) ([]float32, error)

func (*HTTPEmbedding) EmbedBatch 

func (h *HTTPEmbedding) EmbedBatch(texts []string) ([][]float32, error)

func (*HTTPEmbedding) HealthCheck 

func (h *HTTPEmbedding) HealthCheck() (bool, string)

type HTTPEmbeddingOptions 

type HTTPEmbeddingOptions struct {
	BaseURL      string
	Request      string
	ResponsePath string
	Dim          int
	Model        string
	TimeoutS     float64
	// Poster, when non-nil, overrides the default real HTTP client (tests).
	Poster HTTPPoster
}

HTTPEmbeddingOptions configures the generic HTTP embedding provider.

type HTTPPoster 

type HTTPPoster interface {
	Post(url string, payload any, headers map[string]string) (any, error)
}

HTTPPoster is the minimal HTTP contract used by embedding providers so tests can inject a fake and never touch the network.

type HashingEmbedding 

type HashingEmbedding struct {
	// contains filtered or unexported fields
}

HashingEmbedding is the deterministic, offline, dependency-free embedding via feature hashing (unigram + bigram features, L2-normalised).

func NewHashingEmbedding 

func NewHashingEmbedding(dim int) *HashingEmbedding

NewHashingEmbedding returns a HashingEmbedding with the given dim.

func (*HashingEmbedding) Dim 

func (h *HashingEmbedding) Dim() int

func (*HashingEmbedding) Embed 

func (h *HashingEmbedding) Embed(text string) ([]float32, error)

func (*HashingEmbedding) EmbedBatch 

func (h *HashingEmbedding) EmbedBatch(texts []string) ([][]float32, error)

func (*HashingEmbedding) HealthCheck 

func (h *HashingEmbedding) HealthCheck() (bool, string)

type InMemoryVectorIndex 

type InMemoryVectorIndex struct {
	// contains filtered or unexported fields
}

InMemoryVectorIndex is a brute-force cosine index (deterministic, perfect for tests and small workspaces). It replaces the sqlite-vec extension used by the Python port with a pure-Go equivalent; vectors are still persisted as BLOBs in the store so a reopened store can rebuild the index.

func NewInMemoryVectorIndex 

func NewInMemoryVectorIndex(dim int) *InMemoryVectorIndex

NewInMemoryVectorIndex returns an empty index of the given dim.

func (*InMemoryVectorIndex) Delete 

func (ix *InMemoryVectorIndex) Delete(itemID string)

func (*InMemoryVectorIndex) Len 

func (ix *InMemoryVectorIndex) Len() int

func (*InMemoryVectorIndex) Search 

func (ix *InMemoryVectorIndex) Search(query []float32, topK int) []SearchHit

func (*InMemoryVectorIndex) Upsert 

func (ix *InMemoryVectorIndex) Upsert(itemID string, vector []float32) error

type OllamaEmbedding 

type OllamaEmbedding struct {
	// contains filtered or unexported fields
}

OllamaEmbedding targets the Ollama /api/embeddings endpoint. Dim is deferred until the first embed call.

func NewOllamaEmbedding 

func NewOllamaEmbedding(baseURL, model string, timeoutS float64, client HTTPPoster) *OllamaEmbedding

NewOllamaEmbedding builds an OllamaEmbedding (client may be nil for real HTTP).

func (*OllamaEmbedding) Dim 

func (o *OllamaEmbedding) Dim() int

func (*OllamaEmbedding) Embed 

func (o *OllamaEmbedding) Embed(text string) ([]float32, error)

func (*OllamaEmbedding) EmbedBatch 

func (o *OllamaEmbedding) EmbedBatch(texts []string) ([][]float32, error)

func (*OllamaEmbedding) HealthCheck 

func (o *OllamaEmbedding) HealthCheck() (bool, string)

type OpenAIEmbedding 

type OpenAIEmbedding struct {
	// contains filtered or unexported fields
}

OpenAIEmbedding targets the OpenAI (or OpenAI-compatible) embeddings API.

func NewOpenAIEmbedding 

func NewOpenAIEmbedding(model, baseURL, apiKey string, timeoutS float64) *OpenAIEmbedding

NewOpenAIEmbedding builds an OpenAIEmbedding. Empty baseURL defaults to https://api.openai.com/v1.

func (*OpenAIEmbedding) Dim 

func (o *OpenAIEmbedding) Dim() int

func (*OpenAIEmbedding) Embed 

func (o *OpenAIEmbedding) Embed(text string) ([]float32, error)

func (*OpenAIEmbedding) EmbedBatch 

func (o *OpenAIEmbedding) EmbedBatch(texts []string) ([][]float32, error)

func (*OpenAIEmbedding) HealthCheck 

func (o *OpenAIEmbedding) HealthCheck() (bool, string)

type PostgresStore added in v0.4.0 

type PostgresStore struct {
	// contains filtered or unexported fields
}

PostgresStore is the PostgreSQL+pgvector Store implementation. It mirrors SQLiteStore method-by-method; differences are confined to SQL dialect ($N placeholders, ON CONFLICT ... (col), JSONB, vector(dim)) and to the lock/index mechanisms (pg advisory lock, pgvector HNSW index instead of the process-local InMemoryVectorIndex + flock file).

func NewPostgresStore added in v0.4.0 

func NewPostgresStore(dsn string, dim int) (*PostgresStore, error)

NewPostgresStore connects to the PostgreSQL database at dsn (pgxpool) and applies the schema idempotently. dim sizes the pgvector embedding column.

func (*PostgresStore) Close added in v0.4.0 

func (s *PostgresStore) Close() error

Close releases the connection pool.

func (*PostgresStore) Count added in v0.4.0 

func (s *PostgresStore) Count(workspace string) (map[string]int, error)

Count returns a "layer/type" → count map (optionally scoped to workspace).

func (*PostgresStore) CountCodeSymbols added in v0.4.0 

func (s *PostgresStore) CountCodeSymbols() (int, error)

CountCodeSymbols returns the number of code symbol projections.

func (*PostgresStore) CountEdges added in v0.4.0 

func (s *PostgresStore) CountEdges() (int, error)

CountEdges returns the number of edges.

func (*PostgresStore) DeleteMemoriesByTypeSource added in v0.4.0 

func (s *PostgresStore) DeleteMemoriesByTypeSource(typ, source, workspace string) error

DeleteMemoriesByTypeSource deletes memories matching type+source+workspace (code indexer re-index write path).

func (*PostgresStore) DeleteMemory added in v0.4.0 

func (s *PostgresStore) DeleteMemory(id string) error

DeleteMemory deletes a memory (the HNSW index entry goes with the row).

func (*PostgresStore) DeleteSymbolMemories added in v0.4.0 

func (s *PostgresStore) DeleteSymbolMemories(qualifiedName, workspace string) error

DeleteSymbolMemories deletes the code_symbol memories projecting the given qualified name (code indexer re-index write path).

func (*PostgresStore) DeleteUser added in v0.4.0 

func (s *PostgresStore) DeleteUser(username string) error

DeleteUser deletes a user; a missing username is a no-op.

func (*PostgresStore) EpisodicContentsSince added in v0.4.0 

func (s *PostgresStore) EpisodicContentsSince(workspace string, since float64) ([]string, error)

EpisodicContentsSince returns the content strings of episodic events in workspace created at or after since (attention gate recent-duplicate scan).

func (*PostgresStore) FindByHash added in v0.4.0 

func (s *PostgresStore) FindByHash(contentHash, workspace string) (*schema.Memory, error)

FindByHash returns the memory with the given content hash (optionally scoped to workspace), or nil.

func (*PostgresStore) GetIndexedHash added in v0.4.0 

func (s *PostgresStore) GetIndexedHash(filePath string) (string, error)

GetIndexedHash returns the recorded body hash for a file, or "".

func (*PostgresStore) GetMemory added in v0.4.0 

func (s *PostgresStore) GetMemory(id string) (*schema.Memory, error)

GetMemory returns the memory with the given id, or nil.

func (*PostgresStore) GetMeta added in v0.4.0 

func (s *PostgresStore) GetMeta(key string) (string, error)

GetMeta returns the value for a meta key, or "".

func (*PostgresStore) GetUser added in v0.4.0 

func (s *PostgresStore) GetUser(username string) (*schema.User, error)

GetUser returns the user with the given username, or nil.

func (*PostgresStore) InvalidateEdge added in v0.4.0 

func (s *PostgresStore) InvalidateEdge(edgeID string, t float64) error

InvalidateEdge marks an edge no longer current (sets valid_to).

func (*PostgresStore) IterMemories added in v0.4.0 

func (s *PostgresStore) IterMemories(workspace, layer, typ string) ([]*schema.Memory, error)

IterMemories returns memories matching the optional filters.

func (*PostgresStore) ListUsers added in v0.4.0 

func (s *PostgresStore) ListUsers() ([]*schema.User, error)

ListUsers returns all users sorted by username.

func (*PostgresStore) NeighborCounts added in v0.4.0 

func (s *PostgresStore) NeighborCounts() (map[string]int, error)

NeighborCounts returns {memory_id: neighbour_count} for spreading activation.

func (*PostgresStore) Neighbors added in v0.4.0 

func (s *PostgresStore) Neighbors(memID, relation string) ([]*schema.Edge, error)

Neighbors returns valid (valid_to IS NULL) edges touching memID.

func (*PostgresStore) Ping added in v0.4.0 

func (s *PostgresStore) Ping() error

Ping checks storage connectivity (health probe).

func (*PostgresStore) PutCodeRefs added in v0.4.0 

func (s *PostgresStore) PutCodeRefs(refs []*schema.CodeRef) error

PutCodeRefs bulk-inserts cross references atomically: one transaction, so a failure rolls back the whole batch.

func (*PostgresStore) PutCodeSymbol added in v0.4.0 

func (s *PostgresStore) PutCodeSymbol(sym *schema.CodeSymbol) error

PutCodeSymbol inserts or updates a code symbol projection.

func (*PostgresStore) PutEdge added in v0.4.0 

func (s *PostgresStore) PutEdge(e *schema.Edge) error

PutEdge inserts or updates an edge.

func (*PostgresStore) PutMemory added in v0.4.0 

func (s *PostgresStore) PutMemory(mem *schema.Memory, vector []float32) error

func (*PostgresStore) PutUser added in v0.4.0 

func (s *PostgresStore) PutUser(u *schema.User) error

PutUser inserts or updates a user (upsert by username).

func (*PostgresStore) RebuildVectorIndex added in v0.4.0 

func (s *PostgresStore) RebuildVectorIndex(newDim int)

RebuildVectorIndex resets the vector index at a new dim: drops the HNSW index, nulls all embeddings, alters the column to vector(newDim) and recreates the index — one transaction. Mirrors SQLiteStore's reset of the in-memory index; the interface returns no error, so failures are warned.

func (*PostgresStore) RefsForSymbol added in v0.4.0 

func (s *PostgresStore) RefsForSymbol(qualifiedName, direction string) ([]*schema.CodeRef, error)

RefsForSymbol returns cross references for a qualified name.

func (*PostgresStore) SetIndexed added in v0.4.0 

func (s *PostgresStore) SetIndexed(filePath, bodyHash string, now float64) error

SetIndexed records the body hash for a file.

func (*PostgresStore) SetMeta added in v0.4.0 

func (s *PostgresStore) SetMeta(key, value string) error

SetMeta upserts a meta key/value.

func (*PostgresStore) SymbolsForFile added in v0.4.0 

func (s *PostgresStore) SymbolsForFile(filePath string) ([]*schema.CodeSymbol, error)

SymbolsForFile returns code symbols for a file, ordered by line.

func (*PostgresStore) TouchMemories added in v0.4.0 

func (s *PostgresStore) TouchMemories(ids []string, now float64) error

TouchMemories bumps access_count / last_access_at for every listed id in a single UPDATE (recall's access bookkeeping used to issue one UPDATE per id). An empty slice is a no-op.

func (*PostgresStore) TryAcquireIndexLock added in v0.4.0 

func (s *PostgresStore) TryAcquireIndexLock() (func(), error)

TryAcquireIndexLock takes the cross-process code-index lock as a pg advisory lock. Advisory locks are session-scoped, so a dedicated connection is acquired from the pool and held until the returned release function runs (pg_advisory_unlock + conn.Release). Contention fails fast with ErrIndexLockHeld — callers do not queue.

func (*PostgresStore) UpdateMemoryContent added in v0.4.0 

func (s *PostgresStore) UpdateMemoryContent(id, content, summary string, tags []string, vector []float32, now float64) error

UpdateMemoryContent patches content/summary/tags/updated_at in place. A nil vector leaves the embedding column and content_hash alone (unlike the PutMemory upsert, which would NULL them); a non-nil vector rewrites the embedding and recomputes content_hash. A missing id is a no-op.

func (*PostgresStore) VectorSearch added in v0.4.0 

func (s *PostgresStore) VectorSearch(queryVec []float32, topK int) []SearchHit

VectorSearch runs a cosine top-k search via pgvector. Ordering matches InMemoryVectorIndex.Search: similarity desc, id asc as deterministic tiebreak; the vector index is global (workspace filtering happens in the recall layer above). Query failures degrade to nil, mirroring the in-memory index's error-free Search signature (same tolerance as warmIndexFromBlobs).

Degenerate vectors are handled outside the ANN path: pgvector's cosine distance is NaN for zero-norm vectors, and the HNSW index omits them entirely, while the in-memory baseline assigns them similarity 0. A zero-norm query therefore takes a plain id-ordered scan, and zero-norm stored vectors are appended via UNION ALL with a literal 0 similarity.

func (*PostgresStore) Workspaces added in v0.4.0 

func (s *PostgresStore) Workspaces() ([]string, error)

Workspaces lists distinct workspaces.

type RealHTTPPoster 

type RealHTTPPoster struct {
	Timeout time.Duration
}

RealHTTPPoster is the net/http-backed client.

func NewRealHTTPPoster 

func NewRealHTTPPoster(timeoutS float64) *RealHTTPPoster

NewRealHTTPPoster returns a RealHTTPPoster with the given timeout.

func (*RealHTTPPoster) Post 

func (c *RealHTTPPoster) Post(url string, payload any, headers map[string]string) (any, error)

type SQLiteStore 

type SQLiteStore struct {
	DBPath string
	Dim    int
	// contains filtered or unexported fields
}

SQLiteStore is the single persistence layer for LadyM. It owns the SQLite connection and the (in-memory) vector index.

func NewStore 

func NewStore(dbPath string, dim int, preferSQLiteVec bool, enableWAL bool) (*SQLiteStore, error)

NewStore opens (or creates) a SQLite store at dbPath.

NOTE: the Python port used the sqlite-vec loadable extension for persistent ANN search. The Go port always uses the pure-Go InMemoryVectorIndex (the sqlite-vec C extension has no Go binding); embeddings are still persisted as BLOBs and warmed into the index on reopen, so behaviour is identical. The preferSQLiteVec flag is accepted for config parity but has no effect.

func (*SQLiteStore) Close 

func (s *SQLiteStore) Close() error

Close commits and closes the underlying connection.

func (*SQLiteStore) Count 

func (s *SQLiteStore) Count(workspace string) (map[string]int, error)

Count returns a "layer/type" → count map (optionally scoped to workspace).

func (*SQLiteStore) CountCodeSymbols added in v0.4.0 

func (s *SQLiteStore) CountCodeSymbols() (int, error)

CountCodeSymbols returns the number of code symbol projections.

func (*SQLiteStore) CountEdges 

func (s *SQLiteStore) CountEdges() (int, error)

CountEdges returns the number of edges.

func (*SQLiteStore) DeleteMemoriesByTypeSource added in v0.4.0 

func (s *SQLiteStore) DeleteMemoriesByTypeSource(typ, source, workspace string) error

DeleteMemoriesByTypeSource deletes memories matching type+source+workspace (code indexer re-index write path).

func (*SQLiteStore) DeleteMemory 

func (s *SQLiteStore) DeleteMemory(id string) error

DeleteMemory deletes a memory and its vector entry.

func (*SQLiteStore) DeleteSymbolMemories added in v0.4.0 

func (s *SQLiteStore) DeleteSymbolMemories(qualifiedName, workspace string) error

DeleteSymbolMemories deletes the code_symbol memories projecting the given qualified name (code indexer re-index write path).

func (*SQLiteStore) DeleteUser added in v0.4.0 

func (s *SQLiteStore) DeleteUser(username string) error

DeleteUser deletes a user; a missing username is a no-op.

func (*SQLiteStore) EpisodicContentsSince 

func (s *SQLiteStore) EpisodicContentsSince(workspace string, since float64) ([]string, error)

EpisodicContentsSince returns the content strings of episodic events in workspace created at or after since (used by the attention gate's recent-duplicate scan; pushes the time-window cut into SQL).

func (*SQLiteStore) FindByHash 

func (s *SQLiteStore) FindByHash(contentHash, workspace string) (*schema.Memory, error)

FindByHash returns the memory with the given content hash (optionally scoped to workspace), or nil.

func (*SQLiteStore) GetIndexedHash 

func (s *SQLiteStore) GetIndexedHash(filePath string) (string, error)

GetIndexedHash returns the recorded body hash for a file, or "".

func (*SQLiteStore) GetMemory 

func (s *SQLiteStore) GetMemory(id string) (*schema.Memory, error)

GetMemory returns the memory with the given id, or nil.

func (*SQLiteStore) GetMeta 

func (s *SQLiteStore) GetMeta(key string) (string, error)

GetMeta returns the value for a meta key, or "".

func (*SQLiteStore) GetUser added in v0.4.0 

func (s *SQLiteStore) GetUser(username string) (*schema.User, error)

GetUser returns the user with the given username, or nil.

func (*SQLiteStore) InvalidateEdge added in v0.4.0 

func (s *SQLiteStore) InvalidateEdge(edgeID string, t float64) error

InvalidateEdge marks an edge no longer current (sets valid_to).

func (*SQLiteStore) IterMemories 

func (s *SQLiteStore) IterMemories(workspace, layer, typ string) ([]*schema.Memory, error)

IterMemories returns memories matching the optional filters.

func (*SQLiteStore) ListUsers added in v0.4.0 

func (s *SQLiteStore) ListUsers() ([]*schema.User, error)

ListUsers returns all users sorted by username.

func (*SQLiteStore) NeighborCounts 

func (s *SQLiteStore) NeighborCounts() (map[string]int, error)

NeighborCounts returns {memory_id: neighbour_count} for spreading activation.

func (*SQLiteStore) Neighbors 

func (s *SQLiteStore) Neighbors(memID, relation string) ([]*schema.Edge, error)

Neighbors returns valid (valid_to IS NULL) edges touching memID.

func (*SQLiteStore) Ping added in v0.4.0 

func (s *SQLiteStore) Ping() error

Ping checks storage connectivity (health probe).

func (*SQLiteStore) PutCodeRefs 

func (s *SQLiteStore) PutCodeRefs(refs []*schema.CodeRef) error

PutCodeRefs bulk-inserts cross references atomically: one transaction, so a failure rolls back the whole batch (Python: executemany in a single commit).

func (*SQLiteStore) PutCodeSymbol 

func (s *SQLiteStore) PutCodeSymbol(sym *schema.CodeSymbol) error

PutCodeSymbol inserts or updates a code symbol projection.

func (*SQLiteStore) PutEdge 

func (s *SQLiteStore) PutEdge(e *schema.Edge) error

PutEdge inserts or updates an edge.

func (*SQLiteStore) PutMemory 

func (s *SQLiteStore) PutMemory(mem *schema.Memory, vector []float32) error

func (*SQLiteStore) PutUser added in v0.4.0 

func (s *SQLiteStore) PutUser(u *schema.User) error

PutUser inserts or updates a user (upsert by username).

func (*SQLiteStore) RebuildVectorIndex 

func (s *SQLiteStore) RebuildVectorIndex(newDim int)

RebuildVectorIndex resets the index at a new dim (used on dim change).

func (*SQLiteStore) RefsForSymbol 

func (s *SQLiteStore) RefsForSymbol(qualifiedName, direction string) ([]*schema.CodeRef, error)

RefsForSymbol returns cross references for a qualified name.

func (*SQLiteStore) SetIndexed 

func (s *SQLiteStore) SetIndexed(filePath, bodyHash string, now float64) error

SetIndexed records the body hash for a file.

func (*SQLiteStore) SetMeta 

func (s *SQLiteStore) SetMeta(key, value string) error

SetMeta upserts a meta key/value.

func (*SQLiteStore) SymbolsForFile 

func (s *SQLiteStore) SymbolsForFile(filePath string) ([]*schema.CodeSymbol, error)

SymbolsForFile returns code symbols for a file, ordered by line.

func (*SQLiteStore) TouchMemories added in v0.4.0 

func (s *SQLiteStore) TouchMemories(ids []string, now float64) error

TouchMemories bumps access_count / last_access_at for every listed id in a single UPDATE (recall's access bookkeeping used to issue one UPDATE per id). An empty slice is a no-op.

func (*SQLiteStore) TryAcquireIndexLock added in v0.4.0 

func (s *SQLiteStore) TryAcquireIndexLock() (func(), error)

TryAcquireIndexLock takes the cross-process code-index lock on <db>.index.lock and returns the release function. Contention fails fast with ErrIndexLockHeld — callers do not queue.

func (*SQLiteStore) UpdateMemoryContent added in v0.4.0 

func (s *SQLiteStore) UpdateMemoryContent(id, content, summary string, tags []string, vector []float32, now float64) error

UpdateMemoryContent patches content/summary/tags/updated_at in place. A nil vector leaves the embedding column and content_hash alone (unlike the PutMemory upsert, which would NULL them); a non-nil vector rewrites the embedding, recomputes content_hash and re-indexes the vector. A missing id is a no-op.

func (*SQLiteStore) UsingSQLiteVec 

func (s *SQLiteStore) UsingSQLiteVec() bool

UsingSQLiteVec always returns false (see NewStore).

func (*SQLiteStore) VectorSearch added in v0.4.0 

func (s *SQLiteStore) VectorSearch(queryVec []float32, topK int) []SearchHit

VectorSearch runs a cosine top-k search over the in-memory vector index.

func (*SQLiteStore) Workspaces 

func (s *SQLiteStore) Workspaces() ([]string, error)

Workspaces lists distinct workspaces.

type SearchHit 

type SearchHit struct {
	ID         string
	Similarity float64
}

SearchHit is one (id, similarity) pair returned by Search.

type Store added in v0.4.0 

type Store interface {
	// lifecycle
	Close() error

	// Ping checks storage connectivity (health probe): SQLite runs SELECT 1,
	// Postgres pings the pool.
	Ping() error

	// memory CRUD
	PutMemory(mem *schema.Memory, vector []float32) error
	GetMemory(id string) (*schema.Memory, error)
	DeleteMemory(id string) error
	// UpdateMemoryContent patches one memory's content/summary/tags and bumps
	// updated_at (now==0 falls back to the wall clock). Unlike PutMemory's
	// upsert — where a nil vector NULLs the embedding column — a nil vector
	// here leaves embedding and content_hash untouched; a non-nil vector
	// rewrites the embedding and recomputes content_hash from content. A
	// missing id is a no-op (callers 404 beforehand).
	UpdateMemoryContent(id, content, summary string, tags []string, vector []float32, now float64) error
	TouchMemories(ids []string, now float64) error // one batched UPDATE; empty slice is a no-op
	IterMemories(workspace, layer, typ string) ([]*schema.Memory, error)
	FindByHash(contentHash, workspace string) (*schema.Memory, error)
	EpisodicContentsSince(workspace string, since float64) ([]string, error)
	Count(workspace string) (map[string]int, error)

	// vector retrieval (cosine similarity, deterministic top-k; semantics
	// anchored to InMemoryVectorIndex.Search)
	VectorSearch(queryVec []float32, topK int) []SearchHit

	// L4 associative graph
	PutEdge(e *schema.Edge) error
	Neighbors(memID, relation string) ([]*schema.Edge, error)
	CountEdges() (int, error)
	NeighborCounts() (map[string]int, error)
	InvalidateEdge(edgeID string, t float64) error // UPDATE edges SET valid_to=t WHERE id=edgeID

	// code projections
	PutCodeSymbol(sym *schema.CodeSymbol) error
	PutCodeRefs(refs []*schema.CodeRef) error
	SymbolsForFile(filePath string) ([]*schema.CodeSymbol, error)
	RefsForSymbol(qualifiedName, direction string) ([]*schema.CodeRef, error)
	CountCodeSymbols() (int, error)

	// index state / metadata
	GetIndexedHash(filePath string) (string, error)
	SetIndexed(filePath, bodyHash string, now float64) error
	Workspaces() ([]string, error)
	GetMeta(key string) (string, error)
	SetMeta(key, value string) error

	// users (HTTP data-plane Basic auth accounts)
	PutUser(u *schema.User) error                  // upsert by username
	GetUser(username string) (*schema.User, error) // nil when absent
	DeleteUser(username string) error              // missing username is a no-op
	ListUsers() ([]*schema.User, error)            // sorted by username

	// Cross-process mutex for code indexing. The SQLite implementation is the
	// existing flock(<db>.index.lock); it returns the release function. When
	// the lock is already held it returns an error matching ErrIndexLockHeld
	// — the user-facing IndexInProgressError type lives in the code package
	// (code imports storage, so storage cannot reference it) and the caller
	// performs the translation.
	TryAcquireIndexLock() (func(), error)

	// RebuildVectorIndex resets the vector index at a new dim (engine's
	// enforceEmbeddingDim on dim change).
	RebuildVectorIndex(newDim int)

	// The two deletes the code/indexer.go write path needs.
	DeleteMemoriesByTypeSource(typ, source, workspace string) error // DELETE FROM memories WHERE type=? AND source=? AND workspace=?
	DeleteSymbolMemories(qualifiedName, workspace string) error     // DELETE FROM memories WHERE type='code_symbol' AND workspace=? AND id IN (SELECT memory_id FROM code_symbols WHERE qualified_name=?)
}

Store is the persistence contract used by engine / operations / layers / code. SQLiteStore is the reference implementation; the interface exists so no package outside storage depends on the concrete type or the raw *sql.DB.

func OpenStore added in v0.4.0 

func OpenStore(cfg *config.Config, dim int) (Store, error)

OpenStore builds the Store implementation selected by cfg.StoreBackend. "sqlite" (the default) is the personal-edition path and behaves exactly as before; "postgres" requires a DSN (store.dsn / LADYM_STORE_DSN, or store.dsn_env indirection resolved by the config loader).

type VectorIndex 

type VectorIndex interface {
	Upsert(itemID string, vector []float32) error
	Search(query []float32, topK int) []SearchHit
	Delete(itemID string)
	Len() int
}

VectorIndex inserts/queries/deletes vectors keyed by an arbitrary id.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.