Documentation
¶
Index ¶
- func DecodeEmbedding(s string) ([]float32, error)
- func EncodeEmbedding(vec []float32) string
- func WithTimingSink(ctx context.Context, fn func(SearchTiming)) context.Context
- type CellNeighborhood
- type CellSearchResult
- type EmbedMetrics
- type Embedder
- func (e *Embedder) EmbedBatch(ctx context.Context, texts []string) ([][]float32, error)
- func (e *Embedder) EmbedDocument(ctx context.Context, text string) ([]float32, error)
- func (e *Embedder) EmbedQuery(ctx context.Context, text string) ([]float32, error)
- func (e *Embedder) Metrics() EmbedMetrics
- func (e *Embedder) OutputDims() int
- type MemoryCellIndexEntry
- type MemoryCellStore
- func (s *MemoryCellStore) CollectionSize() int64
- func (s *MemoryCellStore) Delete(_ context.Context, id string) error
- func (s *MemoryCellStore) Index(ctx context.Context, doc MemoryCellIndexEntry) ([]float32, error)
- func (s *MemoryCellStore) Neighborhood(id string, topK, edgesPerNeighbor int, keep func(MemoryCellIndexEntry) bool) (*CellNeighborhood, bool)
- func (s *MemoryCellStore) QueryEmbedCacheStats() QueryEmbedCacheStats
- func (s *MemoryCellStore) Search(ctx context.Context, query string, topK int, kindFilter string) ([]CellSearchResult, error)
- func (s *MemoryCellStore) SearchExhaustive(ctx context.Context, query, subject string, kindFilter string) ([]CellSearchResult, error)
- func (s *MemoryCellStore) SearchSubject(ctx context.Context, query, subject string, topK int, kindFilter string) ([]CellSearchResult, error)
- func (s *MemoryCellStore) SeedFromEmbedding(doc MemoryCellIndexEntry, embedding []float32) bool
- func (s *MemoryCellStore) SetScoreGap(gap float32)
- type NeighborEdge
- type QueryEmbedCacheStats
- type SearchTiming
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DecodeEmbedding ¶
DecodeEmbedding deserializes a base64 string produced by EncodeEmbedding back to a float32 vector. Returns nil for an empty string.
func EncodeEmbedding ¶
EncodeEmbedding serializes a float32 vector to a base64 string (little-endian). Returns an empty string for a nil or empty slice.
func WithTimingSink ¶
func WithTimingSink(ctx context.Context, fn func(SearchTiming)) context.Context
WithTimingSink asks Search, SearchSubject and SearchExhaustive to report their phase split to fn.
Types ¶
type CellNeighborhood ¶
type CellNeighborhood struct {
Center MemoryCellIndexEntry
Neighbors []CellSearchResult
Edges []NeighborEdge
}
CellNeighborhood is the semantic neighbourhood of one cell: its nearest neighbours and the links among them that mark sub-clusters.
type CellSearchResult ¶
type CellSearchResult struct {
Entry MemoryCellIndexEntry
Score float32
}
CellSearchResult is a memory cell returned by MemoryCellStore.Search.
type EmbedMetrics ¶
type EmbedMetrics struct {
CacheHits int64 `json:"cacheHits"`
EmbedCalls int64 `json:"embedCalls"`
}
EmbedMetrics holds cache and API call counters.
type Embedder ¶
type Embedder struct {
// contains filtered or unexported fields
}
Embedder generates vector embeddings. It has two modes:
Vertex AI (genai != nil): uses google.golang.org/genai SDK with BackendVertexAI, authenticated via Workload Identity (ADC). Created when endpoint == "".
Gateway (endpoint != ""): routes through an OpenAI-compatible /v1/embeddings endpoint.
func NewEmbedder ¶
func NewEmbedder(ctx context.Context, endpoint, model, project, location string, outputDims, maxInputChars int, logger zerolog.Logger) (*Embedder, error)
NewEmbedder creates an Embedder. When endpoint == "", uses Vertex AI via Workload Identity (ADC); project and location are required. When endpoint != "", routes through the OpenAI-compatible gateway; project and location are unused. outputDims reduces the output vector dimension (0 = model default). Applied to Vertex AI natively, and to the gateway via the OpenAI-compatible "dimensions" param (e.g. Matryoshka models like nomic-embed-text truncate cleanly to 128/256/512). maxInputChars hard-caps input length before the embed call (0 = defaultMaxInputChars); set it below the model's token window × ~4 to avoid the gateway rejecting oversized input.
func (*Embedder) EmbedBatch ¶
EmbedBatch returns embeddings for multiple texts, preserving input order. It issues up to embedBatchConcurrency requests concurrently (see the const) and fails fast on the first error, cancelling the rest.
func (*Embedder) EmbedDocument ¶
EmbedDocument returns the embedding vector for a document to be indexed.
func (*Embedder) EmbedQuery ¶
EmbedQuery returns the embedding vector for a search query.
func (*Embedder) Metrics ¶
func (e *Embedder) Metrics() EmbedMetrics
Metrics returns a snapshot of cache hit and API call counters.
func (*Embedder) OutputDims ¶
OutputDims returns the configured Matryoshka output dimension. 0 means the model's own size, with no truncation.
type MemoryCellIndexEntry ¶
type MemoryCellIndexEntry struct {
ID string
Kind string
Scope string
Subject string
Title string
Content string
Confidence string
Status string
Tags []string
EventDate string // RFC3339, only set for kind=event
}
MemoryCellIndexEntry is the per-cell record indexed in MemoryCellStore: the searchable metadata projection of a memory cell (Title+Content is what gets embedded; the other fields drive filtering). Not a document/file.
type MemoryCellStore ¶
type MemoryCellStore struct {
// contains filtered or unexported fields
}
MemoryCellStore holds a per-subject HNSW index. Each subject's cells are isolated so cross-subject interference is impossible.
func NewMemoryCellStore ¶
func NewMemoryCellStore(embedder *Embedder, logger zerolog.Logger) *MemoryCellStore
NewMemoryCellStore creates an empty MemoryCellStore.
func (*MemoryCellStore) CollectionSize ¶
func (s *MemoryCellStore) CollectionSize() int64
CollectionSize returns total indexed cells across all subjects.
func (*MemoryCellStore) Delete ¶
func (s *MemoryCellStore) Delete(_ context.Context, id string) error
Delete removes a memory cell by ID across all subject indexes. The graph is rebuilt lazily before the next search (see subjectIndex.dirty), so a sweep that deletes hundreds of cells costs one rebuild.
func (*MemoryCellStore) Index ¶
func (s *MemoryCellStore) Index(ctx context.Context, doc MemoryCellIndexEntry) ([]float32, error)
Index embeds and stores a memory cell document.
func (*MemoryCellStore) Neighborhood ¶
func (s *MemoryCellStore) Neighborhood(id string, topK, edgesPerNeighbor int, keep func(MemoryCellIndexEntry) bool) (*CellNeighborhood, bool)
Neighborhood returns the semantic neighbourhood of the cell with the given id, searched in the index of the subject that holds it: subjects never share a graph. The second result is false when the id has no embedding in any index (never seeded, private, or since deleted), so a caller can tell "not indexed" from "no neighbours". keep is an optional per-cell filter; nil keeps all.
Neighbour-to-neighbour edges follow a scale-free rule, not an absolute similarity threshold: an edge survives only when the two cells are more similar to each other than either is to the centre, and then only if it is among each endpoint's edgesPerNeighbor strongest such links (symmetrised, the union, so a hub may end up with a higher degree). Embeddings of same-kind cells crowd into a narrow high band, so any fixed threshold either admits the complete graph or is retuned per corpus and per embedder. "Tighter to each other than to the centre" is what sub-cluster means and holds at any scale. A neighbourhood with no sub-structure returns no edges: a star.
func (*MemoryCellStore) QueryEmbedCacheStats ¶
func (s *MemoryCellStore) QueryEmbedCacheStats() QueryEmbedCacheStats
QueryEmbedCacheStats reads the query-embedding cache counters; zero when the store has no cache.
func (*MemoryCellStore) Search ¶
func (s *MemoryCellStore) Search(ctx context.Context, query string, topK int, kindFilter string) ([]CellSearchResult, error)
Search returns top-k results for the given subject's index only. The subject field in the doc is used to route to the correct index.
func (*MemoryCellStore) SearchExhaustive ¶
func (s *MemoryCellStore) SearchExhaustive(ctx context.Context, query, subject string, kindFilter string) ([]CellSearchResult, error)
SearchExhaustive ranks every entry the caller may see by exact cosine, with no graph and no fetch window: the subject's own cells and the global ones, or every subject's when subject is empty. It exists for one caller. A graph search returns its top-K window, and a caller that drops hits afterwards (a cell deleted or retired on another replica is still indexed here) can see the whole window rejected although other cells would have qualified. This hands back the full ranked population so the caller's filter gets every candidate. No score gap is applied.
func (*MemoryCellStore) SearchSubject ¶
func (s *MemoryCellStore) SearchSubject(ctx context.Context, query, subject string, topK int, kindFilter string) ([]CellSearchResult, error)
SearchSubject searches within the given subject's index and merges results from the global (empty subject) index, returning top-k by score.
func (*MemoryCellStore) SeedFromEmbedding ¶
func (s *MemoryCellStore) SeedFromEmbedding(doc MemoryCellIndexEntry, embedding []float32) bool
SeedFromEmbedding inserts a pre-embedded cell without calling the embedding API. It returns false, inserting nothing, when the vector's length disagrees with the store dimension (a cell embedded by another model): the caller should re-embed it. An empty embedding has nothing to seed and returns true.
The dimension is the embedder's configured output dimension when known, so a stale wrong-dimension cell seeded first cannot lock the store to a wrong shape; a seeded vector only sets it when it is not known.
func (*MemoryCellStore) SetScoreGap ¶
func (s *MemoryCellStore) SetScoreGap(gap float32)
SetScoreGap overrides the semantic relevance cutoff (absolute cosine gap below the top hit). 0 disables the cutoff (return raw top-K). Wired from the --semantic-score-gap flag.
type NeighborEdge ¶
NeighborEdge is one neighbour-to-neighbour similarity link. Score uses the same [0,1] mapping as CellSearchResult.Score, so an edge and a neighbour score compare directly.
type QueryEmbedCacheStats ¶
QueryEmbedCacheStats is a point-in-time counter readout for observability.
Source Files