memory

package
v1.18.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 13, 2026 License: MIT Imports: 31 Imported by: 0

Documentation

Overview

Package memory – embeddings.go implements embedding generation for semantic search. Supports multiple providers: OpenAI, Gemini, Voyage, Mistral, and a zero-cost null fallback. Embeddings are cached by content hash + provider + model to avoid redundant API calls.

Package memory – embeddings_gemini.go implements Google Gemini embedding provider. Uses the Gemini REST API with support for single and batch embedding requests.

Package memory – embeddings_mistral.go implements Mistral embedding provider. Uses the OpenAI-compatible /embeddings endpoint.

Package memory — embeddings_onnx.go implements a local ONNX-based embedding provider using sentence-transformer models (all-MiniLM-L6-v2). Zero API calls — runs entirely on CPU via ONNX Runtime.

Auto-downloads the ONNX runtime shared library and model files on first use to ~/.devclaw/lib/ and ~/.devclaw/models/. Gracefully degrades to NullEmbedder if the runtime is unavailable.

Package memory – embeddings_voyage.go implements Voyage AI embedding provider. Uses the OpenAI-compatible /embeddings endpoint with Voyage-specific input_type field.

Package memory — entity_detector.go implements a lightweight, regex-based entity detector that resolves candidate tokens from turn text against the wings/rooms tables stored in the SQLite store.

The detector is a shared primitive: Room 2.3 uses it inside OnDemandLayer; Sprint 3 will reuse it for Knowledge Graph entity extraction (Kind="kg_entity"). No LLM calls. Zero external dependencies beyond the existing go-sqlite3 driver.

Thread-safety: EntityDetector is safe for concurrent use. The internal snapshot is protected by a RWMutex; cache refresh is serialized with a compare-and-swap on the refreshing field to prevent stampedes.

Package memory — hierarchy.go provides palace-aware wing/hall/room helpers for organizing memories into contextual namespaces.

Sprint 1 (v1.18.0): introduces the Wing/Hall/Room taxonomy per ADR-001. All functions here are PURE GO with no database dependency — they operate on file paths and string normalization only.

Conventions for wing derivation from file path: 

MEMORY.md                               → wing=NULL (legacy, first-class)
{wing}/MEMORY.md                        → wing
{wing}/{room}/MEMORY.md                 → wing + room
{wing}/{hall}/{room}/MEMORY.md          → wing + hall + room
{wing}/YYYY-MM-DD.md                    → wing (daily log)
{wing}/{room}/YYYY-MM-DD.md             → wing + room

Legacy behavior: files at the memory base directory root without a wing prefix have wing=NULL. All queries treat wing=NULL as a first-class citizen per Princípio Zero rule 5 (retrocompat absoluta).

Normalization: wing/room/hall names are stored in canonical form (lowercase, trimmed, accents removed, spaces → kebab-case, non-alnum stripped). Display names can be set separately in the `wings`/`rooms` tables.

References:

  • ADR-001 Wing Naming Scheme
  • ADR-002 Room Auto vs Manual
  • Sprint 0.5 agents-map-v2 §File Inventory

Package memory – indexer.go implements markdown chunking and hash-based delta sync for the memory index. Files are split into chunks, each chunk gets a SHA-256 hash, and only changed chunks are re-embedded.

Package memory — layer_essential.go implements the Sprint 2 L1 EssentialLayer. It renders per-wing "essential stories": deterministic Markdown-ish summaries of the most recently touched rooms and the lead sentences of their top files in a wing. Output is cached in the essential_stories SQLite table with a TTL-based freshness check so that the generation cost is amortised across calls and survives restarts.

Zero LLM calls. Template-only. This implements ADR-005 Option A ("template first"). A future sprint may swap to LLM-based generation behind a flag; until then, Render() is fully deterministic given a stable database snapshot.

Until Room 2.4 wires the layer into the prompt stack, EssentialLayer is dead code at runtime — Render() is safe to call from tests but no production caller exists yet. The cache still gets populated by tests and (eventually) the dream cycle (Sprint 3+), so the schema lives in the DB from day one.

Package memory — layer_identity.go implements the L0 IdentityLayer.

The IdentityLayer loads a user-curated identity fragment from disk and caches it in memory. It hot-reloads on file change via fsnotify, with a 30-second polling fallback when fsnotify is unavailable.

Until Room 2.4 wires this layer into the prompt stack, the layer is dead code at runtime — Render() is safe to call from tests but no production caller exists yet.

Package memory — layer_ondemand.go implements the Sprint 2 L2 OnDemandLayer. It inspects the current user turn, detects entities (via EntityDetector) that match stored wings/rooms, retrieves the top-N memories for those entities from the active wing, and returns a prompt-ready Markdown snippet truncated to a byte budget.

The layer runs on the hot path (every turn). Latency contract: p95 < 10ms on a warm cache. Context timeouts enforce this contract:

  • Detector call: DetectorTimeoutMs (default 3ms)
  • Search calls: SearchTimeoutMs (default 8ms)
  • Overall: DetectorTimeoutMs + SearchTimeoutMs

Until Room 2.4 wires this layer into the prompt stack, OnDemandLayer is dead code at runtime — Render() is safe to call from tests but no production caller exists yet.

Cross-wing fallback: if the active wing yields no results but CrossWingEnabled is true, ONE additional search is run without a wing filter. This surfaces the single best globally-relevant result when the user mentions an entity that lives only in another wing (e.g., a topic from a secondary context).

Package memory — legacy_classifier.go implements pattern-based classification of legacy (wing=NULL) memory files.

Sprint 1 amendment (2026-04-08): per user directive "incremental improvement felt", the dream system will opportunistically classify legacy files into wings using keyword-based pattern matching. This reverses the strict "never backfill" stance of ADR-006 — but only for the safe, free, deterministic path. LLM-based classification remains out of scope (ADR-009 security + cost concerns).

Design principles:

  1. Pattern-only. No LLM calls. Zero token cost. Deterministic.
  2. Conservative. Only classify when the signal is strong; leave ambiguous files as wing=NULL (still first-class).
  3. Idempotent. Running the classifier multiple times on the same file never changes its result.
  4. Auditable. Every classification records the matched keywords in the source field ('auto-legacy') for user review.
  5. One-way. Once a file has wing != NULL, the classifier NEVER touches it again. User's explicit classification is sacred.
  6. Reversible. User can /wing unset or wipe wing=<name> for a file to put it back in the legacy pool.

This file contains the pure classifier (no DB access). The batch runner that applies it to the database lives in legacy_classifier_pass.go.

Package memory — legacy_classifier_pass.go runs the pattern-based legacy classifier over the database, updating files.wing for any legacy file that the classifier can confidently categorize.

This is the DB-touching counterpart to legacy_classifier.go. The classifier itself is pure Go (strings + maps). This file handles:

  • Iterating over files where wing IS NULL (oldest first)
  • Reading their content from the chunks table
  • Calling the classifier
  • Writing files.wing = result.Wing when confidence is high enough
  • Logging every classification for audit
  • Bounding work via a batch size so a single pass does not hog the DB

This function is DESIGNED TO BE CALLED FROM THE DREAM SYSTEM. Sprint 1 exposes it as a public method; Sprint 2 will wire it into dream.go's cycle runner so it fires automatically during idle periods. The manual invocation path (e.g., CLI `devclaw dream run --classify-legacy`) is also valuable for users who want to trigger it explicitly.

Safety rails (Princípio Zero):

  • We NEVER update a file that already has wing != NULL. The user's explicit wing is sacred.
  • We NEVER call LLMs. This is pure pattern matching.
  • A failed classification leaves wing = NULL (still first-class).
  • The batch size is bounded by the caller — a runaway classifier cannot eat the entire DB in one pass.
  • Every mutation is logged with source='auto-legacy' in logs (NOT in files.wing — that column only stores the wing name).
  • No user-facing notification by default. The feature is silent.

Package memory — migration_essential_stories.go installs the Sprint 2 Room 2.2 schema for the L1 EssentialLayer: a cache table keyed by wing holding pre-rendered per-wing essential stories plus a lookup index on generation time.

The migration follows Sprint 1 conventions:

  • ADDITIVE: introduces only a new table and a new index; nothing else in the existing schema is touched.
  • IDEMPOTENT: uses CREATE TABLE IF NOT EXISTS / CREATE INDEX IF NOT EXISTS, safe to run on every startup.
  • REVERSIBLE: a commented-out Down() helper documents the rollback path for Sprint 6 cleanup (kept unused on purpose to avoid any accidental invocation).
  • RETROCOMPAT: does not touch `files`, `chunks`, or any pre-existing palace table; a legacy v1.17.0 upgrade path is unaffected.

The caller (sqlite_store.go:initSchema) treats errors as non-fatal and logs a warning, matching InitHierarchySchema.

Package memory – quantized_embedding.go implements uint8-quantized embeddings for memory-efficient vector search. Inspired by TurboQuant's asymmetric estimation: data is stored in uint8 (1 byte/dim), queries stay float32 for maximum precision.

Compression: float32 (4 bytes/dim) → uint8 (1 byte/dim) = 4x reduction. Quality: correlation ≥ 0.98 with float32 cosine similarity on normalized embeddings.

Package memory – query_expansion.go handles keyword extraction and FTS5 query building. Supports stop words in English, Portuguese, Spanish, and French. Filters out pure numbers, all-punctuation tokens, and very short tokens.

Package memory — sqlite_hierarchy.go adds palace-aware schema to the SQLite memory store without disturbing the existing core schema.

Sprint 1 (v1.18.0): additive schema changes per ADR-003 and Sprint 0.5 doc-02-errata §1.3. All changes are:

  • ADDITIVE: new columns on existing `files` table (nullable, default NULL)
  • IDEMPOTENT: safe to run multiple times
  • RETROCOMPAT: legacy files without wing metadata continue to work
  • REVERSIBLE: DROP TABLE + ALTER TABLE DROP COLUMN (SQLite 3.35+)

NO BACKFILL of existing rows is performed. This preserves Princípio Zero rule 4 ("Backfill nunca automático") and matches ADR-006.

Schema additions: 

files (existing table) gets:
   wing             TEXT   DEFAULT NULL   -- ADR-001
   hall             TEXT   DEFAULT NULL   -- ADR-001
   room             TEXT   DEFAULT NULL   -- ADR-002
   session_id       TEXT   DEFAULT NULL   -- ADR-007-v2
   access_count     INTEGER DEFAULT 0      -- ADR-005
   last_accessed_at DATETIME                -- ADR-005
   deleted_at       DATETIME                -- ADR-003 orphan handling

NEW TABLES:
   wings             -- wing registry with display name / metadata
   rooms             -- room registry per wing with source/confidence
   rooms_archive     -- overflow archive for ADR-002 hard cap
   channel_wing_map  -- (channel, external_id) → wing routing table

This file exposes InitHierarchySchema() which is called from the main initSchema() method in sqlite_store.go via a single added line.

Package memory — sqlite_palace_ops.go provides CRUD operations for the palace-aware tables introduced in Sprint 1: wings, rooms, channel_wing_map.

All functions here are SAFE TO CALL even when the palace-aware feature flag is off — they operate on the registry tables which exist unconditionally (though may be empty). Callers should gate invocation at a higher level.

Sprint 2 Room 2.0b adds AssignWingToFile which touches the `files` table. All other functions deal exclusively with the registry tables.

Package memory – sqlite_store.go implements a SQLite-backed memory store with FTS5 (BM25 ranking) and in-process vector search (cosine similarity). Embeddings are stored as JSON-encoded float32 arrays in the chunks table. This avoids the need for the sqlite-vec extension while still providing hybrid semantic + keyword search.

Package memory — sqlite_store_enriched.go implements the Sprint 3 Room 3.5 SearchEnriched method. It composes the existing HybridSearchWithOpts result with Knowledge Graph facts for entities detected in the query.

The enriched path is completely separate from the legacy search path: HybridSearchWithOpts is NOT modified. Callers opt in by calling HybridSearchEnriched instead of HybridSearchWithOpts.

Package memory implements persistent memory for DevClaw. Provides long-term fact storage and daily conversation logs using the filesystem (MEMORY.md + daily markdown files).

Architecture:

  • MEMORY.md: Long-term facts (append-only, curated by the agent)
  • memory/YYYY-MM-DD.md: Daily conversation summaries (append-only)
  • Search uses simple substring matching (future: BM25 / embeddings)

Package memory — tokenizer_wordpiece.go implements a pure-Go WordPiece tokenizer for BERT/MiniLM models. No external dependencies.

The tokenizer loads a vocab.txt file (one token per line, line number = ID), lowercases input, splits on whitespace and punctuation, and greedily matches the longest vocab prefix for each word. Unknown sub-tokens get [UNK].

Package memory – topic_change_detector.go detects when the user changes conversation topic and provides relevant context for the new topic.

Uses a two-stage cascade with ZERO extra API calls:

  • Stage 1: Entity overlap (free, ~0ms) — compares entity sets between turns
  • Stage 2: Cosine similarity (free, reuses embedding from L2 search)

Both inputs are already computed by the OnDemandLayer pipeline.

Index

Constants

View Source 
const ClassifierDominanceFactor = 2.0

ClassifierDominanceFactor requires the winning wing to have at least this many times the hit count of the second wing. A 2× margin means "work" with 6 hits beats "personal" with 3 hits, but doesn't beat "personal" with 4 hits (4 × 2 = 8 > 6).

View Source 
const ClassifierMinConfidence = 0.65

ClassifierMinConfidence is the threshold below which the classifier refuses to label a file. Files scoring below this stay as wing=NULL.

View Source 
const ClassifierMinHits = 3

ClassifierMinHits is the absolute minimum number of keyword matches for the winning wing. A file with only 1-2 weak signals is always left alone, regardless of margin over the second place.

View Source 
const DefaultLegacyClassificationBatchSize = 20

DefaultLegacyClassificationBatchSize is the number of files processed per pass when no override is given. Chosen to keep a single pass under ~100ms even on slow disks with 500-chunk files.

View Source 
const MaxHallNameLength = 32

MaxHallNameLength caps the length of a normalized hall identifier.

View Source 
const MaxRoomNameLength = 48

MaxRoomNameLength caps the length of a normalized room identifier.

View Source 
const MaxWingNameLength = 32

MaxWingNameLength caps the length of a normalized wing identifier. 32 characters is generous (8-word kebab-case) while preventing abuse.

View Source 
const MemoryFileName = "MEMORY.md"

MemoryFileName is the bare filename of the curated long-term facts file that FileStore.Save writes to and IndexDirectory keys indexed chunks under. Cross-package callers (e.g. copilot.handleMemorySave) MUST reference this constant instead of hardcoding the literal so that a future rename cannot silently break wing assignment, indexing, or decay filtering.

View Source 
const MinAllowedClassifierConfidence = 0.5

MinAllowedClassifierConfidence is the absolute lowest confidence the pass will accept, regardless of caller override. Prevents accidental label pollution from misconfigured callers.

View Source 
const ReservedWingPrefix = "__"

ReservedWingPrefix marks internal/system wings that users cannot create. Any wing name starting with this prefix is rejected by NormalizeWing.

Variables

View Source 
var ErrChannelWingNotFound = errors.New("channel wing mapping not found")

ErrChannelWingNotFound is returned by GetChannelWing when no mapping exists for the given (channel, external_id) pair.

Functions

func BuildPathForCoords added in v1.18.0 

func BuildPathForCoords(c PalaceCoords) string

BuildPathForCoords constructs the relative directory path that corresponds to a PalaceCoords value. The returned path does NOT include the filename.

Examples: 

{}                                → ""
{Wing:"work"}                     → "work"
{Wing:"work", Room:"auth"}        → "work/auth"
{Wing:"work", Hall:"m", Room:"a"} → "work/m/a"

func DefaultTTLForCategory added in v1.18.0 

func DefaultTTLForCategory(category string) time.Duration

DefaultTTLForCategory returns the default TTL for a category. Returns 0 (never expires) for unknown categories.

func FileHash 

func FileHash(path string) (string, error)

FileHash computes the SHA-256 hash of a file's content.

func IndexDirectory 

func IndexDirectory(dir string, cfg ChunkConfig) (map[string][]Chunk, error)

IndexDirectory scans a directory for .md files and chunks them. Returns a map of fileID → []Chunk. The fileID is the relative path.

func InitHierarchySchema added in v1.18.0 

func InitHierarchySchema(db *sql.DB, logger *slog.Logger) error

InitHierarchySchema applies the Sprint 1 palace-aware schema additions to an already-initialized devclaw memory database.

This function is idempotent: running it multiple times against the same database is safe. It checks for existing columns before ALTER TABLE and uses CREATE TABLE IF NOT EXISTS for new tables.

The caller must have already initialized the core schema (files, chunks, embedding_cache). This function does NOT create those.

Retrocompat: this function does not touch existing rows. All new columns default to NULL (or 0 for counters), preserving v1.17.0 behavior when the palace feature flag is off.

func IsLegacyPath added in v1.18.0 

func IsLegacyPath(relPath string) bool

IsLegacyPath reports whether a relative path corresponds to a legacy memory file (no palace coordinates). This is useful for filters that want to distinguish "unclassified" memories from "classified" ones.

func MigrateEssentialStories added in v1.18.0 

func MigrateEssentialStories(db *sql.DB, logger *slog.Logger) error

MigrateEssentialStories installs the essential_stories cache table plus its generated_at lookup index if they do not already exist. Idempotent.

First-run detection: the function checks whether the table already exists via sqlite_master before running the DDL. On the very first run it emits a single INFO log line; subsequent runs stay silent. Errors while reading sqlite_master are not fatal — the function falls through to the CREATE IF NOT EXISTS which is always safe.

func MigrateKgSchema added in v1.18.0 

func MigrateKgSchema(db *sql.DB, logger *slog.Logger) error

MigrateKgSchema installs the Knowledge Graph bitemporal tables (kg_entities, kg_entity_aliases, kg_predicates, kg_triples) plus their indexes if they do not already exist. Idempotent — safe to run on every startup.

First-run detection checks sqlite_master for kg_entities before executing the DDL. On the very first run it emits a single INFO log; subsequent runs stay silent. Errors reading sqlite_master are not fatal — the function falls through to the CREATE IF NOT EXISTS which is always safe.

Follows the same non-fatal policy as InitHierarchySchema and MigrateEssentialStories: the caller logs a warning but never blocks startup.

func NormalizeHall added in v1.18.0 

func NormalizeHall(input string) string

NormalizeHall produces the canonical hall identifier.

func NormalizeRoom added in v1.18.0 

func NormalizeRoom(input string) string

NormalizeRoom produces the canonical room identifier. Same algorithm as NormalizeWing but with a larger length budget.

func NormalizeWing added in v1.18.0 

func NormalizeWing(input string) string

NormalizeWing produces the canonical wing identifier from user input. Returns empty string if the input normalizes to nothing or starts with the reserved prefix "__".

Algorithm:

  1. Trim whitespace
  2. Lowercase (Unicode-aware)
  3. Strip accents (NFD decomposition keeping base characters)
  4. Replace spaces and underscores with hyphens
  5. Drop anything that is not [a-z0-9-]
  6. Collapse consecutive hyphens
  7. Trim leading/trailing hyphens
  8. Reject if result is empty or starts with the reserved prefix
  9. Truncate to MaxWingNameLength at a word boundary

Examples: 

NormalizeWing("Work")       → "work"
NormalizeWing("Trabalho")   → "trabalho"
NormalizeWing("Família")    → "familia"
NormalizeWing("side hustle") → "side-hustle"
NormalizeWing("  ")         → ""
NormalizeWing("__system")   → ""
NormalizeWing("🎉🎉")       → ""

func SeedSuggestedWings added in v1.18.0 

func SeedSuggestedWings(db *sql.DB) error

SeedSuggestedWings populates the `wings` table with the default suggestion list (work, personal, family, finance, learning, health) using INSERT OR IGNORE so it is safe to call multiple times and never overwrites user customizations.

This is called at startup only when the `wings` table is empty — so new users get a starter palette while existing databases are untouched.

Nothing about this is enforcing: users are free to delete these or create completely different wings. They exist purely for discoverability.

func SetL1CacheHitFn added in v1.18.0 

func SetL1CacheHitFn(fn func())

SetL1CacheHitFn sets the callback invoked on each L1 essential-story cache hit. Called once at startup by the copilot package.

func SetL1CacheMissFn added in v1.18.0 

func SetL1CacheMissFn(fn func())

SetL1CacheMissFn sets the callback invoked on each L1 essential-story cache miss (i.e., the generation path is taken). Called once at startup by the copilot package.

func StripAccents added in v1.18.0 

func StripAccents(s string) string

StripAccents removes combining diacritical marks from a string while preserving the base characters. "família" → "familia", "coração" → "coracao".

This is a zero-dependency implementation that handles the common Latin diacritics sufficient for Portuguese, Spanish, French, and English. Exported so the copilot package can reuse the exact same normalization rules for router heuristics and avoid divergence bugs.

func SuggestedWings added in v1.18.0 

func SuggestedWings() []string

SuggestedWings returns a list of wing names suggested to new users on first init. These are NOT reserved and can be deleted or ignored. Users are free to create entirely different wings.

func VectorNorm added in v1.18.0 

func VectorNorm(v []float32) float64

VectorNorm computes the L2 norm of a float32 vector. Useful for precomputing queryNorm before calling CosineSimilarity across many candidates.

Types

type ChannelWingMapping added in v1.18.0 

type ChannelWingMapping struct {
	Channel    string
	ExternalID string
	Wing       string
	Confidence float64
	Source     string // 'manual' | 'heuristic' | 'llm' | 'inherited'
	CreatedAt  time.Time
	UpdatedAt  time.Time
}

ChannelWingMapping describes how the context router should treat messages arriving from a specific channel + external ID.

type Chunk 

type Chunk struct {
	// FileID identifies the source file (relative path).
	FileID string

	// Index is the chunk position within the file (0-based).
	Index int

	// Text is the chunk content.
	Text string

	// Hash is the SHA-256 hex digest of the chunk text.
	Hash string
}

Chunk represents an indexed text fragment from a memory file.

func ChunkMarkdown 

func ChunkMarkdown(text string, cfg ChunkConfig) []Chunk

ChunkMarkdown splits markdown text into semantic chunks. Prefers splitting at heading boundaries, then paragraph boundaries, then sentence boundaries, respecting MaxTokens.

type ChunkConfig 

type ChunkConfig struct {
	// MaxTokens is the approximate max tokens per chunk (default: 500).
	// Uses ~4 chars/token heuristic.
	MaxTokens int

	// Overlap is the number of characters to overlap between chunks (default: 100).
	Overlap int
}

ChunkConfig controls the chunking behavior.

func DefaultChunkConfig 

func DefaultChunkConfig() ChunkConfig

DefaultChunkConfig returns sensible defaults.

type ClassifierResult added in v1.18.0 

type ClassifierResult struct {
	// Wing is the canonical wing identifier (normalized form).
	// Empty string means "could not classify — leave as NULL".
	Wing string

	// Confidence is a score in [0, 1] reflecting how strongly the content
	// matched the wing's keyword set. Classifier calling code should only
	// apply classifications where Confidence >= ClassifierMinConfidence.
	Confidence float64

	// MatchedKeywords lists the keywords that contributed to the decision.
	// Used for audit logging and user-facing explanations.
	MatchedKeywords []string

	// TopWingHits is the number of distinct keyword hits for the chosen wing.
	// Exposed for telemetry and test assertions.
	TopWingHits int

	// SecondWingHits is the number of hits for the next-best wing. Used in
	// the margin check (top must dominate second).
	SecondWingHits int
}

ClassifierResult describes the outcome of classifying a single file's content. When Confidence is 0, the classifier could not decide and the file should remain as wing=NULL.

func ClassifyLegacyContent added in v1.18.0 

func ClassifyLegacyContent(content string, keywords map[string][]string) ClassifierResult

ClassifyLegacyContent applies the pattern-based classifier to a blob of text and returns its best guess at a wing assignment.

The function is stateless and pure — same input always yields same output. It never touches the database or the filesystem.

keywords is the user-provided wing → keyword slice map (typically sourced from HierarchyConfig.LegacyKeywords). If nil or empty, the classifier cannot decide and returns ("", 0.0, nil) — not an error, just "no config". This is intentional: the binary ships zero default keywords to remain locale and domain neutral for open-source deployments.

If no wing has enough signal, the result has Wing="" and Confidence=0. Callers should treat that as "leave as legacy".

type EmbeddingConfig 

type EmbeddingConfig struct {
	// Provider is the embedding provider ("openai", "gemini", "voyage", "mistral", "auto", "none").
	Provider string `yaml:"provider"`

	// Model is the embedding model name (e.g. "text-embedding-3-small").
	Model string `yaml:"model"`

	// Dimensions is the output vector dimensionality (default: auto from model).
	Dimensions int `yaml:"dimensions"`

	// APIKey is the API key for the embedding provider. If empty, falls back to
	// the main LLM API key or provider-specific env vars.
	APIKey string `yaml:"api_key"`

	// BaseURL is the API base URL. If empty, uses the provider default.
	BaseURL string `yaml:"base_url"`

	// Cache enables embedding caching in SQLite (default: true).
	Cache bool `yaml:"cache"`

	// Fallback is the fallback provider when the primary fails ("openai", "gemini", etc., or "none").
	Fallback string `yaml:"fallback"`

	// FallbackAPIKey is the API key for the fallback provider.
	FallbackAPIKey string `yaml:"fallback_api_key"`

	// FallbackBaseURL is the base URL for the fallback provider.
	FallbackBaseURL string `yaml:"fallback_base_url"`

	// FallbackModel is the model for the fallback provider.
	FallbackModel string `yaml:"fallback_model"`

	// Quantize enables uint8 quantization of embeddings for ~4x memory reduction.
	// Default: true. Set to false as escape hatch if quantization degrades search.
	Quantize bool `yaml:"quantize"`

	// QuantizeBits is the quantization bit-width (default: 8). Reserved for future 4-bit support.
	QuantizeBits int `yaml:"quantize_bits"`
}

EmbeddingConfig configures the embedding provider.

func DefaultEmbeddingConfig 

func DefaultEmbeddingConfig() EmbeddingConfig

DefaultEmbeddingConfig returns sensible defaults.

type EmbeddingProvider 

type EmbeddingProvider interface {
	// Embed generates embeddings for a batch of texts.
	// Returns one float32 vector per input text.
	Embed(ctx context.Context, texts []string) ([][]float32, error)

	// Dimensions returns the dimensionality of the output vectors.
	Dimensions() int

	// Name returns the provider name (for cache key derivation).
	Name() string

	// Model returns the model name (for cache key derivation).
	Model() string
}

EmbeddingProvider generates vector embeddings from text.

func NewEmbeddingProvider 

func NewEmbeddingProvider(cfg EmbeddingConfig) EmbeddingProvider

NewEmbeddingProvider creates an embedding provider from config. When a fallback is configured, wraps with FallbackEmbedder for automatic failover.

type EnrichedEntityMatch added in v1.18.0 

type EnrichedEntityMatch struct {
	Name     string
	EntityID int64
	Matched  string
}

type EnrichedSearchResult added in v1.18.0 

type EnrichedSearchResult struct {
	Memories      []SearchResult
	KGFacts       []KGFact
	EntityMatches []EnrichedEntityMatch
}

type EntityCandidate added in v1.18.0 

type EntityCandidate struct {
	// Text is the raw matched token from the turn, preserving the user's
	// casing. Used for display and logging.
	Text string

	// Normalized is the lowercase + accent-stripped form used for SQL
	// lookups. Always a subset of StripAccents(strings.ToLower(Text)).
	Normalized string

	// Offset is the byte offset in the source turn where the match starts.
	// Used for highlighting in future UIs.
	Offset int
}

EntityCandidate is a token extracted from a turn that plausibly matches a stored entity (wing, room, or future KG entity).

type EntityDetector added in v1.18.0 

type EntityDetector struct {
	// contains filtered or unexported fields
}

EntityDetector extracts candidate tokens from turn text and resolves them against the wings/rooms tables. Stateless except for an internal TTL-cached snapshot of known entity names (refreshed on demand).

Thread-safe. Cache refresh is serialized so concurrent turns do not stampede the store.

func NewEntityDetector added in v1.18.0 

func NewEntityDetector(store *SQLiteStore, cfg EntityDetectorConfig, logger *slog.Logger) *EntityDetector

NewEntityDetector constructs a detector bound to the given store. store must NOT be nil. If logger is nil, slog.Default() is used.

func (*EntityDetector) Detect added in v1.18.0 

func (d *EntityDetector) Detect(ctx context.Context, turn string) ([]EntityMatch, error)

Detect scans the turn text and returns resolved entity matches. The turn is expected to be untrusted user input — tokenization uses Unicode letter/digit classes, skips punctuation, and caps the candidate count. Latency target: < 2ms on a warm cache for turns up to 500 chars.

Detection is deterministic for the same (turn, snapshot) pair. Results are ordered by offset ascending.

func (*EntityDetector) Refresh added in v1.18.0 

func (d *EntityDetector) Refresh(ctx context.Context) error

Refresh forces the internal snapshot to reload from the store. Called automatically when CacheTTL elapses; exposed for tests.

type EntityDetectorConfig added in v1.18.0 

type EntityDetectorConfig struct {
	// CacheTTL is how long the in-process entity snapshot survives before a
	// Detect() call triggers a refresh. Default: 30 seconds.
	CacheTTL time.Duration

	// MaxTokens caps how many candidate tokens are extracted from a single
	// turn. Protects against adversarial long inputs. Default: 40.
	MaxTokens int

	// MinTokenLen is the minimum character length (rune count) for a
	// candidate token to be checked against the snapshot. Default: 3.
	MinTokenLen int
}

EntityDetectorConfig configures an EntityDetector instance.

func DefaultEntityDetectorConfig added in v1.18.0 

func DefaultEntityDetectorConfig() EntityDetectorConfig

DefaultEntityDetectorConfig returns sensible defaults.

type EntityMatch added in v1.18.0 

type EntityMatch struct {
	Candidate EntityCandidate

	// Kind is "wing", "room", or (future) "kg_entity".
	Kind string

	// Wing is the wing this entity belongs to. For kind="wing" this equals
	// Candidate.Normalized. For kind="room" this is the room's parent wing.
	// May be empty for legacy NULL-wing rooms.
	Wing string

	// Room is the room name when Kind=="room", empty otherwise.
	Room string
}

EntityMatch is a candidate that successfully resolved to a stored entity after SQL lookup.

type Entry 

type Entry struct {
	Content   string     `json:"content"`
	Source    string     `json:"source"`   // "user", "agent", "system"
	Category  string     `json:"category"` // "fact", "preference", "event", "summary"
	Timestamp time.Time  `json:"timestamp"`
	ExpiresAt *time.Time `json:"expires_at,omitempty"` // nil = never expires
}

Entry represents a single memory fact or event.

type EssentialLayer added in v1.18.0 

type EssentialLayer struct {
	// contains filtered or unexported fields
}

EssentialLayer generates and caches per-wing essential stories from the user's memory database. Stories are deterministic templates composed of room summaries and lead sentences from top memories, truncated to a byte budget. No LLM calls.

Until Room 2.4 wires this layer into the prompt stack, EssentialLayer is dead code at runtime — Render() is safe to call from tests but no production caller exists yet.

func NewEssentialLayer added in v1.18.0 

func NewEssentialLayer(store *SQLiteStore, cfg EssentialLayerConfig, logger *slog.Logger) *EssentialLayer

NewEssentialLayer constructs a layer bound to the given store. store must NOT be nil — the layer is useless without a database. If logger is nil, slog.Default() is used. The passed cfg has its zero fields replaced by package defaults via withDefaults().

func (*EssentialLayer) Generate added in v1.18.0 

func (l *EssentialLayer) Generate(ctx context.Context, wing string) (string, error)

Generate forces an immediate regeneration of the story for a wing, bypassing cache freshness checks. Used by tests and (eventually) by the dream cycle when it decides a wing's story is materially out of date.

Unlike Render, Generate returns an error on SQL failure so the caller can react — tests rely on this to catch regressions early.

func (*EssentialLayer) Invalidate added in v1.18.0 

func (l *EssentialLayer) Invalidate(ctx context.Context, wing string) (int64, error)

Invalidate marks the cached story for a wing as stale, forcing the next Render() call to regenerate. Pass wing="*" to invalidate all cached stories. Returns the count of rows affected.

func (*EssentialLayer) Render added in v1.18.0 

func (l *EssentialLayer) Render(ctx context.Context, wing string) string

Render returns the cached or freshly-generated essential story for the given wing. Wing="" is valid and returns the legacy-NULL wing's story (which may be empty).

On SQL errors the layer logs at WARN and returns an empty string — never an error — so the prompt stack can always call Render() without worrying about hard failures.

Concurrency: per-wing regeneration is serialized with the layer mutex. A cache hit path bypasses the mutex entirely.

type EssentialLayerConfig added in v1.18.0 

type EssentialLayerConfig struct {
	// ByteBudget caps the rendered story's byte length. <=0 uses
	// defaultL1ByteBudget.
	ByteBudget int

	// StaleAfter is the TTL before a cached story is regenerated. <=0
	// uses defaultL1StaleAfter.
	StaleAfter time.Duration

	// RoomsPerWing is the maximum number of rooms walked per wing. <=0
	// uses defaultL1RoomsPerWing.
	RoomsPerWing int

	// LeadSentencesPerRoom is the number of top files per room that
	// contribute a lead sentence. <=0 uses
	// defaultL1LeadSentencesPerRoom.
	LeadSentencesPerRoom int
}

EssentialLayerConfig is the subset of HierarchyConfig fields the layer needs. Passed in explicitly so the layer doesn't depend on the copilot package (avoiding an import cycle).

func DefaultEssentialLayerConfig added in v1.18.0 

func DefaultEssentialLayerConfig() EssentialLayerConfig

DefaultEssentialLayerConfig returns sensible defaults matching the Sprint 2 Room 2.2 spec (400-token budget, 6h TTL, 4 rooms × 3 files).

type FallbackEmbedder added in v1.13.0 

type FallbackEmbedder struct {
	// contains filtered or unexported fields
}

FallbackEmbedder wraps a primary and fallback provider. On primary failure, automatically retries with the fallback. If both fail, returns an error; callers should degrade to FTS-only search.

func NewFallbackEmbedder added in v1.13.0 

func NewFallbackEmbedder(primary, fallback EmbeddingProvider, logger *slog.Logger) *FallbackEmbedder

NewFallbackEmbedder creates a fallback-enabled embedder.

func (*FallbackEmbedder) Dimensions added in v1.13.0 

func (f *FallbackEmbedder) Dimensions() int

Dimensions returns the primary provider's dimensions.

func (*FallbackEmbedder) Embed added in v1.13.0 

func (f *FallbackEmbedder) Embed(ctx context.Context, texts []string) ([][]float32, error)

Embed tries the primary provider, falling back on error.

func (*FallbackEmbedder) Model added in v1.13.0 

func (f *FallbackEmbedder) Model() string

Model returns the primary provider's model.

func (*FallbackEmbedder) Name added in v1.13.0 

func (f *FallbackEmbedder) Name() string

Name returns "fallback:{primary}" — only the primary is used for cache keys since embeddings from different models are not compatible.

type FileStore 

type FileStore struct {
	// contains filtered or unexported fields
}

FileStore implements Store using the filesystem. Uses MEMORY.md for long-term facts and daily markdown files for logs.

func NewFileStore 

func NewFileStore(baseDir string) (*FileStore, error)

NewFileStore creates a file-based memory store at the given directory.

func (*FileStore) Compact added in v1.18.0 

func (fs *FileStore) Compact() (int, error)

Compact rewrites MEMORY.md, removing expired and duplicate entries. Uses atomic write (temp file + rename) to prevent corruption. Returns the number of entries removed.

func (*FileStore) GetAll 

func (fs *FileStore) GetAll() ([]Entry, error)

GetAll reads and parses all entries from MEMORY.md.

func (*FileStore) GetByDate 

func (fs *FileStore) GetByDate(date time.Time) ([]Entry, error)

GetByDate returns entries from the daily log for the given date.

func (*FileStore) GetRecent 

func (fs *FileStore) GetRecent(limit int) ([]Entry, error)

GetRecent returns the most recent entries up to the limit.

func (*FileStore) ListDailyLogs 

func (fs *FileStore) ListDailyLogs() ([]string, error)

ListDailyLogs returns the dates of all daily log files, sorted newest first.

func (*FileStore) RecentFacts 

func (fs *FileStore) RecentFacts(maxFacts int, query string) string

RecentFacts returns a formatted string of recent facts suitable for injection into the system prompt.

func (*FileStore) Save 

func (fs *FileStore) Save(entry Entry) error

Save appends a memory entry to MEMORY.md.

func (*FileStore) SaveDailyLog 

func (fs *FileStore) SaveDailyLog(date time.Time, content string) error

SaveDailyLog appends a conversation summary to the daily log file.

func (*FileStore) SaveIfNotDuplicate added in v1.18.0 

func (fs *FileStore) SaveIfNotDuplicate(entry Entry, contentHash string, isDuplicate func(existing Entry) bool) (bool, string, error)

SaveIfNotDuplicate atomically checks for duplicates and saves under a single lock. Returns (saved bool, existingContent string, error). If saved is false, existingContent contains the matching entry's content.

func (*FileStore) Search 

func (fs *FileStore) Search(query string, maxResults int) ([]Entry, error)

Search returns entries whose content matches the query (case-insensitive). The query is split into words; an entry matches if ALL words appear in its content.

type GeminiEmbedder added in v1.13.0 

type GeminiEmbedder struct {
	// contains filtered or unexported fields
}

GeminiEmbedder generates embeddings using the Google Gemini API.

func NewGeminiEmbedder added in v1.13.0 

func NewGeminiEmbedder(cfg EmbeddingConfig) *GeminiEmbedder

NewGeminiEmbedder creates a Gemini embedding provider.

func (*GeminiEmbedder) Dimensions added in v1.13.0 

func (e *GeminiEmbedder) Dimensions() int

Dimensions returns the output vector dimensionality.

func (*GeminiEmbedder) Embed added in v1.13.0 

func (e *GeminiEmbedder) Embed(ctx context.Context, texts []string) ([][]float32, error)

Embed generates embeddings for a batch of texts. Uses batchEmbedContents for multiple texts, embedContent for a single text.

func (*GeminiEmbedder) Model added in v1.13.0 

func (e *GeminiEmbedder) Model() string

Model returns the model name.

func (*GeminiEmbedder) Name added in v1.13.0 

func (e *GeminiEmbedder) Name() string

Name returns the provider name.

type HybridSearchOptions added in v1.18.0 

type HybridSearchOptions struct {
	// MaxResults caps the number of results returned. Defaults to 6 when 0.
	MaxResults int

	// MinScore drops candidates whose final fused score falls below this
	// threshold. Defaults to 0.1 when 0.
	MinScore float64

	// VectorWeight is the fusion weight for the vector branch. Defaults
	// to 0.7 when 0.
	VectorWeight float64

	// BM25Weight is the fusion weight for the keyword branch. Defaults
	// to 0.3 when 0.
	BM25Weight float64

	// QueryWing, when non-empty, biases the fusion score: candidates whose
	// files.wing equals QueryWing are multiplied by WingBoostMatch, candidates
	// with a different non-empty wing are multiplied by WingBoostPenalty,
	// and candidates with wing IS NULL remain at multiplier 1.0 (neutral).
	//
	// When QueryWing is empty, the boost logic is bypassed entirely and the
	// search returns byte-identical scores and ordering to the legacy
	// HybridSearch signature. This is the Sprint 2 retrocompat contract.
	QueryWing string

	// WingBoostMatch is the score multiplier applied when a candidate's
	// wing matches QueryWing. Defaults to 1.3 when zero. A caller can set
	// this to 1.0 explicitly to disable matching boost while still applying
	// WingBoostPenalty to non-matching files.
	WingBoostMatch float64

	// WingBoostPenalty is the score multiplier applied when a candidate's
	// wing is non-empty but differs from QueryWing. Defaults to 0.4 when
	// zero. Files with wing IS NULL are NEVER penalized regardless of
	// this value — that is the Sprint 1 retrocompat contract.
	WingBoostPenalty float64

	// KGFactsPerEntity caps the number of KG facts returned per detected
	// entity in HybridSearchEnriched. Defaults to 3 when zero.
	// Set to -1 for unlimited.
	KGFactsPerEntity int
}

HybridSearchOptions configures a hybrid (vector + BM25) memory search.

This struct replaces the six-positional-float HybridSearch signature with a single options bag, which is necessary because Sprint 2 Room 2.0c adds wing-aware fusion. Adding "yet another float" was already the wrong shape; the options struct lets callers add new tuning knobs without breaking existing call sites.

Zero-value semantics for the numeric fields preserve the legacy defaults: MaxResults=0 → 6, MinScore=0 → 0.1, VectorWeight=0 → 0.7, BM25Weight=0 → 0.3. Wing fields fall back to 1.3 / 0.4 when zero (see WingBoostMatch godoc).

type IdentityLayer added in v1.18.0 

type IdentityLayer struct {
	// contains filtered or unexported fields
}

IdentityLayer loads a user-curated identity fragment from disk and caches it in memory. Hot-reloads on file change via fsnotify, with a 30s polling fallback when fsnotify is unavailable. Thread-safe.

Until Room 2.4 wires this layer into the prompt stack, the layer is dead code at runtime — Render() is safe to call from tests but no production caller exists yet.

func NewIdentityLayer added in v1.18.0 

func NewIdentityLayer(path string, logger *slog.Logger, budget int) *IdentityLayer

NewIdentityLayer constructs a new IdentityLayer. The path is the absolute file path to load (caller resolves "~"). If logger is nil, slog.Default() is used. Pass budget=0 to use defaultIdentityBudget.

The constructor does NOT touch the filesystem — call Start() to begin watching and Render() to read content.

func (*IdentityLayer) Reload added in v1.18.0 

func (l *IdentityLayer) Reload() error

Reload forces an immediate re-read from disk, bypassing the watcher or poll cadence. Used by tests and the CLI edit subcommand.

func (*IdentityLayer) Render added in v1.18.0 

func (l *IdentityLayer) Render() string

Render returns the cached identity content, truncated to the budget. If the file has not been loaded yet, returns empty string. Never returns an error — file-read failures are logged at WARN and result in empty content.

func (*IdentityLayer) Start added in v1.18.0 

func (l *IdentityLayer) Start() error

Start begins the file watcher (or polling fallback) and performs an initial load. Returns nil even if the file does not exist — a missing identity file is a valid state (renders to empty string). Errors only for unrecoverable conditions (e.g. cannot create temp dir).

Safe to call multiple times — subsequent calls are no-ops.

func (*IdentityLayer) Stop added in v1.18.0 

func (l *IdentityLayer) Stop()

Stop closes the watcher and signals the polling goroutine to exit. Idempotent.

type KGFact added in v1.18.0 

type KGFact struct {
	SubjectName    string
	PredicateName  string
	ObjectText     string
	Confidence     float64
	Wing           string
	SourceMemoryID string
}

type LegacyClassificationConfig added in v1.18.0 

type LegacyClassificationConfig struct {
	// BatchSize caps how many files are inspected per pass. Default 20.
	// Set to 0 to use the default. Negative values are treated as 0.
	BatchSize int

	// MinConfidence overrides ClassifierMinConfidence for this pass.
	// Set to 0 to use the default. Lower bound is 0.5 (hard minimum
	// enforced to prevent accidental misuse).
	MinConfidence float64

	// DryRun reports what WOULD be classified without writing anything
	// to the database. Useful for CLI preview and tests.
	DryRun bool

	// Keywords is the wing → keyword slice map used by the classifier.
	// Sourced from HierarchyConfig.LegacyKeywords. If nil or empty, the
	// pass is a no-op: it logs once at INFO and returns 0 classified.
	// The binary ships zero defaults to stay locale and domain neutral.
	Keywords map[string][]string
}

LegacyClassificationConfig tunes the pass behavior.

type LegacyClassificationStats added in v1.18.0 

type LegacyClassificationStats struct {
	// Scanned is the number of legacy files the pass inspected.
	Scanned int

	// Classified is the number of files that received a new wing.
	Classified int

	// Skipped is Scanned - Classified (files the classifier could not decide).
	Skipped int

	// Errors counts files that failed to read/update due to SQL errors.
	Errors int

	// PerWing tracks how many files landed in each wing this pass.
	PerWing map[string]int
}

LegacyClassificationStats summarizes the outcome of one pass.

type MMRConfig added in v1.12.0 

type MMRConfig struct {
	Enabled bool
	Lambda  float64
}

MMRConfig configures Maximal Marginal Relevance for search diversification.

type MistralEmbedder added in v1.13.0 

type MistralEmbedder struct {
	// contains filtered or unexported fields
}

MistralEmbedder generates embeddings using the Mistral API. Mistral uses a fully OpenAI-compatible request/response format.

func NewMistralEmbedder added in v1.13.0 

func NewMistralEmbedder(cfg EmbeddingConfig) *MistralEmbedder

NewMistralEmbedder creates a Mistral embedding provider.

func (*MistralEmbedder) Dimensions added in v1.13.0 

func (e *MistralEmbedder) Dimensions() int

Dimensions returns the output vector dimensionality.

func (*MistralEmbedder) Embed added in v1.13.0 

func (e *MistralEmbedder) Embed(ctx context.Context, texts []string) ([][]float32, error)

Embed generates embeddings for a batch of texts.

func (*MistralEmbedder) Model added in v1.13.0 

func (e *MistralEmbedder) Model() string

Model returns the model name.

func (*MistralEmbedder) Name added in v1.13.0 

func (e *MistralEmbedder) Name() string

Name returns the provider name.

type NullEmbedder 

type NullEmbedder struct{}

NullEmbedder is a no-op provider that disables semantic search. Used when no embedding provider is configured.

func (*NullEmbedder) Dimensions 

func (e *NullEmbedder) Dimensions() int

Dimensions returns 0.

func (*NullEmbedder) Embed 

func (e *NullEmbedder) Embed(_ context.Context, _ []string) ([][]float32, error)

Embed returns nil (no embeddings).

func (*NullEmbedder) Model 

func (e *NullEmbedder) Model() string

Model returns "none".

func (*NullEmbedder) Name 

func (e *NullEmbedder) Name() string

Name returns "none".

type ONNXEmbedder added in v1.18.0 

type ONNXEmbedder struct {
	// contains filtered or unexported fields
}

ONNXEmbedder generates embeddings using a local ONNX sentence-transformer.

func NewONNXEmbedder added in v1.18.0 

func NewONNXEmbedder(cfg EmbeddingConfig) (*ONNXEmbedder, error)

NewONNXEmbedder creates a local ONNX embedding provider. Auto-downloads runtime and model if not present.

func (*ONNXEmbedder) Close added in v1.18.0 

func (e *ONNXEmbedder) Close() error

Close releases ONNX resources.

func (*ONNXEmbedder) Dimensions added in v1.18.0 

func (e *ONNXEmbedder) Dimensions() int

Dimensions returns the embedding dimensionality.

func (*ONNXEmbedder) Embed added in v1.18.0 

func (e *ONNXEmbedder) Embed(_ context.Context, texts []string) ([][]float32, error)

Embed generates embeddings for a batch of texts.

func (*ONNXEmbedder) Model added in v1.18.0 

func (e *ONNXEmbedder) Model() string

Model returns the model name.

func (*ONNXEmbedder) Name added in v1.18.0 

func (e *ONNXEmbedder) Name() string

Name returns the provider name.

type ONNXPaths added in v1.18.0 

type ONNXPaths struct {
	RuntimeLib string // path to libonnxruntime.so
	ModelFile  string // path to model.onnx
	VocabFile  string // path to vocab.txt
}

ONNXPaths holds resolved paths for ONNX runtime and model files.

type OnDemandLayer added in v1.18.0 

type OnDemandLayer struct {
	// contains filtered or unexported fields
}

OnDemandLayer retrieves per-turn contextual memories based on detected entities. Composes an EntityDetector with a wing-aware search over the SQLite store.

Until Room 2.4 wires this into the prompt stack, the layer is dead code at runtime but fully functional for tests.

func NewOnDemandLayer added in v1.18.0 

func NewOnDemandLayer(store *SQLiteStore, detector *EntityDetector, cfg OnDemandLayerConfig, logger *slog.Logger) *OnDemandLayer

NewOnDemandLayer constructs a layer. detector may be nil — in that case the layer auto-constructs one from store and config defaults. Passing a shared detector allows callers to amortize the entity snapshot cache across multiple layers (Room 2.4 pattern).

store must NOT be nil. If logger is nil, slog.Default() is used.

func (*OnDemandLayer) Render added in v1.18.0 

func (l *OnDemandLayer) Render(ctx context.Context, activeWing string, turn string) string

Render inspects the turn, detects entities, retrieves memories from the active wing first, optionally falls back to ONE cross-wing result when nothing is found in the active wing, and returns a prompt-ready string truncated to the byte budget.

activeWing is the current session's wing ("" = legacy / no wing). turn is the current user message text.

This runs on the hot path. Latency contract: p95 < 10ms warm. Errors are logged at DEBUG and result in an empty string.

func (*OnDemandLayer) SetKG added in v1.18.0 

func (l *OnDemandLayer) SetKG(k *kg.KG, factsPerRender int)

func (*OnDemandLayer) SetTopicDetector added in v1.18.0 

func (l *OnDemandLayer) SetTopicDetector(d *TopicChangeDetector)

SetTopicDetector sets the optional topic change detector.

type OnDemandLayerConfig added in v1.18.0 

type OnDemandLayerConfig struct {
	// ByteBudget caps the total byte length of Render's output.
	// <=0 uses defaultL2ByteBudget (1200 bytes ≈ 300 tokens).
	ByteBudget int

	// MaxResults caps the number of distinct memory results returned.
	// <=0 uses defaultL2MaxResults (5).
	MaxResults int

	// CrossWingEnabled, when true, allows ONE cross-wing fallback result
	// when the active wing yields nothing. Default: true.
	CrossWingEnabled bool

	// DetectorTimeoutMs is the hard latency guard for the entity detector
	// call. <=0 uses defaultL2DetectorTimeoutMs (3ms).
	DetectorTimeoutMs int

	// SearchTimeoutMs is the hard latency guard for store search calls.
	// <=0 uses defaultL2SearchTimeoutMs (8ms).
	SearchTimeoutMs int
}

OnDemandLayerConfig configures the OnDemandLayer hot-path behavior.

func DefaultOnDemandLayerConfig added in v1.18.0 

func DefaultOnDemandLayerConfig() OnDemandLayerConfig

DefaultOnDemandLayerConfig returns sensible defaults for hot-path use.

type OpenAIEmbedder 

type OpenAIEmbedder struct {
	// contains filtered or unexported fields
}

OpenAIEmbedder generates embeddings using the OpenAI Embeddings API.

func NewOpenAIEmbedder 

func NewOpenAIEmbedder(cfg EmbeddingConfig) *OpenAIEmbedder

NewOpenAIEmbedder creates an OpenAI embedding provider.

func (*OpenAIEmbedder) Dimensions 

func (e *OpenAIEmbedder) Dimensions() int

Dimensions returns the output vector dimensionality.

func (*OpenAIEmbedder) Embed 

func (e *OpenAIEmbedder) Embed(ctx context.Context, texts []string) ([][]float32, error)

Embed generates embeddings for a batch of texts.

func (*OpenAIEmbedder) Model 

func (e *OpenAIEmbedder) Model() string

Model returns the model name.

func (*OpenAIEmbedder) Name 

func (e *OpenAIEmbedder) Name() string

Name returns the provider name.

type PalaceCoords added in v1.18.0 

type PalaceCoords struct {
	Wing string // normalized wing identifier, "" = not set (legacy)
	Hall string // normalized hall identifier, "" = no hall
	Room string // normalized room identifier, "" = no room
}

PalaceCoords groups the wing/hall/room triple derived from a path or supplied by a tool call. All fields are optional; empty string means "not set" (which maps to SQL NULL at persistence boundaries).

func ExtractCoordsFromPath added in v1.18.0 

func ExtractCoordsFromPath(relPath string) PalaceCoords

ExtractCoordsFromPath derives wing/hall/room from a file path relative to the memory base directory. The separator is the OS path separator.

Rules (applied to path components BEFORE the final filename):

  • 0 components → all empty (legacy file at root)
  • 1 component → wing only
  • 2 components → wing + room
  • 3+ components → wing + hall + room (middle components collapsed)

The final filename (MEMORY.md, YYYY-MM-DD.md, etc.) is ignored for coordinate derivation.

If any component fails normalization, the whole coordinate set is rejected and empty coords are returned (fail-safe to legacy behavior).

Examples (assuming baseDir="/home/u/.devclaw/memory"): 

"MEMORY.md"                           → {}
"work/MEMORY.md"                      → {Wing:"work"}
"work/auth-migration/MEMORY.md"       → {Wing:"work", Room:"auth-migration"}
"work/meetings/auth/MEMORY.md"        → {Wing:"work", Hall:"meetings", Room:"auth"}
"work/a/b/c/d/MEMORY.md"              → {Wing:"work", Hall:"a", Room:"d"}  (middle collapsed)
"FAMÍLIA/MEMORY.md"                   → {Wing:"familia"}

func (PalaceCoords) IsEmpty added in v1.18.0 

func (c PalaceCoords) IsEmpty() bool

IsEmpty reports whether no palace coordinates are set (legacy memory).

type QuantizedEmbedding added in v1.18.0 

type QuantizedEmbedding struct {
	Data   []uint8 // quantized values in [0, 255]
	Scale  float32 // (max - min) / 255; zero when all values are identical
	MinVal float32 // minimum value before quantization
	Dims   int     // original dimensionality
}

QuantizedEmbedding stores a uint8-quantized embedding with scale/min for reconstruction. Uses the full [0, 255] range for maximum precision (8 effective bits).

func QuantizeFloat32 added in v1.18.0 

func QuantizeFloat32(vec []float32) QuantizedEmbedding

QuantizeFloat32 converts a float32 embedding to uint8 quantized form. Uses affine quantization: q = round((v - min) / (max - min) * 255).

func (QuantizedEmbedding) CosineSimilarity added in v1.18.0 

func (q QuantizedEmbedding) CosineSimilarity(query []float32, queryNorm float64) float64

CosineSimilarity computes cosine similarity between this quantized embedding and a float32 query. queryNorm is the precomputed L2 norm of the query (compute once per search, reuse across all candidates).

func (QuantizedEmbedding) Dequantize added in v1.18.0 

func (q QuantizedEmbedding) Dequantize() []float32

Dequantize reconstructs the float32 embedding from quantized form.

func (QuantizedEmbedding) DotProduct added in v1.18.0 

func (q QuantizedEmbedding) DotProduct(query []float32) float64

DotProduct computes the dot product between this quantized embedding and a float32 query vector. Uses the asymmetric estimator pattern: the query stays in full precision while the data is reconstructed on-the-fly from uint8.

Mathematically: Σ (data[i]*scale + minVal) * query[i] 

= scale * Σ data[i]*query[i]  +  minVal * Σ query[i]

This avoids per-element dequantization by factoring out scale and minVal.

func (QuantizedEmbedding) MarshalBinary added in v1.18.0 

func (q QuantizedEmbedding) MarshalBinary() ([]byte, error)

MarshalBinary serializes the quantized embedding to a compact binary format: [dims:uint32LE][scale:float32LE][minVal:float32LE][data:uint8...]

func (*QuantizedEmbedding) UnmarshalBinary added in v1.18.0 

func (q *QuantizedEmbedding) UnmarshalBinary(data []byte) error

UnmarshalBinary deserializes a quantized embedding from binary format.

type RoomFileSummary added in v1.18.0 

type RoomFileSummary struct {
	FileID      string
	Text        string // concatenated chunk text (first chunk only)
	AccessCount int
	IndexedAt   time.Time
}

RoomFileSummary is a lightweight projection of a memory file used by the L1 EssentialLayer. It carries just enough to render a lead sentence bullet without pulling the entire chunk payload into memory.

type RoomInfo added in v1.18.0 

type RoomInfo struct {
	Wing         string
	Name         string
	Hall         string
	Source       string // 'manual' | 'auto' | 'inferred' | 'legacy'
	Confidence   float64
	ReuseCount   int
	MemoryCount  int
	DisplayName  string
	Description  string
	LastActivity time.Time
	CreatedAt    time.Time
}

RoomInfo describes a registered room inside a wing.

type SQLiteStore 

type SQLiteStore struct {
	// contains filtered or unexported fields
}

SQLiteStore provides persistent memory storage with hybrid search.

func NewSQLiteStore 

func NewSQLiteStore(dbPath string, embedder EmbeddingProvider, logger *slog.Logger) (*SQLiteStore, error)

NewSQLiteStore opens or creates a SQLite memory database with FTS5.

func (*SQLiteStore) ApplyMMR added in v1.12.0 

func (s *SQLiteStore) ApplyMMR(results []SearchResult, cfg MMRConfig, maxResults int) []SearchResult

ApplyMMR applies Maximal Marginal Relevance re-ranking to diversify results. Lambda controls the balance: 0 = max diversity, 1 = max relevance.

func (*SQLiteStore) ApplyTemporalDecay added in v1.12.0 

func (s *SQLiteStore) ApplyTemporalDecay(results []SearchResult, cfg TemporalDecayConfig) []SearchResult

ApplyTemporalDecay applies exponential decay to search results based on file age. Files matching the pattern "memory/YYYY-MM-DD.md" are decayed; evergreen files (MEMORY.md or non-dated) are not decayed.

func (*SQLiteStore) AssignWingToFile added in v1.18.0 

func (s *SQLiteStore) AssignWingToFile(ctx context.Context, fileID string, wing string) error

AssignWingToFile sets the wing column on a file row, but ONLY when the current wing is NULL. This conditional UPDATE is the race barrier that makes concurrent saves and dream classifier passes safe:

  • If memory_save runs first → sets wing=X; classifier's UPDATE becomes a no-op (WHERE wing IS NULL no longer matches).
  • If classifier runs first → sets wing=Y; memory_save's UPDATE becomes a no-op. Both end states are valid; no cross-contamination.

The fileID is the key used by IndexDirectory/IndexChunks — for files written by FileStore.Save this is always "MEMORY.md".

An empty wing is rejected at the caller level (NormalizeWing returns ""). The query uses a parameterized placeholder to prevent injection.

func (*SQLiteStore) ChunkCount 

func (s *SQLiteStore) ChunkCount() int

ChunkCount returns the total number of indexed chunks.

func (*SQLiteStore) Close 

func (s *SQLiteStore) Close() error

Close closes the database connection.

func (*SQLiteStore) DB added in v1.18.0 

func (s *SQLiteStore) DB() *sql.DB

func (*SQLiteStore) DeleteChannelWing added in v1.18.0 

func (s *SQLiteStore) DeleteChannelWing(channel, externalID string) error

DeleteChannelWing removes a channel → wing mapping. Subsequent lookups for the same (channel, external_id) will return ErrChannelWingNotFound.

func (*SQLiteStore) DeleteWing added in v1.18.0 

func (s *SQLiteStore) DeleteWing(name string, force bool) error

DeleteWing removes a wing from the registry. Does NOT cascade to files or rooms — those are left intact but orphaned. Intended for cleanup of suggested wings the user never used.

Returns an error if the wing still has any associated files or rooms, unless force is true.

func (*SQLiteStore) FileCount 

func (s *SQLiteStore) FileCount() int

FileCount returns the total number of indexed files.

func (*SQLiteStore) GetChannelWing added in v1.18.0 

func (s *SQLiteStore) GetChannelWing(channel, externalID string) (ChannelWingMapping, error)

GetChannelWing looks up the wing assigned to a specific channel + external ID. Returns ErrChannelWingNotFound if no mapping exists — callers should treat this as "unmapped" and fall back to heuristics or defaults.

func (*SQLiteStore) GetTaxonomy added in v1.18.0 

func (s *SQLiteStore) GetTaxonomy() ([]TaxonomyEntry, error)

GetTaxonomy returns the full wing → rooms tree with counts. The result is suitable for CLI/WebUI rendering of the palace structure.

LegacyCount on each entry is always 0 — it is computed once at the level of the caller by summing across all wings (see the first return value of TotalLegacyFiles).

func (*SQLiteStore) HybridSearch 

func (s *SQLiteStore) HybridSearch(ctx context.Context, query string, maxResults int, minScore float64, vectorWeight, bm25Weight float64) ([]SearchResult, error)

HybridSearch performs a combined vector + BM25 search with configurable weights. This signature is preserved verbatim for backward compatibility with v1.17.0 callers; it is now a thin wrapper over HybridSearchWithOpts that passes QueryWing="" to take the wing-unaware fast path.

Calls through this entry point are byte-identical to the pre-Sprint-2 implementation: no JOIN against files.wing, no multiplier, no per-result Wing population.

func (*SQLiteStore) HybridSearchEnriched added in v1.18.0 

func (s *SQLiteStore) HybridSearchEnriched(ctx context.Context, query string, opts HybridSearchOptions) (*EnrichedSearchResult, error)

HybridSearchEnriched runs the existing HybridSearchWithOpts and then enriches the result with KG facts for any entities detected in the query.

When s.kg is nil (KG disabled), the method returns memories-only result with no error — identical to calling HybridSearchWithOpts directly plus empty KGFacts and EntityMatches slices.

Entity detection uses a lightweight token-to-kg_entities lookup instead of the full EntityDetector (which is bound to wings/rooms). Each query token is checked against kg_entities.canonical_name and kg_entity_aliases.alias_name.

func (*SQLiteStore) HybridSearchWithOptions added in v1.12.0 

func (s *SQLiteStore) HybridSearchWithOptions(ctx context.Context, query string, maxResults int, minScore float64, vectorWeight, bm25Weight float64, decayCfg TemporalDecayConfig, mmrCfg MMRConfig) ([]SearchResult, error)

HybridSearchWithOptions performs hybrid search with optional temporal decay and MMR. This signature is preserved verbatim for backward compatibility with v1.17.0 callers; it delegates to HybridSearchWithOpts with QueryWing="" to take the wing-unaware fast path.

Sprint 2 callers that need wing-aware fusion should use HybridSearchWithOptsAndPostFilters or HybridSearchWithOpts directly.

func (*SQLiteStore) HybridSearchWithOpts added in v1.18.0 

func (s *SQLiteStore) HybridSearchWithOpts(ctx context.Context, query string, opts HybridSearchOptions) ([]SearchResult, error)

HybridSearchWithOpts is the wing-aware hybrid search implementation introduced in Sprint 2 Room 2.0c. It runs vector + BM25 in parallel, fuses the rankings via the existing weighted-inverse-rank formula (weight * 1/(rank+1)) — NOT standard RRF k=60; that migration is deferred to ADR-010 — and then applies a wing multiplier to the fused score when opts.QueryWing is non-empty.

Wing multiplier rules:

  • candidate.Wing == opts.QueryWing → fused *= WingBoostMatch (1.3 default)
  • candidate.Wing != "" and != opts.QueryWing → fused *= WingBoostPenalty (0.4 default)
  • candidate.Wing == "" (legacy NULL) → fused *= 1.0 (NEVER penalized)

The "wing IS NULL stays neutral" rule is a hard invariant: legacy memories from v1.17.0 must rank exactly the same regardless of whether the query carries a wing. Violating this would silently degrade results for every user who hasn't yet started classifying their memories.

When opts.QueryWing == "", this function is byte-identical to the pre-Sprint-2 HybridSearch — no JOIN against files.wing, no multiplier arithmetic, no per-result Wing field population.

func (*SQLiteStore) HybridSearchWithOptsAndPostFilters added in v1.18.0 

func (s *SQLiteStore) HybridSearchWithOptsAndPostFilters(ctx context.Context, query string, opts HybridSearchOptions, decayCfg TemporalDecayConfig, mmrCfg MMRConfig) ([]SearchResult, error)

HybridSearchWithOptsAndPostFilters runs HybridSearchWithOpts and then applies the temporal-decay and MMR post-filters. It is the wing-aware equivalent of HybridSearchWithOptions and is the entry point used by memory_search and prompt_layers when wing routing is active.

The retrocompat contract is the same as HybridSearchWithOpts: passing opts.QueryWing == "" produces results that are byte-identical to the legacy HybridSearchWithOptions for the same numeric inputs.

func (*SQLiteStore) IndexChunks 

func (s *SQLiteStore) IndexChunks(ctx context.Context, fileID string, chunks []Chunk, fileHash string) error

IndexChunks indexes a set of chunks for a file. Uses delta sync: only re-embeds chunks whose hash has changed.

func (*SQLiteStore) IndexMemoryDir 

func (s *SQLiteStore) IndexMemoryDir(ctx context.Context, memDir string, chunkCfg ChunkConfig) error

IndexMemoryDir indexes all .md files in the memory directory and MEMORY.md.

func (*SQLiteStore) IndexTranscript added in v1.13.0 

func (s *SQLiteStore) IndexTranscript(ctx context.Context, sessionID string, entries []TranscriptEntry) error

IndexTranscript indexes conversation transcript entries as searchable chunks. Each entry is stored as a chunk with file_id "session:<sessionID>". Uses content hashing to avoid re-indexing identical content.

func (*SQLiteStore) KG added in v1.18.0 

func (s *SQLiteStore) KG() *kg.KG

KG returns the Knowledge Graph instance, or nil if KG is not configured.

func (*SQLiteStore) LastQueryEmbedding added in v1.18.0 

func (s *SQLiteStore) LastQueryEmbedding() []float32

LastQueryEmbedding returns a copy of the most recent query embedding from SearchVector. Returns a defensive copy to prevent data races if the embedding provider reuses buffers. Thread-safe.

func (*SQLiteStore) ListChannelWings added in v1.18.0 

func (s *SQLiteStore) ListChannelWings(wingFilter string) ([]ChannelWingMapping, error)

ListChannelWings returns all channel → wing mappings, optionally filtered by wing. Useful for admin/audit views showing "what maps to wing=work".

func (*SQLiteStore) ListRooms added in v1.18.0 

func (s *SQLiteStore) ListRooms(wing string) ([]RoomInfo, error)

ListRooms returns rooms belonging to a wing. Pass an empty wing to list all rooms across all wings (useful for maintenance views).

Results are ordered by memory_count DESC so the most-used rooms surface first.

func (*SQLiteStore) ListRoomsByRecency added in v1.18.0 

func (s *SQLiteStore) ListRoomsByRecency(ctx context.Context, wing string, limit int) ([]RoomInfo, error)

ListRoomsByRecency returns rooms in a wing ordered by last_activity DESC with a stable tie-breaker on name ASC. Pass an empty wing to return rooms whose wing IS NULL (the legacy cidadão set). The limit caps the number of rows returned; pass <= 0 for no cap.

This helper exists for the L1 EssentialLayer which wants the most recently touched rooms per wing, whereas the default ListRooms orders by memory_count DESC to surface heavy-use rooms.

Rooms with NULL last_activity sort LAST (i.e. behind any room that has seen activity) so freshly-created idle rooms do not displace active ones.

func (*SQLiteStore) ListWings added in v1.18.0 

func (s *SQLiteStore) ListWings() ([]WingInfo, error)

ListWings returns all registered wings with their memory and room counts. Counts are computed via JOIN at query time rather than maintained as a cached column, so they always reflect current state.

The `files` table may not yet have a `wing` column if the hierarchy schema failed to initialize — in that case, memory_count is reported as 0 rather than returning an error.

func (*SQLiteStore) PruneEmbeddingCache 

func (s *SQLiteStore) PruneEmbeddingCache(maxEntries int)

PruneEmbeddingCache removes old cache entries exceeding maxEntries.

func (*SQLiteStore) RunLegacyClassificationPass added in v1.18.0 

func (s *SQLiteStore) RunLegacyClassificationPass(ctx context.Context, cfg LegacyClassificationConfig) (*LegacyClassificationStats, error)

RunLegacyClassificationPass scans for legacy files (wing IS NULL), reads their content, runs the pattern-based classifier, and updates files.wing for any that the classifier confidently labels.

This is idempotent: running it multiple times will only label NEW files, because the query filters out files that already have a wing.

Returns a summary of the pass. Individual file errors are logged via the store's logger and counted but do not abort the pass.

Callers typically invoke this from:

  1. The dream system's cycle runner (Sprint 2 integration)
  2. CLI commands (future: `devclaw memory classify-legacy`)
  3. Tests

func (*SQLiteStore) SearchBM25 

func (s *SQLiteStore) SearchBM25(query string, maxResults int) ([]SearchResult, error)

SearchBM25 performs a keyword search using FTS5 BM25 ranking. Falls back to LIKE-based search when FTS5 is not available. Applies query expansion to handle conversational queries.

func (*SQLiteStore) SearchVector 

func (s *SQLiteStore) SearchVector(ctx context.Context, query string, maxResults int) ([]SearchResult, error)

SearchVector performs a vector similarity search using in-memory cosine similarity.

func (*SQLiteStore) SetChannelWing added in v1.18.0 

func (s *SQLiteStore) SetChannelWing(channel, externalID, wing, source string, confidence float64) error

SetChannelWing creates or updates a channel → wing mapping. The wing name is normalized before storage. Source should be one of:

  • "manual" — user explicitly set it (confidence 1.0)
  • "heuristic" — derived from channel pattern (confidence 0.5-0.8)
  • "llm" — classified by LLM (confidence varies)
  • "inherited" — copied from another mapping (confidence inherited)

Passing an empty wing is an error — use DeleteChannelWing to remove a mapping.

func (*SQLiteStore) SetKG added in v1.18.0 

func (s *SQLiteStore) SetKG(k *kg.KG)

SetKG injects a KG reference into the store. Pass nil to disable KG enrichment. The KG instance is inert until someone calls HybridSearchEnriched.

func (*SQLiteStore) SetQuantizeEnabled added in v1.18.0 

func (s *SQLiteStore) SetQuantizeEnabled(enabled bool)

SetQuantizeEnabled enables or disables uint8 embedding quantization. Call before loadVectorCache (typically right after construction).

func (*SQLiteStore) TopFilesInRoom added in v1.18.0 

func (s *SQLiteStore) TopFilesInRoom(ctx context.Context, wing, room string, limit int) ([]RoomFileSummary, error)

TopFilesInRoom returns up to `limit` files belonging to (wing, room) ordered by access_count DESC then indexed_at DESC, with a stable file_id ASC tie-breaker. For each file the first chunk's text is returned as Text — the L1 renderer extracts a lead sentence from it.

Pass wing="" to match the legacy wing IS NULL set. Pass room="" to match files whose room IS NULL as well. The limit must be > 0; callers asking for 0 get an empty slice without hitting the database.

All string parameters are normalized before use to defend the SQL layer even if callers forget to normalize upstream.

func (*SQLiteStore) TotalLegacyFiles added in v1.18.0 

func (s *SQLiteStore) TotalLegacyFiles() (int, error)

TotalLegacyFiles returns the count of files with wing IS NULL — memories that predate the palace hierarchy and have not been classified. This is always >= 0 and is a first-class value per ADR-006 (legacy cidadão).

func (*SQLiteStore) UpsertRoom added in v1.18.0 

func (s *SQLiteStore) UpsertRoom(wing, name, hall, source string, confidence float64) error

UpsertRoom creates or updates a room entry. The source parameter controls how the room was derived; confidence should be 1.0 for manual entries.

If the room already exists, this function increments its reuse_count and bumps last_activity — useful for the ADR-002 Addendum A confidence promotion rule.

func (*SQLiteStore) UpsertWing added in v1.18.0 

func (s *SQLiteStore) UpsertWing(name, displayName, description string) error

UpsertWing creates a wing entry if it doesn't exist, or updates its display metadata if it does. Normalizes the name first; returns an error if normalization yields an empty string.

Does not touch rooms or memories — purely a registry operation.

type SearchResult 

type SearchResult struct {
	FileID string
	Text   string
	Score  float64
	Wing   string
}

SearchResult represents a single search hit with score.

The Wing field (added in Sprint 2 Room 2.0c) carries the originating file's palace wing, if any. It is empty for legacy (wing IS NULL) files and for results coming from the wing-unaware code paths. The field is last in the struct to preserve JSON marshalling compatibility for any caller that decoded SearchResult before Sprint 2.

type Store 

type Store interface {
	// Save persists a new memory entry.
	Save(entry Entry) error

	// Search returns entries matching the query, limited to maxResults.
	Search(query string, maxResults int) ([]Entry, error)

	// GetRecent returns the most recent entries up to limit.
	GetRecent(limit int) ([]Entry, error)

	// GetByDate returns entries for a specific date.
	GetByDate(date time.Time) ([]Entry, error)

	// GetAll returns all stored entries.
	GetAll() ([]Entry, error)

	// SaveDailyLog appends a summary to the daily log file.
	SaveDailyLog(date time.Time, content string) error
}

Store defines the interface for memory persistence.

type TaxonomyEntry added in v1.18.0 

type TaxonomyEntry struct {
	Wing        WingInfo
	Rooms       []RoomInfo
	LegacyCount int // files with wing=NULL that would be "sibling" to this tree
}

TaxonomyEntry describes one wing and its rooms for tree rendering.

type TemporalDecayConfig added in v1.12.0 

type TemporalDecayConfig struct {
	Enabled      bool
	HalfLifeDays float64
}

TemporalDecayConfig configures exponential score decay based on memory age.

type TopicChangeDetector added in v1.18.0 

type TopicChangeDetector struct {
	// contains filtered or unexported fields
}

TopicChangeDetector detects topic changes using a two-stage cascade. Thread-safe via RWMutex.

func NewTopicChangeDetector added in v1.18.0 

func NewTopicChangeDetector(cosineThreshold, entityOverlap float32, kgStore *kg.KG, factsPerInjection int) *TopicChangeDetector

NewTopicChangeDetector creates a detector with the given thresholds. kg may be nil — the detector works without it (no fact injection).

func (*TopicChangeDetector) Detect added in v1.18.0 

func (d *TopicChangeDetector) Detect(currentEntities []EntityMatch, currentEmbedding []float32) TopicChangeResult

Detect uses a two-stage cascade to detect topic changes.

Stage 1 (free): Entity overlap — if overlap >= threshold, same topic (fast path). Stage 2 (free): Cosine similarity of embeddings — confirms topic change.

CRITICAL: Both inputs are already computed by the L2 pipeline:

  • currentEntities from EntityDetector.Detect()
  • currentEmbedding from SearchVector() query embedding

NO extra embedding API calls are made.

func (*TopicChangeDetector) LookupKGFacts added in v1.18.0 

func (d *TopicChangeDetector) LookupKGFacts(ctx context.Context, entities []EntityMatch) []kg.Triple

LookupKGFacts queries the KG for facts related to the current entities. Returns empty if KG is nil or no entities provided.

func (*TopicChangeDetector) UpdateTopic added in v1.18.0 

func (d *TopicChangeDetector) UpdateTopic(entities []EntityMatch, embedding []float32)

UpdateTopic stores the current turn's state for the next comparison. This is a synchronous struct copy — NO API call, NO goroutine needed.

type TopicChangeResult added in v1.18.0 

type TopicChangeResult struct {
	Changed    bool
	Confidence float32 // 1 - cosine similarity (higher = more different)
}

TopicChangeResult holds the detection outcome.

type TranscriptEntry added in v1.13.0 

type TranscriptEntry struct {
	Role    string // "user" or "assistant"
	Content string
}

TranscriptEntry is a conversation entry for transcript indexing.

type VoyageEmbedder added in v1.13.0 

type VoyageEmbedder struct {
	// contains filtered or unexported fields
}

VoyageEmbedder generates embeddings using the Voyage AI API. Voyage uses an OpenAI-compatible format with an additional input_type field that improves relevance by distinguishing queries from documents.

func NewVoyageEmbedder added in v1.13.0 

func NewVoyageEmbedder(cfg EmbeddingConfig) *VoyageEmbedder

NewVoyageEmbedder creates a Voyage AI embedding provider.

func (*VoyageEmbedder) Dimensions added in v1.13.0 

func (e *VoyageEmbedder) Dimensions() int

Dimensions returns the output vector dimensionality.

func (*VoyageEmbedder) Embed added in v1.13.0 

func (e *VoyageEmbedder) Embed(ctx context.Context, texts []string) ([][]float32, error)

Embed generates embeddings for a batch of texts.

func (*VoyageEmbedder) Model added in v1.13.0 

func (e *VoyageEmbedder) Model() string

Model returns the model name.

func (*VoyageEmbedder) Name added in v1.13.0 

func (e *VoyageEmbedder) Name() string

Name returns the provider name.

type WingInfo added in v1.18.0 

type WingInfo struct {
	Name        string
	DisplayName string
	Description string
	Color       string
	IsDefault   bool
	IsSuggested bool
	MemoryCount int // computed from files table at query time
	RoomCount   int // computed from rooms table at query time
	CreatedAt   time.Time
	UpdatedAt   time.Time
}

WingInfo describes a registered wing.

type WordPieceTokenizer added in v1.18.0 

type WordPieceTokenizer struct {
	// contains filtered or unexported fields
}

WordPieceTokenizer tokenizes text for BERT-family models.

func NewWordPieceTokenizer added in v1.18.0 

func NewWordPieceTokenizer(vocabPath string, maxLen int) (*WordPieceTokenizer, error)

NewWordPieceTokenizer loads a vocab.txt file and returns a tokenizer.

func (*WordPieceTokenizer) Tokenize added in v1.18.0 

func (t *WordPieceTokenizer) Tokenize(text string) (inputIDs, attentionMask, tokenTypeIDs []int64)

Tokenize converts text into token IDs, attention mask, and token type IDs. Output is padded/truncated to maxLen.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.