engine

package
v0.1.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 4, 2026 License: MIT Imports: 51 Imported by: 0

Documentation

Overview

Package engine is the core of yaad: it ties storage, the typed graph, dedup, decay, conflict resolution, and hybrid recall together behind the Engine type, the main entry point for remember/recall/forget operations.

Index

Constants

This section is empty.

Variables

View Source
var DefaultDecayConfig = DecayConfig{
	HalfLifeDays:  30,
	MinConfidence: 0.1,
	BoostOnAccess: 0.2,
}

Functions

func BoostNode

func BoostNode(ctx context.Context, store storage.Storage, id string, boost float64) error

BoostNode increases confidence of a node on access (capped at 1.0).

func CacheKey

func CacheKey(opts RecallOpts) string

CacheKey builds a cache key from recall options.

func ContentHash

func ContentHash(content, scope, project, nodeType string) string

ContentHash computes a deterministic SHA-256 hash for node content.

func DecompressSessionData

func DecompressSessionData(data string) ([]byte, error)

DecompressSessionData decompresses session replay data if it was gzip-compressed. Returns the original JSON data. If the data is not compressed, returns it as-is.

func ExpandQuery

func ExpandQuery(query string) string

ExpandQuery adds related terms to improve BM25 recall. Handles common coding synonyms and abbreviations.

Examples:

"auth" → "auth authentication authorize"
"db" → "db database"
"config" → "config configuration settings"

func ExportHTML

func ExportHTML(ctx context.Context, store storage.Storage) (string, error)

ExportHTML generates a standalone HTML file with an interactive memory graph. The file can be shared — no server required, opens in any browser.

func FormatContext

func FormatContext(nodes []*storage.Node) string

FormatContext formats nodes as a markdown context block for injection. Uses tiered injection (Cursor-inspired):

  • Pinned: full content always shown
  • Hot tier (tier 1): full content
  • Warm tier (tier 2): summary/gloss only (full content available via recall)
  • Cold tier: omitted from context

func GarbageCollect

func GarbageCollect(ctx context.Context, store storage.Storage, cfg DecayConfig) (int, error)

GarbageCollect removes nodes below min_confidence (except anchors: file/entity). Processes nodes in pages to avoid loading the entire graph into memory. IDs are collected first, then deleted in a separate pass to avoid offset drift when rows shift during deletion.

func GracefulShutdown

func GracefulShutdown(e *Engine)

GracefulShutdown flushes all pending operations before closing.

func IsValidNodeType

func IsValidNodeType(typ string) bool

IsValidNodeType reports whether typ is a recognized node type.

func IsWellSpaced

func IsWellSpaced(accessTimes []time.Time, config SpacingConfig) bool

func MMRRerank

func MMRRerank(nodes []*storage.Node, scores map[string]float64, lambda float64, k int) []*storage.Node

MMRRerank applies Maximal Marginal Relevance to diversify results. MMR = λ * relevance(doc) - (1-λ) * max_sim(doc, selected) Prevents near-duplicate results from dominating the output.

func NormalizeBM25

func NormalizeBM25(rawScore float64, queryTermCount int) float64

NormalizeBM25 applies sigmoid normalization to raw BM25 scores with query-length-adaptive parameters. Based on Mem0 v3's approach.

Short queries (1-3 terms): midpoint=5.0, steepness=0.7 Medium queries (4-6 terms): midpoint=7.0, steepness=0.6 Long queries (7-14 terms): midpoint=10.0, steepness=0.5 Very long queries (15+): midpoint=12.0, steepness=0.4

func QueryTermCount

func QueryTermCount(query string) int

QueryTermCount counts significant terms in a query (for BM25 normalization).

func RankedNodesToNodes

func RankedNodesToNodes(ranked []*RankedNode) []*storage.Node

RankedNodesToNodes extracts the underlying nodes from ranked results.

func RewriteQuery

func RewriteQuery(ctx context.Context, llm KeygenLLM, strategy KeygenStrategy, query string) string

RewriteQuery turns a raw user query into the search string used for recall, applying the given strategy. llm may be nil; when it is, the function behaves exactly as KeygenSynonyms regardless of strategy, so callers can wire an LLM optionally without branching. The returned string is always non-empty as long as query is non-empty.

Defaults here are yaad's own — deliberately not borrowed from any external framework's RAG config — and the whole step is best-effort: any LLM failure degrades gracefully to local synonym expansion rather than erroring out of a recall.

func RunDecay

func RunDecay(ctx context.Context, store storage.Storage, cfg DecayConfig) error

RunDecay applies half-life decay to all nodes in the store. Orphan nodes (0 edges) and superseded nodes decay 2× faster. Processes nodes in pages to avoid loading the entire graph into memory.

func ShortestPath

func ShortestPath(ctx context.Context, store storage.Storage, fromID, toID string) ([]string, error)

ShortestPath finds the shortest path between two nodes using BFS. Returns the path as a list of node IDs, or nil if no path exists.

BFS proceeds one frontier level at a time and fetches every edge touching the whole frontier in a single GetAllEdgesFor call, rather than two queries (GetEdgesFrom + GetEdgesTo) per node. That turns O(V) round-trips into O(depth), matching the batched pattern already used by IntentBFS/PageRank.

func SpacingScore

func SpacingScore(accessTimes []time.Time, config SpacingConfig) float64

func SpreadAttenuatedBoost

func SpreadAttenuatedBoost(similarity float64, numLinked int, decay float64) float64

SpreadAttenuatedBoost computes entity boost with spread attenuation. Penalizes entities that link to many memories (too generic). Based on Mem0 v3: boost = similarity * 0.5 * 1/(1 + decay*(numLinked-1)²)

func StartupSelfTest

func StartupSelfTest(ctx context.Context, store storage.Storage) error

StartupSelfTest runs quick smoke tests to verify system integrity. Returns nil if all checks pass, error describing what's wrong otherwise.

func SupersedeEdge

func SupersedeEdge(edge *TemporalEdge)

func TimelineFilter

func TimelineFilter(limit int) storage.NodeFilter

TimelineFilter returns a NodeFilter suitable for timeline display.

func TrimToTokenBudget

func TrimToTokenBudget(nodes []*storage.Node, budget int) []*storage.Node

TrimToTokenBudget trims a node list to fit within a token budget. Approximation: 1 token ≈ 4 characters.

Types

type AccessTracker

type AccessTracker struct {
	// contains filtered or unexported fields
}

AccessTracker batches node access events to reduce SQLite UPDATE churn. Instead of updating nodes.access_count on every recall (which causes write contention under concurrent load), it INSERTs lightweight rows into access_log and periodically flushes them in a single aggregated UPDATE.

func NewAccessTracker

func NewAccessTracker(store storage.Storage, interval time.Duration) *AccessTracker

NewAccessTracker creates a tracker that flushes every interval.

func (*AccessTracker) Flush

func (at *AccessTracker) Flush(ctx context.Context)

Flush immediately applies all pending access counts to nodes. Used by the periodic flusher and once more on shutdown so no buffered access is lost.

func (*AccessTracker) Log

func (at *AccessTracker) Log(_ context.Context, nodeID string)

Log records an access for the given node ID (best-effort, non-blocking).

The access is appended to an in-memory buffer and returns immediately — it does NOT touch SQLite on the calling path. The periodic flusher (loop) drains the buffer on its ticker. This is the whole point of the tracker: a single Recall returns many nodes and previously issued one synchronous INSERT per node, N writes per query all contending for the single SQLite writer. Access counts feed decay/recency ranking, not correctness, so deferring them is safe.

On buffer overflow the buffer is drained synchronously in a detached goroutine so a pathological burst can never grow memory without bound.

func (*AccessTracker) Stop

func (at *AccessTracker) Stop()

Stop halts the background flusher and waits for all in-flight background goroutines (the flush loop and any overflow drains) to finish writing to the store. After Stop returns, no AccessTracker goroutine will touch the store, so the caller may safely Close() it. Safe to call multiple times; the wait happens once.

type AffectedNode

type AffectedNode struct {
	Node         *storage.Node `json:"node"`
	RelationType string        `json:"relation_type"`
	Distance     int           `json:"distance"`
	Confidence   float64       `json:"confidence"`
}

AffectedNode is a single affected memory with its relationship to the source.

type AgentFileBridge

type AgentFileBridge struct {
	// contains filtered or unexported fields
}

AgentFileBridge synchronizes yaad memories with agent instruction files (CLAUDE.md, .cursorrules, AGENTS.md). It supports import, export, and bidirectional sync modes.

func NewAgentFileBridge

func NewAgentFileBridge(projectDir string) *AgentFileBridge

NewAgentFileBridge creates a bridge for the given project directory.

func (*AgentFileBridge) Export

func (afb *AgentFileBridge) Export(rules []AgentRule, fileType AgentFileType) error

Export writes rules back to the specified agent file.

func (*AgentFileBridge) Import

func (afb *AgentFileBridge) Import() ([]AgentRule, error)

Import reads agent instruction files and returns extracted rules.

func (*AgentFileBridge) Sync

func (afb *AgentFileBridge) Sync(conventions []string) error

Sync performs bidirectional sync between yaad and agent files. It imports new rules from files and exports yaad conventions back.

func (*AgentFileBridge) Watch

func (afb *AgentFileBridge) Watch() []string

Watch detects changes in agent files and returns modified file paths.

type AgentFileType

type AgentFileType string

AgentFileType represents the type of agent instruction file.

const (
	FileClaudeMD    AgentFileType = "CLAUDE.md"
	FileCursorRules AgentFileType = ".cursorrules"
	FileAgentsMD    AgentFileType = "AGENTS.md"
	FileHawkMD      AgentFileType = ".hawk.md"
)

type AgentRule

type AgentRule struct {
	Content string        `json:"content"`
	Type    string        `json:"type"`   // "convention", "preference", "constraint"
	Source  AgentFileType `json:"source"` // which file it came from
	Line    int           `json:"line"`   // line number in source file
}

AgentRule represents a single rule/convention from an agent file.

type AuditEntry

type AuditEntry struct {
	Timestamp time.Time `json:"ts"`
	Operation string    `json:"op"` // remember, recall, forget, link, feedback
	NodeID    string    `json:"node_id,omitempty"`
	NodeType  string    `json:"node_type,omitempty"`
	Agent     string    `json:"agent,omitempty"`
	Details   string    `json:"details,omitempty"`
}

AuditEntry is a single audit record.

type AuditLog

type AuditLog struct {
	// contains filtered or unexported fields
}

AuditLog records all memory operations for traceability. Stores: who did what, when, on which node. Persists to .yaad/audit.jsonl (append-only).

func NewAuditLog

func NewAuditLog(yaadDir string) (*AuditLog, error)

NewAuditLog creates or opens the audit log file.

func (*AuditLog) Close

func (al *AuditLog) Close()

Close flushes and closes the audit log.

func (*AuditLog) Count

func (al *AuditLog) Count() int

Count returns number of entries logged this session.

func (*AuditLog) Log

func (al *AuditLog) Log(op, nodeID, nodeType, agent, details string)

Log records an operation.

type BloomFilter

type BloomFilter struct {
	// contains filtered or unexported fields
}

BloomFilter provides O(1) negative lookups for memory content. If the filter says "not present", it's guaranteed not in the store. If it says "maybe present", we proceed with the FTS5 query. Eliminates ~40-60% of unnecessary SQLite queries for miss cases.

Parameters tuned for coding memory: - Expected items: 10,000 memories - False positive rate: 1% - Bit array size: ~96KB (minimal memory overhead)

func NewBloomFilter

func NewBloomFilter(expectedItems int, fpRate float64) *BloomFilter

NewBloomFilter creates a bloom filter optimized for the expected number of items.

func (*BloomFilter) Add

func (bf *BloomFilter) Add(term string)

Add inserts a term into the bloom filter.

func (*BloomFilter) AddTerms

func (bf *BloomFilter) AddTerms(content string)

AddTerms indexes all significant terms from content.

func (*BloomFilter) Count

func (bf *BloomFilter) Count() int

Count returns the number of items added.

func (*BloomFilter) FalsePositiveRate

func (bf *BloomFilter) FalsePositiveRate() float64

FalsePositiveRate returns the estimated current FP rate.

func (*BloomFilter) MayContain

func (bf *BloomFilter) MayContain(term string) bool

MayContain returns false if the term is DEFINITELY not in the filter. Returns true if it MIGHT be present (check FTS5 to confirm).

func (*BloomFilter) MayContainAny

func (bf *BloomFilter) MayContainAny(query string) bool

MayContainAny returns true if any of the query terms might be present.

type BoundaryConfig

type BoundaryConfig struct {
	// SimThreshold is the similarity floor below which a topic shift is flagged.
	// For embedding-based detection this is cosine similarity in [-1, 1]; for the
	// lexical fallback it is Jaccard similarity in [0, 1]. Range: [0, 1].
	SimThreshold float64 `json:"sim_threshold"`
	// WindowSize is the number of recent observations the centroid tracks.
	WindowSize int `json:"window_size"`
	// Alpha is the EMA weight for incorporating a new observation into the
	// centroid (0 = ignore new, 1 = replace). Range: (0, 1].
	Alpha float64 `json:"alpha"`
}

BoundaryConfig tunes semantic boundary (topic-shift) detection.

A boundary is flagged when the similarity between incoming content and the running centroid of recent content drops below SimThreshold. The centroid is an exponential moving average over the last WindowSize observations, which keeps the detector responsive to gradual drift while smoothing out single noisy messages.

func DefaultBoundaryConfig

func DefaultBoundaryConfig() BoundaryConfig

DefaultBoundaryConfig returns conservative defaults tuned to avoid flagging normal same-topic variation as a boundary.

type BoundaryDetector

type BoundaryDetector struct {
	// contains filtered or unexported fields
}

BoundaryDetector detects semantic boundaries between two pieces of content. It uses an embeddings.Provider when available and falls back to a lexical (Jaccard) comparison otherwise. The detector itself is stateless; use Tracker for rolling, stateful detection over a stream.

func NewBoundaryDetector

func NewBoundaryDetector(provider embeddings.Provider, cfg BoundaryConfig) *BoundaryDetector

NewBoundaryDetector creates a detector. provider may be nil, in which case the lexical Jaccard fallback is used for all comparisons.

func (*BoundaryDetector) DetectBoundary

func (d *BoundaryDetector) DetectBoundary(prev, next string) bool

DetectBoundary reports whether next represents a topic shift relative to prev. Empty inputs are never treated as boundaries (insufficient signal).

func (*BoundaryDetector) Similarity

func (d *BoundaryDetector) Similarity(a, b string) float64

Similarity returns the similarity between two pieces of content in [0, 1]. It uses the embedding provider when present (cosine, clamped to [0, 1]) and falls back to Jaccard token overlap otherwise.

type CandidateCluster

type CandidateCluster struct {
	Nodes   []*storage.Node
	Score   float64  // combined similarity score
	Reasons []string // human-readable reasons for clustering
}

CandidateCluster is a group of nodes that are potential merge candidates.

type Community

type Community struct {
	ID      int
	NodeIDs []string
	Summary string
	Size    int
	Types   map[string]int // type → count
}

Community represents a cluster of related memories.

type CommunityDetector

type CommunityDetector struct {
	// contains filtered or unexported fields
}

CommunityDetector finds clusters of related memories in the graph using label propagation (lightweight, no external deps). Each community gets a summary enabling "what are the main themes?" global queries.

Based on GraphRAG (Microsoft): community detection → summarization → global retrieval.

func NewCommunityDetector

func NewCommunityDetector(store storage.Storage) *CommunityDetector

NewCommunityDetector creates a detector.

func (*CommunityDetector) Detect

func (cd *CommunityDetector) Detect(ctx context.Context, maxIter int) ([]Community, error)

Detect finds communities using iterative label propagation. Each node starts with its own label. Iteratively, each node adopts the most frequent label among its neighbors. Converges in ~5 iterations.

func (*CommunityDetector) FindCommunityForQuery

func (cd *CommunityDetector) FindCommunityForQuery(communities []Community, query string) *Community

FindCommunityForQuery returns the most relevant community for a query.

func (*CommunityDetector) Summarize

func (cd *CommunityDetector) Summarize(ctx context.Context, communities []Community) []Community

Summarize generates a summary for each community based on its node contents.

type ConflictCandidate

type ConflictCandidate struct {
	NodeID  string
	Content string
	Score   float64
}

ConflictCandidate represents a potential conflicting node found via FTS.

type ConsolidateResult

type ConsolidateResult struct {
	Plans   []*MergePlan
	Applied int
	Skipped int
	DryRun  bool
}

ConsolidateResult holds the outcome of a consolidation run.

type ConsolidationConfig

type ConsolidationConfig struct {
	Enabled        bool    `json:"enabled"`
	MinClusterSize int     `json:"min_cluster_size"`
	MaxClusters    int     `json:"max_clusters"`
	OverlapThres   float64 `json:"overlap_thres"`
}

ConsolidationConfig holds configuration for topic consolidation.

func DefaultConsolidationConfig

func DefaultConsolidationConfig() ConsolidationConfig

DefaultConsolidationConfig returns production-tuned defaults.

type ConsolidationLLM

type ConsolidationLLM interface {
	// SuggestMerge examines a cluster of related nodes and proposes a merge plan.
	SuggestMerge(ctx context.Context, nodes []*storage.Node) (*MergePlan, error)
	// Summarize produces a concise summary of a group of nodes.
	Summarize(ctx context.Context, nodes []*storage.Node) (string, error)
}

ConsolidationLLM is the interface callers must implement to provide LLM capabilities for semantic memory consolidation. This keeps yaad free of any direct LLM dependency.

type ContextPacker

type ContextPacker struct {
	// contains filtered or unexported fields
}

ContextPacker optimizes what gets injected into an agent's context window. Packs maximum information density within a token budget by:

  • Prioritizing by PageRank importance + confidence + recency
  • Deduplicating similar content
  • Compressing verbose memories into concise forms
  • Respecting type quotas (max N conventions, M decisions, etc.)

func NewContextPacker

func NewContextPacker(budget int) *ContextPacker

NewContextPacker creates a packer with the given token budget.

func (*ContextPacker) Pack

func (cp *ContextPacker) Pack(nodes []*storage.Node) *PackedContext

Pack selects and formats the optimal subset of nodes for the context window.

type ContextualScoring

type ContextualScoring struct {
	// contains filtered or unexported fields
}

ContextualScoring boosts memories that match the current session's topic. Detects the session topic from recent queries and boosts related content.

func NewContextualScoring

func NewContextualScoring() *ContextualScoring

NewContextualScoring creates a contextual scorer.

func (*ContextualScoring) LastActivity

func (cs *ContextualScoring) LastActivity() time.Time

LastActivity returns when the last query was recorded.

func (*ContextualScoring) RecordQuery

func (cs *ContextualScoring) RecordQuery(query string)

RecordQuery adds a query to the topic window.

func (*ContextualScoring) Reset

func (cs *ContextualScoring) Reset()

Reset clears topic history (call at session start).

func (*ContextualScoring) TopicBoost

func (cs *ContextualScoring) TopicBoost(node *storage.Node) float64

TopicBoost returns a multiplier for nodes that relate to recent session topics.

type ConversationSummarizer

type ConversationSummarizer struct{}

ConversationSummarizer compresses multi-turn conversations into single memories. Used when ingesting long transcripts — extracts the key takeaways.

func (*ConversationSummarizer) Summarize

func (cs *ConversationSummarizer) Summarize(conversation string, maxMemories int) []SummarizedMemory

Summarize extracts key points from a conversation into concise memories.

type DecayConfig

type DecayConfig struct {
	HalfLifeDays  float64 // default 30
	MinConfidence float64 // default 0.1 — below this, eligible for GC
	BoostOnAccess float64 // default 0.2
}

DecayConfig controls decay behaviour.

type DecayScheduler

type DecayScheduler struct {
	// contains filtered or unexported fields
}

DecayScheduler runs decay and garbage collection on a periodic schedule. It applies half-life decay to memory confidence scores and removes nodes that fall below the minimum confidence threshold.

func NewDecayScheduler

func NewDecayScheduler(store storage.Storage, cfg DecayConfig, interval time.Duration, gcEnabled bool) *DecayScheduler

NewDecayScheduler creates a scheduler that runs decay at the given interval. gcEnabled controls whether garbage collection runs after each decay pass.

func (*DecayScheduler) Start

func (ds *DecayScheduler) Start()

Start begins the periodic decay loop. It runs decay immediately on start, then at the configured interval. Returns immediately (non-blocking).

func (*DecayScheduler) Stop

func (ds *DecayScheduler) Stop()

Stop halts the scheduler. Safe to call multiple times.

type EdgeInput

type EdgeInput struct {
	ToID string
	Type string
}

EdgeInput describes an edge to create alongside a node.

type Engine

type Engine struct {
	DecayConfig DecayConfig
	// contains filtered or unexported fields
}

Engine is the core memory engine wrapping graph + storage.

func New

func New(store storage.Storage, g graph.Graph) *Engine

New creates a memory engine.

func (*Engine) ActiveProspective

func (e *Engine) ActiveProspective() []*cognitive.ProspectiveMemory

ActiveProspective returns all untriggered, unexpired prospective memories.

func (*Engine) AddOnly

func (e *Engine) AddOnly() bool

AddOnly reports whether single-pass ADD-only ingestion is enabled.

func (*Engine) AnalyzeImpact

func (e *Engine) AnalyzeImpact(ctx context.Context, nodeID string, maxDepth int) (*ImpactAnalysis, error)

AnalyzeImpact traces all memories downstream of a changed node and reports what would be affected. Walks both forward (led_to, part_of) and backward (caused_by, supersedes) edges to find the full impact surface.

func (*Engine) Boundary

func (e *Engine) Boundary() *cognitive.BoundaryDetector

Boundary returns the semantic boundary detector (may be nil).

func (*Engine) Branch

func (e *Engine) Branch(ctx context.Context, nodeID, newContent, newType string) (*storage.Node, error)

Branch creates a new node branching off from the given nodeID. The new node inherits the parent's project and scope but carries the provided newContent and newType. An edge of type "caused_by" links the branch node back to its parent, forming a DAG branch.

func (*Engine) CheckTriggers

func (e *Engine) CheckTriggers(messageText string) []*cognitive.ProspectiveMemory

CheckTriggers checks prospective memory triggers against the given text. Returns triggered memories (nil if Prospective engine is not wired).

func (*Engine) Close

func (e *Engine) Close()

Close shuts down the engine and its background workers.

func (*Engine) CognitiveStatus

func (e *Engine) CognitiveStatus() string

CognitiveStatus returns a summary of all cognitive engine states.

func (*Engine) Compact

func (e *Engine) Compact(ctx context.Context, project string) (int, error)

Compact merges low-confidence memories to keep the graph lean.

func (*Engine) CompressSession

func (e *Engine) CompressSession(ctx context.Context, sessionID, project string) (*storage.Node, error)

CompressSession creates a session summary node linking all memories from the session.

func (*Engine) CompressSessionEvents

func (e *Engine) CompressSessionEvents(ctx context.Context, sessionID string) (int, error)

CompressSessionEvents compresses replay events for a session to reduce storage. It reads all events for the session, compresses the data payloads, and stores a single compressed summary event. Returns the number of events compressed.

func (*Engine) ConsolidateTopics

func (e *Engine) ConsolidateTopics(ctx context.Context, project string) ([]*TopicCluster, error)

ConsolidateTopics groups related memories into topic clusters.

func (*Engine) ConsolidationCycles

func (e *Engine) ConsolidationCycles() int64

ConsolidationCycles reports how many consolidation cycles have completed since the loop started. Returns 0 when consolidation has never been started or has been stopped. Primarily for observability and tests.

func (*Engine) Consolidator

func (e *Engine) Consolidator() *TopicConsolidator

Consolidator returns the topic consolidator (may be nil).

func (*Engine) Context

func (e *Engine) Context(ctx context.Context, project string) (*RecallResult, error)

Context returns the hot-tier subgraph for session start injection. Pinned nodes always appear first (guaranteed 500-token budget), followed by hot-tier and active tasks filling the remaining budget.

func (*Engine) CreateEpistemicDirective

func (e *Engine) CreateEpistemicDirective(dtype cognitive.DirectiveType, question, context string, priority float64) *cognitive.EpistemicDirective

CreateEpistemicDirective creates a new knowledge gap directive.

func (*Engine) CreateProspective

func (e *Engine) CreateProspective(triggerCondition, action, sourceSession string, priority float64) *cognitive.ProspectiveMemory

CreateProspective creates a new prospective memory entry.

func (*Engine) Curiosity

func (e *Engine) Curiosity() *cognitive.CuriosityEngine

Curiosity returns the exploration target engine (may be nil).

func (*Engine) DecayScheduler

func (e *Engine) DecayScheduler() *DecayScheduler

DecayScheduler returns the periodic decay scheduler (may be nil).

func (*Engine) DetectTopicBoundaries

func (e *Engine) DetectTopicBoundaries(messages []string) []cognitive.TopicBoundary

DetectTopicBoundaries detects topic shifts in a conversation.

func (*Engine) DiffVersions

func (e *Engine) DiffVersions(ctx context.Context, nodeID string, v1, v2 int) (content1, content2 string, err error)

DiffVersions returns the content of two versions for comparison.

func (*Engine) Epistemic

func (e *Engine) Epistemic() *cognitive.EpistemicEngine

Epistemic returns the knowledge gap detection engine (may be nil).

func (*Engine) EpistemicDirectives

func (e *Engine) EpistemicDirectives() []*cognitive.EpistemicDirective

EpistemicDirectives returns pending knowledge gap directives.

func (*Engine) ExplorationTargets

func (e *Engine) ExplorationTargets(limit int) []*cognitive.ExplorationTarget

ExplorationTargets returns top curiosity-driven exploration targets.

func (*Engine) Feedback

func (e *Engine) Feedback(ctx context.Context, id string, action FeedbackAction, newContent string) error

Feedback applies user feedback to a memory node.

func (*Engine) FilterContent

func (e *Engine) FilterContent(content string) string

FilterContent applies the engine's privacy filter to content. Uses the engine's configured privacy level rather than the package default.

func (*Engine) FindConflictCandidates

func (e *Engine) FindConflictCandidates(ctx context.Context, nodeID, content string, limit int) ([]ConflictCandidate, error)

FindConflictCandidates searches for existing nodes that may conflict with the given content. Uses FTS search scoped to the same node type, returning candidates above a score threshold.

func (*Engine) Forget

func (e *Engine) Forget(ctx context.Context, id string) error

Forget archives a node by setting confidence to 0.

func (*Engine) FusedRecall

func (e *Engine) FusedRecall(ctx context.Context, opts RecallOpts) (*RecallResult, error)

FusedRecall performs multi-signal retrieval combining:

  1. BM25 keyword search (via existing SearchNodes FTS5)
  2. Graph traversal (via existing IntentBFS from seed nodes)
  3. Recency signal (recently accessed/modified nodes)

Then fuses results using Reciprocal Rank Fusion (RRF).

This is inspired by mem0's multi-signal retrieval architecture which combines semantic search, BM25, and entity-graph traversal with RRF to produce higher-quality recall than any single signal alone.

Unlike the existing Recall method (which uses BM25 seeds then graph expansion with a confidence*recency heuristic), FusedRecall treats each signal as an independent ranked list and merges them with a principled rank-fusion algorithm. This avoids the score-space mismatch problem where BM25 scores, graph distances, and timestamps are on different scales.

func (*Engine) GetMemoryStats

func (e *Engine) GetMemoryStats(ctx context.Context) (*MemoryStats, error)

GetMemoryStats returns aggregate statistics about the memory graph.

func (*Engine) GetMetrics

func (e *Engine) GetMetrics() Metrics

GetMetrics returns a copy of the engine's operational metrics.

func (*Engine) GetNodeHistory

func (e *Engine) GetNodeHistory(ctx context.Context, nodeID string) ([]NodeHistoryEntry, error)

GetNodeHistory returns the version history of a memory node.

func (*Engine) GetUserProfile

func (e *Engine) GetUserProfile(ctx context.Context, project string) (*UserProfile, error)

GetUserProfile returns the user profile for a project, built from preference nodes.

func (*Engine) Graph

func (e *Engine) Graph() graph.Graph

Graph returns the underlying graph engine.

func (*Engine) Integrity

func (e *Engine) Integrity() *MemoryIntegrity

Integrity returns the engine's memory integrity checker (may be nil).

func (*Engine) LLMExtractor

func (e *Engine) LLMExtractor() *LLMExtractor

LLMExtractor returns the engine's LLM entity extractor (may be nil).

func (*Engine) MentalModel

func (e *Engine) MentalModel(ctx context.Context, project string) (*mental.Model, error)

MentalModel generates an auto-evolving project summary.

func (*Engine) ObserveWorkflow

func (e *Engine) ObserveWorkflow(tools []string, trigger string, success bool) *cognitive.ProceduralMemory

ObserveWorkflow records a full tool sequence as a learned procedural pattern.

func (*Engine) OpenLoops

func (e *Engine) OpenLoops(limit int) []*cognitive.OpenLoop

OpenLoops returns active (unresolved) Zeigarnik loops. Returns nil if Zeigarnik engine is not wired.

func (*Engine) PendingNodes

func (e *Engine) PendingNodes(ctx context.Context, project string, threshold float64) ([]*storage.Node, error)

PendingNodes returns low-confidence nodes that may need review. Limited to 1000 nodes to prevent unbounded memory use on large graphs.

func (*Engine) PrivacyLevel

func (e *Engine) PrivacyLevel() privacy.FilterLevel

PrivacyLevel returns the current privacy filter level.

func (*Engine) Procedural

func (e *Engine) Procedural() *cognitive.ProceduralEngine

Procedural returns the procedural memory engine (may be nil).

func (*Engine) Profile

func (e *Engine) Profile(ctx context.Context, project string) (*profile.Profile, error)

Profile returns an auto-maintained user/project profile (static facts + dynamic context).

func (*Engine) Prospective

func (e *Engine) Prospective() *cognitive.ProspectiveEngine

Prospective returns the trigger-based future memory engine (may be nil).

func (*Engine) Query

func (e *Engine) Query(ctx context.Context, question string, project string) (*QueryResult, error)

Query answers a natural language question by retrieving relevant memories and synthesizing an answer from them. No LLM required — uses template-based synthesis from high-confidence retrieved nodes.

How it works:

  1. Parse the question — classify intent using the existing intent package
  2. Retrieve — call FusedRecall with the question as the query
  3. Rank and filter — keep only nodes with confidence > 0.3
  4. Synthesize — format retrieved memories into a coherent answer using intent-specific templates
  5. Calculate confidence — average confidence of the used nodes

func (*Engine) RankWithFactors

func (e *Engine) RankWithFactors(ctx context.Context, nodes []*storage.Node, baseScores map[string]float64) []*RankedNode

RankWithFactors re-scores recalled memories using multi-factor ranking.

func (*Engine) Ranker

func (e *Engine) Ranker() *MultiFactorRanker

Ranker returns the multi-factor ranker (may be nil).

func (*Engine) Recall

func (e *Engine) Recall(ctx context.Context, opts RecallOpts) (*RecallResult, error)

Recall performs graph-aware hybrid search: BM25 seed → graph expand → rank.

func (*Engine) RecallWithConfidence

func (e *Engine) RecallWithConfidence(ctx context.Context, opts RecallOpts) (*ScoredRecallResult, error)

RecallWithConfidence performs recall and computes explicit confidence scores for each returned memory. The composite confidence combines:

  • Trust: the node's stored confidence (from feedback, decay, etc.)
  • Recency: exponential decay based on last access/update
  • Centrality: graph connectivity (more connections = more trustworthy)
  • Retrieval rank: how highly the memory scored in retrieval

func (*Engine) RecallWithKeygen

func (e *Engine) RecallWithKeygen(ctx context.Context, llm KeygenLLM, strategy KeygenStrategy, opts RecallOpts) (*RecallResult, error)

RecallWithKeygen rewrites opts.Query via RewriteQuery and then runs Recall on the rewritten query, leaving every other RecallOpts field untouched. It is a thin convenience over Recall for callers that have an LLM available; callers without one can keep using Recall directly.

func (*Engine) Reconsolidation

func (e *Engine) Reconsolidation() *cognitive.ReconsolidationEngine

Reconsolidation returns the memory labile window engine (may be nil).

func (*Engine) RecordSomaticOutcome

func (e *Engine) RecordSomaticOutcome(region string, success bool)

RecordSomaticOutcome records a success/failure outcome for a memory region.

func (*Engine) RecordWorkflowStep

func (e *Engine) RecordWorkflowStep(toolName string)

RecordWorkflowStep records a tool-use step for procedural pattern learning.

func (*Engine) Remember

func (e *Engine) Remember(ctx context.Context, in RememberInput) (*storage.Node, error)

Remember creates a memory node with privacy filtering, dedup, and entity extraction.

func (*Engine) RememberRule

func (e *Engine) RememberRule(ctx context.Context, input RuleInput) (*storage.Node, error)

RememberRule stores a rule file as a yaad convention node. Uses keyed upsert (Key: "rule:<path>") so re-reading the same file updates rather than duplicates. Globs are stored in Tags for searchability.

func (*Engine) ResolveEpistemic

func (e *Engine) ResolveEpistemic(directiveID, resolution string) bool

ResolveEpistemic marks a knowledge gap directive as resolved.

func (*Engine) ResolveLoop

func (e *Engine) ResolveLoop(chunkID string)

ResolveLoop marks a Zeigarnik loop as resolved.

func (*Engine) Rollback

func (e *Engine) Rollback(ctx context.Context, id string, version int) error

Rollback restores a node to a previous version.

func (*Engine) ScanForOpenLoops

func (e *Engine) ScanForOpenLoops(chunks []cognitive.ZeigarnikChunk) int

ScanForOpenLoops scans text chunks for open loops and marks them. Returns the number of new loops detected.

func (*Engine) SearchDecay

func (e *Engine) SearchDecay() *SearchTimeDecay

SearchDecay returns the engine's search-time decay configuration (may be nil).

func (*Engine) SegmentConversation

func (e *Engine) SegmentConversation(messages []string) [][]string

SegmentConversation splits a conversation into topic-coherent segments.

func (e *Engine) SelfLink(ctx context.Context, node *storage.Node)

SelfLink finds existing nodes semantically related to the new node and creates typed edges. Inspired by A-MEM's Zettelkasten self-organizing approach. Runs as a best-effort step during Remember() — failures don't block storage.

NOTE: This runs outside the write lock to avoid blocking concurrent operations. A TOCTOU race exists between the edge existence check and AddEdge. We mitigate by re-checking edges immediately before each AddEdge call. Duplicate edges from truly concurrent SelfLink calls are benign (graph.AddEdge is idempotent for the same from/to/type).

func (*Engine) ShouldSkipSomatic

func (e *Engine) ShouldSkipSomatic(region string) bool

ShouldSkipSomatic returns true if a region should be skipped based on past failures.

func (*Engine) Somatic

func (e *Engine) Somatic() *cognitive.SomaticEngine

Somatic returns the valence/arousal outcome tracking engine (may be nil).

func (*Engine) StartConsolidation

func (e *Engine) StartConsolidation(ctx context.Context, interval time.Duration) bool

StartConsolidation launches a background "sleep-time" consolidation loop that runs during idle periods. Each cycle:

  1. applies half-life decay (RunDecay) over the whole store,
  2. re-runs conflict detection over recently-updated nodes, and
  3. compacts low-confidence memories per active project.

The loop runs at the given interval (the first cycle fires after one interval, not immediately, so startup stays cheap). It is OFF by default and only runs once explicitly started. Calling StartConsolidation while a loop is already running is a no-op and returns false.

Cancellation: the loop stops when ctx is cancelled or StopConsolidation is called, whichever comes first. All writes performed by a cycle reuse the engine's existing locking model (e.mu for the conflict re-scan, the store's WAL single-writer for decay/compaction), so no new write races are introduced.

func (*Engine) StartSession

func (e *Engine) StartSession(ctx context.Context, project, agent string) (string, error)

StartSession creates a new session record and returns its ID.

func (*Engine) Status

func (e *Engine) Status(ctx context.Context, project string) (*Status, error)

func (*Engine) StopConsolidation

func (e *Engine) StopConsolidation()

StopConsolidation halts a running consolidation loop and waits for the current cycle to finish. Safe to call when no loop is running (no-op) and safe to call multiple times.

func (*Engine) Store

func (e *Engine) Store() storage.Storage

Store returns the underlying store.

func (*Engine) SubscribeMemoryEvents

func (e *Engine) SubscribeMemoryEvents() (<-chan MemoryEvent, func())

SubscribeMemoryEvents registers a subscriber for memory mutation events (created/updated/deleted). It returns a receive-only channel and an unsubscribe function the caller must invoke when done. The channel is buffered; events are dropped for subscribers that fall behind.

func (*Engine) SuggestWorkflowSteps

func (e *Engine) SuggestWorkflowSteps(prefix []string, limit int) []string

SuggestWorkflowSteps returns the most likely next steps given observed tool names.

func (*Engine) UpdateUserPreference

func (e *Engine) UpdateUserPreference(ctx context.Context, project, key, value string) error

UpdateUserPreference stores or updates a single preference as a yaad node.

func (*Engine) VectorSearch

func (e *Engine) VectorSearch(query []float32, k int) []string

VectorSearch performs HNSW-accelerated nearest neighbor search. Returns node IDs ranked by vector similarity. Lazily loads the HNSW index from disk if a save path is configured.

func (*Engine) WithAddOnly

func (e *Engine) WithAddOnly(on bool) *Engine

WithAddOnly toggles Mem0-v3-style single-pass ADD ingestion. When enabled, Remember skips conflict resolution / supersede so contradictory memories both persist as independent nodes. Default is false (conflict resolution on).

func (*Engine) WithAuditLog

func (e *Engine) WithAuditLog(al *AuditLog) *Engine

WithAuditLog wires an audit log into the engine for operation tracing.

func (*Engine) WithBoundaryDetector

func (e *Engine) WithBoundaryDetector(cfg cognitive.BoundaryConfig) *Engine

WithBoundaryDetector wires the semantic boundary detector.

func (*Engine) WithConsolidator

func (e *Engine) WithConsolidator(cfg ConsolidationConfig) *Engine

WithConsolidator wires the topic consolidation engine.

func (*Engine) WithCuriosity

func (e *Engine) WithCuriosity(cfg cognitive.CuriosityConfig) *Engine

WithCuriosity wires the exploration target engine.

func (*Engine) WithDecayScheduler

func (e *Engine) WithDecayScheduler(interval time.Duration, gcEnabled bool) *Engine

WithDecayScheduler wires a periodic decay and garbage collection scheduler. The scheduler runs decay at the configured interval and optionally runs GC after each decay pass. Call Start() on the returned scheduler to begin.

func (*Engine) WithEmbedder

func (e *Engine) WithEmbedder(p embeddings.Provider) *Engine

WithEmbedder wires an embedding provider so the fused-recall vector path can embed the query text itself (true semantic retrieval). Without it, the vector signal falls back to using the top BM25 hit's stored embedding as a proxy, which biases the vector results toward what BM25 already found.

func (*Engine) WithEpistemic

func (e *Engine) WithEpistemic(cfg cognitive.EpistemicConfig) *Engine

WithEpistemic wires the knowledge gap detection engine.

func (*Engine) WithHNSWPath

func (e *Engine) WithHNSWPath(path string) *Engine

WithHNSWPath sets the path for persisting the HNSW vector index to disk. If a saved index exists at this path, it is loaded at Engine creation time. The index is saved after every insert and periodically to this path.

func (*Engine) WithIntegrity

func (e *Engine) WithIntegrity(mi *MemoryIntegrity) *Engine

WithIntegrity sets the memory integrity checker for signature persistence.

func (*Engine) WithLLMExtractor

func (e *Engine) WithLLMExtractor(x *LLMExtractor) *Engine

WithLLMExtractor wires an LLM-based entity extractor. When set, Remember uses ExtractEntitiesWithLLM (LLM results merged with regex, deduped) for anchor-node extraction, falling back to regex-only when the LLM call fails. When nil (the default), entity extraction is regex-only — no behavior change for users without an LLM configured.

func (*Engine) WithMemoryFile

func (e *Engine) WithMemoryFile(path string) *Engine

WithMemoryFile sets a path for a human-readable MEMORY.md hot-pointer file. When set, Remember and Forget update this file with a summary of hot-tier memories, making memory inspectable without MCP tools or the database.

func (*Engine) WithMultiFactorRanker

func (e *Engine) WithMultiFactorRanker(cfg RankingWeights) *Engine

WithMultiFactorRanker wires the multi-factor re-ranking engine.

func (*Engine) WithPrivacyConfig

func (e *Engine) WithPrivacyConfig(cfg PrivacyConfig) *Engine

WithPrivacyConfig sets the full privacy configuration on the engine.

func (*Engine) WithPrivacyLevel

func (e *Engine) WithPrivacyLevel(level privacy.FilterLevel) *Engine

WithPrivacyLevel sets the privacy filter level on the engine. Levels: privacy.Strict, privacy.Moderate, privacy.Minimal.

func (*Engine) WithProcedural

func (e *Engine) WithProcedural(cfg cognitive.ProceduralConfig) *Engine

WithProcedural wires the procedural/workflow memory engine.

func (*Engine) WithProspective

func (e *Engine) WithProspective(cfg cognitive.ProspectiveConfig) *Engine

WithProspective wires the trigger-based future memory engine.

func (*Engine) WithReconsolidation

func (e *Engine) WithReconsolidation(cfg cognitive.ReconsolidationConfig) *Engine

WithReconsolidation wires the memory labile window engine.

func (*Engine) WithSearchDecay

func (e *Engine) WithSearchDecay(halfLife time.Duration) *Engine

WithSearchDecay enables non-destructive search-time decay re-ranking. When set, FusedRecall applies half-life exponential decay to result scores at query time WITHOUT modifying stored confidence values. This allows different queries to use different decay profiles and preserves the original confidence for audit and export.

func (*Engine) WithSomatic

func (e *Engine) WithSomatic(cfg cognitive.SomaticConfig) *Engine

WithSomatic wires the valence/arousal outcome tracking engine.

func (*Engine) WithSummarizer

func (e *Engine) WithSummarizer(s compact.Summarizer) *Engine

WithSummarizer sets a custom summarizer for compaction (e.g., LLM-backed).

func (*Engine) WithZeigarnik

func (e *Engine) WithZeigarnik(cfg cognitive.ZeigarnikConfig) *Engine

WithZeigarnik wires the open-loop detection engine.

func (*Engine) WorkflowPatterns

func (e *Engine) WorkflowPatterns(limit int) []*cognitive.ProceduralMemory

WorkflowPatterns returns all learned procedural patterns.

func (*Engine) Zeigarnik

func (e *Engine) Zeigarnik() *cognitive.ZeigarnikEngine

Zeigarnik returns the open-loop detection engine (may be nil).

type Entity

type Entity struct {
	Name string
	Type string // "file", "entity", "person", "project", "technology"
}

Entity represents an auto-extracted entity from content.

func ExtractEntities

func ExtractEntities(content string) []Entity

ExtractEntities pulls file paths, packages, functions, classes, people, technologies, and projects from content.

func ExtractEntitiesWithLLM

func ExtractEntitiesWithLLM(ctx context.Context, content string, extractor *LLMExtractor) []Entity

ExtractEntitiesWithLLM is the unified entity extraction entry point. When a non-nil extractor is supplied and its LLM call succeeds, the LLM-extracted entities are merged with the regex ExtractEntities output and deduplicated by (Type, normalized Name). If the extractor is nil or its call fails, it falls back to regex-only extraction. This preserves the exact regex behavior for users who have not configured an LLM.

Ordering is deterministic: regex entities first (preserving ExtractEntities order), then any LLM entities not already present.

type EntityIndex

type EntityIndex struct {
	// contains filtered or unexported fields
}

EntityIndex maintains a parallel index of entities (functions, files, libraries, classes, APIs) extracted from memory content. During retrieval, query entities are matched against this index to boost relevant memories.

Based on Mem0 v3's approach: extract entities on ingestion, boost on retrieval. But faster: regex-based extraction (no LLM call), O(1) lookup via map.

func NewEntityIndex

func NewEntityIndex() *EntityIndex

NewEntityIndex creates a new entity index.

func (*EntityIndex) BoostScores

func (ei *EntityIndex) BoostScores(query string) map[string]float64

BoostScores takes a query, extracts entities from it, and returns boost multipliers for node IDs that share entities with the query. Returns map[nodeID] → boost factor (1.0 = no boost, higher = more entity overlap).

func (*EntityIndex) IndexNode

func (ei *EntityIndex) IndexNode(node *storage.Node)

IndexNode extracts entities from a node and adds them to the index.

func (*EntityIndex) LoadFromStore

func (ei *EntityIndex) LoadFromStore(ctx context.Context, store storage.Storage) error

LoadFromStore rebuilds the entity index from all nodes in the store.

func (*EntityIndex) RemoveNode

func (ei *EntityIndex) RemoveNode(nodeID string)

RemoveNode removes a node from the entity index.

func (*EntityIndex) Size

func (ei *EntityIndex) Size() int

Size returns the number of unique entities tracked.

type FeedbackAction

type FeedbackAction string

FeedbackAction represents what to do with a pending memory.

const (
	FeedbackApprove FeedbackAction = "approve"
	FeedbackEdit    FeedbackAction = "edit"
	FeedbackDiscard FeedbackAction = "discard"
)

type FeedbackSignal

type FeedbackSignal struct {
	// contains filtered or unexported fields
}

FeedbackSignal tracks which memories were useful vs irrelevant. Enables: demote nodes explicitly marked irrelevant, boost nodes that led to successful outcomes. Learns over time which memories matter.

func NewFeedbackSignal

func NewFeedbackSignal(store storage.Storage) *FeedbackSignal

NewFeedbackSignal creates a feedback tracker.

func (*FeedbackSignal) ApplyToNodes

func (fs *FeedbackSignal) ApplyToNodes(ctx context.Context) int

ApplyToNodes adjusts confidence based on accumulated feedback.

func (*FeedbackSignal) BoostMultiplier

func (fs *FeedbackSignal) BoostMultiplier(nodeID string) float64

BoostMultiplier returns a score multiplier based on feedback history. Positive feedback → boost (up to 1.5x), negative → penalty (down to 0.5x).

func (*FeedbackSignal) MarkIrrelevant

func (fs *FeedbackSignal) MarkIrrelevant(nodeID string)

MarkIrrelevant indicates a memory was not useful (noise).

func (*FeedbackSignal) MarkRelevant

func (fs *FeedbackSignal) MarkRelevant(nodeID string)

MarkRelevant indicates a memory was useful in context.

type GitLearner

type GitLearner struct {
	// contains filtered or unexported fields
}

GitLearner extracts memories from git history automatically. Scans commit messages, diffs, and blame data to discover:

  • Architecture decisions (from commit messages with "chose", "decided", "switched")
  • Conventions (from repeated patterns across commits)
  • Bug patterns (from fix/hotfix commits)
  • File purposes (from first-commit messages)

No other memory tool does this. Your project's entire history becomes memory.

func NewGitLearner

func NewGitLearner(dir string, engine *Engine) *GitLearner

NewGitLearner creates a learner for a project directory.

func (*GitLearner) LearnFromBlame

func (gl *GitLearner) LearnFromBlame(ctx context.Context, filePath string) error

LearnFromBlame extracts file-level knowledge from git blame. Discovers who owns what and when major changes happened.

func (*GitLearner) LearnFromHistory

func (gl *GitLearner) LearnFromHistory(ctx context.Context, limit int, since time.Time) (*LearnResult, error)

LearnFromHistory scans git log and extracts memories. limit: max commits to scan (0 = all). since: only commits after this date.

func (*GitLearner) Suggest

func (gl *GitLearner) Suggest(ctx context.Context) ([]MemorySuggestion, error)

Suggest analyzes recent changes and suggests memories to store.

type GraphDiff

type GraphDiff struct {
	Added    []*storage.Node
	Modified []*storage.Node
	Removed  []string // IDs of removed/archived nodes
}

GraphDiff shows what memories changed between two points in time.

func DiffSince

func DiffSince(ctx context.Context, store storage.Storage, sinceVersion int) (*GraphDiff, error)

DiffSince returns nodes added or modified since the given node count snapshot.

type HNSW

type HNSW struct {
	M int // max connections per layer
	// contains filtered or unexported fields
}

HNSW implements Hierarchical Navigable Small World graph for approximate nearest neighbor search. Pure Go, zero dependencies.

Based on: Malkov & Yashunin, "Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs" (2018).

Parameters tuned for coding memory workloads (1K-50K vectors, 384D):

  • M = 16 (max connections per layer)
  • efConstruct = 200 (construction beam width)
  • efSearch = 100 (search beam width)
  • mL = 1/ln(M) (level generation factor)

The index can be persisted to disk via Save() / LoadHNSW() for survival across restarts. Set autoSavePath to enable debounced auto-save (at most once per second, or every 100 inserts).

func LoadHNSW

func LoadHNSW(path string) (*HNSW, error)

LoadHNSW restores an HNSW index from a gob-encoded file at path.

func NewHNSW

func NewHNSW(dim int) *HNSW

NewHNSW creates an HNSW index with default parameters for coding memory.

func (*HNSW) Compact

func (h *HNSW) Compact()

Compact removes dead nodes and remaps all neighbor indices. Returns early if dead ratio is below 20% (no compaction needed).

func (*HNSW) Contains

func (h *HNSW) Contains(id string) bool

Contains checks if an ID is in the index.

func (*HNSW) DeadNodeRatio

func (h *HNSW) DeadNodeRatio() float64

DeadNodeRatio returns the fraction of nodes with nil vectors (logically deleted).

func (*HNSW) Insert

func (h *HNSW) Insert(id string, vector []float32)

Insert adds a vector to the index.

func (*HNSW) Remove

func (h *HNSW) Remove(id string)

Remove marks a node as deleted (logical delete, doesn't restructure).

func (*HNSW) Save

func (h *HNSW) Save(path string) error

Save writes the HNSW index to a gob-encoded file at path.

func (*HNSW) Search

func (h *HNSW) Search(query []float32, k int) ([]string, []float32)

Search finds the k nearest neighbors to the query vector. Returns (nodeIDs, distances) sorted by distance ascending.

func (*HNSW) Size

func (h *HNSW) Size() int

Size returns the number of vectors in the index.

type HandoffSummary

type HandoffSummary struct {
	SessionID    string   `json:"session_id"`
	Duration     string   `json:"duration"`
	Accomplished []string `json:"accomplished"`
	Decisions    []string `json:"decisions"`
	Conventions  []string `json:"conventions"`
	BugsFound    []string `json:"bugs_found"`
	InProgress   []string `json:"in_progress"`
	NextSteps    []string `json:"next_steps"`
}

HandoffSummary is the structured output of session analysis.

type HealthCheck

type HealthCheck struct {
	Name    string        `json:"name"`
	Status  HealthStatus  `json:"status"`
	Message string        `json:"message,omitempty"`
	Latency time.Duration `json:"latency"`
}

HealthCheck is an individual health check result.

type HealthChecker

type HealthChecker struct {
	// contains filtered or unexported fields
}

HealthChecker runs diagnostic checks on the system.

func NewHealthChecker

func NewHealthChecker(store storage.Storage) *HealthChecker

NewHealthChecker creates a health checker.

func (*HealthChecker) Check

func (hc *HealthChecker) Check(ctx context.Context) *HealthReport

Check runs all health checks and returns a report.

type HealthReport

type HealthReport struct {
	Status     HealthStatus  `json:"status"`
	Uptime     time.Duration `json:"uptime"`
	Checks     []HealthCheck `json:"checks"`
	NodeCount  int           `json:"node_count"`
	EdgeCount  int           `json:"edge_count"`
	DBSize     int64         `json:"db_size_bytes"`
	CacheHits  int           `json:"cache_hits"`
	LastRecall time.Duration `json:"last_recall_latency"`
}

HealthReport provides detailed system health information.

type HealthStatus

type HealthStatus string

HealthStatus represents the overall system health.

const (
	HealthHealthy   HealthStatus = "healthy"
	HealthDegraded  HealthStatus = "degraded"
	HealthUnhealthy HealthStatus = "unhealthy"
)

type HierarchicalMemory

type HierarchicalMemory struct {
	// contains filtered or unexported fields
}

HierarchicalMemory implements RAPTOR-style multi-level abstraction. Memories are organized in layers:

  • Level 0: Individual memories (leaf nodes)
  • Level 1: Topic clusters (grouped by entity overlap)
  • Level 2: Domain summaries (architecture, testing, deployment, etc.)

Retrieval can target specific levels: detailed queries hit level 0, broad queries hit level 2, default hits all levels with RRF fusion.

func NewHierarchicalMemory

func NewHierarchicalMemory(store storage.Storage) *HierarchicalMemory

NewHierarchicalMemory creates the hierarchy builder.

func (*HierarchicalMemory) Build

func (hm *HierarchicalMemory) Build(ctx context.Context) error

Build constructs the hierarchy from current memory state. Call periodically (e.g., at session end) to keep hierarchy fresh.

func (*HierarchicalMemory) FormatLevel

func (hm *HierarchicalMemory) FormatLevel(level int) string

FormatLevel returns a readable summary of a hierarchy level.

func (*HierarchicalMemory) RetrieveAdaptive

func (hm *HierarchicalMemory) RetrieveAdaptive(query string) int

RetrieveAdaptive picks the right level based on query specificity. Specific queries (file names, function names) → Level 0 Topic queries ("auth", "testing") → Level 1 Global queries ("what are the main patterns?") → Level 2

func (*HierarchicalMemory) RetrieveAtLevel

func (hm *HierarchicalMemory) RetrieveAtLevel(query string, level int) []MemoryCluster

RetrieveAtLevel searches within a specific abstraction level.

type HierarchyLevel

type HierarchyLevel struct {
	Level    int
	Clusters []MemoryCluster
}

HierarchyLevel represents one abstraction layer.

type HybridSearch

type HybridSearch struct {
	// contains filtered or unexported fields
}

HybridSearch performs 3-stage retrieval: BM25 → vector → graph expansion → RRF fusion.

func NewHybridSearch

func NewHybridSearch(store storage.Storage, g graph.Graph, provider embeddings.Provider) *HybridSearch

NewHybridSearch creates a hybrid search engine.

func (*HybridSearch) Search

func (h *HybridSearch) Search(ctx context.Context, query string, opts RecallOpts) ([]*ScoredNode, error)

Search runs hybrid search and returns ranked nodes. 4-path retrieval: BM25 + vector + graph (intent-aware) + temporal recency Based on Hindsight's multi-strategy approach.

func (*HybridSearch) SetHNSW

func (h *HybridSearch) SetHNSW(index *HNSW)

SetHNSW attaches an HNSW index for fast approximate nearest neighbor search.

type ImpactAnalysis

type ImpactAnalysis struct {
	SourceNode *storage.Node       `json:"source_node"`
	Affected   []*AffectedNode     `json:"affected"`
	TotalCount int                 `json:"total_count"`
	StaleCount int                 `json:"stale_count"`
	ByRelType  map[string][]string `json:"by_relationship_type"`
	AnalyzedAt time.Time           `json:"analyzed_at"`
}

ImpactAnalysis reports how a memory change cascades through the graph. Given a node that changed, it identifies all downstream memories that may be affected, grouped by relationship type.

type IngestResult

type IngestResult struct {
	Source      string
	Conventions int
	Decisions   int
	Bugs        int
	Specs       int
	Tasks       int
	Preferences int
	Files       int
	Total       int
	Skipped     int
}

IngestResult tracks what was extracted from a source.

type Ingester

type Ingester struct {
	// contains filtered or unexported fields
}

Ingester handles bulk memory creation from various sources: conversations, markdown files, code files, CLAUDE.md, .cursorrules.

func NewIngester

func NewIngester(engine *Engine) *Ingester

NewIngester creates an ingester bound to an engine.

func (*Ingester) DetectStack

func (ing *Ingester) DetectStack(ctx context.Context, projectDir string) (*IngestResult, error)

DetectStack scans package files and returns the detected tech stack.

func (*Ingester) IngestClaudeMD

func (ing *Ingester) IngestClaudeMD(ctx context.Context, path string) (*IngestResult, error)

IngestClaudeMD imports from a CLAUDE.md file (conventions, rules, patterns).

func (*Ingester) IngestCodeFile

func (ing *Ingester) IngestCodeFile(ctx context.Context, path string) (*IngestResult, error)

IngestCodeFile extracts specs from source code (functions, types, package purpose).

func (*Ingester) IngestConversation

func (ing *Ingester) IngestConversation(ctx context.Context, text string) (*IngestResult, error)

IngestConversation parses a conversation transcript (alternating Human/Assistant) and extracts decisions, conventions, bugs mentioned.

func (*Ingester) IngestCursorRules

func (ing *Ingester) IngestCursorRules(ctx context.Context, path string) (*IngestResult, error)

IngestCursorRules imports from a .cursorrules file.

func (*Ingester) IngestDirectory

func (ing *Ingester) IngestDirectory(ctx context.Context, dir string) (*IngestResult, error)

IngestDirectory scans a project directory and ingests all relevant files.

func (*Ingester) IngestFile

func (ing *Ingester) IngestFile(ctx context.Context, path string) (*IngestResult, error)

IngestFile parses a markdown file from disk.

func (*Ingester) IngestMarkdown

func (ing *Ingester) IngestMarkdown(ctx context.Context, content, source string) (*IngestResult, error)

IngestMarkdown parses a markdown file and extracts memories from headers and lists.

type InjectConfig

type InjectConfig struct {
	// Threshold is the minimum combined score (recency + relevance) a memory
	// must reach to be auto-injected into the conversation context.
	// Range: 0.0-1.0. Default: 0.3.
	Threshold float64

	// MaxInject is the maximum number of memories to inject per check.
	// Default: 5.
	MaxInject int

	// TokenBudget is the maximum tokens for injected context.
	// Default: 1000.
	TokenBudget int

	// Cooldown is the minimum time between injection rounds to avoid flooding.
	// Default: 30s.
	Cooldown time.Duration

	// RecencyWeight controls how much recency influences the combined score.
	// RelevanceWeight is (1 - RecencyWeight). Default: 0.4.
	RecencyWeight float64
}

InjectConfig controls proactive context injection behavior.

func DefaultInjectConfig

func DefaultInjectConfig() InjectConfig

DefaultInjectConfig returns production-tuned injection parameters.

type InjectedMemory

type InjectedMemory struct {
	Node   *storage.Node
	Score  float64 // combined recency + relevance score
	Reason string  // human-readable explanation (e.g., "recently accessed + high relevance")
}

InjectedMemory is a memory selected for proactive injection along with its score and the reason it was surfaced.

type InjectionResult

type InjectionResult struct {
	Injected []InjectedMemory
	Skipped  int           // memories that were candidates but below threshold
	Duration time.Duration // how long the injection check took
}

InjectionResult holds the output of a proactive injection check.

type KeygenLLM

type KeygenLLM interface {
	GenerateKeywords(ctx context.Context, query string) (string, error)
}

KeygenLLM is the interface a caller implements to let yaad rewrite a natural- language query into retrieval keywords with an LLM before searching. Like ConsolidationLLM, it keeps yaad free of any direct LLM dependency — the engine only ever sees this small interface.

GenerateKeywords receives the user's raw query and returns a space-separated (or newline-separated) set of keywords/phrases to search for. Returning an empty string or an error signals "no rewrite"; the caller falls back to the query unchanged (after synonym expansion).

type KeygenStrategy

type KeygenStrategy int

KeygenStrategy controls how RewriteQuery turns a raw query into search terms.

const (
	// KeygenSynonyms uses only the built-in dependency-free synonym expansion
	// (ExpandQuery). This is the default and requires no LLM.
	KeygenSynonyms KeygenStrategy = iota
	// KeygenLLMThenSynonyms asks the LLM to extract keywords first, then unions
	// the result with synonym expansion of the original query. This mirrors the
	// "generate keywords before retrieval" pattern: separate the LLM-driven
	// keyword step from the retrieval step so a verbose question becomes tight
	// search terms without the retriever ever seeing the prose.
	KeygenLLMThenSynonyms
	// KeygenLLMOnly uses only the LLM keyword output, falling back to synonym
	// expansion if the LLM is absent or returns nothing.
	KeygenLLMOnly
)

type LLMConsolidationConfig

type LLMConsolidationConfig struct {
	// Enabled turns the consolidator on/off.
	Enabled bool `json:"enabled"`
	// MinClusterSize is the minimum number of nodes in a candidate cluster.
	MinClusterSize int `json:"min_cluster_size"`
	// MaxBatchSize caps how many nodes are sent to the LLM in a single call.
	MaxBatchSize int `json:"max_batch_size"`
	// KeySimilarityThreshold (0-1) controls how similar keys must be to
	// group nodes together. 0.5 means at least 50% token overlap.
	KeySimilarityThreshold float64 `json:"key_similarity_threshold"`
	// ContentKeywordThreshold (0-1) controls keyword overlap required.
	ContentKeywordThreshold float64 `json:"content_keyword_threshold"`
	// EdgeCrossRefThreshold is the minimum number of shared edges between
	// two nodes to consider them candidates.
	EdgeCrossRefThreshold int `json:"edge_cross_ref_threshold"`
	// DryRun when true causes Consolidate to return merge plans without
	// modifying storage.
	DryRun bool `json:"dry_run"`
}

LLMConsolidationConfig holds tuning knobs for the LLM consolidator.

func DefaultLLMConsolidationConfig

func DefaultLLMConsolidationConfig() LLMConsolidationConfig

DefaultLLMConsolidationConfig returns production-tuned defaults.

type LLMConsolidator

type LLMConsolidator struct {
	// contains filtered or unexported fields
}

LLMConsolidator uses an LLM to find semantically duplicate or closely related memory nodes and merge them, reducing graph noise.

func NewLLMConsolidator

func NewLLMConsolidator(store storage.Storage, llm ConsolidationLLM, config LLMConsolidationConfig) *LLMConsolidator

NewLLMConsolidator creates an LLM-backed consolidator.

func (*LLMConsolidator) Consolidate

func (lc *LLMConsolidator) Consolidate(ctx context.Context, project string) (*ConsolidateResult, error)

Consolidate finds candidate clusters, asks the LLM for merge suggestions, and applies the plans to storage. If DryRun is set in the config, it returns merge plans without modifying anything.

func (*LLMConsolidator) FindCandidateClusters

func (lc *LLMConsolidator) FindCandidateClusters(ctx context.Context, project string) ([]*CandidateCluster, error)

FindCandidateClusters scans all nodes of a project and groups them by type+project, then refines by key similarity, content keyword overlap, and cross-reference density.

type LLMExtractor

type LLMExtractor struct {
	// contains filtered or unexported fields
}

LLMExtractor uses an LLM to extract entities from content. Falls back to regex extraction if LLM is unavailable.

func NewLLMExtractor

func NewLLMExtractor(apiKey, baseURL, model string) *LLMExtractor

NewLLMExtractor creates an LLM-based entity extractor. baseURL: "https://api.openai.com" or any OpenAI-compatible endpoint.

func (*LLMExtractor) Extract

func (e *LLMExtractor) Extract(ctx context.Context, content string) []Entity

Extract returns entities from content using the LLM. Returns regex-extracted entities if LLM call fails.

func (*LLMExtractor) ExtractWithError

func (e *LLMExtractor) ExtractWithError(ctx context.Context, content string) ([]Entity, error)

ExtractWithError runs the LLM extraction and surfaces the underlying error instead of silently falling back. Use Extract for the fallback-on-error behavior; use this when the caller wants to merge or branch on success.

type LearnResult

type LearnResult struct {
	Decisions   int
	Conventions int
	Bugs        int
	Files       int
	Skipped     int
	Duration    time.Duration
}

LearnResult reports what was extracted from git history.

type MemoryCluster

type MemoryCluster struct {
	ID       string
	Summary  string
	NodeIDs  []string
	Keywords []string
	Size     int
	AvgConf  float64
}

MemoryCluster is a group of related memories with a summary.

type MemoryEvent

type MemoryEvent struct {
	Kind      MemoryEventKind
	NodeID    string
	Content   string
	Project   string
	Timestamp int64 // unix milliseconds
}

MemoryEvent is an in-process notification about a memory mutation. It carries the same payload the gRPC WatchMemories RPC (api/proto/yaad.proto) would stream, so a transport (SSE today, gRPC later) can forward it verbatim.

type MemoryEventKind

type MemoryEventKind string

MemoryEventKind classifies a memory mutation emitted on the engine's event bus.

const (
	// MemoryCreated is emitted when a new node is stored.
	MemoryCreated MemoryEventKind = "created"
	// MemoryUpdated is emitted when an existing node is overwritten (keyed upsert).
	MemoryUpdated MemoryEventKind = "updated"
	// MemoryDeleted is emitted when a node is archived/forgotten.
	MemoryDeleted MemoryEventKind = "deleted"
)

type MemoryExpiry

type MemoryExpiry struct {
	// contains filtered or unexported fields
}

MemoryExpiry handles automatic deletion of memories past their TTL.

func NewMemoryExpiry

func NewMemoryExpiry(defaultTTL time.Duration) *MemoryExpiry

NewMemoryExpiry creates an expiry manager.

func (*MemoryExpiry) IsExpired

func (me *MemoryExpiry) IsExpired(createdAt time.Time, pinned bool) bool

IsExpired checks if a node has exceeded its TTL.

type MemoryIntegrity

type MemoryIntegrity struct {
	// contains filtered or unexported fields
}

MemoryIntegrity provides HMAC-SHA256 signing and verification for memory nodes. Detects if memories were tampered with outside yaad (e.g., direct SQLite edits). The signing key is stored in .yaad/integrity.key (auto-generated on first use). The key is immutable after construction, so no mutex is needed.

func NewMemoryIntegrity

func NewMemoryIntegrity(yaadDir string) (*MemoryIntegrity, error)

NewMemoryIntegrity creates an integrity checker, loading or generating the key. The raw key material is stored on disk, but the actual HMAC key is derived from it using HKDF-SHA256 with the machine's hostname and username as context. This binds the key to the originating machine so the on-disk bytes are not directly usable elsewhere.

func (*MemoryIntegrity) Sign

func (mi *MemoryIntegrity) Sign(node *storage.Node) string

Sign generates an HMAC-SHA256 signature for a memory node's content.

func (*MemoryIntegrity) Verify

func (mi *MemoryIntegrity) Verify(node *storage.Node, expectedSig string) bool

Verify checks if a node's content matches its expected signature.

func (*MemoryIntegrity) VerifyBatch

func (mi *MemoryIntegrity) VerifyBatch(nodes []*storage.Node, signatures map[string]string) []string

VerifyBatch checks multiple nodes and returns IDs of tampered ones.

type MemoryStats

type MemoryStats struct {
	TotalNodes  int
	NodesByType map[string]int
	TotalEdges  int
	LastUpdated time.Time
	TopTopics   []string // top 5 most-connected node subjects
}

MemoryStats holds aggregate statistics about the memory graph.

type MemorySuggestion

type MemorySuggestion struct {
	Content    string  `json:"content"`
	Type       string  `json:"type"`
	Confidence float64 `json:"confidence"`
	Reason     string  `json:"reason"`
}

MemorySuggestion is a recommended memory to store.

type MergePlan

type MergePlan struct {
	// NodeIDs lists the IDs of nodes to merge.
	NodeIDs []string `json:"node_ids"`
	// NewContent is the unified content replacing the merged nodes.
	NewContent string `json:"new_content"`
	// NewSummary is a short summary of the merged content.
	NewSummary string `json:"new_summary"`
	// NewKey is the key for the resulting consolidated node.
	NewKey string `json:"new_key"`
	// NewType is the node type for the consolidated node.
	NewType string `json:"new_type"`
}

MergePlan describes how a cluster of nodes should be merged into one.

type Metrics

type Metrics struct {
	Remembers        int64
	Recalls          int64
	Errors           int64
	NodesStored      int64
	InjectChecks     int64 // proactive injection checks performed
	InjectsPerformed int64 // memories actually injected
}

Metrics tracks basic engine operation counters.

type MinHashLSH

type MinHashLSH struct {
	// contains filtered or unexported fields
}

MinHashLSH implements Locality-Sensitive Hashing using MinHash with banding. Uses 128 MinHash permutations divided into 16 bands of 8 rows each. For a threshold of 0.85, this gives ~97% probability of being a candidate for true positives and ~0.01% for items below threshold.

func NewMinHashLSH

func NewMinHashLSH(threshold float64) *MinHashLSH

NewMinHashLSH creates a MinHash LSH index with the given Jaccard similarity threshold. A threshold of 0.85 is standard for near-duplicate detection.

func (*MinHashLSH) Insert

func (lsh *MinHashLSH) Insert(id string, terms map[string]bool)

Insert computes the MinHash signature for a set of terms and adds the ID to the appropriate band buckets.

func (*MinHashLSH) Query

func (lsh *MinHashLSH) Query(terms map[string]bool) []string

Query returns candidate IDs whose term sets are likely above the similarity threshold. The candidates may include false positives but will have very few false negatives.

func (*MinHashLSH) Size

func (lsh *MinHashLSH) Size() int

Size returns the number of indexed documents.

type MultiFactorRanker

type MultiFactorRanker struct {
	// contains filtered or unexported fields
}

MultiFactorRanker re-scores recalled memories using configurable factor weights.

func NewMultiFactorRanker

func NewMultiFactorRanker(cfg RankingWeights) *MultiFactorRanker

NewMultiFactorRanker creates a ranker with the given weights (normalized).

func (*MultiFactorRanker) Rank

func (r *MultiFactorRanker) Rank(ctx context.Context, nodes []*storage.Node, baseScores map[string]float64, store storage.Storage) []*RankedNode

Rank re-scores nodes using multi-factor ranking and returns them sorted by combined score descending. baseScores provides pre-computed relevance scores (e.g., from BM25 or RRF fusion).

type NodeHistoryEntry

type NodeHistoryEntry struct {
	Version   int    `json:"version"`
	Content   string `json:"content"`
	ChangedBy string `json:"changed_by"`
	Reason    string `json:"reason"`
	ChangedAt string `json:"changed_at"`
}

NodeHistoryEntry represents one version of a memory node.

type PackedContext

type PackedContext struct {
	Content    string
	TokensUsed int
	NodesUsed  int
	TypeCounts map[string]int
}

PackedContext is the optimized output ready for prompt injection.

type PageRank

type PageRank struct {
	// contains filtered or unexported fields
}

PageRank computes importance scores for memory nodes based on graph structure. Nodes with more inbound links from important nodes score higher. Used to surface the most "central" memories in the graph.

Algorithm: iterative power method with damping factor 0.85. Converges in ~20 iterations for typical memory graphs (<10K nodes).

func NewPageRank

func NewPageRank(store storage.Storage) *PageRank

NewPageRank creates a PageRank computer.

func (*PageRank) Compute

func (pr *PageRank) Compute(ctx context.Context, limit int) ([]RankedNode, error)

Compute runs PageRank and returns nodes sorted by importance.

type PrivacyConfig

type PrivacyConfig struct {
	Level        privacy.FilterLevel `json:"level"`
	StripPrivate bool                `json:"strip_private"` // strip <private>...</private> tags
}

PrivacyConfig holds the privacy filtering configuration for the engine. Controls how aggressively sensitive information is redacted from memories before they are stored.

func DefaultPrivacyConfig

func DefaultPrivacyConfig() PrivacyConfig

DefaultPrivacyConfig returns the standard configuration (moderate filtering).

type ProactiveContext

type ProactiveContext struct {
	// contains filtered or unexported fields
}

ProactiveContext predicts what context the agent will likely need next by analyzing recent access patterns and graph connectivity.

func NewProactiveContext

func NewProactiveContext(eng *Engine, search *HybridSearch) *ProactiveContext

NewProactiveContext creates a proactive context predictor.

func (*ProactiveContext) Predict

func (p *ProactiveContext) Predict(ctx context.Context, project string, budget int) ([]*storage.Node, error)

Predict returns nodes likely needed in the next session based on: 1. Recently accessed nodes and their neighbors 2. Active tasks and their dependencies 3. High-centrality nodes in the project subgraph

type ProactiveInjector

type ProactiveInjector struct {
	// contains filtered or unexported fields
}

ProactiveInjector monitors conversation context and proactively surfaces relevant memories using recency + relevance scoring.

func NewProactiveInjector

func NewProactiveInjector(eng *Engine, cfg InjectConfig) *ProactiveInjector

NewProactiveInjector creates a proactive injector with the given config.

func (*ProactiveInjector) Inject

func (pi *ProactiveInjector) Inject(ctx context.Context, conversationText string, project string) (*InjectionResult, error)

Inject evaluates the current conversation context and proactively surfaces relevant memories that exceed the configured threshold. The conversationText is the recent conversation content used to find relevant memories.

type QueryCache

type QueryCache struct {
	// contains filtered or unexported fields
}

QueryCache is an LRU cache for recall results. Eliminates redundant SQLite queries for frequently-asked questions within a session. Auto-invalidates on writes (Remember, Forget, Feedback).

func NewQueryCache

func NewQueryCache(maxSize int, ttl time.Duration) *QueryCache

NewQueryCache creates a cache with configurable size and TTL.

func (*QueryCache) Clear

func (c *QueryCache) Clear()

Clear empties the cache entirely.

func (*QueryCache) Get

func (c *QueryCache) Get(key string) *RecallResult

Get retrieves a cached result. Returns nil if miss or stale.

func (*QueryCache) Invalidate

func (c *QueryCache) Invalidate()

Invalidate bumps the version, making all cached entries stale. Called after any write operation (Remember, Forget, Feedback, etc).

func (*QueryCache) Put

func (c *QueryCache) Put(key string, result *RecallResult)

Put stores a result in the cache.

func (*QueryCache) Stats

func (c *QueryCache) Stats() (size int, version uint64)

Stats returns cache hit/miss info.

type QueryMetrics

type QueryMetrics struct {
	Query       string
	Duration    time.Duration
	ResultCount int
	SignalsUsed []string
	WasUseful   bool
}

QueryMetrics tracks retrieval performance for auto-tuning.

type QueryPlan

type QueryPlan struct {
	UseBM25      bool
	UseVector    bool
	UseGraph     bool
	UseRecency   bool
	UseEntity    bool
	GraphDepth   int
	BM25Limit    int
	Intent       intent.Intent
	TemporalOnly bool
}

QueryPlan describes which retrieval signals to use.

type QueryPlanner

type QueryPlanner struct {
	// contains filtered or unexported fields
}

QueryPlanner implements intelligent query routing based on Qdrant's cardinality estimation and Cognee's auto-routing approach. Instead of always running all 5 retrieval signals, it selects the optimal subset based on query characteristics.

Techniques from:

  • Qdrant: Cardinality-based path selection
  • Cognee: Intent-aware routing to specialized retrievers
  • Mem0: Query-adaptive parameters

func NewQueryPlanner

func NewQueryPlanner(nodeCount int, hasVectors, hasGraph bool) *QueryPlanner

NewQueryPlanner creates a planner with context about the index.

func (*QueryPlanner) EstimateCardinality

func (qp *QueryPlanner) EstimateCardinality(query string) float64

EstimateCardinality estimates how many results a query will return. Used to decide if expensive signals (vector, graph) are worth running. Based on Qdrant's Agresti-Coull sampling approach (simplified).

func (*QueryPlanner) Plan

func (qp *QueryPlanner) Plan(query string, opts RecallOpts) QueryPlan

Plan analyzes a query and decides the optimal retrieval strategy.

type QueryResult

type QueryResult struct {
	Answer     string          // synthesized answer from retrieved memories
	Sources    []*storage.Node // the memories used to form the answer
	Confidence float64         // 0-1, based on relevance scores of retrieved nodes
}

QueryResult holds the answer to a natural language query.

type Quiz

type Quiz struct {
	// contains filtered or unexported fields
}

Quiz tests how well an agent (or developer) remembers project knowledge. Generates questions from stored memories and checks answers. Useful for validating that the memory system is capturing the right things.

func NewQuiz

func NewQuiz(store storage.Storage) *Quiz

NewQuiz creates a quiz generator.

func (*Quiz) CheckAnswer

func (q *Quiz) CheckAnswer(question QuizQuestion, answer string) bool

CheckAnswer verifies if a given answer matches the stored memory.

func (*Quiz) Generate

func (q *Quiz) Generate(ctx context.Context, count int) ([]QuizQuestion, error)

Generate creates quiz questions from stored memories.

type QuizQuestion

type QuizQuestion struct {
	Question string `json:"question"`
	Answer   string `json:"answer"`
	Type     string `json:"type"`
	NodeID   string `json:"node_id"`
}

QuizQuestion is a generated question with its expected answer.

type QuizResult

type QuizResult struct {
	Total   int     `json:"total"`
	Correct int     `json:"correct"`
	Score   float64 `json:"score"` // 0-100
}

QuizResult tracks how well the quiz was answered.

type RankedNode

type RankedNode struct {
	Node    *storage.Node
	Score   float64
	Factors map[string]float64
}

RankedNode pairs a memory node with its multi-factor score and a breakdown of individual factor contributions.

type RankingWeights

type RankingWeights struct {
	Recency    float64
	Relevance  float64
	Frequency  float64
	Confidence float64
	Tier       float64
	Centrality float64
	Pinned     float64
}

RankingWeights configures the relative importance of each ranking factor. Weights are normalized to sum to 1.0 before use.

func DefaultRankingWeights

func DefaultRankingWeights() RankingWeights

DefaultRankingWeights returns balanced weights tuned for coding memory workloads.

type RateLimiter

type RateLimiter struct {
	// contains filtered or unexported fields
}

RateLimiter implements a token bucket rate limiter for the REST API. Prevents abuse and ensures fair usage for developers running locally multiple agents simultaneously.

func NewRateLimiter

func NewRateLimiter(ratePerSec float64, maxBurst int) *RateLimiter

NewRateLimiter creates a limiter with given rate (req/sec) and burst capacity.

func (*RateLimiter) Allow

func (rl *RateLimiter) Allow() bool

Allow checks if a request is allowed. Returns true if within limits.

func (*RateLimiter) Remaining

func (rl *RateLimiter) Remaining() int

Remaining returns how many requests are available right now.

type RecallOpts

type RecallOpts struct {
	Query              string
	Depth              int
	Limit              int
	Budget             int // max tokens in response (0 = no cap)
	Type               string
	Tier               int
	Project            string
	Alpha              float64           // hybrid retrieval blend: -1=RRF (default), 0=pure BM25, 1=pure vector, else alpha blend
	GraphTraverseDepth int               // if > 0, enrich results by traversing the graph from top results (0 = disabled)
	MetadataFilters    map[string]string // key-value metadata filters (AND semantics)
}

RecallOpts configures a recall search.

type RecallResult

type RecallResult struct {
	Nodes []*storage.Node
	Edges []*storage.Edge
}

RecallResult holds search results.

type RememberInput

type RememberInput struct {
	Type     string // convention|decision|bug|spec|task|preference
	Content  string
	Summary  string // doubles as short searchable title (Engram Title)
	Scope    string // global|project
	Project  string
	Tier     int
	Tags     string
	Key      string // optional unique key per project (upsert: same key → update, not duplicate)
	TopicKey string // topic-based upsert dedup key (Engram pattern); stored in Tags as "topic:<key>"
	Pinned   bool   // pinned nodes always appear in context output
	Session  string
	Agent    string
	Metadata map[string]string // optional structured key-value metadata persisted alongside the node
	// Optional: explicit edges to create
	Edges []EdgeInput
}

RememberInput is the input for creating a memory node.

type ReviewRecord

type ReviewRecord struct {
	NodeID      string    `json:"node_id"`
	Stability   float64   `json:"stability"`  // days until retrieval strength drops to 90%
	Difficulty  float64   `json:"difficulty"` // 0=easy, 1=hard
	LastReview  time.Time `json:"last_review"`
	ReviewCount int       `json:"review_count"`
}

ReviewRecord tracks a single memory's retrieval strength.

type RuleInput

type RuleInput struct {
	Path        string   // file path of the rule file
	Content     string   // markdown content (body, without frontmatter)
	Description string   // from frontmatter
	Globs       []string // glob patterns from frontmatter
	AlwaysApply bool     // from frontmatter
	Project     string   // project scope
}

RuleInput stores a rule file as a yaad convention node with glob metadata.

type ScoredMemory

type ScoredMemory struct {
	Node       *storage.Node `json:"node"`
	Confidence float64       `json:"confidence"` // [0, 1] composite confidence
	Recency    float64       `json:"recency"`    // [0, 1] recency factor
	Centrality float64       `json:"centrality"` // [0, 1] graph centrality factor
	Trust      float64       `json:"trust"`      // [0, 1] base node confidence
}

ScoredMemory is a memory node with an explicit confidence score reflecting how reliable/relevant the memory is for the given context.

type ScoredNode

type ScoredNode struct {
	Node  *storage.Node
	Score float64
}

ScoredNode is a node with a combined relevance score.

func Rerank

func Rerank(ctx context.Context, nodes []*ScoredNode, store storage.Storage) []*ScoredNode

Rerank re-scores nodes combining RRF score, graph centrality, recency, and confidence.

type ScoredRecallResult

type ScoredRecallResult struct {
	Nodes []*ScoredMemory `json:"nodes"`
	Edges []*storage.Edge `json:"edges,omitempty"`
	Count int             `json:"count"`
	Query string          `json:"query,omitempty"`
}

ScoredRecallResult extends RecallResult with per-node confidence scores.

type ScoringConfig

type ScoringConfig struct {
	BM25Weight     float64 // weight for BM25 signal in fusion (default 0.4)
	VectorWeight   float64 // weight for vector signal (default 0.3)
	GraphWeight    float64 // weight for graph signal (default 0.2)
	RecencyWeight  float64 // weight for recency signal (default 0.1)
	EntityBoostCap float64 // max entity boost multiplier (default 2.0)
	MMRLambda      float64 // diversity vs relevance tradeoff (default 0.85)
	SpreadDecay    float64 // entity spread attenuation factor (default 0.001)
}

ScoringConfig holds tunable parameters for retrieval scoring.

func DefaultScoringConfig

func DefaultScoringConfig() ScoringConfig

DefaultScoringConfig returns production-tuned scoring parameters.

type SearchTimeDecay

type SearchTimeDecay struct {
	HalfLife time.Duration
}

SearchTimeDecay applies non-destructive decay scoring at query time. Unlike RunDecay which modifies stored confidence values, SearchTimeDecay computes decay-adjusted scores purely at retrieval time, leaving stored data untouched. This allows different queries to use different decay profiles and preserves the original confidence for audit and export.

func DefaultSearchTimeDecay

func DefaultSearchTimeDecay() SearchTimeDecay

DefaultSearchTimeDecay returns a SearchTimeDecay with a 30-day half-life, matching the default destructive decay configuration.

func (SearchTimeDecay) ApplySearchTimeDecay

func (d SearchTimeDecay) ApplySearchTimeDecay(nodes []*storage.Node, now time.Time) []ScoredMemory

ApplySearchTimeDecay computes decay-adjusted scores for a set of nodes at query time WITHOUT modifying stored confidence values.

Uses the half-life exponential decay formula:

adjustedScore = confidence * exp(-lambda * age)

where:

  • lambda = ln(2) / halfLife
  • age = time since the most recent of AccessedAt or UpdatedAt
  • confidence = the node's stored Confidence value (never mutated)

Returns a slice of ScoredMemory (reusing the type from confidence.go). The Confidence field holds the decay-adjusted score; Trust holds the original stored confidence. Nodes with zero confidence or nodes whose reference time is in the future (clock skew) are assigned age=0 so they receive no penalty.

type SessionHandoff

type SessionHandoff struct {
	// contains filtered or unexported fields
}

SessionHandoff implements MemGPT-style automatic session summary generation. At session end, it produces a concise handoff summary that captures:

  • What was accomplished
  • Decisions made
  • Conventions established
  • Bugs found/fixed
  • What's in progress

The next session gets this injected automatically so the agent picks up exactly where it left off.

func NewSessionHandoff

func NewSessionHandoff(store storage.Storage) *SessionHandoff

NewSessionHandoff creates a handoff generator.

func (*SessionHandoff) FormatForInjection

func (sh *SessionHandoff) FormatForInjection(summary *HandoffSummary) string

FormatForInjection converts the handoff into a markdown string for prompt injection.

func (*SessionHandoff) Generate

func (sh *SessionHandoff) Generate(ctx context.Context, sessionID string, duration time.Duration) (*HandoffSummary, error)

Generate produces a handoff summary from memories created during a session.

func (*SessionHandoff) GetLastSessionSummary

func (sh *SessionHandoff) GetLastSessionSummary(ctx context.Context, project string) string

GetLastSessionSummary retrieves and formats the previous session's handoff.

type SpacedRepetition

type SpacedRepetition struct {
	// contains filtered or unexported fields
}

SpacedRepetition provides FSRS-6 style memory decay and review scheduling. Memories lose retrieval strength over time unless reviewed. Frequently accessed memories persist; unused ones fade.

func NewSpacedRepetition

func NewSpacedRepetition(store *storage.Store) *SpacedRepetition

NewSpacedRepetition creates a spaced repetition tracker.

func (*SpacedRepetition) RecordReview

func (sr *SpacedRepetition) RecordReview(ctx context.Context, nodeID string) error

RecordReview records an access to a memory node, updating its retrieval strength.

func (*SpacedRepetition) RetrievalStrength

func (sr *SpacedRepetition) RetrievalStrength(ctx context.Context, nodeID string) (float64, error)

RetrievalStrength returns the current retrieval strength (0-1) for a memory.

func (*SpacedRepetition) ShouldReview

func (sr *SpacedRepetition) ShouldReview(ctx context.Context, nodeID string) (bool, error)

ShouldReview returns true if the memory is due for review (strength below threshold).

type SpacingConfig

type SpacingConfig struct {
	OptimalInterval time.Duration `json:"optimal_interval"`
	MaxBonus        float64       `json:"max_bonus"`
	CrammingPenalty float64       `json:"cramming_penalty"`
}

func DefaultSpacingConfig

func DefaultSpacingConfig() SpacingConfig

type Sparsifier

type Sparsifier struct {
	// contains filtered or unexported fields
}

Sparsifier implements Memory³-style memory sparsification. As the graph grows, it:

  1. Merges near-duplicate memories (same entities, similar content)
  2. Compresses low-value clusters into single summary nodes
  3. Prunes truly orphaned nodes with no edges and low confidence

This keeps the memory graph lean and fast as it scales.

func NewSparsifier

func NewSparsifier(store storage.Storage) *Sparsifier

NewSparsifier creates a memory sparsification engine.

func (*Sparsifier) Run

func (s *Sparsifier) Run(ctx context.Context) (*SparsifyResult, error)

Run executes all sparsification passes. Safe to call periodically (e.g., weekly).

type SparsifyResult

type SparsifyResult struct {
	Merged     int
	Compressed int
	Pruned     int
}

SparsifyResult reports what was cleaned up.

type StaleMemory

type StaleMemory struct {
	NodeID    string    `json:"node_id"`
	File      string    `json:"file"`
	ChangedAt time.Time `json:"changed_at"`
	Reason    string    `json:"reason"`
}

StaleMemory represents a memory flagged as stale due to a file change.

type StalenessManager

type StalenessManager struct {
	// contains filtered or unexported fields
}

StalenessManager marks memories as stale when their related files change in git. It integrates the git watcher with the memory engine to automatically flag memories that may be outdated.

func NewStalenessManager

func NewStalenessManager(store storage.Storage, g graph.Graph, dir string) (*StalenessManager, error)

NewStalenessManager creates a manager for a project directory.

func (*StalenessManager) MarkStale

func (sm *StalenessManager) MarkStale(ctx context.Context, since time.Time, penalty float64) ([]StaleMemory, error)

MarkStale detects files changed since the given time and flags affected memories. For each affected node, confidence is reduced by the staleness penalty and a staleness tag is added so retrieval can account for potential outdatedness. Pinned nodes are never penalized — they're manually curated.

func (*StalenessManager) StaleSince

func (sm *StalenessManager) StaleSince(ctx context.Context, since time.Time) ([]git.StaleReport, error)

StaleSince returns stale reports without modifying any nodes (read-only).

func (*StalenessManager) WatchFile

func (sm *StalenessManager) WatchFile(ctx context.Context, filePath, nodeID, gitHash string) error

WatchFile registers a file-to-node mapping for staleness tracking.

type Status

type Status struct {
	Nodes    int
	Edges    int
	Sessions int
}

Status returns basic stats.

type SuggestedEdge

type SuggestedEdge struct {
	FromID   string `json:"from_id"`
	FromType string `json:"from_type"`
	ToID     string `json:"to_id"`
	ToType   string `json:"to_type"`
	EdgeType string `json:"edge_type"`
	Reason   string `json:"reason"`
}

SuggestedEdge is a recommended edge to create.

func SuggestLinks(ctx context.Context, store storage.Storage) ([]SuggestedEdge, error)

SuggestLinks finds orphan nodes and suggests edges to connect them.

type SummarizedMemory

type SummarizedMemory struct {
	Content string
	Type    string
}

SummarizedMemory is a compressed memory extracted from conversation.

type TemplateMemory

type TemplateMemory struct {
	Content string
	Type    string
}

TemplateMemory is a pre-built memory.

type TemplateSet

type TemplateSet struct {
	Name     string
	Stack    string
	Memories []TemplateMemory
}

TemplateSet is a collection of starter memories for a stack.

type Templates

type Templates struct{}

Templates provides pre-built memory sets for common tech stacks.

func (*Templates) Apply

func (t *Templates) Apply(ctx context.Context, eng *Engine, templateName string) (int, error)

Apply stores a template's memories into the engine.

func (*Templates) Available

func (t *Templates) Available() []TemplateSet

Available returns all available templates.

type TemporalEdge

type TemporalEdge struct {
	SourceID     string     `json:"source_id"`
	TargetID     string     `json:"target_id"`
	RelationType string     `json:"relation_type"`
	ValidFrom    *time.Time `json:"valid_from,omitempty"`
	ValidUntil   *time.Time `json:"valid_until,omitempty"`
	Confidence   float64    `json:"confidence"`
	Evidence     string     `json:"evidence,omitempty"`
}

func FilterActiveEdges

func FilterActiveEdges(edges []*TemporalEdge, at time.Time) []*TemporalEdge

func FilterCurrentEdges

func FilterCurrentEdges(edges []*TemporalEdge) []*TemporalEdge

func (*TemporalEdge) IsActiveAt

func (e *TemporalEdge) IsActiveAt(t time.Time) bool

func (*TemporalEdge) IsCurrentlyActive

func (e *TemporalEdge) IsCurrentlyActive() bool

type TemporalFilter

type TemporalFilter struct {
	// contains filtered or unexported fields
}

TemporalFilter enables time-bounded memory retrieval. Based on Zep's valid_at/invalid_at interval model layered on top of yaad's temporal backbone.

Supports queries like:

  • "What was true at time X?" (point-in-time)
  • "What changed between X and Y?" (range)
  • "What's currently valid?" (active only)

func NewTemporalFilter

func NewTemporalFilter(store storage.Storage) *TemporalFilter

NewTemporalFilter creates a temporal filter.

func (*TemporalFilter) ActiveEdges

func (tf *TemporalFilter) ActiveEdges(ctx context.Context, nodeID string) ([]*storage.Edge, error)

ActiveEdges returns only currently-valid edges (invalid_at is zero).

func (*TemporalFilter) ChangedBetween

func (tf *TemporalFilter) ChangedBetween(ctx context.Context, start, end time.Time, limit int) ([]*storage.Node, error)

ChangedBetween returns nodes that were created or modified in a time range.

func (*TemporalFilter) FilterByTime

func (tf *TemporalFilter) FilterByTime(ctx context.Context, nodes []*storage.Node, query TemporalQuery) []*storage.Node

FilterByTime returns nodes that match the temporal constraints.

func (*TemporalFilter) IsSuperseded

func (tf *TemporalFilter) IsSuperseded(ctx context.Context, nodeID string) (bool, string)

IsSuperseded checks if a node has been superseded by another via edge.

func (*TemporalFilter) RecentSince

func (tf *TemporalFilter) RecentSince(ctx context.Context, since time.Time, limit int) ([]*storage.Node, error)

RecentSince returns nodes created or updated since the given time.

type TemporalQuery

type TemporalQuery struct {
	After      time.Time // only nodes created/updated after this time
	Before     time.Time // only nodes created/updated before this time
	ActiveOnly bool      // only nodes not superseded (no invalid_at on edges)
}

TemporalQuery specifies time constraints for retrieval.

type TierLoader

type TierLoader struct {
	// contains filtered or unexported fields
}

TierLoader implements three-tier memory loading (hot/warm/cold) for performance-optimized context window assembly.

Hot tier (tier 1): Always loaded — pinned nodes, recently accessed, high-confidence memories. Fits in ~500 tokens.

Warm tier (tier 2): Loaded on demand — moderately accessed memories, active tasks, recent decisions. Loaded when a session starts.

Cold tier (tier 3): Loaded only on explicit query — older memories, low-confidence nodes, archived items. Loaded via Recall() when needed.

This approach is inspired by Letta's memory hierarchy and MemGPT's tiered context management, adapted for yaad's graph-based architecture.

func NewTierLoader

func NewTierLoader(store storage.Storage) *TierLoader

NewTierLoader creates a tier loader with default TTLs.

func (*TierLoader) InvalidateAll

func (tl *TierLoader) InvalidateAll()

InvalidateAll forces all tier caches to refresh on next access.

func (*TierLoader) InvalidateHot

func (tl *TierLoader) InvalidateHot()

InvalidateHot forces hot-tier cache refresh on next access.

func (*TierLoader) InvalidateWarm

func (tl *TierLoader) InvalidateWarm()

InvalidateWarm forces warm-tier cache refresh on next access.

func (*TierLoader) LoadAll

func (tl *TierLoader) LoadAll(ctx context.Context, project string) (*TierResult, error)

LoadAll loads all three tiers and returns them combined.

func (*TierLoader) LoadCold

func (tl *TierLoader) LoadCold(ctx context.Context, project string, limit int) ([]*storage.Node, error)

LoadCold returns cold-tier nodes (low confidence, old, tier 3). Always hits the database — no caching.

func (*TierLoader) LoadHot

func (tl *TierLoader) LoadHot(ctx context.Context, project string) ([]*storage.Node, error)

LoadHot returns hot-tier nodes (pinned + high-confidence + recently accessed). Cached in memory with a short TTL for performance.

func (*TierLoader) LoadWarm

func (tl *TierLoader) LoadWarm(ctx context.Context, project string) ([]*storage.Node, error)

LoadWarm returns warm-tier nodes (active tasks, recent decisions, moderate confidence). Loaded on session start with a medium TTL.

type TierResult

type TierResult struct {
	Hot  []*storage.Node `json:"hot"`
	Warm []*storage.Node `json:"warm"`
	Cold []*storage.Node `json:"cold"`
}

TierResult holds the combined result of loading multiple tiers.

type TopicCluster

type TopicCluster struct {
	ID        string    `json:"id"`
	Topic     string    `json:"topic"`
	Summary   string    `json:"summary"`
	MemoryIDs []string  `json:"memory_ids"`
	Keywords  []string  `json:"keywords"`
	Size      int       `json:"size"`
	Coherence float64   `json:"coherence"`
	CreatedAt time.Time `json:"created_at"`
}

TopicCluster represents a consolidated group of related memories.

type TopicConsolidator

type TopicConsolidator struct {
	// contains filtered or unexported fields
}

TopicConsolidator groups related memories into topic clusters based on keyword and entity overlap. This helps organize the memory graph by creating cluster summaries that represent related knowledge.

func NewTopicConsolidator

func NewTopicConsolidator(store storage.Storage, config ConsolidationConfig) *TopicConsolidator

NewTopicConsolidator creates a topic consolidator.

func (*TopicConsolidator) Consolidate

func (tc *TopicConsolidator) Consolidate(ctx context.Context, project string) ([]*TopicCluster, error)

Consolidate finds groups of related memories and returns topic clusters. It operates on the given project's memories, computing pairwise keyword overlap to identify coherent groups.

func (*TopicConsolidator) FindRelated

func (tc *TopicConsolidator) FindRelated(ctx context.Context, node *storage.Node, limit int) ([]string, error)

FindRelated returns memory IDs that belong to the same topic cluster as the given node, based on keyword overlap.

type Tracker

type Tracker struct {
	// contains filtered or unexported fields
}

Tracker is a stateful, rolling boundary detector for streaming content. It maintains an exponential moving-average centroid of recent observations and flags an observation as a boundary when it diverges from that centroid.

Tracker is NOT safe for concurrent use; guard with a mutex if shared.

func NewTracker

func NewTracker(det *BoundaryDetector) *Tracker

NewTracker creates a stateful Tracker backed by the given detector.

func (*Tracker) Observe

func (t *Tracker) Observe(content string) (isBoundary bool)

Observe ingests the next piece of content and reports whether it marks a semantic boundary relative to recent content. The very first non-empty observation establishes the baseline and is never a boundary.

func (*Tracker) Reset

func (t *Tracker) Reset()

Reset clears all rolling state, returning the Tracker to its initial baseline.

type UserProfile

type UserProfile struct {
	Preferences map[string]string // e.g., "indent": "tabs", "test_framework": "testify"
	Patterns    []string          // observed patterns: "always runs tests after edit"
	UpdatedAt   time.Time
}

UserProfile holds user preferences and observed patterns for a project.

Directories

Path Synopsis
Package cognitive implements yaad's higher-level memory subsystems — epistemic state, curiosity, boundary detection, reconsolidation, and related processes layered on top of the core engine.
Package cognitive implements yaad's higher-level memory subsystems — epistemic state, curiosity, boundary detection, reconsolidation, and related processes layered on top of the core engine.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL