yaad

package module
v0.1.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 4, 2026 License: MIT Imports: 7 Imported by: 0

README

याद Yaad

Give your coding agent persistent memory.

One config line. Works with any MCP agent. Zero setup.

Go Pure Go Tests CI Discord


The Problem

Every coding agent forgets everything when the session ends.

Session 1: You explain your stack, conventions, architecture. Agent writes great code.
Session 2: Agent has forgotten everything. You start over.
Session 50: You've wasted hours re-teaching the same context.

The Fix

Yaad is the memory engine behind the hawk coding agent. Run hawk and your agent gets persistent, graph-native memory — across sessions, across models, across projects — with no separate install or daemon to manage.

Yaad is a Go library, not a standalone binary. It ships no yaad command of its own; its memory features are surfaced through the host agent (hawk). To embed Yaad in your own Go program, import it directly:

import (
    yaad "github.com/GrayCodeAI/yaad/engine"
    "github.com/GrayCodeAI/yaad/storage"
)

Ecosystem Boundaries

Yaad is a Hawk support engine. Keep the dependency edge one-way:

  • yaad uses local-only types (memory/retrieval types are yaad-scoped, not shared contracts)
  • do not import hawk/internal/*
  • do not import removed legacy path hawk/shared/types
  • do not import other engines (eyrie, tok, trace, sight, inspect) — engines are peers, not dependencies

What Happens Next

Your agent starts a session → Yaad injects context from previous sessions:

## Project Memory (Yaad)

### Conventions (always follow)
- Use `jose` library, not `jsonwebtoken` (Edge compatibility)
- Named exports only, no default exports
- Run `pnpm test --coverage` before committing

### Active Tasks
- ✓ JWT token issuance endpoint
- → Rate limiting on /auth/token (in progress)

### ⚠ Stale Warnings
- Auth subgraph outdated: src/middleware/auth.ts modified 2h ago

### Previous Session
- Implemented rate limiting skeleton, hit NATS backpressure issue

Your agent works → stores decisions, bugs, conventions automatically.
Session ends → Yaad compresses and links everything in a memory graph.
Next session → picks up exactly where you left off. Zero re-explaining.


How It Works

Yaad is a memory layer — it doesn't call LLMs. Your agent handles the LLM. Yaad handles memory.

Your Agent                          Yaad
   │                                  │
   ├─ starts session ──────────────▶  │ returns hot-tier context (~2K tokens)
   │                                  │
   ├─ needs context ───────────────▶  │ graph-aware search (BM25 + vector + graph + temporal)
   │  "auth middleware"               │ returns: decisions + conventions + bugs + specs
   │                                  │
   ├─ learns something ────────────▶  │ stores node, extracts entities, links edges
   │  "Use RS256 for JWT"            │ auto-detects: file refs, libraries, functions
   │                                  │
   ├─ ends session ────────────────▶  │ compresses → summary node → links to graph
   │                                  │
   └─ next session ────────────────▶  │ picks up from summary. zero re-explaining.
Under the Hood

Relaxed DAG — memories are nodes, relationships are edges:

[decision: "Use RS256"] ──led_to──▶ [convention: "Always RS256"]
        │                                     │
        │ led_to                              │ touches
        ▼                                     ▼
[spec: "Auth subsystem"] ◀──part_of── [file: src/middleware/auth.ts]
        │
        │ relates_to
        ▼
[bug: "Token refresh race"] ──supersedes──▶ [bug: "Token expiry (FIXED)"]

Intent-aware retrieval — "why" queries traverse causal edges, "when" queries traverse temporal edges:

"why did we choose NATS?"  → Intent: Why  → boost caused_by, led_to edges
"when did we fix auth?"    → Intent: When → boost temporal backbone
"what is the auth spec?"   → Intent: What → boost spec, part_of edges

4-path search — BM25 + vector + graph (intent-aware) + temporal recency, fused with RRF.


Memory Types

Type What it stores Example
convention Coding rules & patterns "Use jose not jsonwebtoken"
decision Architecture choices + why "Chose NATS for backpressure"
bug Symptom → Cause → Fix "Token race → use mutex"
spec How a subsystem works "Auth: RS256 JWT with jose"
task Done / in-progress / blocked "✓ auth, → rate limiting"
skill Reusable step sequences "Deploy: test → build → fly"
preference User coding style "Functional style, tabs"
file File/module anchor "src/middleware/auth.ts"
entity Auto-extracted entity "jose", "PostgreSQL"

Key Features

Graph-Native Memory (Relaxed DAG)

Not a flat list of memories. A directed graph with 8 edge types:

  • Causal (acyclic): led_to, supersedes, caused_by, learned_in, part_of
  • Relational (cycles OK): relates_to, depends_on, touches

Enables: subgraph extraction, impact analysis, causal chain traversal.

Intent-Aware 4-Path Search

Based on MAGMA (arxiv:2601.03236):

  1. BM25 (FTS5) — keyword matching
  2. Vector (optional) — semantic similarity
  3. Graph (intent-aware BFS) — edge weights boosted by query intent
  4. Temporal — recency-aware for "when" queries

Fused with Reciprocal Rank Fusion (RRF).

Dual-Stream Ingestion

Based on MAGMA + GAM research:

  • Fast path (sync): store node + temporal edge, return in <1ms
  • Slow path (async goroutine): infer causal edges, link entities

Agent is never blocked waiting for memory processing.

Git-Aware Staleness

When source files change, Yaad walks the graph backwards to flag stale subgraphs:

"Auth subgraph may be stale: src/auth.ts modified 2h ago. Affected: [decision: RS256], [convention: jose], [bug: token refresh]"

Impact Analysis

"What memories break if I change schema.sql?" → reverse graph traversal → "3 decisions + 2 specs + 1 convention affected"

Auto-Decay & Compaction
  • Half-life decay: unused memories fade automatically
  • Compaction: low-confidence memories merge into summaries
  • Pinned memories never decay (core architecture decisions, deploy process)
  • Auto-decay runs on every session start — zero maintenance
Privacy & Security
  • API keys, tokens, secrets auto-stripped on ingest (regex + entropy detection)
  • Localhost-only binding (127.0.0.1)
  • HTTPS with auto self-signed cert generation
  • All data stays local (SQLite, your machine)
  • No LLM API calls — Yaad never sends your code anywhere
Worktree-Aware Memory Sharing

Memory is scoped to the git repository root (git rev-parse --show-toplevel), so every worktree, subdirectory, or linked checkout of the same repo shares one .yaad/ memory store. Falls back to the working directory when not in a git repo.

LLM Entity Extraction (opt-in)

Optionally extract entities with an OpenAI-compatible LLM, merged and deduplicated with the regex extractor. Default off — enable with YAAD_LLM_EXTRACT=1 + YAAD_LLM_API_KEY. Falls back to regex-only on any failure, so behavior is unchanged when unset.

Temporal Fact-Validity Windows

Zep-style valid_at / invalid_at intervals on edges let recall filter to facts that were active at a given point in time, distinguishing currently-valid from superseded knowledge.

Live Memory Streaming (gRPC / SSE)

WatchMemories (and WatchStale) stream Remember/Forget mutations in real time over gRPC, with identical semantics exposed over HTTP/SSE at /yaad/events for broad client compatibility.

Versions & Rollback

Every edit is versioned. The engine can list a node's version history and restore a prior state — the rollback itself is recorded as a new version.

ADD-Only Single-Pass Mode (opt-in)

Mem0-v3-style single-pass ingestion that appends memories without the conflict-resolution pass. Default off — enable with YAAD_ADD_ONLY=1.

Subagent-Scoped Memory

A subagent can request its own isolated memory store (<base>/agent-memory/<id>/) via an agent-id env var, keeping parallel agents' memories separate.

Sleep-Time Consolidation (opt-in)

A background loop consolidates memories on the long-running serve / mcp paths. Default off — enable by setting YAAD_CONSOLIDATE_INTERVAL to a positive duration (e.g. 30m).

Semantic Boundary Detection

Detects topic boundaries in a session to segment memories into coherent units before consolidation.


Memory Tools (23 tools)

When Yaad is embedded in a host agent (such as hawk), it exposes these operations to the agent. Yaad implements the logic; the host surfaces them (e.g. as MCP tools):

Tool What it does
yaad_remember Store a memory (convention, decision, bug, spec, task, skill, preference)
yaad_recall Graph-aware search with intent classification
yaad_hybrid_recall 4-path search: BM25 + vector + graph + temporal
yaad_context Get hot-tier context for session injection
yaad_link Create typed edge between memories
yaad_forget Archive a memory (sets confidence to 0)
yaad_feedback Approve / edit / discard a memory
yaad_pin Pin/unpin a memory (pinned = always in context)
yaad_stale Find memories invalidated by git changes
yaad_proactive Predict what context the agent needs next
yaad_compact Merge low-confidence memories into summaries
yaad_mental_model Auto-generated project summary
yaad_skill_store Save a reusable step sequence
yaad_skill_get Retrieve and replay a skill
yaad_session_recap Summary of the previous session
yaad_subgraph Extract neighborhood around a memory
yaad_impact What memories are affected by a file change?
yaad_status Graph stats (nodes, edges, sessions)
yaad_decay Manually trigger confidence decay
yaad_gc Garbage collect archived memories
yaad_embed Generate vector embedding for a node
yaad_export Export graph as JSON/Markdown/Obsidian
yaad_import Import graph from JSON

Architecture

┌─────────────────────────────────────────────────────────────────┐
│              YOUR CODING AGENT                                   │
│  Hawk · Claude Code · Cursor · Gemini CLI · Any MCP Agent       │
└──────┬───────────────┬──────────────────────────────────────────┘
       │ MCP (stdio)   │ REST/HTTPS (127.0.0.1:3456)
       ▼               ▼
┌─────────────────────────────────────────────────────────────────┐
│                       YAAD                                      │
│  Memory Engine · Graph Engine · 4-Path Search · Dual-Stream     │
├─────────────────────────────────────────────────────────────────┤
│  SQLite (WAL mode) · FTS5 · Embeddings (optional)              │
└─────────────────────────────────────────────────────────────────┘

Zero dependencies. Pure Go. No CGO. No Docker. No cloud. Embedded by the host agent; all data stays local (SQLite, WAL mode).


Library API

Yaad is consumed as a Go library. The core entry points:

import (
    yaad "github.com/GrayCodeAI/yaad/engine"
    "github.com/GrayCodeAI/yaad/storage"
)

// Open (or create) a memory store rooted at the repo / working dir.
store, err := storage.Open(".yaad")
eng := yaad.New(store)

// Store, recall, link, and inspect impact — the same operations the
// host agent exposes as tools (see the table above).
eng.Remember(ctx, yaad.Note{Type: "decision", Text: "Chose NATS for backpressure"})
hits, _ := eng.Recall(ctx, "why did we choose NATS?")

See ARCHITECTURE.md and api/openapi.yaml for the full surface. Versioning/rollback, export/import, decay, and GC are all available as engine methods (the host agent decides which to expose).


Configuration

Generated at .yaad/config.toml:

[server]
port = 3456
host = "127.0.0.1"

[memory]
hot_token_budget = 800
warm_token_budget = 800
max_memories = 10000

[search]
bm25_weight = 0.5
vector_weight = 0.5
default_limit = 10

[decay]
enabled = true
half_life_days = 30
min_confidence = 0.1
boost_on_access = 0.2

[git]
watch = true
auto_stale = true

Development

git clone https://github.com/GrayCodeAI/yaad.git
cd yaad
make build           # go build ./... (library packages; no standalone binary)
make test            # Run all tests
make cover           # Coverage report (coverage.out + coverage.html)
make ci              # Everything CI runs (tidy, fmt, vet, lint, test-race, security)

Documentation

Doc What
ARCHITECTURE.md Technical architecture
COMPARISON.md vs Mem0, Letta, Engram, agentmemory
CONTRIBUTING.md How to contribute
CHANGELOG.md Release notes
api/openapi.yaml OpenAPI spec

Community


MIT © 2026 GrayCodeAI

yaad (याद) — Hindi/Urdu for memory, remembrance

Documentation

Overview

Package yaad provides a graph-native persistent memory engine for AI coding agents. It stores, indexes, and retrieves contextual memories using a DAG with typed nodes and edges, a 4-signal hybrid search (BM25 + vector + graph + temporal) fused via reciprocal-rank fusion, and tiered spatial memory (hot/warm/cold) with promotion thresholds and token budgets.

Spatial memory management (this file) divides entries into three access tiers, promotes entries based on recent hit counts, demotes idle entries, and enforces per-tier token budgets by evicting the least-recently accessed entries first. The system is safe for concurrent use.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ImportMemory

func ImportMemory(sm *SpatialMemory, export *MemoryExport) (int, error)

ImportMemory copies the entries from export into sm, overwriting any existing entries that share the same ID. Each imported entry retains its original tier, access count, and timestamps so the restored state matches the exported state. Entries with an empty ID or an unrecognized tier are skipped. It returns the number of entries successfully imported.

Types

type ExportConfig

type ExportConfig struct {
	// IncludeColdTier includes cold-tier (rarely accessed) entries when true.
	// When false, cold entries are omitted, reducing export size.
	IncludeColdTier bool

	// MaxEntries caps the number of entries in the export. Entries are sorted
	// by LastAccessed descending before truncation, so the most recently
	// accessed entries are retained. Zero or negative means unlimited.
	MaxEntries int

	// Format declares the intended serialization format: "json" (default) or
	// "gzipped". ExportMemory validates this value; the actual compression is
	// the caller's responsibility (e.g. wrapping the writer in gzip.NewWriter).
	Format string
}

ExportConfig controls what ExportMemory includes in a MemoryExport and in which on-wire format the export is intended to be written.

type MemoryEntry

type MemoryEntry struct {
	ID           string
	Content      string
	Tier         TierType
	AccessCount  int
	LastAccessed time.Time
	CreatedAt    time.Time
	TokenCount   int
}

MemoryEntry is a single memory unit tracked by the spatial memory system.

type MemoryExport

type MemoryExport struct {
	Version    string        `json:"version"`
	ExportedAt time.Time     `json:"exported_at"`
	Entries    []MemoryEntry `json:"entries"`
	Stats      TierStatsMap  `json:"stats"`
}

MemoryExport is a serializable snapshot of a SpatialMemory's entries and tier statistics. It is produced by ExportMemory and consumed by ImportMemory to back up and restore spatial memory state.

func ExportMemory

func ExportMemory(sm *SpatialMemory, config ExportConfig) (*MemoryExport, error)

ExportMemory captures the current state of sm into a MemoryExport, shaped by config. The returned Stats describe only the entries that made it into the export (after tier filtering and MaxEntries truncation).

func ReadMemoryExport

func ReadMemoryExport(r io.Reader) (*MemoryExport, error)

ReadMemoryExport decodes a MemoryExport from the JSON format produced by WriteTo.

func (*MemoryExport) Summary

func (me *MemoryExport) Summary() string

Summary returns a one-line human-readable description of the export, including its version, timestamp, and per-tier entry breakdown.

func (*MemoryExport) WriteTo

func (me *MemoryExport) WriteTo(w io.Writer) (int64, error)

WriteTo serializes the memory export as indented JSON and writes it to w. It implements io.WriterTo.

type SpatialMemory

type SpatialMemory struct {
	// contains filtered or unexported fields
}

SpatialMemory manages a tiered set of memory entries with token budgets per tier. It is safe for concurrent use.

func NewSpatialMemory

func NewSpatialMemory(hotBudget, warmBudget, coldBudget int) *SpatialMemory

NewSpatialMemory creates a SpatialMemory with the given token budgets per tier.

func (*SpatialMemory) Access

func (sm *SpatialMemory) Access(id string) (MemoryEntry, bool)

Access records an access to the memory entry with the given id, updating its tier according to the promotion rules. Returns a copy of the entry, or false if the id was not found.

func (*SpatialMemory) Add

func (sm *SpatialMemory) Add(entry MemoryEntry)

Add inserts a new memory entry. If the entry already exists it is overwritten. New entries always land in TierCold; LastAccessed defaults to now and CreatedAt fills in if zero.

func (*SpatialMemory) Compact

func (sm *SpatialMemory) Compact()

Compact demotes entries that violate idle thresholds and enforces token budgets by demoting the least-recently-accessed entries first.

func (*SpatialMemory) Remove

func (sm *SpatialMemory) Remove(id string) bool

Remove deletes the memory entry with the given id.

func (*SpatialMemory) TierStats

func (sm *SpatialMemory) TierStats() map[TierType]struct{ Count, Tokens int }

TierStats returns the count of entries and total token usage for each tier. It snapshots entry pointers under the lock and computes stats outside it, reducing lock hold time for large entry sets.

type TierStats

type TierStats struct {
	Count  int `json:"count"`
	Tokens int `json:"tokens"`
}

TierStats holds the aggregate entry count and token usage for one tier.

type TierStatsMap

type TierStatsMap map[TierType]TierStats

TierStatsMap maps each TierType to its aggregate statistics.

type TierType

type TierType int

TierType represents the access tier of a memory entry.

const (
	// TierHot is the fast path for frequently accessed memories.
	TierHot TierType = iota
	// TierWarm holds recently accessed memories.
	TierWarm
	// TierCold holds rarely accessed memories, ready for archival.
	TierCold
)

Directories

Path Synopsis
benchmark
Package browse presents stored memories as a navigable virtual filesystem (directories per category, files per memory) for tree/search/get access.
Package browse presents stored memories as a navigable virtual filesystem (directories per category, files per memory) for tree/search/get access.
Package compact implements memory compaction — auto-summarize when the graph exceeds a token budget.
Package compact implements memory compaction — auto-summarize when the graph exceeds a token budget.
Package config loads, validates, and provides defaults for yaad's configuration (server, storage, embeddings, and related settings).
Package config loads, validates, and provides defaults for yaad's configuration (server, storage, embeddings, and related settings).
Package conflict detects and resolves contradictory memories.
Package conflict detects and resolves contradictory memories.
Package dedup implements rolling-window deduplication.
Package dedup implements rolling-window deduplication.
Package embeddings provides a pluggable embedding provider interface.
Package embeddings provides a pluggable embedding provider interface.
Package engine is the core of yaad: it ties storage, the typed graph, dedup, decay, conflict resolution, and hybrid recall together behind the Engine type, the main entry point for remember/recall/forget operations.
Package engine is the core of yaad: it ties storage, the typed graph, dedup, decay, conflict resolution, and hybrid recall together behind the Engine type, the main entry point for remember/recall/forget operations.
cognitive
Package cognitive implements yaad's higher-level memory subsystems — epistemic state, curiosity, boundary detection, reconsolidation, and related processes layered on top of the core engine.
Package cognitive implements yaad's higher-level memory subsystems — epistemic state, curiosity, boundary detection, reconsolidation, and related processes layered on top of the core engine.
Package exportimport handles JSON round-trip, Markdown, and Obsidian vault export.
Package exportimport handles JSON round-trip, Markdown, and Obsidian vault export.
Package git provides git-awareness for memory staleness: it watches the working tree and HEAD so memories tied to changed files can be flagged or decayed.
Package git provides git-awareness for memory staleness: it watches the working tree and HEAD so memories tied to changed files can be flagged or decayed.
Package graph implements yaad's typed, acyclic memory graph: nodes connected by typed edges, with cycle-checked insertion and traversal helpers used by recall and link suggestion.
Package graph implements yaad's typed, acyclic memory graph: nodes connected by typed edges, with cycle-checked insertion and traversal helpers used by recall and link suggestion.
Package hooks implements auto-capture hooks for lifecycle events.
Package hooks implements auto-capture hooks for lifecycle events.
Package ingest - semantic code chunker.
Package ingest - semantic code chunker.
Package intent classifies query intent to route graph traversal.
Package intent classifies query intent to route graph traversal.
internal
bench
Package bench implements a LongMemEval-style evaluation harness for Yaad.
Package bench implements a LongMemEval-style evaluation harness for Yaad.
daemon
Package daemon manages yaad's background server lifecycle: PID file tracking, health checks, and auto-start logic.
Package daemon manages yaad's background server lifecycle: PID file tracking, health checks, and auto-start logic.
proactive
Package proactive implements proactive context preloading for yaad.
Package proactive implements proactive context preloading for yaad.
search
Package search provides intent-aware retrieval routing for yaad's graph.
Package search provides intent-aware retrieval routing for yaad's graph.
server
Package server exposes yaad over the network: the REST API, the MCP server, and the browser dashboard, all backed by the same engine.
Package server exposes yaad over the network: the REST API, the MCP server, and the browser dashboard, all backed by the same engine.
telemetry
Package telemetry provides OpenTelemetry metric instruments for yaad.
Package telemetry provides OpenTelemetry metric instruments for yaad.
temporal
Package temporal provides bi-temporal validity windows for memory nodes.
Package temporal provides bi-temporal validity windows for memory nodes.
tls
Package tls builds the server's TLS configuration, generating and auto-renewing a self-signed localhost certificate when none is supplied.
Package tls builds the server's TLS configuration, generating and auto-renewing a self-signed localhost certificate when none is supplied.
tui
Package tui implements the Yaad terminal UI using Bubbletea.
Package tui implements the Yaad terminal UI using Bubbletea.
version
Package version provides the canonical version string for yaad.
Package version provides the canonical version string for yaad.
Package mental implements auto-generated project mental models.
Package mental implements auto-generated project mental models.
Package privacy strips secrets and sensitive material from memory content before storage, via pattern matching, entropy heuristics, and <private> tag removal.
Package privacy strips secrets and sensitive material from memory content before storage, via pattern matching, entropy heuristics, and <private> tag removal.
Package profile implements auto-maintained user/project profiles.
Package profile implements auto-maintained user/project profiles.
Package skill implements procedural memory — reusable step sequences.
Package skill implements procedural memory — reusable step sequences.
Package storage is yaad's persistence layer: a CGO-free SQLite store with FTS5 search, prepared-statement caching, and busy-retry, plus the Storage interface the rest of the system depends on.
Package storage is yaad's persistence layer: a CGO-free SQLite store with FTS5 search, prepared-statement caching, and busy-retry, plus the Storage interface the rest of the system depends on.
Package temporal implements an immutable temporal backbone.
Package temporal implements an immutable temporal backbone.
Package utils holds small, dependency-free helpers shared across yaad.
Package utils holds small, dependency-free helpers shared across yaad.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL