Documentation
¶
Overview ¶
Package memory loads optional context files that are auto-injected into the system prompt at session start.
Layout:
~/.yottacode/USER.md — global preferences (curated, human-only)
<cwd>/.yottacode/YOTTACODE.md — per-repo context (curated, agent-writable)
~/.yottacode/memory/user/<name>.md — user-scope agent-managed memories
~/.yottacode/memory/projects/<project_slug>/<name>.md — project-scope agent-managed memories
~/.yottacode/memory/projects/<project_slug>/subagents/ — that project's subagent run transcripts
(owned by internal/subagents; subdirs are
invisible to scanMemoryDir)
USER.md and YOTTACODE.md are the trust anchor: they always inject in full. The agent-managed memory dirs are managed via the memory_save / memory_forget tools — the agent decides in-band when something is worth remembering. MEMORY.md inside each memory dir is an auto-generated index that always renders too; the per-file bodies are filtered per turn by the retrieval orchestrator (see retrieve.go).
The project_slug is derived by ProjectSlug(cwd) — see path.go.
Package memory — retrieval orchestrator.
Agent-managed memory grows over time. By v0.4 a heavy user can accumulate dozens of typed memory files; concatenating every body into the system prompt on every turn taxes both the context window and the model's attention.
This file scores each MemoryEntry against the current user prompt and returns a ranked, length-bounded subset. It deliberately does NOT touch USER.md / YOTTACODE.md — those are the trust anchor: short, curated, and always injected in full. USER.md is human-only; YOTTACODE.md is human-seeded and the agent keeps it fresh as the project evolves (parity with how Claude Code maintains CLAUDE.md).
MEMORY.md (the per-scope index) is also unfiltered — it's the table of contents, and the model needs to know what files exist even when their bodies aren't injected. Only per-entry bodies pass through Select.
Index ¶
- Constants
- func ArchivePrior(memPath, stamp string) (string, error)
- func AtomicWrite(path string, data []byte, perm os.FileMode) error
- func CosineSimilarity(a, b []float32) float64
- func DeleteVec(mdPath string)
- func EnsureProjectMemoryDir(cwd string) (string, error)
- func EnsureUserMemoryDir() (string, error)
- func Expand(stemmed string) []string
- func MemoryFilePath(scope, name, cwd string) (string, error)
- func MemoryRoot() (string, error)
- func NeedsReembed(path, currentModel string) bool
- func ProjectMemoryDir(cwd string) (string, error)
- func ProjectSlug(cwd string) string
- func ReadVec(path string) ([]float32, error)
- func RegenerateMemoryIndex(scope, cwd string) error
- func RenderFrontmatter(name, memType, description string, created time.Time) string
- func RenderMemoryIndex(scope string, entries []MemoryEntry) string
- func Score(entry MemoryEntry, query string) float64
- func Stem(word string) string
- func StemExpandTokenize(s string) []string
- func SystemPrompt(base string, l Loaded) string
- func SystemPromptFor(base string, l Loaded, query string, cfg config.RetrievalConfig) string
- func SystemPromptForSemantic(ctx context.Context, base string, l Loaded, query string, ...) string
- func UserMemoryDir() (string, error)
- func VecPath(mdPath string) string
- func WriteVec(path string, vec []float32) error
- func WriteVecWithModel(path string, vec []float32, model string) error
- type Corpus
- type EmbedClient
- type Frontmatter
- type Loaded
- type MemoryEntry
- type Scored
- type Summary
- type VecMeta
Constants ¶
const ArchiveDirName = ".archive"
ArchiveDirName is the subdirectory inside a memory directory where prior versions of overwritten memories are kept. It is a directory, so scanMemoryDir skips it — archived versions never appear in the index, retrieval, or `memory list`; they exist only for recovery.
const InteractiveEmbedTimeout = 2 * time.Second
InteractiveEmbedTimeout bounds a single embedding call made on the synchronous, user-facing path: per-turn retrieval (rebuilding the system prompt before a turn fires) and memory_save. The default client Timeout (30s) is correct for batch `memory reindex`, but on the interactive path a mid-session Ollama hang (model unloaded, swapped, or busy) would otherwise freeze the TUI for up to 30s every turn. Interactive callers set Timeout to this short bound; on timeout the retriever falls back to BM25 and memory_save just skips the .vec.
const SubagentsDirName = "subagents"
SubagentsDirName is the directory inside each project memory dir that holds that project's subagent run transcripts. Exported (like ArchiveDirName) because internal/subagents joins it in TranscriptDirFor and the save-proactivity eval skips it when counting memories — one const keeps every site in agreement.
Variables ¶
This section is empty.
Functions ¶
func ArchivePrior ¶ added in v0.3.0
ArchivePrior copies the current contents of memPath (if the file exists) into <dir>/.archive/<name>.<stamp>.md so an overwrite can never silently destroy a different memory that happened to share the name. stamp must be unique per call (the caller passes a timestamp). Returns the archive path written, or "" when there was nothing to archive (the file did not exist). Durable like every other memory write (staged + fsync'd via AtomicWrite).
func AtomicWrite ¶ added in v0.3.0
AtomicWrite writes data to path durably and atomically.
It stages to a UNIQUE temp file in the same directory (via os.CreateTemp), fsyncs the data, renames onto the destination, then fsyncs the directory so the rename survives a crash. The temp file is removed on every error path; a successful rename consumes it, making the deferred Remove a no-op.
This replaces the older "<path>.tmp" + os.WriteFile + os.Rename pattern, which had two latent defects the memory layer relied on:
- A DETERMINISTIC temp name ("<path>.tmp"): two writers targeting the same destination (two yottacode processes in one repo, or a parent loop and a detached background subagent sharing the same tool) staged through the same file, so their bytes interleaved into one descriptor or one rename fired mid-write — corrupting the destination. A unique temp name makes concurrent writes last-writer-wins on a valid file instead.
- No fsync before rename: on a crash/power-loss just after the call returned "saved", the file could come back zero-length or stale.
Using os.CreateTemp also closes a staging-file symlink-follow gap: it refuses to open through a pre-planted symlink at the temp path.
func CosineSimilarity ¶ added in v0.3.0
CosineSimilarity returns the cosine similarity between two vectors. Returns 0 if either vector is zero-length or the vectors have different dimensions.
func DeleteVec ¶ added in v0.3.0
func DeleteVec(mdPath string)
DeleteVec removes a sidecar vector file if it exists.
func EnsureProjectMemoryDir ¶
EnsureProjectMemoryDir creates the project-scope memory directory.
func EnsureUserMemoryDir ¶
EnsureUserMemoryDir creates the user-scope memory directory if missing.
func Expand ¶ added in v0.3.0
Expand returns the stemmed input token plus any known synonyms. The first element is always the input itself. Tokens not in any synonym group return a single-element slice. All entries are pre-stemmed so callers must stem before calling.
func MemoryFilePath ¶
MemoryFilePath validates a memory name and returns the absolute file path under the chosen scope. Scope must be "user" or "project".
func MemoryRoot ¶ added in v0.3.0
MemoryRoot returns the root of the agent-managed memory tree: $YOTTACODE_HOME/memory when the override is set — the shared ychome.Dir resolution skills, plans, and agent definitions also use, so all global state follows the same root — or ~/.yottacode/memory otherwise.
func NeedsReembed ¶ added in v0.3.0
NeedsReembed reports whether a .vec file should be re-embedded, either because it uses the legacy format (no model recorded) or because it was embedded with a different model.
func ProjectMemoryDir ¶
ProjectMemoryDir returns ~/.yottacode/memory/projects/<slug>/ — the per-project, per-user agent-managed memory directory. The project's subagent transcripts nest under <dir>/subagents (see internal/subagents.TranscriptDirFor); scanMemoryDir ignores subdirectories, so the two cohabit without the loader seeing transcript files.
func ProjectSlug ¶
ProjectSlug returns a stable, slug-safe identifier for the project rooted at cwd. Strategy (highest-priority first):
- Git remote URL of `origin`. Survives renames of cwd, and is the same string across machines and clones — project memory follows the project, not the directory it happens to live in. Parsed from formats like `https://github.com/user/repo.git`, `git@github.com:user/repo.git`, `ssh://git@host/path/repo`.
- `filepath.Base(cwd)` slugified. Used when no git remote is configured (uncommitted scratch dir, vendored copy without git, brand-new project pre-init).
Returns "default" when both lookups fail (empty cwd, root-only path, exotic edge cases). The returned string is guaranteed to match projectSlugPattern, so callers can use it directly as a path component without further sanitization.
Two unrelated repos with the same basename and no git remote will collide. The collision is documented; users who care can either initialize a git repo (which gets them a unique remote-derived slug) or live with the merge.
func ReadVec ¶ added in v0.3.0
ReadVec reads a float32 vector from a .vec file. Handles both the new header format and legacy raw float32 files. Returns nil, nil if the file does not exist.
func RegenerateMemoryIndex ¶
RegenerateMemoryIndex re-scans the chosen scope's memory dir and rewrites MEMORY.md atomically. Used by memory_save and memory_forget after every mutation. If the dir is empty or missing after the mutation, MEMORY.md is removed instead of being written as an empty stub.
func RenderFrontmatter ¶
RenderFrontmatter writes the four-field header. Description is expected to already be one line (caller strips newlines before passing); the writer doesn't re-validate. Created is rendered as RFC3339 in UTC.
func RenderMemoryIndex ¶
func RenderMemoryIndex(scope string, entries []MemoryEntry) string
RenderMemoryIndex builds the MEMORY.md text for one scope: a header, a stable preamble warning humans not to edit it, and a per-type section listing each entry as a markdown link with its description. Empty entries produce a minimal index (just the preamble) so the scanner still recognizes the file as ours.
func Score ¶
func Score(entry MemoryEntry, query string) float64
Score returns a relevance score in [0.0, 1.0] for the given entry against the query using the legacy keyword strategy. Pure and deterministic. Kept for backward compatibility with strategy="keyword".
func Stem ¶ added in v0.3.0
Stem returns the Porter-stemmed form of a single lowercase English token. Implements the Porter 1980 suffix-stripping algorithm (5 steps). Pure, deterministic, zero allocations beyond the result.
func StemExpandTokenize ¶ added in v0.3.0
StemExpandTokenize tokenizes, stems, and expands synonyms for the query side. Synonym expansion runs on queries only (not documents) to increase recall without inflating document frequencies.
func SystemPrompt ¶
SystemPrompt composes the base prompt with every loaded memory section. Each section is framed as background reference — not as a topic to describe.
func SystemPromptFor ¶
SystemPromptFor is the per-turn variant of SystemPrompt: USER.md and YOTTACODE.md inject in full as before, both MEMORY.md indexes inject in full (they're the table of contents), and per-entry bodies pass through Select(query, cfg) first.
func SystemPromptForSemantic ¶ added in v0.3.0
func SystemPromptForSemantic(ctx context.Context, base string, l Loaded, query string, cfg config.RetrievalConfig, embedClient *EmbedClient) string
SystemPromptForSemantic is like SystemPromptFor but accepts an optional EmbedClient for semantic retrieval. Pass nil to use keyword/bm25 scoring only. ctx bounds the embed call — callers on an interactive path should pass a cancelable context (the TUI uses the turn context) so retrieval never outlives the work it serves.
func UserMemoryDir ¶
UserMemoryDir returns ~/.yottacode/memory/user/ — the cross-project agent-managed memory directory. Memories saved here apply to every session for this user.
func WriteVec ¶ added in v0.3.0
WriteVec writes a float32 vector to disk with a header that records the embedding model name and dimension count.
func WriteVecWithModel ¶ added in v0.3.0
WriteVecWithModel writes a vector with an explicit model name header.
Staging uses a UNIQUE temp file (os.CreateTemp) and fsyncs before rename, for the same reasons as memory.AtomicWrite: a fixed "<path>.tmp" name let a concurrent writer's deferred cleanup delete this writer's in-flight temp (silently dropping the .vec), and the missing fsync left a crash window. The streaming header+binary layout can't go through AtomicWrite's single-buffer API, so the same guarantees are applied inline here.
Types ¶
type Corpus ¶ added in v0.3.0
Corpus holds precomputed statistics for BM25 scoring. Built once per retrieval pass from the current memory set via BuildCorpus.
func BuildCorpus ¶ added in v0.3.0
func BuildCorpus(entries []MemoryEntry) *Corpus
BuildCorpus tokenizes and stems all entries, computing document frequencies and average document length. O(n * avg_tokens).
func (*Corpus) BM25 ¶ added in v0.3.0
BM25 scores equal-weighted query stems against document at docIdx. Retained for the public/legacy path; the agent's retrieval uses the weighted variant (bm25Weighted) so synonyms count for less than exact terms.
func (*Corpus) Rank ¶ added in v0.3.0
Rank scores all documents against equal-weighted query stems and returns them sorted by descending score with alphabetical tie-breaking by entry name. The agent's retrieval path uses rankWeighted so synonym terms are down-weighted; Rank stays equal-weighted for the public/CLI surfaces.
type EmbedClient ¶ added in v0.3.0
EmbedClient talks to a local Ollama embedding endpoint.
func NewEmbedClient ¶ added in v0.3.0
func NewEmbedClient(baseURL, model string) *EmbedClient
NewEmbedClient returns a client configured for local Ollama embeddings. BaseURL defaults to http://localhost:11434 (or $OLLAMA_HOST). Model defaults to "nomic-embed-text".
func (*EmbedClient) Available ¶ added in v0.3.0
func (c *EmbedClient) Available(ctx context.Context) bool
Available reports whether the configured embedding model is installed on the Ollama server. Uses a short probe timeout.
func (*EmbedClient) Embed ¶ added in v0.3.0
Embed returns the embedding vector for a single text input. Calls POST /api/embeddings on the configured Ollama server.
func (*EmbedClient) Status ¶ added in v0.3.0
func (c *EmbedClient) Status(ctx context.Context) (reachable, installed bool)
Status probes the Ollama server and reports separately whether the server is reachable and whether the configured model is installed. Distinguishes "Ollama isn't running" (reachable=false) from "Ollama is running but the model was removed" (reachable=true, installed=false) so callers can surface a targeted warning.
type Frontmatter ¶
Frontmatter is the YAML-ish header on every agent-managed memory file. Real YAML is overkill for four flat fields — a tolerant key:value scanner keeps the writer one-line-per-field and the reader tiny.
func ParseFrontmatter ¶
func ParseFrontmatter(data []byte) (fm Frontmatter, body string, ok bool)
ParseFrontmatter splits frontmatter from body. Returns ok=false when the frontmatter block is missing or malformed; in that case fm is zero and body equals the input. Tolerant of files written by hand (no frontmatter at all is fine — caller defaults Type to "reference" and Name to the basename).
type Loaded ¶
type Loaded struct {
UserPath string
UserText string
ProjectPath string // YOTTACODE.md
ProjectText string
// User-scope agent-managed memories.
UserMemoryDir string
UserMemoryIndex string // raw MEMORY.md text (rendered if missing on disk)
UserMemories []MemoryEntry
// Project-scope agent-managed memories (per ProjectSlug(cwd)).
ProjectMemoryDir string
ProjectMemoryIndex string
ProjectMemories []MemoryEntry
}
Loaded carries the raw contents (and resolved paths) of every memory source read at session start. The two trust anchors (USER.md and YOTTACODE.md) sit alongside the user-scope and project-scope agent-managed memory directories. ProjectPath / ProjectText refer to YOTTACODE.md (the field name is historical from the YOTTACODE.md era).
type MemoryEntry ¶
type MemoryEntry struct {
Path string
Scope string // "user" | "project"
Name string // basename without .md
Type string // user | feedback | project | reference
Description string
Body string
}
MemoryEntry is one parsed memory file: enough to render the prompt and the MEMORY.md index. Body is raw markdown (frontmatter stripped).
func Select ¶
func Select(entries []MemoryEntry, query string, cfg config.RetrievalConfig) []MemoryEntry
Select ranks entries against the query and returns at most cfg.TopK with score >= cfg.MinScore, further capped so the combined entry bodies stay within cfg.MaxBytes (0 = unlimited). Strategy selects the scoring algorithm: "keyword" uses the legacy exact-token scorer, "bm25" (default) uses BM25 with stemming and synonyms.
func SelectWithEmbeddings ¶ added in v0.3.0
func SelectWithEmbeddings(ctx context.Context, entries []MemoryEntry, query string, cfg config.RetrievalConfig, embedClient *EmbedClient) []MemoryEntry
SelectWithEmbeddings is like Select but accepts an optional EmbedClient for semantic scoring. When embedClient is non-nil and strategy is "semantic", BM25 scores are combined with cosine similarity from vector embeddings. ctx bounds the embed call: the TUI passes the turn context so Esc cancels an in-flight embed instead of waiting out its timeout.
type Scored ¶
type Scored struct {
Entry MemoryEntry
Score float64
}
Scored pairs a memory entry with the relevance score the orchestrator assigned it for a particular query. Score is in [0.0, 1.0]; deterministic across runs.
func SelectWithEmbeddingsScored ¶ added in v0.3.0
func SelectWithEmbeddingsScored(ctx context.Context, entries []MemoryEntry, query string, cfg config.RetrievalConfig, embedClient *EmbedClient) []Scored
SelectWithEmbeddingsScored is like SelectWithEmbeddings but returns Scored entries with their relevance scores preserved. Used by memory_search so the agent can see how well each memory matched.
type Summary ¶
Summary is the short tag used by the TUI status bar — a human-friendly one-liner describing which memory sources are active.
Source Files