memmy-eval

Overview ¶

Command memmy-eval is the validation harness CLI for memmy.

It owns four phases of the framework:

ingest   — extract Claude Code session JSONL into a versioned dataset
queries  — generate (or top-up) labeled queries against an ingested corpus
run      — replay the corpus into a fresh memmy db, run the query battery,
           write per-run metrics
sweep    — run a parameter matrix over the same corpus + query set
ls       — list datasets in the configured root

Datasets live OUTSIDE this repo at $MEMMY_EVAL_HOME (default ~/.local/share/memmy-eval/<name>/).