benchmark/

Yaad Benchmark Publication Workflow

This directory is the publication surface for Yaad benchmarks.

It complements:

• yaad/internal/bench for retrieval-quality evaluation
• yaad/benchmark/benchmark_test.go for microbenchmarks
• yaad/benchmark/compare.md for architecture/performance narrative
• yaad/benchmark/manifests/ for machine-readable suite definitions

Official suite ids

• yaad-longmem-core
• yaad-microbench
• yaad-locomo-beam-publication (planned)

See the Hawk-side benchmark registry at hawk/docs/benchmarks/SUITES.md.

Current published baselines

• yaad-longmem-core: results/yaad-longmem-core/2026-06-27/
• yaad-microbench: results/yaad-microbench/2026-06-27/

Current runnable commands

/bin/zsh -lc 'GOCACHE=$PWD/.gocache go run ./benchmark/cmd/longmemreport --suite default'
go test ./benchmark/ -bench=. -benchmem -benchtime=1s
go test ./benchmark/ -bench=BenchmarkRemember -benchmem
go test ./benchmark/ -bench=BenchmarkRecall -benchmem
go test ./benchmark/ -bench=BenchmarkGraphBFS -benchmem
/bin/zsh -lc 'GOCACHE=$PWD/.gocache go test ./internal/bench -count=1'

Publication layout

Recommended committed layout:

benchmark/
  README.md
  compare.md
  results/
    yaad-longmem-core/
      2026-06-27/
        report.md
        result.txt
        notes.md
    yaad-microbench/
      2026-06-27/
        report.md
        result.txt
        notes.md

Required metrics

yaad-longmem-core

• R@1
• R@3
• R@5
• R@10
• MRR
• avg tokens/query
• duration

yaad-microbench

• latency
• allocs/op
• bytes/op where emitted
• benchmark environment notes

Promotion rule

Do not cite a Yaad benchmark as evidence in cross-project comparison docs unless:

1. the command used is recorded
2. the commit is recorded
3. the metric table is committed or linked as an artifact
4. the benchmark scope is stated clearly