eval-gate

Overview ¶

eval-gate is a CI regression gate that compares the latest `make eval` output against the committed baseline. Exits non-zero when any tracked metric drifts beyond the per-metric tolerance.

This is the LLM-free half of the eval surface (Stage B is the LLM-driven half — too expensive to run on every PR). It runs in ~2 minutes end-to-end and gates:

• eval/baseline/retrieval.json vs eval/results/latest/retrieval.json Aggregate recall, precision, f1 must each stay within the tolerance. Drop below tolerance → regression.
• eval/baseline/validate.json vs eval/results/latest/validate.json Issue count must not increase. The schema validator is meant to converge on zero; new findings should land as code fixes, not as silent baseline drift.

Run: go run ./cmd/eval-gate (after `make eval` has written the `eval/results/latest/` artefacts)