mcpeval

Overview ¶

Command mcpeval drives the gen-eval corpus through the REAL served stack — it spawns the shipped binary's `mcp` subcommand per project and talks JSON-RPC over stdio, exactly like an agent host does.

DECISION(2026-07): cmd/eval measures the retrieval library; four eval-vs-served config divergences (intent ranker, rerank pool, adaptive, protect fix) lived undetected because nothing scored the served path. This harness closes that class: if the MCP server's ranking drifts from the library's, the numbers diverge here first.

Usage:

mcpeval -bin dist/contextmaxxer.exe \
        -manifest internal/eval/testdata/manifest.public.json [-holdout]