release-parity

Overview ¶

Command release-parity runs two mache binaries through the same MCP tool calls on the same corpus and reports every difference in what they answer.

It is the behavior half of a release gate. `task check` proves the tree is internally consistent; it cannot say what changed for someone upgrading from the last release. This does: the release binary and the candidate each serve the corpus over stdio, every call in a committed plan goes to both, and the answers are compared. Each difference must then be explained by a CHANGELOG entry, or it is a regression.

Three properties carry the weight:

• EXHAUSTIVE: the run fails if the candidate registers a tool the plan never calls, so a new tool cannot ship unexercised.
• NOISE IS MEASURED, NOT GUESSED: each binary runs the plan [runsPerSide] times, and any JSON path whose value (or presence) differs between runs of the SAME binary is masked before the cross-version comparison. No list of "volatile-looking" field names.
• ISOLATED: each binary gets its own HOME, so neither reads the other's parse cache. The version-namespaced leyline binaries are linked in, so no run downloads anything.

Usage:

release-parity -base ./mache-v0.21.1 -head bin/mache \
  -corpus testdata/snapshots/small-go-golden \
  -plan tools/release-parity/plans/small-go-golden.json -out .parity/small-go
release-parity -head bin/mache -corpus ... -list   # print the registry