benchcmp

Overview ¶

Command benchcmp compares two sets of `go test -bench` results using only the standard library. It is the repository's dependency-compliant stand-in for benchstat: for every benchmark and unit present in both inputs it reports the old and new medians, the relative change and a two-sided Mann-Whitney U p-value. A change whose p-value exceeds alpha is reported as "~" (no significant difference), never as a speedup or slowdown. Rows whose old and new observations are all identical (typically 0 allocs/op) are omitted.

Usage:

go run ./tools/benchcmp [-alpha 0.05] old.txt new.txt

Run each side with -count of at least 6 (preferably 10, interleaved) so the test has enough power; with fewer samples every row reports "~".