perfconvert

Overview ¶

Command perfconvert turns the offline scenario harness's per-scenario JSON KPIs (the []kpi.ScenarioResult that `task perf:scenarios` writes to $MECATL_PERF_JSON) into the THREE github-action-benchmark custom-format files the perf workflow feeds to the trend dashboard + alert gate. See docs/adr/0019-perf-tracking.md (Phase 3) and .github/workflows/perf.yml.

THREE suites (a "customSmallerIsBetter" gate, a "customBiggerIsBetter" gate, and a "customSmallerIsBetter" ADVISORY suite):

• SMALLER (gated, fail-on-alert) — per scenario allocs_per_op (EXCEPT the render benches; see RENDER below), plus tokens_total and goroutine_delta for ALL scenarios (the tui render benches' tokens/goroutines are deterministic — always 0 — so they gate safely).
• BIGGER (gated, fail-on-alert) — cache_hit_rate for the cacheHitWhitelist.
• RENDER (ADVISORY, fail-on-alert:false) — allocs_per_op for ONLY the renderAllocAdvisory scenarios (the tui_scrollback_view* benches). These are split out because their allocs/op is NOT deterministic the way the loop scenarios' is: it carries a b.N residual (the streaming bench re-joins the whole scrollback every op) and process-wide runtime.ReadMemStats background noise (perf/kpi cannot goroutine-scope the MemStats read), so the gh-pages history showed view ~4.6% / steady ~30% swings on commits that touched no TUI code at all. Gating them at the shared 2% threshold false-positives; carrying them as advisory keeps the trend visible without failing a PR. (This is the escalation the SMALLER suite's comment anticipated — now shipped.)

It imports ONLY perf/kpi + the standard library — same leaf posture as the kpi package itself; it never reaches into engine/... or internal/....

Aggregation: under `go test -count=N` a scenario emits N rows that share (Name, GitSHA) and differ only in Sample. perfconvert groups by Name and takes the MEDIAN of each metric across the samples, so a single noisy sample cannot move the gated value. The deterministic metrics (allocs/op, tokens) are identical across samples anyway; the median is belt-and-braces and matters most for the advisory render-allocs metric routed through here.