Help us shape the future of Go! Take the 2026 Go Developer Survey

bench

package
v0.16.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 31, 2026 License: MIT Imports: 4 Imported by: 0

Documentation

Overview

Package bench holds the pure-logic measurement core for the codegraph benchmark harness (PERF-01, INDX-06). Peak RSS is always measured externally, via the OS-level rusage of a child process the caller spawned (exec.Cmd.ProcessState.SysUsage()) — never via in-process Go runtime memory statistics: an externally observed child-process rusage figure is the only peak-RSS number that stays comparable across runs and machines, because an in-process Go runtime statistic measures the harness rather than the subject (D-05). This package has no network or crypto surface and does not shell out itself; callers own the exec.Cmd. Measurement helpers never panic — they return (T, error) so a single bad run fails its CI step loudly instead of crashing the whole gate.

Index

Constants

View Source
const DefaultRSSTolerance = 0.15

DefaultRSSTolerance is the maximum allowed relative growth in peak RSS before the gate fails: 15% larger than the committed baseline. This is a D-05 starting point, deliberately tune-able by callers that need a different budget.

View Source
const DefaultThroughputTolerance = 0.10

DefaultThroughputTolerance is the maximum allowed relative regression in indexing throughput (files/sec) before the gate fails: 10% slower than the committed baseline. This is a D-05 starting point, deliberately tune-able by callers that need a different budget.

Variables

This section is empty.

Functions

func CheckRegression

func CheckRegression(baseline, current Metrics, ceilingBytes int64) error

CheckRegression compares a candidate run's Metrics (current) against a committed baseline and fails the gate (PERF-02) when any of:

  • baseline and current were measured on different platforms (GOOS/GOARCH), which makes every numeric comparison below meaningless — see the platform-mismatch check for why, or
  • indexing throughput regresses beyond DefaultThroughputTolerance, or
  • peak RSS grows beyond DefaultRSSTolerance relative to the baseline, or
  • peak RSS exceeds ceilingBytes, an absolute bounded-memory budget (INDX-06) enforced independently of the relative RSS delta — a baseline that itself already used a lot of memory must not let further growth hide behind a large denominator.

CheckRegression NEVER mutates baseline or current, and it never panics: a degenerate baseline (zero or negative FilesPerSec) returns a plain error instead of dividing by zero. A degenerate CURRENT reading (zero or negative FilesPerSec or PeakRSSBytes) is refused the same way, rather than being read as "no regression" or misattributed as a real one — backlog 999.4 recorded a current.PeakRSSBytes of 0 silently passing both the relative RSS check and the absolute INDX-06 ceiling. Re-blessing the baseline (updating baseline.json to accept a new normal) is a separate, explicit action — an operator-invoked `-rebless` flag on the runner (Plan 08-07) — and is never a side effect of running this check. That separation is D-05's point: an accidental auto-rewrite here would silently defeat the gate.

func PeakRSSBytes

func PeakRSSBytes(state *os.ProcessState) (int64, error)

PeakRSSBytes returns the peak resident set size, in bytes, of the process described by state. state must come from a completed exec.Cmd (i.e. after Wait() returns) so ProcessState.SysUsage() is populated. The raw ru_maxrss value is normalized to bytes based on runtime.GOOS (see normalizeMaxrss) since its unit differs by OS.

Types

type Metrics

type Metrics struct {
	Subject string `json:"subject"` // provenance label, e.g. "go" — not part of a measurement's identity
	Repo    string `json:"repo"`
	GOOS    string `json:"goos"`
	GOARCH  string `json:"goarch"`

	// Runner records the CI runner identity this Metrics was measured on
	// (e.g. a Namespace runs-on profile label such as
	// "namespace-profile-linux-amd64-4x8"), captured verbatim from the
	// workflow that ran it. It sits alongside GOOS/GOARCH because a
	// runner-class change (e.g. GitHub-hosted ubuntu-latest ->
	// Namespace) can hold GOOS/GOARCH constant while still changing the
	// measurement environment — namespace-profile-linux-amd64-4x8 IS
	// linux/amd64, so the GOOS/GOARCH guard alone is structurally blind
	// to that migration. Empty means "recorded before this field
	// existed" (e.g. any baseline.json predating this change), not an
	// error — measurement must not require it, since it is only
	// meaningful in CI. Runner does NOT yet participate in
	// CheckRegression's comparison; that lands separately, once a
	// Runner-labelled baseline exists to compare against.
	Runner string `json:"runner"`

	// ScratchFS records which filesystem class the regression corpus +
	// Pebble store were materialized on for THIS measurement: "tmpfs" or
	// "disk" (see tools/bench/runner's scratchFSTmpfs/scratchFSDisk).
	// Same philosophy as Runner: the storage frame is exactly as
	// load-bearing to a throughput comparison as the runner class is —
	// PERF-02's synthetic corpus is ~18MB of raw content but ~770MB of
	// actual on-disk footprint once small-file block overhead is
	// accounted for (120k files at ~4KB/block minimum each), and at that
	// file count the workload is IOPS/metadata-bound, not
	// bandwidth-bound. Comparing a tmpfs-recorded baseline against a
	// disk-recorded gate run (or vice versa) would be exactly the kind
	// of measurement-frame mismatch that produced the original
	// fictitious 10.6% "regression" — a different location for the same
	// failure class. Empty means "recorded before this field existed"
	// or "the caller supplied an explicit -scratch-dir and its
	// filesystem class wasn't characterized" — not an error. ScratchFS
	// does NOT yet participate in CheckRegression's comparison; wiring
	// that in, like Runner's, is a deliberately separate, later change.
	ScratchFS string `json:"scratch_fs"`

	// MedianOfTrials records how many INDEPENDENT measurement sessions
	// these numbers are the median of — where a session is a full
	// corpus-materialize + init + measure cycle, not a repeat of the
	// measured command inside one session (that inner repetition is
	// D-05's fixed median-of-5 and is not represented here).
	//
	// This is provenance, deliberately not a gate. Unlike GOOS/GOARCH, a
	// differing trial count is not a category error: median-of-3 and
	// median-of-5 estimate the same population median, just with
	// different precision, so refusing to compare them would be enforcing
	// a rule that isn't a validity rule. It is recorded and printed so
	// that a human reviewing a candidate baseline can see the procedure
	// that produced it — the thing nobody could see when a darwin
	// baseline was gating a linux runner.
	MedianOfTrials int `json:"median_of_trials"`

	FilesPerSec          float64 `json:"files_per_sec"`
	BytesPerSec          float64 `json:"bytes_per_sec"`
	QueryLatencyMedianMS float64 `json:"query_latency_median_ms"`
	PeakRSSBytes         int64   `json:"peak_rss_bytes"`
	ColdStartMS          float64 `json:"cold_start_ms"`
}

Metrics is a plain json-tagged data holder for one benchmark run's results — no methods with side effects, no network, no shelling out (mirrors internal/version.VersionInfo's discipline). Consumed by the committed-baseline regression gate and the absolute-numbers publisher (bench.yml's publish job).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL