Documentation
¶
Overview ¶
Package slowtests is the machine behind cmd/nova-ci's `slowtests` verb in docs/SPEC-CI.md, "The per-package test time budget". It reads the newline-delimited TestEvent objects `go test -json` writes, sums the package-level Elapsed for each package, keeps the slowest few test-level rows so a finding can name where the time went, and reports every package whose total is over the budget. The one file it reads is ci.yml, which names the runners a measured row may name, and it never touches the network or the clock: the events and the budget come from the caller, so a test feeds canned lines and asserts a verdict.
Index ¶
Constants ¶
const MaxHeadroom = 3.0
MaxHeadroom is the most a row's budget may exceed its own measurement: three times. The same tests take about 1.0-1.2 s on a busy runner against about 0.9 s idle, well inside it.
const SleepsMarker = "SLEEPS:"
SleepsMarker is the text a unit test's t.Skip starts with when it is skipped for a sleep or a wall-clock wait; go test -json carries it as an output event of the skipped test.
Variables ¶
This section is empty.
Functions ¶
func Benches ¶
Benches reads the labels a row may name as where it was measured from ci.yml's own text, so the inventory has the one home ci.yml is (docs/STANDARD.md section 4): the machine label of every self-hosted `runs-on` list and every runner group a step's groupFlag names, in the order the file names them and once each. An expression is no label: it names the matrix that fills it in.
func Seconds ¶
Seconds renders a duration in seconds to one decimal, with the unit: 3.2 becomes "3.2s" and 70.0 stays "70.0s", so two adjacent rows compare without a reader doing arithmetic.
func Verdict ¶
Verdict is what the check prints and its exit code. Every CI-SLOW line and the CI-LOAD line are printed on every leg, so the time is always measured. The exit code never reads the load, and a no is 1 (the check ran and said no), apart from the 2 of a run that could not read its input:
- an unledgered SLEEPS skip (a CI-SLEEPS line) is 1 on every leg: it is what the test does, a static fact;
- a package with a start event and no package-level terminal event (pass, fail or skip) is 1 on every leg: the finding is `truncated: <pkg> started and never ended`, because a stream cut by a pipe must not read as a clean run;
- a CI-SLOW line is 1 only when enforce is set, which one caller does: the nightly whole-tree run on the idle reference leg (ci.yml's schedule branch of the test step, `make test SLOWTESTS_ENFORCE=1`). Everywhere else it is a printed measurement and exit 0.
Types ¶
type Budgets ¶
Budgets is what Judge holds a run to. Package is the default seconds for a package's total; Test is the default seconds for one top-level test, zero meaning tests are not judged one by one. Rows are the allowlist: a row whose Test is empty is a package's own budget, any other row one test's. Sleeps is the ledger of the tests already skipped with the SLEEPS marker; a SLEEPS skip not on it is a finding whatever the budgets or the load.
type Event ¶
type Event struct {
Action string `json:"Action"`
Package string `json:"Package"`
Test string `json:"Test"`
Elapsed float64 `json:"Elapsed"`
Output string `json:"Output"`
}
Event is one decoded TestEvent line. The field names are `go test -json`'s own; a package-level event has Test == "" and carries the package's total Elapsed, while a test-level event carries that one test's Elapsed. Output is the text of an output (or build-output) event: the budget check never reads it, and `nova-ci local` prints it under a red test so the failure is named with its own words.
func Parse ¶
Parse decodes newline-delimited TestEvent JSON. Blank lines are skipped; a line that is not an object, or an object with no Action (every event go test -json writes has one, the bookkeeping ones included), is an error naming its 1-based line, never a silent skip. Parse itself makes no completeness claim: it only reads the lines. A package that starts and never gets its own package-level pass, fail or skip is truncatedPackages, which Judge applies to the events Parse returned.
type Load ¶
Load is the host's run-queue load average and its logical CPU count when a run is judged. It is a MEASUREMENT printed beside the times, never an input to the verdict: a budget verdict is the same on any machine, so the load only tells a reader what the box was doing while the times were taken. Known is false when the host has no load average to read (Windows) or the read failed; Why then says so.
type OverTest ¶
OverTest is one top-level test over its budget: the default per-test budget, or its allowlist row.
type Package ¶
Package is one package's summed elapsed time and its slowest tests, sorted worst first and capped at testsKept. Budget is the seconds it was judged against: the default, or its allowlist row.
type Report ¶
type Report struct {
Packages int
Over []Package
OverTests []OverTest
Sleepers []Sleeper
Truncated []string
Slowest Package
Budget time.Duration
}
Report is one run of the budget check: how many packages were seen, the ones over budget (worst first), the tests over budget (worst first), the single slowest package overall, the package budget they were judged against (kept only so a finding can print it), and the packages that started and never ended.
func Judge ¶
Judge folds the events into a report. A package's total is the sum of its package-level Elapsed (Test == ""), judged against its allowlist row or b.Package. With b.Test > 0 every top-level test (no "/" in its name: a subtest's time is already its parent's) is judged against its row or b.Test. Over-budget packages and tests are returned worst first, ties by name, so the output never depends on map iteration. A test that skips with the SLEEPS marker in its output and is not on b.Sleeps is a Sleeper, in first-seen order, one per top-level test.
func (Report) OKLine ¶
OKLine is the one line a run inside budget prints, naming the single slowest package overall. With no packages at all the slot is "none", not empty.
func (Report) OverLines ¶
OverLines is one line per over-budget package, worst first, then one per over-budget test, worst first.
func (Report) SleepsLines ¶
SleepsLines is one line per unledgered SLEEPS skip. It is red on every leg: a test skipped for a wall-clock wait is what the test does, not how busy the runner was.
func (Report) TruncatedLines ¶
TruncatedLines is one finding per package with a start event and no package-level terminal event (pass, fail or skip): `truncated: <pkg> started and never ended`.
type Row ¶
Row is one allowlist line: `pkg<TAB>test<TAB>seconds<TAB>measured`, with `-` in the test column for a package's own row. Package is module-relative (internal/ci) and matches an event's import path by its trailing path elements. Seconds is the budget the row enforces; Measured is the time the row was cut from and Where the CI run (`run<id>`) or the bench label Benches reads from ci.yml that measured it, so no number on the list is a guess: the measured column is `<seconds>s@<where>`, and the budget may not exceed MaxHeadroom times the measurement.
func ParseAllowlist ¶
ParseAllowlist reads `pkg<TAB>test<TAB>seconds<TAB>measured` rows. Blank lines and lines starting with # are skipped; `-` in the test column is the package's own row. The measured column is `<seconds>s@<where>`: the time the row was cut from and where it was measured, a CI run or a runner label ciBenches reads from ci.yml. A malformed row, a budget that is not a positive number, a row with no measurement, a budget under its measurement or over MaxHeadroom times it, a package column that is not the full module-relative path, or a row written twice is an error naming its 1-based line.
type SleepRow ¶
SleepRow is one line of the SLEEPS ledger: `pkg<TAB>test<TAB>where`, a top-level test that skips itself with the SLEEPS marker because it waits on the wall clock, and the run or issue that found the wait.
func ParseSleeps ¶
ParseSleeps reads the SLEEPS ledger: `pkg<TAB>test<TAB>where` rows, blank lines and # comments skipped. A malformed row, a package column that is not the full module-relative path, or a row written twice is an error naming its 1-based line.
Source Files