e2eshard

Overview ¶

Command e2eshard plans and maintains the sharding of forge's `-tags e2e` suite (.github/workflows/e2e-suite.yml). It is CI plumbing, run with `go run ./internal/e2eshard`, and no part of the forge binary.

e2eshard list     [-dir internal/cli]
    The e2e tests, read from source: every top-level Test function in a
    file built only under the e2e tag. This is the set the naming guard
    (naming_test.go) holds to the TestE2E prefix.

e2eshard plan     -list FILE -timings FILE -shards N -select pr|all -out DIR [-summary FILE]
    Split the tests named in FILE (the e2e test binary's
    `-test.list '^TestE2E'` output) into N shards of roughly equal
    recorded duration. Writes DIR/selected-tests.txt and
    DIR/shard-<i>.txt.

e2eshard timings  [-merge FILE] [-dir internal/cli] SHARD.json...
    Fold `go test -json` streams from shard runs into a timings file on
    stdout: each test's median duration across the streams, over the
    entries of -merge for tests the streams did not run. With -dir,
    entries for tests that no longer exist are dropped.
    scripts/e2e-timings.sh drives this against a CI run.

WHY A NAMING CONVENTION. The shards must run the e2e tests and nothing else: internal/cli's ~3,000 untagged unit tests are already run, under -race, by ci.yml's Test (internal/cli) job, and running them again in the shards cost each shard 50-200 test-seconds for no signal. A build tag cannot be selected at run time; a name can. So a test is an e2e test exactly when its name starts with TestE2E, and naming_test.go fails the moment a test in an e2e-tagged file is named otherwise, or a TestE2E test lives in an untagged file. The shards select `-test.list '^TestE2E'` from the compiled binary, which by that guard is precisely the tagged set.

WHY RECORDED DURATIONS. The shards used to split the sorted test list round-robin (`NR % 4`), which balanced test COUNTS: one run's slowest shard carried 3,162 test-seconds while its fastest carried 1,544, and the lane waits for the slowest. The plan is longest-processing-time-first over the committed timings, so every shard's expected load is close to the mean. A test with no recorded timing yet (a new one) is planned at the median of the known ones; a stale timing only reorders work, it never drops a test.

NIGHTLY TESTS. A test named TestE2ENightly* is part of the suite but not of a pull request's run: `-select pr` leaves it out and nightly.yml runs the whole suite with `-select all`. Move a test there only when it is slow and a regression in it is cheap to bisect a day late; say why in its comment.