Documentation
¶
Overview ¶
Package acceptance is GitHub issue #108's tier: every cohort estate under live/e2e/estates is applied against the floci emulator, its state file is deleted, and the plan rebuilt from ownership markers alone is asserted empty - the definition of done a user can check for themselves, run per cohort as a measurement.
The result is recorded per cohort in live/cohort-acceptance.json - pass, or the first phase that failed - and the committed artifact is a ratchet: a cohort recorded as passing must keep passing, and a run that widens the passing set is expected to commit the regenerated artifact. A cohort's resource count may not fall below what is committed either, pass or fail: issue #539 found a fixture that lost its failing resources and converted a red cohort to green with no other signal, and ratchetViolations (acceptance_live_test.go) fails the run on that just as hard as on a regression - see its doc comment for why a shrink is a hard failure rather than a warning. What this package deliberately does NOT report is admitted-type counts. #105 measured why: admitting six types moved unadmitted-type sites from 961 to 845 and moved the number of configurations that actually work by zero. The only number here is cohorts that apply and replan empty.
Nothing in live/e2e/run.sh is reused. That script is 2515 lines around one demo fixture; this tier is built on internal/live/flocitest's primitives (CohortDirs, CopyFixtureDir, StartFloci, the cross-process init lock) precisely so that adding a cohort means adding a directory, not editing a harness.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func IsStale ¶ added in v0.5.0
func IsStale(r CohortResult, currentEmulator string) bool
IsStale reports whether r's last recorded run measured against a different emulator image than currentEmulator - the same fact tools/gauntlet.IsStale computes for live/gauntlet.json's estate rows. A row with no LastRun at all (r.LastRun == nil) is a different fact from "measured, but against a superseded pin" and callers should check for it separately; this function only answers the second question.
Nothing in this package queues stale cohorts as work yet - unlike tools/gauntlet's `next`, there is no `cohort next` command, and wiring one is left as a follow-up (issue #433's PR says so explicitly) rather than done blind alongside the schema change. This is the primitive a future "cohort next" would need, published now so that follow-up does not have to invent the field names or the comparison a second time.
Types ¶
type Artifact ¶
type Artifact struct {
GeneratedBy string `json:"generated_by"`
// Image is the emulator image this run's checkout is pinned to,
// digest included - CONFIGURATION, not a claim about what any one row
// was measured against. Read per-row LastRun.Emulator for evidence (see
// LastRun's doc comment).
Image string `json:"image"`
// Provider is the AWS provider release the fixtures pin.
Provider string `json:"provider"`
Totals struct {
Cohorts int `json:"cohorts"`
Pass int `json:"pass"`
Fail int `json:"fail"`
} `json:"totals"`
Cohorts []CohortResult `json:"cohorts"`
}
Artifact is live/cohort-acceptance.json's whole shape.
type CohortResult ¶
type CohortResult struct {
Name string `json:"name"`
// Status is "pass" or "fail". "unsupported" is reserved for a cohort a
// capability probe rules out before any apply; #99's probe reported
// every listable admitted type implemented, so no cohort carries it
// today and the field earns its place only when the emulator loses
// something.
Status string `json:"status"`
// Phase is PhasePass, or the first phase that failed.
Phase Phase `json:"phase"`
// Resources is the number of resource blocks the cohort declares.
Resources int `json:"resources"`
// FailedResources names the addresses the failing phase reported, when
// it reported any: apply errors name the resource that refused,
// replan-not-empty names the addresses with proposed changes.
FailedResources []string `json:"failed_resources,omitempty"`
// Detail is the first error line of the failing phase, enough to find
// the full output in the test log without pasting it all here.
Detail string `json:"detail,omitempty"`
// TimedOut is set when the failing phase was killed at its deadline
// rather than exiting - the API Gateway availability waiter against
// floci is the known shape.
TimedOut bool `json:"timed_out,omitempty"`
// LastRun is this row's own provenance: the commit and emulator digest
// this cohort was actually measured against, the same per-row idiom
// live/gauntlet.json's estates carry (last_run.commit since #413,
// last_run.emulator since #414/f27f19d443). See LastRun's doc comment
// for why this lives on the row and not only at the artifact's
// top-level Image/GeneratedBy.
LastRun *LastRun `json:"last_run,omitempty"`
}
CohortResult is one cohort's verdict.
type LastRun ¶ added in v0.5.0
type LastRun struct {
Commit string `json:"commit"`
Date string `json:"date"`
Emulator string `json:"emulator"`
}
LastRun records the run that produced one cohort's verdict.
TestCohortAcceptance writes the whole artifact in one pass today - every row in one test run, all-or-nothing (see enforceRatchet's refusal to write a partial artifact) - so Commit/Emulator are identical across every row in a given write. That is an artifact of how the runner happens to work today, not a property this schema may assume: #414 (f27f19d443) found the same "true for one instant, false after an incremental re-run" trap one layer up, in live/gauntlet.json, where per-estate runs made the board-wide claim silently stale. A per-row stamp costs nothing when runs are monolithic and is the only thing that stays honest if a future change ever runs cohorts incrementally (a `-run` subset, a per-cohort retry) the way gauntlet.json's estates already do - so a board-wide claim must always be able to say it derived from these rows, never from the top-level Image/GeneratedBy fields, which describe the checkout's CURRENT configuration (the pin the NEXT run will use), not evidence about what any past row was actually measured against.
type Phase ¶
type Phase string
Phase names how far a cohort got. The values are ordered: each phase requires every earlier one.
const ( // PhaseInit: terraform/tofu init failed - a harness or registry // problem, not a fixture verdict. PhaseInit Phase = "init" // PhaseApply: the stock apply against the emulator did not create // every resource. The fixture, the provider or the emulator refused; // FailedResources carries what the apply output named. PhaseApply Phase = "apply" // PhaseReplan: live-plan errored after the state was deleted. PhaseReplan Phase = "replan" // PhaseEmpty: live-plan exited cleanly but the run could not confirm // an empty plan - it proposed changes (FailedResources carries the // addresses), or printed output this harness does not recognize, which // is recorded as a failure to assert rather than presumed empty. PhaseEmpty Phase = "empty" // PhasePass: applied, state deleted, replanned empty. PhasePass Phase = "pass" )
Source Files