Documentation
¶
Overview ¶
Package flake turns the test verdicts assay already produces into evidence about test reliability. A flaky test is one whose verdict changed while the code did not: the same test, at the same package fingerprint, both passing and failing. That definition is what separates a flake from a regression — a verdict that changed *across* fingerprints is just the code changing.
Index ¶
Constants ¶
const ( // DefaultQuarantinePath is committed, unlike the journal: it is curated // policy about which tests may not gate anything, not derived data. DefaultQuarantinePath = ".assay/flaky-tests.json" )
Variables ¶
This section is empty.
Functions ¶
func ParseVerboseOutput ¶
ParseVerboseOutput extracts top-level test verdicts from `go test -v` style output. Only unindented result lines count: subtest verdicts are indented, and a failing subtest already fails its top-level parent, which is the name -test.run selects.
Types ¶
type Evidence ¶
type Evidence struct {
Fingerprint string
Passes int
Fails int
Sources []string
FirstUnix int64
LastUnix int64
Detail string
}
Evidence is the conflicting record at one fingerprint: the same code state observed both passing and failing.
type Flake ¶
Flake is a test with at least one fingerprint of conflicting verdicts.
func Detect ¶
func Detect(observations []Observation) []Flake
Detect finds every test with conflicting verdicts at the same fingerprint. Only the conflicting fingerprints are reported as evidence: a fingerprint where the test always failed is a plain red test, not a flake.
type Journal ¶
type Journal struct {
// contains filtered or unexported fields
}
Journal is the append-only observation log for one workspace. It lives under the machine-local cache — evidence is derived data — while the curated quarantine file lives in the repository. Appends from concurrent assay processes are serialised by a lock file, so a watch session and a mutation run can both record without tearing each other's lines.
func OpenJournal ¶
OpenJournal places the workspace's journal under cacheDir, keyed by the workspace root so two repositories never mix evidence.
func (*Journal) Load ¶
func (j *Journal) Load() ([]Observation, error)
Load reads every observation. Malformed lines are skipped rather than fatal: a torn line from a crashed process must not make the whole history unreadable.
func (*Journal) Record ¶
func (j *Journal) Record(observations []Observation) error
Record appends observations, compacting first when the journal has grown past the threshold. Nil-safe: a nil journal records nothing, so callers running with --no-cache need no branching.
type Observation ¶
type Observation struct {
Package string `json:"package"`
Test string `json:"test"`
Fingerprint string `json:"fingerprint"`
Verdict Verdict `json:"verdict"`
Source string `json:"source"`
Detail string `json:"detail,omitempty"`
Unix int64 `json:"unix"`
}
Observation is one test verdict at one code state. The fingerprint is the package's content fingerprint — sources, dependencies, build tags — which is exactly the identity under which a verdict is expected to be reproducible.
type Quarantine ¶
type Quarantine struct {
Version int `json:"version"`
Tests []QuarantineEntry `json:"tests"`
// contains filtered or unexported fields
}
Quarantine is the committed list of tests whose verdicts may not be trusted to judge anything — most importantly, mutants: a flaky covering test marks mutants killed for reasons that have nothing to do with the mutation, and every such false kill inflates the score.
func LoadQuarantine ¶
func LoadQuarantine(path string) (*Quarantine, error)
func NewQuarantine ¶
func NewQuarantine() *Quarantine
func (*Quarantine) Add ¶
func (q *Quarantine) Add(entries ...QuarantineEntry) int
Add records entries that are not already present and reports how many were new.
func (*Quarantine) Blocked ¶
func (q *Quarantine) Blocked() map[string][]string
Blocked renders the quarantine in the shape mutation exclusion consumes.
func (*Quarantine) Contains ¶
func (q *Quarantine) Contains(pkg, test string) bool
func (*Quarantine) Len ¶
func (q *Quarantine) Len() int
func (*Quarantine) Save ¶
func (q *Quarantine) Save(path string) error
Source Files