Documentation
¶
Overview ¶
Package detector implements flakehunter's flaky-test detection.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Conclusion ¶
type Conclusion string
Conclusion is the outcome GitHub reports for a job.
const ( ConclusionSuccess Conclusion = "success" ConclusionFailure Conclusion = "failure" ConclusionCancelled Conclusion = "cancelled" ConclusionSkipped Conclusion = "skipped" ConclusionTimedOut Conclusion = "timed_out" )
Job conclusions flakehunter cares about. Cancelled and skipped jobs carry no signal about the code and are excluded from detection entirely.
func (Conclusion) Decisive ¶
func (c Conclusion) Decisive() bool
Decisive reports whether a conclusion says anything about the code under test.
func (Conclusion) Passed ¶
func (c Conclusion) Passed() bool
Passed reports whether the conclusion counts as green.
type FlakyOccurrence ¶
type FlakyOccurrence struct {
ID string `json:"id"`
HeadSHA string `json:"head_sha"`
Branch string `json:"branch"`
WorkflowName string `json:"workflow_name"`
WorkflowFile string `json:"workflow_file"`
JobName string `json:"job_name"`
FailedRuns []JobResult `json:"failed_runs"`
PassedRuns []JobResult `json:"passed_runs"`
// TotalAttempts counts every decisive attempt observed for this signature.
TotalAttempts int `json:"total_attempts"`
// FailureRate is failures over decisive attempts, in [0,1].
FailureRate float64 `json:"failure_rate"`
}
FlakyOccurrence is one job that produced both a pass and a fail on a single commit — the statistical signature of a flaky test.
func Detect ¶
func Detect(jobs []JobResult, opts Options) []FlakyOccurrence
Detect finds flaky occurrences in a set of job results.
The rule is deliberately strict: a job is flaky only when the *same job name*, in the *same workflow*, on the *same commit SHA*, produced both a pass and a failure. Anything looser produces false positives that destroy trust in the tool faster than the flakes it is meant to find:
- "the job failed then passed on a later commit" is a fix, not a flake
- "a different job failed" is unrelated
- "the run was cancelled" says nothing about the code
func (FlakyOccurrence) PrimaryFailure ¶
func (o FlakyOccurrence) PrimaryFailure() (JobResult, bool)
PrimaryFailure returns the failing run best suited to log analysis: the most recent one, whose logs are least likely to have been evicted by retention.
type JobResult ¶
type JobResult struct {
RunID int64 `json:"run_id"`
RunAttempt int `json:"run_attempt"`
JobID int64 `json:"job_id"`
JobName string `json:"job_name"`
WorkflowName string `json:"workflow_name"`
WorkflowFile string `json:"workflow_file"`
HeadSHA string `json:"head_sha"`
Branch string `json:"branch"`
Conclusion Conclusion `json:"conclusion"`
StartedAt time.Time `json:"started_at"`
URL string `json:"url"`
}
JobResult is one job execution within one workflow run.
type Options ¶
type Options struct {
// MinAttempts is how many decisive attempts a signature needs before it can
// be called flaky. Two is the minimum that can possibly disagree.
MinAttempts int
}
Options tunes detection.
func DefaultOptions ¶
func DefaultOptions() Options
DefaultOptions returns the standard detection settings.
type Summary ¶
type Summary struct {
TotalJobs int `json:"total_jobs"`
DistinctCommits int `json:"distinct_commits"`
FlakyOccurrences int `json:"flaky_occurrences"`
AffectedJobs int `json:"affected_jobs"`
MeanFailureRate float64 `json:"mean_failure_rate"`
}
Summary aggregates a scan for reporting.
func Summarise ¶
func Summarise(jobs []JobResult, occurrences []FlakyOccurrence) Summary
Summarise builds aggregate statistics for a scan result.
Source Files