Documentation
¶
Overview ¶
Package report renders the result of a workflow run as files for machines and for people: JSON, JUnit XML, Markdown and the GitHub Actions job summary.
Index ¶
- Constants
- Variables
- func FormatMs(ms float64) string
- func NearestRank(sorted []float64, p float64) float64
- type BaselineRun
- type BaselineSteps
- type Contract
- type Count
- type Failure
- type Format
- type Job
- type Latency
- type Repeat
- type RepeatBaseline
- type RepeatTest
- type Report
- func (r *Report) AddJob(job Job)
- func (r *Report) BaselineSteps(jobID string) *BaselineSteps
- func (r *Report) Compare(base *Report)
- func (r *Report) Mask(m *mask.Masker)
- func (r *Report) Write(t Target) error
- func (r *Report) WriteJSON(w io.Writer) error
- func (r *Report) WriteJUnit(w io.Writer) error
- func (r *Report) WriteMarkdown(w io.Writer) error
- type Retry
- type Step
- type Summary
- type Target
- type Violation
Constants ¶
const ( Passed = "passed" Failed = "failed" Skipped = "skipped" Untested = "untested" // A step that ran but has no test expression )
Report statuses for the workflow, jobs and steps.
const ( FailureAssertion = "assertion" // The test expression evaluated to false FailureTestError = "test_error" // The test expression could not be evaluated FailureTestType = "test_type" // The test expression did not evaluate to a boolean FailureAction = "action" // The action itself returned an error FailureTemplate = "template" // A template in the step's with, vars or name could not be evaluated // The response broke the contract the action checked it against, such // as an OpenAPI document. FailureContractResponse = "contract_response" // The request broke the contract the action checked it against: the // workflow sent what the contract does not allow. FailureContractRequest = "contract_request" // The guard of the run, such as --read-only, refused what the step was // to do. FailureRefused = "refused" )
Failure kinds, the reasons a step fails.
Variables ¶
var ErrNoStepSummary = errors.New("GITHUB_STEP_SUMMARY is not set; skipping the github-summary report")
ErrNoStepSummary is returned for a github-summary report when no path was given and GITHUB_STEP_SUMMARY is not set, as on a machine outside GitHub Actions. It is not a failure: the same command line can run anywhere.
var ErrStepSummaryFull = errors.New("the job summary has no room left under GitHub's 1 MiB limit; skipping the github-summary report")
ErrStepSummaryFull is returned when earlier writes have left no room in the job summary for even a shortened page. Writing anyway would push it past GitHub's limit and lose what the earlier steps wrote, so nothing is written.
Functions ¶
func FormatMs ¶ added in v1.28.0
FormatMs writes a time in milliseconds as a person reads it: in milliseconds below a second, with one decimal below 100ms, and in seconds from a second up.
func NearestRank ¶ added in v1.29.0
NearestRank returns the p-th percentile of sorted, which is in ascending order and not empty: the smallest value that at least p percent of the values are less than or equal to.
Types ¶
type BaselineRun ¶ added in v1.29.0
type BaselineRun struct {
RunID string `json:"run_id,omitempty"`
StartedAt time.Time `json:"started_at"`
}
BaselineRun is the run a report's response times were compared to.
type BaselineSteps ¶ added in v1.29.0
type BaselineSteps struct {
// contains filtered or unexported fields
}
BaselineSteps are the repeated steps of a job of a baseline report, which Next gives out by their ids in order.
func (*BaselineSteps) All ¶ added in v1.29.0
func (s *BaselineSteps) All() map[string][]*RepeatBaseline
All returns the outcomes of the repeated steps that Next has not given out, by their ids, each in order.
func (*BaselineSteps) Next ¶ added in v1.29.0
func (s *BaselineSteps) Next(id string) *RepeatBaseline
Next returns the outcome of the next repeated step of id, or nil when there is none left.
type Contract ¶
type Contract struct {
Spec string `json:"spec"` // The contract, such as the path of an OpenAPI document
Operation string `json:"operation"` // Such as GET /users/{id}
Response string `json:"response,omitempty"` // Such as 200, 2XX or default; empty when none is declared
}
Contract is what a response was matched to in a contract, which a report of the coverage of the contract counts.
type Count ¶
type Count struct {
Total int `json:"total"`
Passed int `json:"passed"`
Failed int `json:"failed"`
Skipped int `json:"skipped"`
Untested int `json:"untested,omitempty"`
}
Count is a tally by status.
type Failure ¶
type Failure struct {
Kind string `json:"kind"`
Message string `json:"message"`
Request map[string]any `json:"request,omitempty"`
Response map[string]any `json:"response,omitempty"`
// Violations are what broke the contract, when Kind is a contract's.
Violations []Violation `json:"violations,omitempty"`
}
Failure says why a step failed.
type Job ¶
type Job struct {
ID string `json:"id"`
Name string `json:"name"`
Status string `json:"status"`
StartedAt time.Time `json:"started_at"`
DurationMs int64 `json:"duration_ms"`
Steps []Step `json:"steps"`
// RepeatTest is the outcome of the test of a repeated job over its
// runs, which decides whether the job passed; nil when it has none.
RepeatTest *RepeatTest `json:"repeat_test,omitempty"`
// BaselineTest is the outcome of the test of a repeated job against the
// run compared to, which the job passes only when it holds or is not
// checked; nil when the job has none.
BaselineTest *RepeatTest `json:"baseline_test,omitempty"`
}
Job is one job of a Report.
type Latency ¶ added in v1.28.0
type Latency struct {
Count int `json:"count"`
MinMs float64 `json:"min_ms"`
// MeanMs is the arithmetic mean.
MeanMs float64 `json:"mean_ms"`
P50Ms float64 `json:"p50_ms"`
P90Ms float64 `json:"p90_ms"`
P95Ms float64 `json:"p95_ms"`
P99Ms float64 `json:"p99_ms"`
MaxMs float64 `json:"max_ms"`
}
Latency is the distribution of the times the action of a repeated step took, over the runs in which it returned one. A run in which the action failed to run returns none, and one in which the step was skipped does not run it; neither is counted. A run whose test failed is, so a limit on the time does not take the slow runs out of what it is measured against.
The percentiles are by nearest rank: the time that the given share of the runs took at most, which is always one of the times taken.
func NewLatency ¶ added in v1.28.0
NewLatency returns the distribution of secs, the times of the runs in seconds, or nil when there are none.
type Repeat ¶
type Repeat struct {
Total int `json:"total"`
Success int `json:"success"`
Failure int `json:"failure"`
// Skipped is the iterations in which the step was not run: after a
// step that failed in a job with fail_fast, or for an iteration that
// came to no entry.
Skipped int `json:"skipped,omitempty"`
// Warmup is the runs before those counted here, which are counted in
// nothing.
Warmup int `json:"warmup,omitempty"`
// Latency is the distribution of the times the step's action took, nil
// when it returned none, as an action that failed to run does.
Latency *Latency `json:"latency,omitempty"`
// Baseline is the outcome of the step in the run compared to; nil when
// none was compared, or the run had no step of the id.
Baseline *RepeatBaseline `json:"baseline,omitempty"`
}
Repeat records the outcome of every iteration of a repeated job.
type RepeatBaseline ¶ added in v1.29.0
type RepeatBaseline struct {
Total int `json:"total"`
Success int `json:"success"`
Latency *Latency `json:"latency,omitempty"`
}
RepeatBaseline is the outcome of a repeated step in the run compared to.
func (*RepeatBaseline) Summary ¶ added in v1.29.0
func (b *RepeatBaseline) Summary(now *Latency) string
Summary writes on one line the times of the step in the baseline, and how much now, the times of this run, changed from them, as the terminal shows it under now. A nil baseline is a step the baseline has none of.
type RepeatTest ¶ added in v1.29.0
type RepeatTest struct {
Test string `json:"test"`
Status string `json:"status"` // Passed, Failed or Skipped
Failure *Failure `json:"failure,omitempty"`
// Reason is why the test was skipped, which a test against the run
// compared to is when there is nothing to compare to.
Reason string `json:"reason,omitempty"`
}
RepeatTest is the outcome of the test of a repeated job over its runs.
type Report ¶
type Report struct {
Name string `json:"name"`
// RunID names the run, as actions are told it, such as in the trace
// header of the http action.
RunID string `json:"run_id,omitempty"`
Description string `json:"description,omitempty"`
Status string `json:"status"`
StartedAt time.Time `json:"started_at"`
FinishedAt time.Time `json:"finished_at"`
DurationMs int64 `json:"duration_ms"`
Summary Summary `json:"summary"`
Jobs []Job `json:"jobs"`
// Baseline is the run the response times were compared to; nil when
// they were compared to none. See Compare.
Baseline *BaselineRun `json:"baseline,omitempty"`
}
Report is the result of a workflow run in a form meant for machines: the JSON file is this structure as is, and the other formats are rendered from it.
func (*Report) AddJob ¶
AddJob appends a job, counts it and its steps in the summary, and fails the report when the job failed.
func (*Report) BaselineSteps ¶ added in v1.29.0
func (r *Report) BaselineSteps(jobID string) *BaselineSteps
BaselineSteps returns the repeated steps of the first job of id in r; nil, which has none, when r is nil or has no job of the id.
func (*Report) Compare ¶ added in v1.29.0
Compare gives each repeated step whose response times were measured the outcome of the step of the same id, in the job of the same id, in base, the report of an earlier run. When a job has several steps of an id, as the entries of an iteration have, they are matched in order. A step that base has no step for, as a step added since has not, is given none.
func (*Report) Mask ¶
Mask hides secrets in every text the report carries, and credential headers in failed steps' requests and responses. It masks the data before it is rendered, because JSON and XML escape quotes, backslashes and ampersands, and a secret containing one would no longer match in the rendered file.
func (*Report) Write ¶
Write renders the report in the target's format and writes it to the target's path, creating parent directories as needed.
func (*Report) WriteJUnit ¶
WriteJUnit writes the report as JUnit XML.
func (*Report) WriteMarkdown ¶
WriteMarkdown writes the report as a Markdown page: a summary line, a table of jobs, a table of the response times of the repeated steps, a table of how they changed from the baseline when there is one, and a section for every failed step with its test, the reason it failed, and the request and response. The page is meant to be read as is, for example as a GitHub job summary or by a coding agent.
type Step ¶
type Step struct {
Index int `json:"index"`
// ID is the id of the step, or step_ and where it is written in the job
// when it has none, by which a later run's report finds it.
ID string `json:"id,omitempty"`
Name string `json:"name"`
Status string `json:"status"`
Test string `json:"test,omitempty"`
DurationMs int64 `json:"duration_ms"`
Retry *Retry `json:"retry,omitempty"`
Repeat *Repeat `json:"repeat,omitempty"`
Echo string `json:"echo,omitempty"`
Failure *Failure `json:"failure,omitempty"`
// Contract is what the action matched the response to in the contract
// it checked it against, such as an operation of an OpenAPI document.
Contract *Contract `json:"contract,omitempty"`
}
Step is one step of a Job.
type Target ¶
Target is one report file to write after a run.
func ParseTargets ¶
ParseTargets parses a comma separated list of format[=path] entries, such as "junit=out/junit.xml,json". A format without a path is written to its default file name in the current directory.
type Violation ¶
type Violation struct {
In string `json:"in"` // Where it was found: request or response
Field string `json:"field,omitempty"` // The field, such as /items/0/id
Reason string `json:"reason,omitempty"`
Message string `json:"message"`
}
Violation is one thing a contract does not allow.