report

package
v1.29.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 16, 2026 License: MIT Imports: 12 Imported by: 0

Documentation

Overview

Package report renders the result of a workflow run as files for machines and for people: JSON, JUnit XML, Markdown and the GitHub Actions job summary.

Index

Constants

View Source
const (
	Passed   = "passed"
	Failed   = "failed"
	Skipped  = "skipped"
	Untested = "untested" // A step that ran but has no test expression
)

Report statuses for the workflow, jobs and steps.

View Source
const (
	FailureAssertion = "assertion"  // The test expression evaluated to false
	FailureTestError = "test_error" // The test expression could not be evaluated
	FailureTestType  = "test_type"  // The test expression did not evaluate to a boolean
	FailureAction    = "action"     // The action itself returned an error
	FailureTemplate  = "template"   // A template in the step's with, vars or name could not be evaluated
	// The response broke the contract the action checked it against, such
	// as an OpenAPI document.
	FailureContractResponse = "contract_response"
	// The request broke the contract the action checked it against: the
	// workflow sent what the contract does not allow.
	FailureContractRequest = "contract_request"
	// The guard of the run, such as --read-only, refused what the step was
	// to do.
	FailureRefused = "refused"
)

Failure kinds, the reasons a step fails.

Variables

View Source
var ErrNoStepSummary = errors.New("GITHUB_STEP_SUMMARY is not set; skipping the github-summary report")

ErrNoStepSummary is returned for a github-summary report when no path was given and GITHUB_STEP_SUMMARY is not set, as on a machine outside GitHub Actions. It is not a failure: the same command line can run anywhere.

View Source
var ErrStepSummaryFull = errors.New("the job summary has no room left under GitHub's 1 MiB limit; skipping the github-summary report")

ErrStepSummaryFull is returned when earlier writes have left no room in the job summary for even a shortened page. Writing anyway would push it past GitHub's limit and lose what the earlier steps wrote, so nothing is written.

Functions

func FormatMs added in v1.28.0

func FormatMs(ms float64) string

FormatMs writes a time in milliseconds as a person reads it: in milliseconds below a second, with one decimal below 100ms, and in seconds from a second up.

func NearestRank added in v1.29.0

func NearestRank(sorted []float64, p float64) float64

NearestRank returns the p-th percentile of sorted, which is in ascending order and not empty: the smallest value that at least p percent of the values are less than or equal to.

Types

type BaselineRun added in v1.29.0

type BaselineRun struct {
	RunID     string    `json:"run_id,omitempty"`
	StartedAt time.Time `json:"started_at"`
}

BaselineRun is the run a report's response times were compared to.

type BaselineSteps added in v1.29.0

type BaselineSteps struct {
	// contains filtered or unexported fields
}

BaselineSteps are the repeated steps of a job of a baseline report, which Next gives out by their ids in order.

func (*BaselineSteps) All added in v1.29.0

func (s *BaselineSteps) All() map[string][]*RepeatBaseline

All returns the outcomes of the repeated steps that Next has not given out, by their ids, each in order.

func (*BaselineSteps) Next added in v1.29.0

func (s *BaselineSteps) Next(id string) *RepeatBaseline

Next returns the outcome of the next repeated step of id, or nil when there is none left.

type Contract

type Contract struct {
	Spec      string `json:"spec"`               // The contract, such as the path of an OpenAPI document
	Operation string `json:"operation"`          // Such as GET /users/{id}
	Response  string `json:"response,omitempty"` // Such as 200, 2XX or default; empty when none is declared
}

Contract is what a response was matched to in a contract, which a report of the coverage of the contract counts.

type Count

type Count struct {
	Total    int `json:"total"`
	Passed   int `json:"passed"`
	Failed   int `json:"failed"`
	Skipped  int `json:"skipped"`
	Untested int `json:"untested,omitempty"`
}

Count is a tally by status.

type Failure

type Failure struct {
	Kind     string         `json:"kind"`
	Message  string         `json:"message"`
	Request  map[string]any `json:"request,omitempty"`
	Response map[string]any `json:"response,omitempty"`
	// Violations are what broke the contract, when Kind is a contract's.
	Violations []Violation `json:"violations,omitempty"`
}

Failure says why a step failed.

type Format

type Format string

Format names a report file format.

const (
	JSON     Format = "json"
	JUnit    Format = "junit"
	Markdown Format = "markdown"
	// GitHubSummary appends the Markdown page to the GitHub Actions job
	// summary, the file GITHUB_STEP_SUMMARY names.
	GitHubSummary Format = "github-summary"
)

type Job

type Job struct {
	ID         string    `json:"id"`
	Name       string    `json:"name"`
	Status     string    `json:"status"`
	StartedAt  time.Time `json:"started_at"`
	DurationMs int64     `json:"duration_ms"`
	Steps      []Step    `json:"steps"`
	// RepeatTest is the outcome of the test of a repeated job over its
	// runs, which decides whether the job passed; nil when it has none.
	RepeatTest *RepeatTest `json:"repeat_test,omitempty"`
	// BaselineTest is the outcome of the test of a repeated job against the
	// run compared to, which the job passes only when it holds or is not
	// checked; nil when the job has none.
	BaselineTest *RepeatTest `json:"baseline_test,omitempty"`
}

Job is one job of a Report.

type Latency added in v1.28.0

type Latency struct {
	Count int     `json:"count"`
	MinMs float64 `json:"min_ms"`
	// MeanMs is the arithmetic mean.
	MeanMs float64 `json:"mean_ms"`
	P50Ms  float64 `json:"p50_ms"`
	P90Ms  float64 `json:"p90_ms"`
	P95Ms  float64 `json:"p95_ms"`
	P99Ms  float64 `json:"p99_ms"`
	MaxMs  float64 `json:"max_ms"`
}

Latency is the distribution of the times the action of a repeated step took, over the runs in which it returned one. A run in which the action failed to run returns none, and one in which the step was skipped does not run it; neither is counted. A run whose test failed is, so a limit on the time does not take the slow runs out of what it is measured against.

The percentiles are by nearest rank: the time that the given share of the runs took at most, which is always one of the times taken.

func NewLatency added in v1.28.0

func NewLatency(secs []float64) *Latency

NewLatency returns the distribution of secs, the times of the runs in seconds, or nil when there are none.

func (*Latency) Summary added in v1.28.0

func (l *Latency) Summary() string

Summary writes the distribution on one line, as the terminal and the Markdown report show it.

type Repeat

type Repeat struct {
	Total   int `json:"total"`
	Success int `json:"success"`
	Failure int `json:"failure"`
	// Skipped is the iterations in which the step was not run: after a
	// step that failed in a job with fail_fast, or for an iteration that
	// came to no entry.
	Skipped int `json:"skipped,omitempty"`
	// Warmup is the runs before those counted here, which are counted in
	// nothing.
	Warmup int `json:"warmup,omitempty"`
	// Latency is the distribution of the times the step's action took, nil
	// when it returned none, as an action that failed to run does.
	Latency *Latency `json:"latency,omitempty"`
	// Baseline is the outcome of the step in the run compared to; nil when
	// none was compared, or the run had no step of the id.
	Baseline *RepeatBaseline `json:"baseline,omitempty"`
}

Repeat records the outcome of every iteration of a repeated job.

type RepeatBaseline added in v1.29.0

type RepeatBaseline struct {
	Total   int      `json:"total"`
	Success int      `json:"success"`
	Latency *Latency `json:"latency,omitempty"`
}

RepeatBaseline is the outcome of a repeated step in the run compared to.

func (*RepeatBaseline) Summary added in v1.29.0

func (b *RepeatBaseline) Summary(now *Latency) string

Summary writes on one line the times of the step in the baseline, and how much now, the times of this run, changed from them, as the terminal shows it under now. A nil baseline is a step the baseline has none of.

type RepeatTest added in v1.29.0

type RepeatTest struct {
	Test    string   `json:"test"`
	Status  string   `json:"status"` // Passed, Failed or Skipped
	Failure *Failure `json:"failure,omitempty"`
	// Reason is why the test was skipped, which a test against the run
	// compared to is when there is nothing to compare to.
	Reason string `json:"reason,omitempty"`
}

RepeatTest is the outcome of the test of a repeated job over its runs.

type Report

type Report struct {
	Name string `json:"name"`
	// RunID names the run, as actions are told it, such as in the trace
	// header of the http action.
	RunID       string    `json:"run_id,omitempty"`
	Description string    `json:"description,omitempty"`
	Status      string    `json:"status"`
	StartedAt   time.Time `json:"started_at"`
	FinishedAt  time.Time `json:"finished_at"`
	DurationMs  int64     `json:"duration_ms"`
	Summary     Summary   `json:"summary"`
	Jobs        []Job     `json:"jobs"`
	// Baseline is the run the response times were compared to; nil when
	// they were compared to none. See Compare.
	Baseline *BaselineRun `json:"baseline,omitempty"`
}

Report is the result of a workflow run in a form meant for machines: the JSON file is this structure as is, and the other formats are rendered from it.

func New

func New(name, description string, startedAt, finishedAt time.Time) *Report

New returns a report of a run with no jobs yet; AddJob adds them.

func Read added in v1.29.0

func Read(path string) (*Report, error)

Read reads a JSON report, as WriteJSON writes it.

func (*Report) AddJob

func (r *Report) AddJob(job Job)

AddJob appends a job, counts it and its steps in the summary, and fails the report when the job failed.

func (*Report) BaselineSteps added in v1.29.0

func (r *Report) BaselineSteps(jobID string) *BaselineSteps

BaselineSteps returns the repeated steps of the first job of id in r; nil, which has none, when r is nil or has no job of the id.

func (*Report) Compare added in v1.29.0

func (r *Report) Compare(base *Report)

Compare gives each repeated step whose response times were measured the outcome of the step of the same id, in the job of the same id, in base, the report of an earlier run. When a job has several steps of an id, as the entries of an iteration have, they are matched in order. A step that base has no step for, as a step added since has not, is given none.

func (*Report) Mask

func (r *Report) Mask(m *mask.Masker)

Mask hides secrets in every text the report carries, and credential headers in failed steps' requests and responses. It masks the data before it is rendered, because JSON and XML escape quotes, backslashes and ampersands, and a secret containing one would no longer match in the rendered file.

func (*Report) Write

func (r *Report) Write(t Target) error

Write renders the report in the target's format and writes it to the target's path, creating parent directories as needed.

func (*Report) WriteJSON

func (r *Report) WriteJSON(w io.Writer) error

WriteJSON writes the report as indented JSON.

func (*Report) WriteJUnit

func (r *Report) WriteJUnit(w io.Writer) error

WriteJUnit writes the report as JUnit XML.

func (*Report) WriteMarkdown

func (r *Report) WriteMarkdown(w io.Writer) error

WriteMarkdown writes the report as a Markdown page: a summary line, a table of jobs, a table of the response times of the repeated steps, a table of how they changed from the baseline when there is one, and a section for every failed step with its test, the reason it failed, and the request and response. The page is meant to be read as is, for example as a GitHub job summary or by a coding agent.

type Retry

type Retry struct {
	Attempts int `json:"attempts"`
	Max      int `json:"max"`
}

Retry records how many attempts a retried step took.

type Step

type Step struct {
	Index int `json:"index"`
	// ID is the id of the step, or step_ and where it is written in the job
	// when it has none, by which a later run's report finds it.
	ID         string   `json:"id,omitempty"`
	Name       string   `json:"name"`
	Status     string   `json:"status"`
	Test       string   `json:"test,omitempty"`
	DurationMs int64    `json:"duration_ms"`
	Retry      *Retry   `json:"retry,omitempty"`
	Repeat     *Repeat  `json:"repeat,omitempty"`
	Echo       string   `json:"echo,omitempty"`
	Failure    *Failure `json:"failure,omitempty"`
	// Contract is what the action matched the response to in the contract
	// it checked it against, such as an operation of an OpenAPI document.
	Contract *Contract `json:"contract,omitempty"`
}

Step is one step of a Job.

type Summary

type Summary struct {
	Jobs  Count `json:"jobs"`
	Steps Count `json:"steps"`
}

Summary counts jobs and steps by status.

type Target

type Target struct {
	Format Format
	Path   string
}

Target is one report file to write after a run.

func ParseTargets

func ParseTargets(s string) ([]Target, error)

ParseTargets parses a comma separated list of format[=path] entries, such as "junit=out/junit.xml,json". A format without a path is written to its default file name in the current directory.

type Violation

type Violation struct {
	In      string `json:"in"`              // Where it was found: request or response
	Field   string `json:"field,omitempty"` // The field, such as /items/0/id
	Reason  string `json:"reason,omitempty"`
	Message string `json:"message"`
}

Violation is one thing a contract does not allow.

func (Violation) String

func (v Violation) String() string

String is the violation on one line: the field, what is wrong with it and why.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL