report

package
v0.9.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 15, 2026 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Overview

Package report implements FR-RPT-01 through 04 and FR-CLI-05: turning a run's recorded iterations into the structured data a self-contained HTML report, a JSON summary, and (eventually) the MCP get_results tool all read from. It implements engine.Recorder, exactly the seam internal/engine's own doc comment on Recorder sets aside for it, and knows nothing about protocol drivers, scripting, or the CLI. A protocol call arrives as a name, the same way a step does.

Index

Constants

View Source
const LiveMaxHold = 10 * time.Second

LiveMaxHold is the longest a bucket is held back for calls that started in it (or earlier) and have not ended: this long past the end of its second. After that it is handed out with what it has, so that one hung call cannot freeze a live view. The final Result.TimeSeries stays authoritative.

View Source
const MaxCheckNames = 100

MaxCheckNames bounds how many distinct check names a run keeps. Further names are counted together under OtherChecksName.

View Source
const MaxStepNames = 100

MaxStepNames bounds how many distinct step names a run keeps. Further names are counted together under OtherStepsName.

View Source
const OtherChecksName = "(other checks)"

OtherChecksName is the bucket for check names past MaxCheckNames.

View Source
const OtherStepsName = "(other steps)"

OtherStepsName is the bucket for step names past MaxStepNames.

Variables

This section is empty.

Functions

func AppendStepSummary added in v0.4.0

func AppendStepSummary(path string, res *Result) error

AppendStepSummary appends MarkdownSummary(res) to the file at path. A step summary file is shared by every step of a job, so it is appended to, never replaced.

func FormatByteCount added in v0.9.0

func FormatByteCount(n int64) string

FormatByteCount renders a byte count as B, KiB, MiB, GiB, TiB, or PiB.

func JUnitXML added in v0.4.0

func JUnitXML(res *Result) ([]byte, error)

JUnitXML renders res as JUnit XML.

func MarkdownSummary added in v0.4.0

func MarkdownSummary(res *Result) string

MarkdownSummary renders res as a short Markdown page: the key numbers and the verdicts (FR-RPT-05). `vegaload run` appends it to the file GitHub Actions names in GITHUB_STEP_SUMMARY, so it shows on the job's page. The verdicts are the words PASS and FAIL, not colour or symbols.

func ShowProtocols added in v0.9.0

func ShowProtocols(res *Result) bool

ShowProtocols reports whether the text, Markdown, and HTML views should print the protocol breakdown. JSON always keeps the data. A run that only called http, http1, or http2 keeps the layout it had before this breakdown existed.

func WriteHTML

func WriteHTML(path string, res *Result) error

WriteHTML implements FR-RPT-01: one self-contained HTML file for res, with no server and nothing fetched at render time — every number is inlined, and the two charts are hand-drawn inline SVG (see the artifact-diagramming conventions this follows: currentColor strokes so the report reads in both light and dark browser themes, a viewBox sized to the content, labelled axes instead of a legend). The file opens correctly from disk with no network access at all, which is what "self-contained" means for FR-RPT-03's offline-renderable guarantee.

func WriteJSON

func WriteJSON(path string, res *Result) error

WriteJSON implements FR-CLI-05's structured JSON output mode for a finished run's result: res marshalled as indented JSON to path.

func WriteJUnit added in v0.4.0

func WriteJUnit(path string, res *Result) error

WriteJUnit writes res as JUnit XML to path.

Types

type AbortInfo added in v0.6.0

type AbortInfo struct {
	// Threshold is the name of the threshold that stopped the run.
	Threshold string `json:"threshold"`
	// Observed is what the run measured when it was stopped.
	Observed string `json:"observed"`
	// At is how far into the run it was stopped.
	At time.Duration `json:"at_ns"`
}

AbortInfo says why and when a run was stopped early (FR-CLI-13).

type BaselineMetric added in v0.4.0

type BaselineMetric struct {
	Name      string  `json:"name"`
	Baseline  float64 `json:"baseline"`
	Candidate float64 `json:"candidate"`
	Unit      string  `json:"unit"`
	Regressed bool    `json:"regressed"`
}

BaselineMetric is one gated metric: its baseline and this run's value, in the unit given ("ns" for p95, "ratio" for the error rate).

type BaselineResult added in v0.4.0

type BaselineResult struct {
	Path string `json:"path"`
	// MaxRegressionPercent is how much worse than the baseline each gated
	// metric may be before the gate fails.
	MaxRegressionPercent float64          `json:"max_regression_percent"`
	Passed               bool             `json:"passed"`
	Metrics              []BaselineMetric `json:"metrics"`
	Notes                []string         `json:"notes,omitempty"`
}

BaselineResult is the verdict of a run's baseline gate (FR-CLI-14). It only covers the two gated metrics, p95 latency and error rate; the full comparison is what `vegaload compare` prints.

type BucketStat added in v0.9.1

type BucketStat struct {
	Offset      time.Duration // start of the bucket
	Requests    int64
	Failed      int64
	RPS         float64
	ErrorRate   float64
	P50         time.Duration
	P95         time.Duration
	P99         time.Duration
	Total       int64 // calls in this and every earlier bucket, as emitted
	TotalFailed int64
	// Capped is true when the bucket was handed out while a call that started
	// in it, or in an earlier second, was still running, because the hold hit
	// its limit (LiveMaxHold). The bucket then lacks that call; the final
	// Result.TimeSeries has it.
	Capped bool
}

BucketStat is what one one-second bucket of a run holds: the calls that started in that second. Requests, Failed, RPS and ErrorRate are the same numbers as the matching Point of Result.TimeSeries, worked out the same way. The latency percentiles use the same nearest-rank rule as Finish.

type CheckResult added in v0.4.0

type CheckResult struct {
	Name   string `json:"name"`
	Passes int64  `json:"passes"`
	Fails  int64  `json:"fails"`
}

CheckResult is one named check's totals across the whole run.

type Collector

type Collector struct {
	// contains filtered or unexported fields
}

Collector implements engine.Recorder, accumulating every iteration of a run so Finish can compute the percentiles and time-bucketed series a Result needs. It is safe for concurrent use — Run calls Record from every VU goroutine at once.

func NewCollector

func NewCollector() *Collector

NewCollector returns a Collector ready to record a run that is starting now.

func (*Collector) Begin added in v0.9.1

func (c *Collector) Begin(start time.Time)

Begin and End implement engine.IterationTracker. They keep the start of each running iteration, only while a live cursor exists (see LiveBuckets), so the cursor can hold a bucket back until the calls that started in it have ended.

func (*Collector) Counts

func (c *Collector) Counts() (total, failed int64)

Counts returns the total and failed iteration counts recorded so far. Safe to call concurrently with Record, including while the run it belongs to is still in progress.

func (*Collector) End added in v0.9.1

func (c *Collector) End(start time.Time)

End undoes the Begin made with the same start. The engine calls it after the iteration was recorded.

func (*Collector) Finish

func (c *Collector) Finish(executor string, elapsed time.Duration) *Result

Finish computes a Result from every sample recorded so far. Call it once, after the executor has returned; executor identifies the load shape (ex.Name()) and elapsed is the run's measured wall-clock time, both of which the caller already has from runScenario.

func (*Collector) LiveBuckets added in v0.9.1

func (c *Collector) LiveBuckets() *LiveBuckets

LiveBuckets returns a cursor over the collector's buckets, from the first. Use one cursor per run, from one goroutine. Creating it makes the collector keep the start of every running iteration from then on, so create it before the executor starts.

func (*Collector) Record

func (c *Collector) Record(r engine.IterationResult)

Record implements engine.Recorder.

func (*Collector) RecordCall added in v0.9.0

func (c *Collector) RecordCall(protocol, op string, d time.Duration, failed bool, sent, received int64)

RecordCall counts one finished protocol call. A nil collector counts nothing. protocol and op are redacted before they are stored.

func (*Collector) RecordCheck added in v0.4.0

func (c *Collector) RecordCheck(name string, passed bool)

RecordCheck counts one evaluation of the named check (FR-CLI-12). It implements netapi.CheckRecorder and is safe for concurrent use.

func (*Collector) RecordMeasure added in v0.9.0

func (c *Collector) RecordMeasure(protocol, name string, d time.Duration)

RecordMeasure counts one phase time inside a protocol. A measure is not a call and does not change that protocol's totals.

func (*Collector) RecordStep added in v0.6.0

func (c *Collector) RecordStep(name string, d time.Duration, failed bool)

RecordStep counts one run of the named step (FR-CLI-18). It implements netapi.StepRecorder and is safe for concurrent use.

func (*Collector) Start added in v0.9.1

func (c *Collector) Start() time.Time

Start is the instant the run's clock started. Every offset in the series (and in a live point) counts from it.

type DataFile added in v0.9.0

type DataFile struct {
	Name string `json:"name"`
	Rows int    `json:"rows"`
}

DataFile is one data file a protocol run read. Name is the file's base name. Rows is how many rows it has. A cell is not stored.

type Latency

type Latency struct {
	Min  time.Duration `json:"min_ns"`
	Mean time.Duration `json:"mean_ns"`
	Max  time.Duration `json:"max_ns"`
	P50  time.Duration `json:"p50_ns"`
	P90  time.Duration `json:"p90_ns"`
	P95  time.Duration `json:"p95_ns"`
	P99  time.Duration `json:"p99_ns"`
}

Latency holds the percentiles and summary stats FR-RPT-01 asks for.

type LiveBuckets added in v0.9.1

type LiveBuckets struct {
	// contains filtered or unexported fields
}

LiveBuckets hands out the one-second buckets of a run that is still going, in order, each exactly once. It reads the collector's samples without copying them all: a bucket's calls are found by scanning back from the end of the slice, which holds the newest.

A call is recorded when it ends, but it belongs to the second it started in. A bucket is therefore handed out only when its second has closed, a grace period has passed (for a call that is just starting or just ending), and no call that started in that second or an earlier one is still running (the collector learns this from the engine through Begin and End). A call that runs longer than LiveMaxHold past the end of its second no longer holds the bucket: it is handed out without that call, marked Capped, and the final Result.TimeSeries, which has the call, is authoritative.

The final series ends at the last second a recorded call started in. A bucket past that second is therefore not handed out while the run goes on (a quiet second is held until a later one has a call, or the run ends), so the points never cover a second the series does not. Rest hands out what is left. A Capped bucket is the one exception: it goes out as it is.

func (*LiveBuckets) Next added in v0.9.1

func (l *LiveBuckets) Next(now, grace time.Duration) (BucketStat, bool)

Next returns the next bucket once it is due: now is the time since the collector started, and the bucket is due when now has passed the end of its second by grace and no call that started in it or earlier is running. A hold ends at the latest maxHold past the end of the second, and the bucket is then Capped. ok is false when it is not due yet.

func (*LiveBuckets) Rest added in v0.9.1

func (l *LiveBuckets) Rest() []BucketStat

Rest returns every bucket not handed out yet, up to the last second a call started in. Call it once the run has ended, before Finish's result is used, so that the points and the final time series cover the same seconds.

func (*LiveBuckets) SetMaxHold added in v0.9.1

func (l *LiveBuckets) SetMaxHold(d time.Duration)

SetMaxHold changes the hold limit (LiveMaxHold by default). It is for tests.

type MeasureResult added in v0.9.0

type MeasureResult struct {
	Name    string  `json:"name"`
	Count   int64   `json:"count"`
	Latency Latency `json:"latency"`
}

MeasureResult is a phase time inside a protocol, such as ftp upload_time. A measure is not a call: it does not change the protocol's totals.

type OperationResult added in v0.9.0

type OperationResult struct {
	Name          string  `json:"name"`
	Total         int64   `json:"total"`
	Failed        int64   `json:"failed"`
	ErrorRate     float64 `json:"error_rate"`
	Latency       Latency `json:"latency"`
	BytesSent     int64   `json:"bytes_sent"`
	BytesReceived int64   `json:"bytes_received"`
}

OperationResult is one operation inside a protocol, such as ftp roundtrip.

type Point

type Point struct {
	Offset    time.Duration `json:"offset_ns"`
	Requests  int64         `json:"requests"`
	Failed    int64         `json:"failed"`
	RPS       float64       `json:"rps"`
	ErrorRate float64       `json:"error_rate"`
}

Point is one bucket of Result.TimeSeries.

type ProtocolResult added in v0.9.0

type ProtocolResult struct {
	Name          string            `json:"name"`
	Total         int64             `json:"total"`
	Failed        int64             `json:"failed"`
	ErrorRate     float64           `json:"error_rate"`
	Latency       Latency           `json:"latency"`
	BytesSent     int64             `json:"bytes_sent"`
	BytesReceived int64             `json:"bytes_received"`
	BytesPerSec   float64           `json:"bytes_per_second"`
	Operations    []OperationResult `json:"operations"`
	Measures      []MeasureResult   `json:"measures,omitempty"`
}

ProtocolResult is one protocol's totals across the whole run. Latency is computed from every call, not from the operation percentiles. BytesPerSec is (sent + received) divided by the run's elapsed time.

func (ProtocolResult) Operation added in v0.9.0

func (p ProtocolResult) Operation(name string) (OperationResult, bool)

Operation returns the operation with the given name.

type Result

type Result struct {
	Executor  string        `json:"executor"`
	StartedAt time.Time     `json:"started_at"`
	Elapsed   time.Duration `json:"elapsed_ns"`
	Total     int64         `json:"total"`
	Failed    int64         `json:"failed"`
	ErrorRate float64       `json:"error_rate"`
	Latency   Latency       `json:"latency"`
	// TimeSeries is RPS and error rate bucketed by one-second intervals
	// from the run's start (FR-RPT-01's "RPS over time" and "error rate
	// over time"). It stays small regardless of request volume — one
	// point per second of run duration, not one per request — which is
	// what lets the HTML report embed it directly even for a
	// 1,000,000-request run (NFR-03).
	TimeSeries []Point `json:"time_series"`

	// Thresholds is the verdict of each pass/fail threshold the run was
	// given (FR-CLI-11), in the order they were given. It is omitted when
	// the run had none, so reports from before thresholds existed, and
	// runs that do not use them, look exactly as before. ThresholdsPassed
	// is true only when every threshold passed.
	Thresholds       []ThresholdResult `json:"thresholds,omitempty"`
	ThresholdsPassed *bool             `json:"thresholds_passed,omitempty"`

	// Aborted is set when the run was stopped early because a threshold
	// was broken (-abort-on-breach, FR-CLI-13). It is omitted for a run
	// that ran to its end.
	Aborted *AbortInfo `json:"aborted,omitempty"`

	// Baseline is the verdict of the baseline gate (FR-CLI-14): this run
	// compared with a baseline report the user supplied. It is omitted
	// when the run had no baseline.
	Baseline *BaselineResult `json:"baseline,omitempty"`

	// Steps is the latency and error rate of each named step the scenario
	// ran with step() (FR-CLI-18), in the order each name was first seen.
	// It is omitted when the scenario used no steps, so reports from
	// before steps existed look exactly as before. A step that throws is a
	// failed step, and it also fails its iteration.
	Steps []StepResult `json:"steps,omitempty"`

	// Protocols is the latency, error rate, and byte totals of each
	// protocol the run called, in the order each name was first seen.
	// It is omitted when the run recorded no call, so a report written
	// before this field existed, and a run that makes no recorded call,
	// look exactly as before.
	Protocols []ProtocolResult `json:"protocols,omitempty"`

	// Checks is the pass and fail count of each named check the scenario
	// made with check() (FR-CLI-12), in the order each name was first
	// seen. It is omitted when the scenario made no checks. A failed
	// check does not fail an iteration or count as a failed request: it
	// is its own measure, which a threshold can judge through the
	// check_rate metric.
	Checks []CheckResult `json:"checks,omitempty"`

	// DataFiles is the name and row count of each data file a protocol
	// run read. It is omitted when the run had no -data, so that JSON
	// stays the same. A cell is never recorded.
	DataFiles []DataFile `json:"data_files,omitempty"`
}

Result is a run's structured outcome: everything the HTML report renders, and the same shape FR-RPT-04 says the MCP get_results tool (Phase 2) must return, so an agent host can render its own view from exactly the same data a human reads in the HTML report.

func (*Result) CheckRate added in v0.4.0

func (r *Result) CheckRate() (float64, bool)

CheckRate is the share of all checks that passed, from 0 to 1. The second result is false when the run made no checks, since a rate of nothing is not a rate of 100%.

func (*Result) Protocol added in v0.9.0

func (r *Result) Protocol(name string) (ProtocolResult, bool)

Protocol returns the protocol with the given name, if the run recorded one.

func (*Result) Step added in v0.6.0

func (r *Result) Step(name string) (StepResult, bool)

Step returns the step with the given name, if the run had one.

type StepResult added in v0.6.0

type StepResult struct {
	Name      string  `json:"name"`
	Total     int64   `json:"total"`
	Failed    int64   `json:"failed"`
	ErrorRate float64 `json:"error_rate"`
	Latency   Latency `json:"latency"`
}

StepResult is one named step's totals across the whole run: how many times it ran, how many of those failed, and its latency.

type ThresholdResult added in v0.4.0

type ThresholdResult struct {
	Name   string `json:"name"`
	Metric string `json:"metric"`
	// Step is the step the threshold judges (FR-CLI-18). It is empty for a
	// threshold on the whole run.
	Step string `json:"step,omitempty"`
	// Protocol and Op are the protocol, and optionally one operation, the
	// threshold judges. They are empty for a threshold on the whole run.
	Protocol string `json:"protocol,omitempty"`
	Op       string `json:"op,omitempty"`
	Operator string `json:"operator"`
	Value    string `json:"value"`
	Observed string `json:"observed"`
	Passed   bool   `json:"passed"`
}

ThresholdResult is one threshold's verdict: what was asked (a name, a metric, an operator and a value) and what the run observed. The shape is deliberately plain — a name, a metric, an operator, a value — so anything that reads a saved report can read it without knowing how VegaLoad evaluated it. Value and Observed are strings so durations ("300ms") and plain numbers share one field.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL