report

package
v0.9.0-rc.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 13, 2026 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Overview

Package report implements FR-RPT-01 through 04 and FR-CLI-05: turning a run's recorded iterations into the structured data a self-contained HTML report, a JSON summary, and (eventually) the MCP get_results tool all read from. It implements engine.Recorder, exactly the seam internal/engine's own doc comment on Recorder sets aside for it, and knows nothing about protocol drivers, scripting, or the CLI. A protocol call arrives as a name, the same way a step does.

Index

Constants

View Source
const MaxCheckNames = 100

MaxCheckNames bounds how many distinct check names a run keeps. Further names are counted together under OtherChecksName.

View Source
const MaxStepNames = 100

MaxStepNames bounds how many distinct step names a run keeps. Further names are counted together under OtherStepsName.

View Source
const OtherChecksName = "(other checks)"

OtherChecksName is the bucket for check names past MaxCheckNames.

View Source
const OtherStepsName = "(other steps)"

OtherStepsName is the bucket for step names past MaxStepNames.

Variables

This section is empty.

Functions

func AppendStepSummary added in v0.4.0

func AppendStepSummary(path string, res *Result) error

AppendStepSummary appends MarkdownSummary(res) to the file at path. A step summary file is shared by every step of a job, so it is appended to, never replaced.

func FormatByteCount added in v0.9.0

func FormatByteCount(n int64) string

FormatByteCount renders a byte count as B, KiB, MiB, GiB, TiB, or PiB.

func JUnitXML added in v0.4.0

func JUnitXML(res *Result) ([]byte, error)

JUnitXML renders res as JUnit XML.

func MarkdownSummary added in v0.4.0

func MarkdownSummary(res *Result) string

MarkdownSummary renders res as a short Markdown page: the key numbers and the verdicts (FR-RPT-05). `vegaload run` appends it to the file GitHub Actions names in GITHUB_STEP_SUMMARY, so it shows on the job's page. The verdicts are the words PASS and FAIL, not colour or symbols.

func ShowProtocols added in v0.9.0

func ShowProtocols(res *Result) bool

ShowProtocols reports whether the text, Markdown, and HTML views should print the protocol breakdown. JSON always keeps the data. A run that only called http, http1, or http2 keeps the layout it had before this breakdown existed.

func WriteHTML

func WriteHTML(path string, res *Result) error

WriteHTML implements FR-RPT-01: one self-contained HTML file for res, with no server and nothing fetched at render time — every number is inlined, and the two charts are hand-drawn inline SVG (see the artifact-diagramming conventions this follows: currentColor strokes so the report reads in both light and dark browser themes, a viewBox sized to the content, labelled axes instead of a legend). The file opens correctly from disk with no network access at all, which is what "self-contained" means for FR-RPT-03's offline-renderable guarantee.

func WriteJSON

func WriteJSON(path string, res *Result) error

WriteJSON implements FR-CLI-05's structured JSON output mode for a finished run's result: res marshalled as indented JSON to path.

func WriteJUnit added in v0.4.0

func WriteJUnit(path string, res *Result) error

WriteJUnit writes res as JUnit XML to path.

Types

type AbortInfo added in v0.6.0

type AbortInfo struct {
	// Threshold is the name of the threshold that stopped the run.
	Threshold string `json:"threshold"`
	// Observed is what the run measured when it was stopped.
	Observed string `json:"observed"`
	// At is how far into the run it was stopped.
	At time.Duration `json:"at_ns"`
}

AbortInfo says why and when a run was stopped early (FR-CLI-13).

type BaselineMetric added in v0.4.0

type BaselineMetric struct {
	Name      string  `json:"name"`
	Baseline  float64 `json:"baseline"`
	Candidate float64 `json:"candidate"`
	Unit      string  `json:"unit"`
	Regressed bool    `json:"regressed"`
}

BaselineMetric is one gated metric: its baseline and this run's value, in the unit given ("ns" for p95, "ratio" for the error rate).

type BaselineResult added in v0.4.0

type BaselineResult struct {
	Path string `json:"path"`
	// MaxRegressionPercent is how much worse than the baseline each gated
	// metric may be before the gate fails.
	MaxRegressionPercent float64          `json:"max_regression_percent"`
	Passed               bool             `json:"passed"`
	Metrics              []BaselineMetric `json:"metrics"`
	Notes                []string         `json:"notes,omitempty"`
}

BaselineResult is the verdict of a run's baseline gate (FR-CLI-14). It only covers the two gated metrics, p95 latency and error rate; the full comparison is what `vegaload compare` prints.

type CheckResult added in v0.4.0

type CheckResult struct {
	Name   string `json:"name"`
	Passes int64  `json:"passes"`
	Fails  int64  `json:"fails"`
}

CheckResult is one named check's totals across the whole run.

type Collector

type Collector struct {
	// contains filtered or unexported fields
}

Collector implements engine.Recorder, accumulating every iteration of a run so Finish can compute the percentiles and time-bucketed series a Result needs. It is safe for concurrent use — Run calls Record from every VU goroutine at once.

func NewCollector

func NewCollector() *Collector

NewCollector returns a Collector ready to record a run that is starting now.

func (*Collector) Counts

func (c *Collector) Counts() (total, failed int64)

Counts returns the total and failed iteration counts recorded so far. Safe to call concurrently with Record, including while the run it belongs to is still in progress.

func (*Collector) Finish

func (c *Collector) Finish(executor string, elapsed time.Duration) *Result

Finish computes a Result from every sample recorded so far. Call it once, after the executor has returned; executor identifies the load shape (ex.Name()) and elapsed is the run's measured wall-clock time, both of which the caller already has from runScenario.

func (*Collector) Record

func (c *Collector) Record(r engine.IterationResult)

Record implements engine.Recorder.

func (*Collector) RecordCall added in v0.9.0

func (c *Collector) RecordCall(protocol, op string, d time.Duration, failed bool, sent, received int64)

RecordCall counts one finished protocol call. A nil collector counts nothing. protocol and op are redacted before they are stored.

func (*Collector) RecordCheck added in v0.4.0

func (c *Collector) RecordCheck(name string, passed bool)

RecordCheck counts one evaluation of the named check (FR-CLI-12). It implements netapi.CheckRecorder and is safe for concurrent use.

func (*Collector) RecordMeasure added in v0.9.0

func (c *Collector) RecordMeasure(protocol, name string, d time.Duration)

RecordMeasure counts one phase time inside a protocol. A measure is not a call and does not change that protocol's totals.

func (*Collector) RecordStep added in v0.6.0

func (c *Collector) RecordStep(name string, d time.Duration, failed bool)

RecordStep counts one run of the named step (FR-CLI-18). It implements netapi.StepRecorder and is safe for concurrent use.

type DataFile added in v0.9.0

type DataFile struct {
	Name string `json:"name"`
	Rows int    `json:"rows"`
}

DataFile is one data file a protocol run read. Name is the file's base name. Rows is how many rows it has. A cell is not stored.

type Latency

type Latency struct {
	Min  time.Duration `json:"min_ns"`
	Mean time.Duration `json:"mean_ns"`
	Max  time.Duration `json:"max_ns"`
	P50  time.Duration `json:"p50_ns"`
	P90  time.Duration `json:"p90_ns"`
	P95  time.Duration `json:"p95_ns"`
	P99  time.Duration `json:"p99_ns"`
}

Latency holds the percentiles and summary stats FR-RPT-01 asks for.

type MeasureResult added in v0.9.0

type MeasureResult struct {
	Name    string  `json:"name"`
	Count   int64   `json:"count"`
	Latency Latency `json:"latency"`
}

MeasureResult is a phase time inside a protocol, such as ftp upload_time. A measure is not a call: it does not change the protocol's totals.

type OperationResult added in v0.9.0

type OperationResult struct {
	Name          string  `json:"name"`
	Total         int64   `json:"total"`
	Failed        int64   `json:"failed"`
	ErrorRate     float64 `json:"error_rate"`
	Latency       Latency `json:"latency"`
	BytesSent     int64   `json:"bytes_sent"`
	BytesReceived int64   `json:"bytes_received"`
}

OperationResult is one operation inside a protocol, such as ftp roundtrip.

type Point

type Point struct {
	Offset    time.Duration `json:"offset_ns"`
	Requests  int64         `json:"requests"`
	Failed    int64         `json:"failed"`
	RPS       float64       `json:"rps"`
	ErrorRate float64       `json:"error_rate"`
}

Point is one bucket of Result.TimeSeries.

type ProtocolResult added in v0.9.0

type ProtocolResult struct {
	Name          string            `json:"name"`
	Total         int64             `json:"total"`
	Failed        int64             `json:"failed"`
	ErrorRate     float64           `json:"error_rate"`
	Latency       Latency           `json:"latency"`
	BytesSent     int64             `json:"bytes_sent"`
	BytesReceived int64             `json:"bytes_received"`
	BytesPerSec   float64           `json:"bytes_per_second"`
	Operations    []OperationResult `json:"operations"`
	Measures      []MeasureResult   `json:"measures,omitempty"`
}

ProtocolResult is one protocol's totals across the whole run. Latency is computed from every call, not from the operation percentiles. BytesPerSec is (sent + received) divided by the run's elapsed time.

func (ProtocolResult) Operation added in v0.9.0

func (p ProtocolResult) Operation(name string) (OperationResult, bool)

Operation returns the operation with the given name.

type Result

type Result struct {
	Executor  string        `json:"executor"`
	StartedAt time.Time     `json:"started_at"`
	Elapsed   time.Duration `json:"elapsed_ns"`
	Total     int64         `json:"total"`
	Failed    int64         `json:"failed"`
	ErrorRate float64       `json:"error_rate"`
	Latency   Latency       `json:"latency"`
	// TimeSeries is RPS and error rate bucketed by one-second intervals
	// from the run's start (FR-RPT-01's "RPS over time" and "error rate
	// over time"). It stays small regardless of request volume — one
	// point per second of run duration, not one per request — which is
	// what lets the HTML report embed it directly even for a
	// 1,000,000-request run (NFR-03).
	TimeSeries []Point `json:"time_series"`

	// Thresholds is the verdict of each pass/fail threshold the run was
	// given (FR-CLI-11), in the order they were given. It is omitted when
	// the run had none, so reports from before thresholds existed, and
	// runs that do not use them, look exactly as before. ThresholdsPassed
	// is true only when every threshold passed.
	Thresholds       []ThresholdResult `json:"thresholds,omitempty"`
	ThresholdsPassed *bool             `json:"thresholds_passed,omitempty"`

	// Aborted is set when the run was stopped early because a threshold
	// was broken (-abort-on-breach, FR-CLI-13). It is omitted for a run
	// that ran to its end.
	Aborted *AbortInfo `json:"aborted,omitempty"`

	// Baseline is the verdict of the baseline gate (FR-CLI-14): this run
	// compared with a baseline report the user supplied. It is omitted
	// when the run had no baseline.
	Baseline *BaselineResult `json:"baseline,omitempty"`

	// Steps is the latency and error rate of each named step the scenario
	// ran with step() (FR-CLI-18), in the order each name was first seen.
	// It is omitted when the scenario used no steps, so reports from
	// before steps existed look exactly as before. A step that throws is a
	// failed step, and it also fails its iteration.
	Steps []StepResult `json:"steps,omitempty"`

	// Protocols is the latency, error rate, and byte totals of each
	// protocol the run called, in the order each name was first seen.
	// It is omitted when the run recorded no call, so a report written
	// before this field existed, and a run that makes no recorded call,
	// look exactly as before.
	Protocols []ProtocolResult `json:"protocols,omitempty"`

	// Checks is the pass and fail count of each named check the scenario
	// made with check() (FR-CLI-12), in the order each name was first
	// seen. It is omitted when the scenario made no checks. A failed
	// check does not fail an iteration or count as a failed request: it
	// is its own measure, which a threshold can judge through the
	// check_rate metric.
	Checks []CheckResult `json:"checks,omitempty"`

	// DataFiles is the name and row count of each data file a protocol
	// run read. It is omitted when the run had no -data, so that JSON
	// stays the same. A cell is never recorded.
	DataFiles []DataFile `json:"data_files,omitempty"`
}

Result is a run's structured outcome: everything the HTML report renders, and the same shape FR-RPT-04 says the MCP get_results tool (Phase 2) must return, so an agent host can render its own view from exactly the same data a human reads in the HTML report.

func (*Result) CheckRate added in v0.4.0

func (r *Result) CheckRate() (float64, bool)

CheckRate is the share of all checks that passed, from 0 to 1. The second result is false when the run made no checks, since a rate of nothing is not a rate of 100%.

func (*Result) Protocol added in v0.9.0

func (r *Result) Protocol(name string) (ProtocolResult, bool)

Protocol returns the protocol with the given name, if the run recorded one.

func (*Result) Step added in v0.6.0

func (r *Result) Step(name string) (StepResult, bool)

Step returns the step with the given name, if the run had one.

type StepResult added in v0.6.0

type StepResult struct {
	Name      string  `json:"name"`
	Total     int64   `json:"total"`
	Failed    int64   `json:"failed"`
	ErrorRate float64 `json:"error_rate"`
	Latency   Latency `json:"latency"`
}

StepResult is one named step's totals across the whole run: how many times it ran, how many of those failed, and its latency.

type ThresholdResult added in v0.4.0

type ThresholdResult struct {
	Name   string `json:"name"`
	Metric string `json:"metric"`
	// Step is the step the threshold judges (FR-CLI-18). It is empty for a
	// threshold on the whole run.
	Step string `json:"step,omitempty"`
	// Protocol and Op are the protocol, and optionally one operation, the
	// threshold judges. They are empty for a threshold on the whole run.
	Protocol string `json:"protocol,omitempty"`
	Op       string `json:"op,omitempty"`
	Operator string `json:"operator"`
	Value    string `json:"value"`
	Observed string `json:"observed"`
	Passed   bool   `json:"passed"`
}

ThresholdResult is one threshold's verdict: what was asked (a name, a metric, an operator and a value) and what the run observed. The shape is deliberately plain — a name, a metric, an operator, a value — so anything that reads a saved report can read it without knowing how VegaLoad evaluated it. Value and Observed are strings so durations ("300ms") and plain numbers share one field.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL