Documentation
¶
Overview ¶
Package report implements FR-RPT-01 through 04 and FR-CLI-05: turning a run's recorded iterations into the structured data a self-contained HTML report, a JSON summary, and (eventually) the MCP get_results tool all read from. It implements engine.Recorder, exactly the seam internal/engine's own doc comment on Recorder sets aside for it, and knows nothing about protocols, scripting, or the CLI.
Index ¶
- Constants
- func AppendStepSummary(path string, res *Result) error
- func JUnitXML(res *Result) ([]byte, error)
- func MarkdownSummary(res *Result) string
- func WriteHTML(path string, res *Result) error
- func WriteJSON(path string, res *Result) error
- func WriteJUnit(path string, res *Result) error
- type BaselineMetric
- type BaselineResult
- type CheckResult
- type Collector
- type Latency
- type Point
- type Result
- type ThresholdResult
Constants ¶
const MaxCheckNames = 100
MaxCheckNames bounds how many distinct check names a run keeps. Further names are counted together under OtherChecksName.
const OtherChecksName = "(other checks)"
OtherChecksName is the bucket for check names past MaxCheckNames.
Variables ¶
This section is empty.
Functions ¶
func AppendStepSummary ¶ added in v0.4.0
AppendStepSummary appends MarkdownSummary(res) to the file at path. A step summary file is shared by every step of a job, so it is appended to, never replaced.
func MarkdownSummary ¶ added in v0.4.0
MarkdownSummary renders res as a short Markdown page: the key numbers and the verdicts (FR-RPT-05). `vegaload run` appends it to the file GitHub Actions names in GITHUB_STEP_SUMMARY, so it shows on the job's page. The verdicts are the words PASS and FAIL, not colour or symbols.
func WriteHTML ¶
WriteHTML implements FR-RPT-01: one self-contained HTML file for res, with no server and nothing fetched at render time — every number is inlined, and the two charts are hand-drawn inline SVG (see the artifact-diagramming conventions this follows: currentColor strokes so the report reads in both light and dark browser themes, a viewBox sized to the content, labelled axes instead of a legend). The file opens correctly from disk with no network access at all, which is what "self-contained" means for FR-RPT-03's offline-renderable guarantee.
func WriteJSON ¶
WriteJSON implements FR-CLI-05's structured JSON output mode for a finished run's result: res marshalled as indented JSON to path.
func WriteJUnit ¶ added in v0.4.0
WriteJUnit writes res as JUnit XML to path.
Types ¶
type BaselineMetric ¶ added in v0.4.0
type BaselineMetric struct {
Name string `json:"name"`
Baseline float64 `json:"baseline"`
Candidate float64 `json:"candidate"`
Unit string `json:"unit"`
Regressed bool `json:"regressed"`
}
BaselineMetric is one gated metric: its baseline and this run's value, in the unit given ("ns" for p95, "ratio" for the error rate).
type BaselineResult ¶ added in v0.4.0
type BaselineResult struct {
Path string `json:"path"`
// MaxRegressionPercent is how much worse than the baseline each gated
// metric may be before the gate fails.
MaxRegressionPercent float64 `json:"max_regression_percent"`
Passed bool `json:"passed"`
Metrics []BaselineMetric `json:"metrics"`
Notes []string `json:"notes,omitempty"`
}
BaselineResult is the verdict of a run's baseline gate (FR-CLI-14). It only covers the two gated metrics, p95 latency and error rate; the full comparison is what `vegaload compare` prints.
type CheckResult ¶ added in v0.4.0
type CheckResult struct {
Name string `json:"name"`
Passes int64 `json:"passes"`
Fails int64 `json:"fails"`
}
CheckResult is one named check's totals across the whole run.
type Collector ¶
type Collector struct {
// contains filtered or unexported fields
}
Collector implements engine.Recorder, accumulating every iteration of a run so Finish can compute the percentiles and time-bucketed series a Result needs. It is safe for concurrent use — Run calls Record from every VU goroutine at once.
func NewCollector ¶
func NewCollector() *Collector
NewCollector returns a Collector ready to record a run that is starting now.
func (*Collector) Counts ¶
Counts returns the total and failed iteration counts recorded so far. Safe to call concurrently with Record, including while the run it belongs to is still in progress.
func (*Collector) Finish ¶
Finish computes a Result from every sample recorded so far. Call it once, after the executor has returned; executor identifies the load shape (ex.Name()) and elapsed is the run's measured wall-clock time, both of which the caller already has from runScenario.
func (*Collector) Record ¶
func (c *Collector) Record(r engine.IterationResult)
Record implements engine.Recorder.
func (*Collector) RecordCheck ¶ added in v0.4.0
RecordCheck counts one evaluation of the named check (FR-CLI-12). It implements netapi.CheckRecorder and is safe for concurrent use.
type Latency ¶
type Latency struct {
Min time.Duration `json:"min_ns"`
Mean time.Duration `json:"mean_ns"`
Max time.Duration `json:"max_ns"`
P50 time.Duration `json:"p50_ns"`
P90 time.Duration `json:"p90_ns"`
P95 time.Duration `json:"p95_ns"`
P99 time.Duration `json:"p99_ns"`
}
Latency holds the percentiles and summary stats FR-RPT-01 asks for.
type Point ¶
type Point struct {
Offset time.Duration `json:"offset_ns"`
Requests int64 `json:"requests"`
Failed int64 `json:"failed"`
RPS float64 `json:"rps"`
ErrorRate float64 `json:"error_rate"`
}
Point is one bucket of Result.TimeSeries.
type Result ¶
type Result struct {
Executor string `json:"executor"`
StartedAt time.Time `json:"started_at"`
Elapsed time.Duration `json:"elapsed_ns"`
Total int64 `json:"total"`
Failed int64 `json:"failed"`
ErrorRate float64 `json:"error_rate"`
Latency Latency `json:"latency"`
// TimeSeries is RPS and error rate bucketed by one-second intervals
// from the run's start (FR-RPT-01's "RPS over time" and "error rate
// over time"). It stays small regardless of request volume — one
// point per second of run duration, not one per request — which is
// what lets the HTML report embed it directly even for a
// 1,000,000-request run (NFR-03).
TimeSeries []Point `json:"time_series"`
// Thresholds is the verdict of each pass/fail threshold the run was
// given (FR-CLI-11), in the order they were given. It is omitted when
// the run had none, so reports from before thresholds existed, and
// runs that do not use them, look exactly as before. ThresholdsPassed
// is true only when every threshold passed.
Thresholds []ThresholdResult `json:"thresholds,omitempty"`
ThresholdsPassed *bool `json:"thresholds_passed,omitempty"`
// Baseline is the verdict of the baseline gate (FR-CLI-14): this run
// compared with a baseline report the user supplied. It is omitted
// when the run had no baseline.
Baseline *BaselineResult `json:"baseline,omitempty"`
// Checks is the pass and fail count of each named check the scenario
// made with check() (FR-CLI-12), in the order each name was first
// seen. It is omitted when the scenario made no checks. A failed
// check does not fail an iteration or count as a failed request: it
// is its own measure, which a threshold can judge through the
// check_rate metric.
Checks []CheckResult `json:"checks,omitempty"`
}
Result is a run's structured outcome: everything the HTML report renders, and the same shape FR-RPT-04 says the MCP get_results tool (Phase 2) must return, so an agent host can render its own view from exactly the same data a human reads in the HTML report.
type ThresholdResult ¶ added in v0.4.0
type ThresholdResult struct {
Name string `json:"name"`
Metric string `json:"metric"`
Operator string `json:"operator"`
Value string `json:"value"`
Observed string `json:"observed"`
Passed bool `json:"passed"`
}
ThresholdResult is one threshold's verdict: what was asked (a name, a metric, an operator and a value) and what the run observed. The shape is deliberately plain — a name, a metric, an operator, a value — so anything that reads a saved report can read it without knowing how VegaLoad evaluated it. Value and Observed are strings so durations ("300ms") and plain numbers share one field.