compare

package
v0.5.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 7, 2026 License: Apache-2.0 Imports: 3 Imported by: 0

Documentation

Overview

Package compare diffs two report.Result values (typically JSON from `vegaload run -out`) so a later run can be checked against a baseline without a baseline registry or history store — just two files.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Gate added in v0.4.0

func Gate(baseline, candidate *report.Result, path string, pct float64) *report.BaselineResult

Gate judges a finished run against a baseline report (FR-CLI-14). It uses Compare, so `vegaload run -baseline` and `vegaload compare` agree. A run that completed no requests fails the gate: with nothing measured, "no worse than the baseline" would pass trivially.

Types

type Metric

type Metric struct {
	Name      string  `json:"name"`
	Baseline  float64 `json:"baseline"`
	Candidate float64 `json:"candidate"`
	Delta     float64 `json:"delta"` // candidate - baseline
	Unit      string  `json:"unit"`  // "ratio", "ns", "rps", "count"
	Regressed bool    `json:"regressed"`
}

Metric is one compared field: baseline vs candidate, with a delta and whether that field alone counts as a regression under Options.

type Options

type Options struct {
	// ErrorRateDelta is absolute slack on error rate (0.01 = one
	// percentage point). Candidate may exceed baseline by this much
	// without counting as a regression.
	ErrorRateDelta float64
	// P95Ratio is the maximum allowed candidate/baseline p95 ratio.
	// 1.0 means any p95 increase is a regression; 1.2 allows 20% headroom.
	// Values below 1.0 are treated as 1.0.
	P95Ratio float64
}

Options tunes what counts as a regression. Zero values mean strict compare: any increase in error rate or p95 latency is a regression.

func GateOptions added in v0.4.0

func GateOptions(baseline *report.Result, pct float64) Options

GateOptions turns "no more than pct percent worse than the baseline" into Compare's options. p95 may rise by pct percent. The error rate may rise by pct percent of the baseline's error rate, so a baseline with no errors allows none: use a -threshold on error_rate for an absolute limit.

type Result

type Result struct {
	BaselinePath  string   `json:"baseline_path,omitempty"`
	CandidatePath string   `json:"candidate_path,omitempty"`
	Metrics       []Metric `json:"metrics"`
	Regressed     bool     `json:"regressed"`
	Notes         []string `json:"notes"`
}

Result is the structured outcome of Compare — the same shape printed as text or JSON by `vegaload compare`, and returned by the MCP compare_reports tool.

func Compare

func Compare(baseline, candidate *report.Result, opts Options) Result

Compare diffs candidate against baseline. Paths are optional labels filled in by the CLI; the comparison itself is purely numeric.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL