threshold

package
v0.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 13, 2026 License: Apache-2.0 Imports: 8 Imported by: 0

Documentation

Overview

Package threshold implements FR-CLI-11: pass/fail thresholds on a run's own results, so a run can gate a CI pipeline.

A threshold is a name, a metric, an operator and a value, for example

fast-api: p95 < 300ms

It is judged once, against the finished run's numbers — the latency percentiles, error rate, request rate and counts VegaLoad itself measured. It never looks outside the run, and it produces a verdict (pass or fail) and nothing else.

A threshold can judge one named step of a scenario (FR-CLI-18), one protocol, or one operation of a protocol, instead of the whole run, by adding a selector after the metric:

login-fast: p95{step="login"} < 300ms
ftp-fast: p95{protocol="ftp"} < 500ms
rt-ok: error_rate{protocol="ftp",op="roundtrip"} < 1%

Index

Constants

View Source
const MinLiveSamples = 20

MinLiveSamples is how many requests (or checks, or runs of a step) a statistic needs before a live judgement counts. A p95 of ten requests says very little.

Variables

This section is empty.

Functions

func AllPassed

func AllPassed(rs []report.ThresholdResult) bool

AllPassed reports whether every result passed. It is true for an empty list, since a run with no thresholds has nothing to breach.

func Evaluate

func Evaluate(ts []Threshold, res *report.Result) []report.ThresholdResult

Evaluate judges every threshold against a finished run's result, in order. A run that completed no requests fails every threshold: with nothing measured, "p95 < 300ms" would otherwise pass trivially.

func MetricNames

func MetricNames() []string

MetricNames lists the metrics a threshold can use, for help text and error messages.

Types

type LiveKind added in v0.6.0

type LiveKind int

LiveKind says how far a threshold can be judged while a run is still in progress (FR-CLI-13, -abort-on-breach).

const (
	// LiveNone thresholds are judged only on the finished run. An example
	// is rps, which is an average over the whole run, or "total >= 1000",
	// which cannot be broken until the run is over.
	LiveNone LiveKind = iota
	// LiveSticky thresholds cannot recover once broken. The number of
	// failed requests only goes up, so "failed < 5" is broken for good the
	// moment the fifth one fails. The same holds for "max" and "total"
	// with an upper limit, and for "min" with a lower limit.
	LiveSticky
	// LiveSustained thresholds are statistics, such as p95 or error_rate.
	// They move up and down, so a breach early in the run may go away. A
	// run is stopped on one only when it stays broken (see the watcher in
	// cmd/vegaload).
	LiveSustained
)

type Threshold

type Threshold struct {
	Name   string
	Metric string
	// Step is the named step this threshold judges (FR-CLI-18). It is empty
	// for a threshold on the whole run.
	Step string
	// Protocol and Op judge one protocol, or one operation of it. They are
	// empty for a threshold on the whole run. A threshold names a step or a
	// protocol, never both. Op is set only together with Protocol.
	Protocol string
	Op       string
	Operator string
	// Value is the canonical text of the limit: a duration such as
	// "300ms", or a plain number such as "0.01".
	Value string
	// contains filtered or unexported fields
}

Threshold is one parsed pass/fail threshold.

func LoadFile

func LoadFile(path string) ([]Threshold, error)

LoadFile reads thresholds from a JSON file (the -thresholds flag).

Three shapes are accepted, so a file can come straight from another command's output:

[{"name": "fast", "metric": "p95", "operator": "<", "value": "300ms"}]
[{"metric": "p95", "step": "login", "operator": "<", "value": "300ms"}]
{"thresholds": [ ...same list... ]}
{"latency_p95": "300ms", "error_rate": 0.015}

The last is what `vegaload diagnose -output json` suggests, under a "suggested_thresholds" key, which is also accepted. The suggested values are ceilings, so they become "<=" thresholds.

func NotLive added in v0.6.0

func NotLive(ts []Threshold) []Threshold

NotLive returns the thresholds that cannot be judged during a run, so the caller can say they wait for the end.

func Parse

func Parse(expr string) (Threshold, error)

Parse parses one threshold expression: an optional "name:" prefix, then metric, operator and value, e.g. "p95 < 300ms" or "fast-api: error_rate <= 1%".

func ParseAll

func ParseAll(exprs []string) ([]Threshold, error)

ParseAll parses several expressions, failing on the first bad one.

func ParseFile

func ParseFile(data []byte) ([]Threshold, error)

ParseFile parses the contents of a thresholds file. See LoadFile.

func (Threshold) Expression

func (t Threshold) Expression() string

Expression returns the threshold as text that Parse accepts again, including its name, e.g. "fast-api: p95 < 300ms". It is what other commands print as a ready-to-paste -threshold flag.

func (Threshold) Live added in v0.6.0

func (t Threshold) Live() LiveKind

Live reports how far t can be judged during a run.

func (Threshold) LiveBreach added in v0.6.0

func (t Threshold) LiveBreach(res *report.Result) (observed string, breached bool)

LiveBreach judges t on a snapshot of a run in progress. It reports breached only when there is enough evidence: a step that has not run yet, or too few samples for a statistic, is not a breach. Those are settled on the finished run, as before.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL