streambench

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 3, 2026 License: MIT Imports: 9 Imported by: 0

Documentation

Overview

Package streambench holds the result schema, statistics helpers and the regression comparison used by the streaming benchmarks in internal/nzbfilesystem and by cmd/compare.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AnyRegressed

func AnyRegressed(d []Delta) bool

func FormatTable

func FormatTable(d []Delta) string

func Save

func Save(path string, r *Result) error

Types

type Delta

type Delta struct {
	Scenario       string
	Metric         string
	Unit           string
	Base, New      float64
	Pct            float64
	HigherIsBetter bool
	Informational  bool
	Regressed      bool
}

func Compare

func Compare(base, nw *Result, threshold float64) []Delta

Compare pairs metrics by scenario and name. A metric regresses when it moves more than threshold (fraction, e.g. 0.05) in its bad direction; a metric-level Tolerance on either side raises that bar for that metric. Metrics present in only one result are reported with a zero delta.

type Metric

type Metric struct {
	Name           string  `json:"name"`
	Unit           string  `json:"unit"`
	Value          float64 `json:"value"`
	HigherIsBetter bool    `json:"higher_is_better"`
	// Informational metrics are reported but never fail the regression gate:
	// tail percentiles from small samples and counts that depend on timing.
	Informational bool `json:"informational,omitempty"`
	// Tolerance, when non-zero, replaces the global threshold for this metric.
	// It records measured run-to-run noise so a naturally jittery number does
	// not fail the gate on noise alone.
	Tolerance float64 `json:"tolerance,omitempty"`
}

Metric is one measured value. HigherIsBetter drives the regression rule: throughput is better when higher, latency and byte waste when lower.

func Median

func Median(runs [][]Metric) []Metric

Median returns the per-name median of several runs' metrics. Every run must report the same metric names in the same order.

type Result

type Result struct {
	GitSHA    string     `json:"git_sha"`
	Timestamp time.Time  `json:"timestamp"`
	Profile   string     `json:"profile,omitempty"`
	Scenarios []Scenario `json:"scenarios"`
}

func Load

func Load(path string) (*Result, error)

func (*Result) Add

func (r *Result) Add(scenario string, m ...Metric)

Add appends metrics to the named scenario, creating it on first use so benchmarks can report in any order.

type Samples

type Samples struct {
	// contains filtered or unexported fields
}

Samples collects durations from concurrent goroutines and answers percentile queries over the sorted set.

func (*Samples) Add

func (s *Samples) Add(d time.Duration)

func (*Samples) Count

func (s *Samples) Count() int

func (*Samples) Mean

func (s *Samples) Mean() time.Duration

func (*Samples) P

func (s *Samples) P(q float64) time.Duration

P returns the q-quantile (q in [0,1]) using nearest-rank on the sorted samples. Zero when there are no samples.

type Scenario

type Scenario struct {
	Name    string   `json:"name"`
	Metrics []Metric `json:"metrics"`
}

Directories

Path Synopsis
cmd
compare command
Command compare diffs two streaming benchmark result files and exits 1 when any metric regressed past the threshold.
Command compare diffs two streaming benchmark result files and exits 1 when any metric regressed past the threshold.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL