benchparse

package module
v0.4.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 5, 2020 License: MIT Imports: 10 Imported by: 2

README

GoTest GoBuild GoDoc

benchparse

Utilities for parsing go benchmark results. Parsed results are split by sub-benchmarks, with support for sub-benchmarks with names of the form var_name=var_value.

See godoc for full package details and examples.

Documentation

Overview

Package benchparse provides utilities for parsing benchmark results. Parsed results are split by sub-benchmarks, with support for sub-benchmarks with names of the form 'var_name=var_value'

Index

Examples

Constants

This section is empty.

Variables

View Source 
var ErrNotMeasured = errors.New("not measured")

ErrNotMeasured indicates that a specific output was not measured.

Functions

This section is empty.

Types

type BenchInputs 

type BenchInputs struct {
	VarValues []BenchVarValue // sub-benchmark names of the form some_var=some_val
	Subs      []BenchSub      // remaining components of a sub-benchmark
	MaxProcs  int             // the value of GOMAXPROCS when the benchmark was run
}

BenchInputs define a sub-benchmark. For example a benchmark with a full name 'BenchmarkMyType/some_method/foo=2/bar=baz-4' would be defined by the Subs=[some_method], the VarValues=[foo=2 bar=baz], and MaxProcs=4.

func (BenchInputs) String 

func (b BenchInputs) String() string

String returns the string representation of the BenchInputs. This should be equivalent to the portion of the benchmark name following the name of the top-level benchmark, but formatting of VarValues may vary slightly.

type BenchOutputs 

type BenchOutputs interface {
	GetIterations() int
	GetNsPerOp() (float64, error)
	GetAllocedBytesPerOp() (uint64, error) // measured if either '-test.benchmem' is set of if testing.B.ReportAllocs() is called
	GetAllocsPerOp() (uint64, error)       // measured if either '-test.benchmem' is set of if testing.B.ReportAllocs() is called
	GetMBPerS() (float64, error)           // measured if testing.B.SetBytes() is called
}

BenchOutputs are the outputs of a single benchmark run.

Since not all output values are measured on each benchmark run, the getter for these values will return ErrNotMeasured if this is the case.

type BenchRes 

type BenchRes struct {
	Inputs  BenchInputs  // the input variables
	Outputs BenchOutputs // the output result
}

BenchRes represents a result from a single benchmark run. This corresponds to one line from the testing.B output.

type BenchResults added in v0.4.0 

type BenchResults []BenchRes

BenchResults represents a list of benchmark results

func (BenchResults) Filter added in v0.4.0 

func (b BenchResults) Filter(filterExpr string) (BenchResults, error)

Filter returns a subset of the BenchResults matching the provided filter expr. For example filtering by the expression 'var1<=2' will return the results where the input variable named 'var1' has a value less than or equal to 2.

Example 
r := strings.NewReader(`
			BenchmarkMath/areaUnder/y=sin(x)/delta=0.001000/start_x=-2/end_x=1/abs_val=true-4         	   21801	     55357 ns/op	       0 B/op	       0 allocs/op
			BenchmarkMath/areaUnder/y=2x+3/delta=1.000000/start_x=-1/end_x=2/abs_val=false-4          	88335925	        13.3 ns/op	       0 B/op	       0 allocs/op
			BenchmarkMath/max/y=2x+3/delta=0.001000/start_x=-2/end_x=1-4                              	   56282	     20361 ns/op	       0 B/op	       0 allocs/op
			BenchmarkMath/max/y=sin(x)/delta=1.000000/start_x=-1/end_x=2-4                            	16381138	        62.7 ns/op	       0 B/op	       0 allocs/op
			`)
benches, err := ParseBenchmarks(r)
if err != nil {
	log.Fatal(err)
}

filtered, err := benches[0].Results.Filter("y==sin(x)")
if err != nil {
	log.Fatal(err)
}

for _, res := range filtered {
	nsPerOp, err := res.Outputs.GetNsPerOp()
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("ns per op = %v\n", nsPerOp)
}

Output:

ns per op = 55357
ns per op = 62.7

func (BenchResults) Group added in v0.4.0 

func (b BenchResults) Group(groupBy []string) GroupedResults

Group groups a benchmarks results by a specified set of input variable names. For example a Benchmark with Results corresponding to the cases [/foo=1/bar=baz /foo=2/bar=baz /foo=1/bar=qux /foo=2/bar=qux] grouped by ['foo'] would have 2 groups of results (those with Inputs where

Example 
r := strings.NewReader(`
			BenchmarkMath/areaUnder/y=sin(x)/delta=0.001000/start_x=-2/end_x=1/abs_val=true-4         	   21801	     55357 ns/op	       0 B/op	       0 allocs/op
			BenchmarkMath/areaUnder/y=2x+3/delta=1.000000/start_x=-1/end_x=2/abs_val=false-4          	88335925	        13.3 ns/op	       0 B/op	       0 allocs/op
			BenchmarkMath/max/y=2x+3/delta=0.001000/start_x=-2/end_x=1-4                              	   56282	     20361 ns/op	       0 B/op	       0 allocs/op
			BenchmarkMath/max/y=sin(x)/delta=1.000000/start_x=-1/end_x=2-4                            	16381138	        62.7 ns/op	       0 B/op	       0 allocs/op
			`)
benches, err := ParseBenchmarks(r)
if err != nil {
	log.Fatal(err)
}

groupedResults := benches[0].Results.Group([]string{"y"})

// sort by key names to ensure consistent iteration order
groupNames := make([]string, len(groupedResults))
i := 0
for k := range groupedResults {
	groupNames[i] = k
	i++
}
sort.Strings(groupNames)

for _, k := range groupNames {
	fmt.Println(k)
	v := groupedResults[k]

	times := make([]float64, len(v))
	for i, res := range v {
		nsPerOp, err := res.Outputs.GetNsPerOp()
		if err != nil {
			log.Fatal(err)
		}
		times[i] = nsPerOp
	}
	fmt.Printf("ns per op = %v\n", times)
}

Output:

y=2x+3
ns per op = [13.3 20361]
y=sin(x)
ns per op = [55357 62.7]

type BenchSub 

type BenchSub struct {
	Name string
	// contains filtered or unexported fields
}

BenchSub represents an input to the benchmark represented by a sub-benchmark with a name NOT of the form 'var_name=var_value'.

func (BenchSub) String 

func (b BenchSub) String() string

type BenchVarValue 

type BenchVarValue struct {
	Name  string
	Value interface{}
	// contains filtered or unexported fields
}

BenchVarValue represents an input to the benchmark represented by a sub-benchmark with a name of the form 'var_name=var_value'.

func (BenchVarValue) String 

func (b BenchVarValue) String() string

String returns the string representation of the BenchVarValue with the form 'var_name=var_value'.

The string representation of a BenchVarValue may vary slightly from the original input due to things like floating point precision and alternate string representations of various types.

Currently the '%f' verb is used for floating point values in order to guarantee that they can be distinguished from integer values. For everything else the default '%v' verb is used for simplicities sake.

type Benchmark 

type Benchmark struct {
	Name    string
	Results BenchResults
}

Benchmark represents a single top-level benchmark and it's results.

func ParseBenchmarks 

func ParseBenchmarks(r io.Reader) ([]Benchmark, error)

ParseBenchmarks extracts a list of Benchmarks from testing.B output.

Example 
r := strings.NewReader(`
			BenchmarkMath/areaUnder/y=sin(x)/delta=0.001000/start_x=-2/end_x=1/abs_val=true-4         	   21801	     55357 ns/op	       0 B/op	       0 allocs/op
			BenchmarkMath/areaUnder/y=2x+3/delta=1.000000/start_x=-1/end_x=2/abs_val=false-4          	88335925	        13.3 ns/op	       0 B/op	       0 allocs/op
			BenchmarkMath/max/y=2x+3/delta=0.001000/start_x=-2/end_x=1-4                              	   56282	     20361 ns/op	       0 B/op	       0 allocs/op
			BenchmarkMath/max/y=sin(x)/delta=1.000000/start_x=-1/end_x=2-4                            	16381138	        62.7 ns/op	       0 B/op	       0 allocs/op
			`)
benches, err := ParseBenchmarks(r)
if err != nil {
	log.Fatal(err)
}

for _, bench := range benches {
	fmt.Printf("bench name: %s\n", bench.Name)
	for _, res := range bench.Results {
		var (
			varValues = make([]string, len(res.Inputs.VarValues))
			otherSubs = make([]string, len(res.Inputs.Subs))
		)

		for i, varVal := range res.Inputs.VarValues {
			varValues[i] = varVal.String()
		}
		for i, sub := range res.Inputs.Subs {
			otherSubs[i] = sub.String()
		}

		fmt.Printf("var values = %q\n", varValues)
		fmt.Printf("other subs = %q\n", otherSubs)
		nsPerOp, err := res.Outputs.GetNsPerOp()
		if err != nil {
			log.Fatal(err)
		}
		fmt.Printf("ns per op = %.2f\n", nsPerOp)
	}
}

Output:

bench name: BenchmarkMath
var values = ["y=sin(x)" "delta=0.001000" "start_x=-2" "end_x=1" "abs_val=true"]
other subs = ["areaUnder"]
ns per op = 55357.00
var values = ["y=2x+3" "delta=1.000000" "start_x=-1" "end_x=2" "abs_val=false"]
other subs = ["areaUnder"]
ns per op = 13.30
var values = ["y=2x+3" "delta=0.001000" "start_x=-2" "end_x=1"]
other subs = ["max"]
ns per op = 20361.00
var values = ["y=sin(x)" "delta=1.000000" "start_x=-1" "end_x=2"]
other subs = ["max"]
ns per op = 62.70

func (Benchmark) String 

func (b Benchmark) String() string

String returns the string representation of the benchmark. This follows the same format as the testing.B output.

type Comparison added in v0.4.0 

type Comparison string

Comparison represents a comparison operation. 

const (
	Eq Comparison = "=="
	Ne Comparison = "!="
	Lt Comparison = "<"
	Gt Comparison = ">"
	Le Comparison = "<="
	Ge Comparison = ">="
)

The available comparison operations.

type GroupedResults 

type GroupedResults map[string]BenchResults

GroupedResults represents a grouping of benchmark results.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.