Documentation
¶
Overview ¶
Package testscaling measures how the cost of a hot path grows with the size of the input that grows in a session (history length, message size, transcript lines, keys) and fails a test when the growth exceeds the declared bound.
A gate runs the path at a size N and at 10N and compares the costs. The slope log10(cost(10N)/cost(N)) is about 0 for a constant path, 1 for a linear path and 2 for a quadratic path. Allocation count and allocated bytes are the gated costs because they do not depend on the load other processes put on the host; wall time is measured and reported only. Hardware instruction counters are not used: perf_event_open is refused on the shared hosts (kernel.perf_event_paranoid=4), and valgrind instruction counts belong to the Durable counts gate.
A path that allocates nothing at either size has slopes of 0 and passes any bound: its CPU growth is reported in the wall-time slope but not gated, so a gate on such a path proves only that the path stays allocation-free.
Only test code imports this package.
Index ¶
Constants ¶
const ReportEnv = "PIG_SCALING_REPORT"
ReportEnv names a file each measured case appends one JSON line to, for building the complexity inventory.
const Tolerance = 0.35
Tolerance is the slope a measurement may exceed its bound by. Over a tenfold size step it admits a factor of about 2.2 beyond the bound, which covers an n log n path declared linear and slice growth rounding, and it still fails a path that is one whole order worse than declared (a factor of 10).
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Case ¶
type Case struct {
// Name identifies the path in failures and in the report.
Name string
// Size is N, the smaller of the two measured sizes; the larger is 10N.
Size int
// Bound is the declared growth of allocations and allocated bytes.
Bound Order
// BytesBound, when not zero, is the declared growth of allocated bytes where it differs from Bound: a path whose
// allocation count is linear but that copies a slice of one 16-byte header per line on every frame allocates bytes
// that grow a little faster than its size. Zero means Bound.
BytesBound Order
// Prepare builds the state for size n outside the measurement and returns
// the operation to measure. Each call must return a fresh operation, since
// an operation may consume its state.
Prepare func(n int) func()
// Runs is the number of fresh operations measured at each size; the least
// cost of the runs is kept, so allocations of goroutines other tests left
// running only add to a run and do not count. Zero means 3.
Runs int
}
Case is one gated hot path.
type Order ¶
type Order float64
Order is the exponent of a cost bound in the growing size: 0 is constant, 1 is linear and 2 is quadratic. A logarithmic factor is within the tolerance of the next lower whole exponent.
type Result ¶
type Result struct {
Name string `json:"name"`
Bound string `json:"bound"`
BytesBound string `json:"bytesBound,omitempty"`
N int `json:"n"`
Small Cost `json:"small"`
Large Cost `json:"large"`
AllocsSlope float64 `json:"allocsSlope"`
BytesSlope float64 `json:"bytesSlope"`
TimeSlope float64 `json:"timeSlope"`
}
Result is the measurement of one case at N and 10N.
func Check ¶
Check measures c and fails t when the allocation or byte slope exceeds the declared bound by more than Tolerance.
func (Result) Violations ¶
Violations describes each gated cost whose slope exceeds bound by more than Tolerance.
func (Result) ViolationsWith ¶
ViolationsWith is Violations with a separate bound for allocated bytes.