Documentation
¶
Overview ¶
Package reporttest builds synthetic Gatling logs for tests and benchmarks.
A recorded log is a few kilobytes; the memory goal is about logs of gigabytes. Replay yields a text log of any size from one recording without writing a file or holding the log in memory, which is what lets a test measure the reader rather than the fixture.
Index ¶
- Constants
- Variables
- func AtRank(sorted []int64, rank float64) int64
- func Compare(observed Observed, recorded Recorded, durations [3][]int64) (differences []string)
- func ComparePercentiles(observed Observed, etalon Etalon, recorded Recorded, reference bool, ...) (differences, notes []string)
- func Durations(rd simlog.RunReader) ([3][]int64, error)
- func EtalonSamples(w io.Writer, rd simlog.RunReader) error
- func GlobalInformation(console string) (string, error)
- func Interpolated(sorted []int64, rank float64) (value float64, lower, upper int64)
- func IsReference(version string) bool
- func Items(run model.Run, handed *int, items ...model.Item) simlog.RunReader
- func RankMisplacement(sorted []int64, value int64, rank float64) float64
- func RankTolerance(rank float64, n int) float64
- func Replay(header, body []byte, n int) io.Reader
- func Samples(rd simlog.RunReader, yield func(outcome model.Outcome, ms int64)) error
- func Split(log []byte) (header, body []byte, err error)
- type Etalon
- type Figures
- type Observed
- type Recorded
- type Value
Constants ¶
const OrderOfEqualCentroids = "the difference caio's order of equal centroids makes"
OrderOfEqualCentroids ends the note that describes the one difference from Gatling 3.11's digest constitution Principle II admits, so that a test of recordings known to need no such description can refuse it.
Variables ¶
var GatlingRanks = [4]float64{50, 75, 95, 99}
GatlingRanks are the percentile ranks Gatling reports by default, which no recorded run changed.
var Versions = []string{"3.11.5", "3.12.0", "3.13.1", "3.14.9", "3.15.1"}
Versions are the Gatling releases the corpus holds a run of, oldest first.
Functions ¶
func AtRank ¶ added in v0.14.0
AtRank returns the request standing at the percentile's rank: the ⌈rank/100·n⌉-th smallest response time.
func Compare ¶ added in v0.14.0
Compare returns, one sentence each, every whole-run figure of observed that differs from what Gatling recorded and every percentile that breaks the rank rule over the run's own response times, durations holding them for all, successful and failed requests. Gatling's own percentiles are not read here: ComparePercentiles holds this tool's to them.
func ComparePercentiles ¶ added in v0.14.0
func ComparePercentiles(observed Observed, etalon Etalon, recorded Recorded, reference bool, durations [3][]int64) (differences, notes []string)
ComparePercentiles holds the percentiles of observed to Gatling 3.11's (constitution Principle II). Each must be a value the etalon's AVL digest gives for the run, or differ from one across a boundary between two runs of equal response times (see acrossEqualRuns), which is described and not held. Where recorded is what Gatling 3.11.x or 3.12.x printed for the run — reference — each printed percentile must equal this tool's, or be another value the digest gives, which its unseeded generator can draw. It returns a sentence for each percentile that does not hold, and notes that describe, against the run's own response times in durations, what a later Gatling printed and what MergingDigest gives where either differs.
func Durations ¶ added in v0.14.0
Durations walks the run rd yields and returns the response times, in whole milliseconds, of all, successful and failed requests, each sorted. These are the exact values a percentile estimate is held to.
func EtalonSamples ¶ added in v0.14.0
EtalonSamples writes what the etalon, testdata/etalon/Etalon.java, reads from the run rd yields: one line for each request whose end the source recorded, in log order — "ok" or "failed", a tab, and its response time in whole milliseconds. That is the order Gatling feeds its digests in.
func GlobalInformation ¶ added in v0.14.0
GlobalInformation returns the last Global Information block Gatling printed on its console, byte for byte: from the rule of = signs that opens it to the one that closes it, with the response-time distribution and the errors between. A live recording keeps that much of the console and nothing around it.
func Interpolated ¶ added in v0.14.0
Interpolated returns the value at rank, a percentile rank above 0 and at most 100, by linear interpolation between the two sorted response times around the position rank/100·(n−1), and those two neighbours. It is how a t-digest that has merged no centroid reads a quantile, computed the same way.
func IsReference ¶ added in v0.14.0
IsReference reports whether a recording of this Gatling version is one whose printed percentiles this tool's must equal: 3.11.x and 3.12.x compute them over t-digest 3.1, and later versions over the miscount of tdunning/t-digest#230 (research §18).
func Items ¶ added in v0.14.0
Items returns a reader over items, as a run reader yields them, for the run shapes no Gatling log produces. Handed will hold how many it has yielded when it is not nil, which a test that reads the walk midway asks for.
func RankMisplacement ¶ added in v0.14.0
RankMisplacement returns how far the quantile rank/100 lies outside the range value occupies among the sorted response times — from the share strictly below it to the share at or below it — and zero when it lies inside.
func RankTolerance ¶ added in v0.14.0
RankTolerance is how far the rule lets an estimate misplace its rank among n requests: 4·q·(1−q)/100 of them, the most a t-digest at compression 100 merges around the quantile q, plus one request for the step between two recorded values.
func Replay ¶
Replay returns a reader that yields header once and then body n times. Reads are served from the recording's bytes directly; nothing is copied ahead of time, so a replay of gigabytes costs the size of one recording.
A negative n yields the header alone, as zero does.
func Samples ¶ added in v0.14.0
Samples walks the run rd yields and calls yield with the outcome and the response time, in whole milliseconds, of every request that counts towards a figure: one the source recorded an outcome and an end for. It is the one reading of what a run holds, so that the etalon, the rank rule and a test's own digest are never fed different sets of requests.
func Split ¶
Split divides a text simulation.log into its header — every line through the RUN record — and its body, the events after it. Both keep their trailing newline, so header followed by body is the log again.
The header is capped to its own length, so appending to it cannot reach into the body it was cut from. A log whose last line is unterminated is refused: its body would end mid-record, and repeating it would splice that record onto the next copy's first one.
Types ¶
type Etalon ¶ added in v0.14.0
Etalon is what the real t-digest libraries give for one run, for all, successful and failed requests and each of GatlingRanks: every value Gatling 3.11's digest — AVLTreeDigest(100) of t-digest 3.1 — gave over the seeds of its generator, ascending, and the value MergingDigest(100) of t-digest 3.3 gave. An outcome no request reached has neither. Samples and SHA256 say what the etalon was run over — how many lines of EtalonSamples it read, and their digest — which ties an etalon.tsv to the log beside it.
func ReadEtalon ¶ added in v0.14.0
ReadEtalon reads the output of the etalon: a comment, the samples it read and their SHA-256, a heading, and one line for each column and rank holding the AVL digest's values, comma-separated, and the MergingDigest value, or "-" for both where the column is empty. It refuses an etalon that does not say what it was run over, and one that gives a column and rank twice.
type Figures ¶ added in v0.14.0
type Figures interface {
Count() int
Min() (int64, bool)
Max() (int64, bool)
Mean() (int64, bool)
StdDev() (int64, bool)
Percentile(rank float64) (int64, bool)
}
Figures is what a comparison reads of one outcome's figures.
type Observed ¶ added in v0.14.0
Observed is what a summary reported about a run, in the shape a comparison with Gatling reads it: all, successful and failed requests in that order.
type Recorded ¶ added in v0.14.0
type Recorded struct {
Count [3]int64
Min, Max, Mean, StdDev [3]Value
Percentiles [4][3]Value
Rate [3]float64
Bands [4]int64
// Rounded says the rates and shares were printed to two decimals, as a
// console prints them, rather than written as the doubles themselves.
Rounded bool
}
Recorded is what Gatling itself printed or wrote about a run as a whole: each figure for all, successful and failed requests, in that order, and the four response-time bands. A figure Gatling left empty is marked absent.
func ParseConsole ¶ added in v0.14.0
ParseConsole reads the last Global Information block and the response-time distribution after it from what Gatling printed on its console. It reads both layouts: the parenthesised one up to 3.13.x, and the table with | separators from 3.14.0.
func ReadGlobalStats ¶ added in v0.14.0
ReadGlobalStats reads the global_stats.json Gatling wrote up to 3.13.x. The file writes 0 for every figure of an outcome no request reached, so such an outcome's timing figures are marked absent.
Source Files