Documentation
¶
Overview ¶
Package report reads a finished load-test run and summarises it.
It is glue over github.com/galax-io/parsec plus one pass: it finds the run, opens its log whatever version wrote it, and walks the stream once, counting what the log holds into a Summary. It computes the statistics of the run itself, because parsec yields primitives and exports no statistic; it writes no data format, and it declares no type for anything parsec already defines — the position of a sample, the bounds of a run, the outcome recorded, an optional value, and what a source can never record are all parsec's own.
The stats.json writer of galaxio-cli#52 folds over this same pass.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrNoPath error = noPath{}
ErrNoPath refuses an argument that was given but is empty. The zero value of every configuration field and every unset shell variable is the empty string, and guessing a results root for one would answer confidently about a run nobody asked for.
It wraps parsec's refusal of the same thing rather than restating it, so that errors.Is against run.ErrNoPath holds for what this package returns. The wording is this command's, because it can say the one thing a library cannot: that omitting the argument is how the default root is asked for.
var ErrSpent = errors.New("the run has already been read: a reader yields its items once")
ErrSpent refuses a second walk over a run that has already been read.
Functions ¶
func Locate ¶
Locate finds the run to read. An empty path means the results root Maven and sbt write to, resolved against the working directory; anything else is handed to parsec unchanged: a run directory or its simulation.log is the run, and any other directory is a results root searched by lastRun.txt, then by the most recently modified run.
A path the caller named that does not exist, or that exists and is not a directory, is reported as what it is, never as an absence of runs: those are different things and the wrong one sends a caller to debug a place rather than a spelling. The default root is not treated that way: the caller never typed it, so "no run under target/gatling" is the honest answer for a project that has not run a load test.
Types ¶
type Bands ¶ added in v0.14.0
Bands are the two boundaries of the response-time bands, in whole milliseconds. A successful response falls under Lower, from Lower to under Upper, or at or above Upper; a failed one counts as failed whatever its time.
type ExportBand ¶ added in v0.15.0
ExportBand carries an exact population count and its binary64 percentage.
type ExportColumn ¶ added in v0.15.0
type ExportColumn struct {
Count int
Min int64
Max int64
Mean int64
StdDev int64
Percentiles [4]int64
Rate float64
}
ExportColumn uses Count == 0 to distinguish legacy empty-column placeholders from populated measurements. Empty columns contain only numeric zeros.
type ExportStats ¶ added in v0.15.0
type ExportStats struct {
Columns [3]ExportColumn
Bands [4]ExportBand
Bounds Bands
}
ExportStats is the validated numeric view used by both legacy products. Columns are total, OK, KO; Bands are under, between, over, failed. It is not an intermediate serialized format.
type Figures ¶ added in v0.14.0
type Figures struct {
// contains filtered or unexported fields
}
Figures are the response-time figures of one outcome of a run — its successful requests or its failed ones — accumulated in one pass without keeping a single request.
Every figure here is exact but the percentiles, which a t-digest estimates (see Percentile). Durations are whole milliseconds; the sum is kept in 128 bits and the sum of squares in 192, and the mean and the deviation are divided and rooted in integers only when they are read. A duration is at most 2⁶³−1 ns, under 2⁴⁴ ms, and a count is at most 2⁶³, so the sum stays under 2¹⁰⁷ and the sum of squares under 2¹⁵¹: neither can wrap, whatever a log holds.
A figure that does not exist — any timing figure of an outcome no request with a recorded end reached — is reported as absent, never as zero, because zero is a real response time.
func (Figures) Count ¶ added in v0.14.0
Count returns how many requests of this outcome the run holds, with or without a recorded end.
func (Figures) Max ¶ added in v0.14.0
Max returns the slowest recorded response time in milliseconds, and false when no request of this outcome has a recorded end.
func (Figures) Mean ¶ added in v0.14.0
Mean returns the arithmetic mean of the recorded response times, rounded half up to a whole millisecond as Gatling rounds it — 0.5 ms is 1 — and false when no request of this outcome has a recorded end.
func (Figures) Min ¶ added in v0.14.0
Min returns the fastest recorded response time in milliseconds, and false when no request of this outcome has a recorded end.
func (Figures) Percentile ¶ added in v0.14.0
Percentile returns the response time at rank, a percentile rank above 0 and at most 100, rounded half up to a whole millisecond, and false when no request of this outcome has a recorded end or rank is not a percentile rank.
Unlike every other figure, a percentile is an estimate: it is read from a t-digest, github.com/caio/go-tdigest at its defaults. It is the number Gatling 3.11 and 3.12 print for the same log, whose digest — AVLTreeDigest(100) of t-digest 3.1, read as Math.round(quantile) — groups and interpolates the same way; the tests hold every percentile of every recorded run to that digest (constitution Principle II). The quantile interpolates between neighbouring recorded values, so where response times have a gap it can be a value no request had — 1427 ms for the 95th percentile of the recorded 3.13.1 run, whose request at that rank took 1502 ms, and 1427 is what Gatling 3.11's digest gives for it too. caio/go-tdigest#42 proposes a read by rank that returns the recorded value, which would part from Gatling 3.11's numbers. The same requests in the same order always give the same percentiles.
How far it may differ is a rule the tests hold every percentile to. While an outcome holds at most 200 requests, the digest has merged nothing and the percentile is the interpolation between the two recorded response times around the position rank/100·(n−1): it lies between them. At any size it misplaces its rank among the recorded requests by at most 4·q·(1−q)/100 of them plus one request, q being rank/100.
func (Figures) StdDev ¶ added in v0.14.0
StdDev returns the population standard deviation of the recorded response times, taken about their unrounded mean and rounded half up to a whole millisecond, and false when no request of this outcome has a recorded end. These are the definitions Gatling's recorded figures reproduce: a sample deviation, or one about the rounded mean, does not.
type Node ¶ added in v0.15.0
type Node struct {
Position model.Position
Parent model.Position
// contains filtered or unexported fields
}
Node summarizes only its own requests or group traversals, never descendants. The zero Parent denotes the root. Position owns all retained name storage.
type Options ¶ added in v0.14.0
type Options struct {
// Percentiles are the ranks to report, each above 0 and at most 100. The
// summary keeps each once, in increasing order, whatever order they came in.
Percentiles []float64
// Bands are the boundaries of the response-time bands.
Bands Bands
}
Options is what a run is summarised at: the percentile ranks to report and the two boundaries of the response-time bands. A caller chooses them before the walk, and the walk never changes them.
func DefaultOptions ¶ added in v0.14.0
func DefaultOptions() Options
DefaultOptions returns Gatling's own settings, unchanged from 3.11.5 to 3.15.1: the 50th, 75th, 95th and 99th percentile, and bands at 800 ms and 1200 ms. A run whose author changed them in gatling.conf records that nowhere in its log, so these are what a summary is compared against by default.
func (Options) Normalize ¶ added in v0.14.0
Normalize returns o with its ranks sorted and each kept once, or an error naming the rank or the boundaries no summary can be computed at. The ranks are copied, so the caller's slice is never reordered under it.
A caller that reads a run normalizes its options before it opens one: the refusal is the caller's fault and not the log's, and a command turns it into a usage error. Scan normalizes again, so a caller that does not is still refused rather than answered with a summary at options nobody chose.
type Source ¶
type Source struct {
Format gatling.Format
Reader simlog.RunReader
// contains filtered or unexported fields
}
Source is an open run: the log format it holds and the reader that yields its items. Close it when the read is over.
func Open ¶
Open opens the run at loc. The format is identified from the log's leading bytes, never from its name, and parsec applies its own version gate; a refusal comes back naming the log.
The accepted version range is parsec's own. Nothing here narrows or widens it, so the error for an unsupported version quotes the range in force.
func (*Source) BytesRead ¶ added in v0.14.0
BytesRead returns how many bytes of the log the reader has taken so far. The reader buffers, so it runs ahead of the items walked by at most its buffer. A source no file was opened for has read nothing, as it has no size.
func (*Source) Close ¶
Close releases the log file, and spends the source: the reader over a closed file answers from its latched end rather than failing, so a walk afterwards would look like a run that recorded nothing.
func (*Source) Scan ¶
Scan walks the run once and summarises it at opts, and refuses to walk it again. Options no summary can be computed at are refused before the walk is spent, so that a caller can correct them and try again.
A reader yields its items once: both codecs latch their end and answer every later call with it, so a second pass returns a summary of nothing and no error — which is exactly what a run that recorded nothing returns, and no caller could tell the two apart. A closed source is the same: the latched end means the file is never touched, so reading one succeeds too.
This is why the summary of galaxio-cli#51 and the stats.json writer of galaxio-cli#52 fold over one pass rather than one each: the first caller to fold twice would have shipped a report of zero requests for a full run and exited 0.
type Summary ¶ added in v0.14.0
type Summary struct {
// Tally counts what the log holds, as milestone v0.13.0 counted it.
Tally Tally
// Options is what the run was summarised at, its ranks sorted and each
// kept once.
Options Options
// OK and Failed are the figures of the requests the source recorded as
// successful and as failed.
OK Figures
Failed Figures
// Under, Between and Over are the response-time bands: how many successful
// responses took less than the lower boundary, from the lower boundary to
// less than the upper one, and the upper boundary or more. Failed requests
// are a band of their own, Failed.Count(), whatever their time, and a
// successful request with no recorded end is in no band, so the three add
// up to the successful requests that have one.
Under, Between, Over int
// contains filtered or unexported fields
}
Summary is what one pass over a run produced: the tally of what the log holds, the options it was summarised at, and the figures of the run as a whole.
A request counts toward the figures of the outcome the source recorded for it, and toward nothing else: a failure never reaches a figure of successful requests, a request whose outcome the source lost reaches no figure at all, and a group traversal is not a request.
func Scan ¶
func Scan(ctx context.Context, rd simlog.RunReader, opts Options, tick func(Summary)) (Summary, error)
Scan walks the run rd yields once and returns its summary at opts: what the log holds, counted, with the run's bounds extended as it goes, and the figures of its successful and failed requests. Nothing is retained, so memory does not grow with the log. Options no summary can be computed at are refused before anything is read.
A log cut short — a run killed mid-flight — returns the summary of everything it did hold together with an error, because a partial run must not be mistaken for a complete one. One shape cannot be detected: a run killed exactly on a record boundary leaves a log neither format carries an end marker for, so parsec ends it with io.EOF and this reports it as complete. The format cannot express the difference and neither can this. Any other read failure returns the summary walked so far and an error saying the run was not read completely. A cancelled ctx returns its error.
tick, when not nil, is called with the summary so far every cancelCheckInterval items, in the goroutine that walks, before the context is looked at: a progress display reads the walk there and costs it nothing between calls. The summary it is given shares the walk's digests, so it must be read inside the call and never kept.
rd must not have been walked already. A reader yields its items once, so a second walk returns an empty summary and no error, which no caller can tell from a run that recorded nothing. Source.Scan holds that precondition for the open runs this package hands out; a caller holding a bare reader owns it.
func (Summary) All ¶ added in v0.14.0
All returns the figures of every request of the run, successful and failed together: the exact figures of the two outcomes combined, and the percentiles of the digest the walk fed with both.
func (Summary) Export ¶ added in v0.15.0
func (s Summary) Export() (ExportStats, error)
Export returns a complete legacy numeric view or refuses a missing populated measurement. It reuses the exact moments and existing deterministic sketches, and does not validate group metrics which global statistics do not represent.
func (Summary) Rate ¶ added in v0.14.0
Rate returns count requests per second of the run: count divided by the run's span in whole seconds, rounded up, which is the one divisor Gatling uses for every rate of a run. It returns false when the run cannot be timed or spans no time, so that a rate is never shown as infinite and a span is never replaced by a second nobody measured. A count of zero over a known span is a rate of 0.
func (Summary) Share ¶ added in v0.14.0
Share returns count as a percentage of all requests of the run, and false when the run holds none. It divides before it multiplies, as Gatling does: 12 of 36 is 33.33333333333333, where 100·12/36 would be 33.333333333333336.
type Tally ¶
type Tally struct {
// Requests is every sample walked; Successes, Failures and Unknown sum to
// it. Unknown counts a sample whose outcome the source lost, which is
// neither a success nor a failure and must not be silently made either.
Requests int
Successes int
Failures int
Unknown int
// Groups counts group traversals and Users virtual-user events, not
// distinct groups or virtual users: a run emits one traversal per entry
// and a start and an end per user.
Groups int
Users int
Errors int
// Assertions counts declared-assertion payloads a source yielded among its
// events rather than ahead of them. Gatling writes its own before the
// events, where they land on the run description instead, so this is zero
// for a Gatling log — but a payload that does arrive here must be counted
// rather than walked past in silence.
Assertions int
// Other counts items of a kind this package does not know, which a later
// parsec release could introduce. Nothing can be said about them except
// that they were there, and saying nothing at all would under-report the
// log.
Other int
// Bounds is where the run begins and ends, as parsec defines it. It may
// report nothing when an item could not be placed in time.
Bounds model.Bounds
}
Tally is what one pass over a run counted, beside the bounds parsec derives from the same pass.
These are counts of records walked, not statistics: nothing here is a mean, a percentile, a range or a rate. Successes and failures are kept apart from the outcome the source recorded so that the figures of a Summary inherit a split that was never conflated.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package legacy renders Gatling's former stats.json products.
|
Package legacy renders Gatling's former stats.json products. |
|
Package reporttest is what the tests and benchmarks of internal/report share: synthetic Gatling logs, the comparison of a summary with what Gatling recorded, and the service the live Gatling runs load.
|
Package reporttest is what the tests and benchmarks of internal/report share: synthetic Gatling logs, the comparison of a summary with what Gatling recorded, and the service the live Gatling runs load. |