compare

type Options struct {
	// DPI both renderers are asked for.
	DPI float64
	// MaxDuration bounds our own renderer. The judge is bounded by
	// poppler.Timeout, which is a repository-wide setting rather than an
	// option because it is not a property of the comparison: it is the point
	// at which the other implementation is declared to have hung.
	MaxDuration time.Duration
	// Pages is how many of each document to judge; 0 means the first only.
	Pages int
	// Super is the factor both pages are drawn OVERSIZE by and then reduced
	// by before anything is compared. 0 and 1 both mean "draw at DPI and
	// compare what comes out".
	//
	// It exists because most of what two rasterisers disagree about is not
	// the page, it is the SAMPLING of the page: every glyph edge and every
	// curve falls between pixels, and each side rounds it its own way.
	// Drawing bigger and averaging back down makes both converge on the same
	// coverage, and what survives is content.
	//
	// Measured on one text-heavy form, pixels identical BIT FOR BIT between
	// the two renderers:
	//
	//	1x   85.8%   mean |diff| 9.37
	//	2x   90.4%   mean |diff| 5.07
	//	4x   94.0%   mean |diff| 2.69
	//	8x   97.1%   mean |diff| 1.90
	//
	// The cost is the square of it: 4x draws sixteen times the pixels. What
	// it buys is a comparison sharp enough to be read strictly, which at 1x
	// it is not -- a seventh of every page differs there, and none of that
	// seventh is a defect.
	Super int
}

type Result struct {
	Path string
	Page int
	// Share is the fraction of pixels that differ materially, or -1 when the
	// two could not be compared at all.
	Share float64
	// Note says why, when they could not be.
	Note string
	// Ours and Theirs are how long each renderer took, which is the other
	// thing worth knowing: a page drawn in no time may be a page drawn blank.
	Ours, Theirs time.Duration
	// Identical is the fraction of pixels that are the same BYTE FOR BYTE,
	// which is the strictest reading there is, and Mean is the mean absolute
	// per-channel difference in levels of 255. Both are -1 when the pages
	// could not be compared.
	//
	// At [Options.Super] of 1 these say little: two rasterisers disagree
	// about the sampling of every edge, so a seventh of an ordinary page is
	// not identical and none of that seventh is a defect. Drawn oversize and
	// reduced they become sharp -- 97.1% identical at 8x on the page those
	// figures came from -- and then what is NOT identical is worth opening.
	Identical, Mean float64
	// Colour is how far apart the two pages are in COLOUR, which Share
	// cannot say. Share reduces both to grey and asks how many pixels are
	// more than a quarter of the range apart, so a page tinted by nineteen
	// levels reads as nought -- and so does a scanned page that lost its
	// whole ink layer, which moves the page average by twelve levels and
	// almost no single pixel by sixty-four. Measured: such a page scored
	// 0.14% on Share.
	//
	// It is the mean per-channel difference inside the WORST square of
	// [tile] pixels, in levels of 255, and -1 when the pages could not be
	// compared.
	//
	// It is a MAGNITUDE and not a criterion. Share stays the pass criterion,
	// because "is this the same page to somebody looking at it" is the
	// question the corpus is judged on; this is the term that moves when a
	// colour space is read differently or a layer goes missing, so that such
	// a change is visible at all rather than invisible by construction.
	Colour float64
	// ColourWorst is the worst single blurred per-channel difference on the
	// page, which says whether Colour is a tint over a whole square or one
	// hard edge inside it.
	ColourWorst float64
	// Tool is the poppler tool that did not finish within poppler.Timeout,
	// empty when none did. A page the judge would not answer about is not a
	// page that disagreed, and only the named ones can be gone and looked at.
	Tool string
}

type Summary struct {
	Compared, NotCompared int
	Median, P90, P99, Max float64
	Under                 map[float64]int
	Notes                 map[string]int
	// IdenticalMean is the mean of [Result.Identical] over the pages
	// compared, and MeanDiff the mean of [Result.Mean].
	IdenticalMean, MeanDiff float64
	// ColourMean is the MEAN over the pages compared of each page's worst
	// square, and ColourMax the largest of them. Both are needed and the
	// difference between them matters: one page of 250 at fifty levels moves
	// the mean by a fifth of a level and is invisible in it, which is how a
	// scanned page drawn without its ink layer hid inside a population that
	// read 14.98 before the fix and 14.83 after.
	//
	// ColourWorst is the worst single blurred pixel any page reached.
	//
	// They are magnitudes beside the criterion, never a criterion.
	ColourMean, ColourMax, ColourWorst float64
	// Slowest is the longest our own renderer took on any one page.
	Slowest time.Duration
	// Over is how many pages took longer than the threshold given.
	Over int
	// Slow names the pages that took longest, worst first.
	//
	// A count of pages over a threshold says a corpus has a problem and gives
	// nobody a way to go and look at it. What is wanted is the document and
	// the page, so this keeps them.
	Slow []Result
	// Worst names the pages that disagree most, worst first.
	//
	// The same argument Slow makes: a count of pages over a threshold says a
	// corpus has a problem and gives nobody a way to go and look at it. This
	// file has reported "9 pages at or over 10%" without saying WHICH nine
	// since it was written, and a disagreement nobody can open is a number
	// rather than a finding.
	Worst []Result
	// Hung names every page the judge would not finish drawing. Unlike Slow
	// it is not capped: a cap is a way of dropping names, and a hang that is
	// not named is indistinguishable from a page that scored badly, which is
	// the whole reason the bound exists.
	Hung []Result

	// Timed is how many pages BOTH renderers were timed on, and it is smaller
	// than Compared: a page the judge would not finish has a duration that is
	// the bound rather than a measurement, and counting it would flatter us by
	// exactly the bound. A page neither could draw has no timing at all.
	Timed int
	// OursTotal and TheirsTotal are the sums over those pages. A total is worth
	// having and is not the headline: it is dominated by the few heaviest pages,
	// so a renderer can win it while losing most of the corpus, or lose it while
	// winning most.
	OursTotal, TheirsTotal time.Duration
	// RatioMedian and RatioP90 are quantiles of ours/theirs PER PAGE. Below one
	// is faster. The median says what a typical page does, which the totals
	// cannot.
	RatioMedian, RatioP90 float64
	// Faster is how many of the Timed pages we drew in less time than the judge.
	Faster int
	// WorstRatio names the pages where ours/theirs is largest, largest first --
	// the same argument Slow and Worst make: "we are slower on 14 pages" is a
	// number, the documents are a finding.
	//
	// Largest is not the same as slower. On a population we win everywhere the
	// worst ratio is still below one, and naming these pages "losses" said
	// otherwise for as long as it took to read one report.
	//
	// It holds a POOL of worstRatioPool rows rather than the few a report prints,
	// because a list selected on the maximum of one sample per page is biased and
	// a caller cannot repair a selection that has already been truncated. Three
	// of the eight pages this project published as its worst were inflated by
	// 1.75x to 2.66x when drawn again, and the inflation was not bad luck: taking
	// the top N by ratio from single samples preferentially picks the pages whose
	// sample came out unluckily high, and leaves out the ones whose sample came
	// out luckily low. A caller that redraws these (cmd/compare -confirm) fixes
	// both halves; one that does not should print the first few and call them
	// candidates.
	WorstRatio []Result
}