Documentation
¶
Overview ¶
Package survey counts what a population of PDFs is actually made of.
It exists because a decision was nearly taken on a number that measured the corpus rather than the world: JBIG2 appears in six of sixteen hundred government forms, which says only that government forms are not scanned books. What a survey has to answer is per population, and it has to distinguish a filter that appears from a filter whose absence leaves a page blank — those are very different amounts of harm.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Counts ¶
type Counts struct {
// Documents is how many opened. Files is how many were looked at, so the
// two together say how many refused to open at all.
Files, Documents int
// Pages is the total across the documents that opened.
Pages int
// UsedBy is, per image filter, how many DOCUMENTS carry at least one image
// in it. This is the number a prevalence is quoted from.
UsedBy map[string]int
// Images is, per image filter, how many images there are. A single
// document can hold hundreds, so this is not a prevalence.
Images map[string]int
// BlankWithout is, per image filter, how many PAGES have nothing on them
// but images in that filter — the pages that come out blank when it is not
// decoded. This is the harm, as against the presence.
BlankWithout map[string]int
// MaskedBy is, per image filter, how many images are shaped by a MASK in
// that filter — and it is here because its absence cost a release.
//
// A modern scanned page is not one image. It is a low-resolution colour
// background with a high-resolution bitonal ink layer over it, and the ink
// layer is a dark rectangle whose shape comes entirely from a JBIG2
// stencil in /Mask. Counting filters by what they encode as CONTENT said
// JBIG2 was in twenty documents of three thousand and blanked three pages,
// so it was set aside — while in truth it shapes the text of every page of
// every book those scanners produce, and without it that dark rectangle is
// painted over the whole page.
//
// A filter can matter enormously while appearing almost nowhere in the
// count that was being kept.
MaskedBy map[string]int
// Refused is, per reason, how many documents would not open. A population's
// refusals are part of what it is, and their REASONS decide whether they
// are ours to fix: eleven per cent of scanned books refuse because they
// carry Adobe's EBX lending DRM, which is not a defect and not something to
// implement.
Refused map[string]int
// Profiles is, per SHAPE of embedded ICC profile, how many DOCUMENTS
// carry at least one -- where the shape is what gfx/color makes of the
// profile rather than which tag it happens to use, because that is the
// question a reader actually has: can this be converted by arithmetic,
// or does it need a table.
//
// It follows the spaces that name ANOTHER space. The picture that made
// this necessary is a `Separation` whose alternate is a CMYK press
// profile; a census of the spaces a page names DIRECTLY would not have
// seen it, and the profile it misses was for two versions the largest
// mean squared error in the corpus.
Profiles map[string]int
}
Counts is what one population is made of.
func Survey ¶
Survey reads every document it is given and adds up what they are made of. A document that will not open is counted in Files and not in Documents: a population's refusals are part of what it is.