README
¶
extract
Reads a PDF page back: the text on it, with where each piece of it sits, and the images it places.
text, err := extract.Text(doc, 1) // the page as words, lines put back together
runs, err := extract.Runs(doc, 1) // every piece, with its place and its size
imgs, err := extract.Images(doc, 1) // every picture, and where it lands
Pure Go, no C. It builds for GOOS=js/wasm and every architecture the fleet
targets.
What a page actually holds
Not text. A page holds instructions for drawing glyphs, and the text is what
those glyphs were meant to say — which the document says only if it was asked
to. Where it does not, this says so rather than guessing: a run whose codes
could not be worked out comes back marked Unreadable, so a caller can tell
this page says nothing from this page could not be read.
Three things are tried, in order:
- the font's
/ToUnicodemap, which is the document's own word on what its text says; - the name its encoding gives the code — but only a name the document chose itself, since a symbolic font read through an assumed encoding gives the wrong letter with nothing to say so;
- what the embedded font program calls the code, read with
go-opentype— which is the honest answer for a symbolic font, and the reason a page of mathematics comes back as mathematics.
Text drawn in the mode that puts no ink on the page — how a scanner writes
what it read underneath the picture it read it from — is returned and marked
Invisible.
Images
A picture comes back as the file holds it, with the box it covers on the page.
A JPEG comes out as a JPEG: re-encoding it would lose something for nothing.
Turning the other kinds into pixels means reading their colour space, which is
render's work rather than this
package's.
How it is checked
100% of statements, including every branch that handles a page saying something it should not.
Then on the corpus — 118 833 real PDFs, 121 946 pages:
pages read 121946
with text 94865 (77.8%)
saying nothing 27081, of which 4812 did draw glyphs
characters 33976414 (17272226 of them letters)
runs 10797045
unreadable 203607 (1.89%)
invisible 29917
images 300685
panics 0
The 22% of pages that say nothing are figures — this corpus is arXiv's, and
most of it is plots. Of the pages that did draw glyphs and still said
nothing, nearly all are Type 3 fonts with no /ToUnicode: dvips bitmap fonts
whose glyphs are called a1 and s32, which no reader can turn back into
letters.
Licence
BSD-3-Clause.
Documentation
¶
Overview ¶
Package extract reads a PDF page back: the text on it, with where each piece of it sits, and the images it places.
A page does not hold text. It holds instructions for drawing glyphs, and the text is what those glyphs were meant to say — which the document says only if it was asked to. Where it does not, this says so rather than guessing: a run whose characters could not be worked out comes back marked, so that a caller can tell "this page says nothing" from "this page could not be read".
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Image ¶
type Image struct {
// Name is what the page's resources call it, or empty for one written
// into the content stream.
Name string
// Width and Height are the picture's own, in samples.
Width, Height int
// X and Y are where its bottom left corner lands on the page, and
// DrawnWidth and DrawnHeight how large it is drawn, in points. A picture
// turned or skewed by the page still reports the box it covers.
X, Y float64
DrawnWidth, DrawnHeight float64
// Filter is the last filter still on the data: DCTDecode for a JPEG that
// comes out as one, JPXDecode for a JPEG 2000, and empty for data this
// has unfiltered into plain samples.
Filter reader.Name
// Data is the picture's bytes, with every filter this understands undone.
Data []byte
// Dict is the image's own dictionary, which says how to read the data:
// its colour space, how many bits a sample takes, and whether it is a
// mask.
Dict reader.Dict
// Inline reports a picture written into the content stream rather than
// held as a resource of its own.
Inline bool
}
An Image is a picture the page places, and where it puts it.
What comes back is the picture as the file holds it — the bytes and the dictionary describing them — rather than pixels. A JPEG comes out as a JPEG, which is what anyone pulling pictures out of a document wants: re-encoding it would lose something for nothing. Turning the other kinds into pixels means reading their colour space, which is the renderer's work rather than this package's.
type Run ¶
type Run struct {
// Text is what the run says.
Text string
// X and Y are where it starts, in the page's own coordinates — up from
// the bottom left corner, in points.
X, Y float64
// Width is how far the pen moved drawing it, and Size how tall the text
// is, both in points.
Width, Size float64
// Font is the name the page's resources give the font.
Font string
// Space is how wide a space would be here, which is what says whether a
// gap between two runs is a word break or only kerning.
Space float64
// Invisible reports text drawn in the mode that puts no ink on the page.
// It is how a scanner puts what it read underneath the picture it read it
// from, so it is text worth having — and worth knowing about.
Invisible bool
// Unreadable reports a run some of whose codes said nothing about which
// characters they were: the font carried no map and named its glyphs
// after numbers. What Text holds is then only the part that could be read.
Unreadable bool
}
A Run is a piece of text drawn in one go: one font, one size, one place.
Source Files