README
¶
reader
A pure-Go, zero-C PDF reader — the parsing half of
go-pdfkit. Where
pdfkit authors new PDF 1.7, this
module takes existing PDF bytes apart: lexer, object model, cross-reference
tables and streams, stream filters, the standard security handler, the page
tree, and content streams. It also writes the same object graph back out,
which is what every operation on an existing file needs.
Nothing outside the Go standard library is required, so it builds for
GOOS=js/wasm and every 64-bit architecture the fleet targets.
Status
Landing in waves. This module currently provides the object layer and the document structure:
- Lexer — the full PDF object syntax: integers and reals in the shapes
producers actually write, literal strings with their escapes, hex strings,
names with
#xxescapes, arrays, dictionaries, streams, and keywords. - Objects —
Null,Bool,Integer,Real,String,Name,Array,Dict,Stream, and indirectRef, with theN G R/N G objlookahead resolved the way the specification requires. - Filters —
FlateDecode,LZWDecode(with PDF'sEarlyChange, which the standard library's LZW does not implement),ASCIIHexDecode,ASCII85Decode,RunLengthDecode,Crypt, plus the PNG and TIFF predictors. Image filters (DCTDecode,JPXDecode,CCITTFaxDecode,JBIG2Decode) are reported rather than applied, so an image consumer gets the encoded bytes. - Filters that fail — a chain that cannot be run to the end is not the end
of the stream.
DecodeRecoveringreturns what the filters before the failure produced — the prefix a damaged Flate stream did inflate, not its compressed bytes — and says so throughDecoded.RecoveredandDecoded.Cause. Bytes no filter decoded arrive inDecoded.Undecoded, never inDecoded.Data, so a compressed stream cannot be painted as samples or tokenised as content by a caller that forgot to check a flag.Decodeis the strict reading and refuses such a stream outright, so a caller that must not act on damaged data says which it wants by which one it calls. Page content takes the lenient reading, because a page whose content will not decode is still a page;PageContentDecodedreports whether any salvaging happened. Cross-reference streams take the strict one: a table that has to be guessed at is worse than no table, and the repair below is the answer to one. - Cross-references — classic tables, cross-reference streams, object
streams,
/Prevchains, and the/XRefStmof a hybrid file, newest definition winning. - Repair — a file whose tables are missing, truncated or simply wrong is
rebuilt by scanning it for object headers, trailers and, failing those, for
a catalogue; a file that kept its pages but lost its catalogue gets one.
This is not an exceptional path: it is what makes a reader usable. The
rebuild establishes the file key before it reads object streams, so a file
whose catalogue lives in an encrypted one can be rebuilt at all; it takes
/Encryptfrom a cross-reference stream's dictionary when the file has no trailer keyword to name it in. Where two object streams claim the same object, the later one in the file wins, the same rule the header scan applies to objects written directly — so the same damaged file reads the same way twice. When a rebuild fails, the error says what the rebuild found and not merely that the tables could not be read. - Documents —
Open, object resolution with cycle and recursion guards, the trailer, the catalogue, and the page tree with the four attributes a page inherits from its ancestors. - Encryption — the standard security handler in every revision, both ways:
reading RC4 at 40 and 128 bits, AESV2, and the AES-256 of
/R 5and/R 6, with crypt filters,/Identityand/EncryptMetadata; and writing AES-256 or AES-128, with permissions, a user password and an owner password. A password is tried as the user password and as the owner password, andOpenuses the empty one, so a file protected only against editing opens with no password at all. and steps over rubbish rather than losing the operations around it, with inline images read whole. Where an inline image ends is the one genuinely ambiguous thing in a content stream, since its data may spell EI itself; the length is computed from the image's own geometry where it can be, and otherwise every candidate EI is tried until the data before one actually decodes. - Writing — the exact inverse of the parser: objects rendered in PDF
syntax with stable dictionary ordering and strings in whichever of the two
forms is shorter, and a
Writerthat copies whole object graphs out of one or more documents, renumbering as it goes, then lays down a cross-reference table and a trailer.NewPackedWriterinstead puts what it can into compressed object streams and ends in a cross-reference stream, which is what makes a file small. Measured against a corpus of 118 863 real PDFs — Matplotlib, cairo, pdfTeX, Ghostscript, Adobe, R, Apache FOP, PDF 1.3 through 1.7 —Opensucceeds on 118 833 of them and finds 138 337 pages, in 8 seconds, with no panics. Of the thirty it refuses, twenty-seven are PNG files with a.pdfextension, one is PostScript, and two are PDFs truncated past recovery.
The one encrypted PDF the corpus holds — /V 4 /R 4, written by macOS
Quartz — decrypts to readable metadata and a valid content stream, which is
the only genuinely independent check of the key derivation there is here;
the other revisions are round-tripped against a test encryptor written from
the same algorithms.
The content-stream tokeniser reads 1 536 769 753 operations across those
138 337 pages in a minute and a half, with no panics, no page failing to
decode, and no operator outside the seventy the format defines — beyond four
arc and six nan written by a producer that was simply wrong.
Rewriting is checked the same way: every one of the 118 833 files that open is copied object by object into a new file, which is then re-read and compared on page count, media boxes and the bytes of every content stream. All 118 833 match, 35.5 GB in and 35.2 GB out, in twenty-eight seconds — and all 118 833 match again when the packed writer is used instead, which brings the output down to 94.6% of the input.
Next wave: the operations built on all of this.
Install
go get github.com/go-pdfkit/reader
Testing
go test -covermode=set ./...
CI gates on exact 100% statement coverage, go vet, and a cross-compile
across linux/{amd64,arm64,riscv64,loong64,ppc64le,s390x}, js/wasm,
darwin/arm64 and windows/amd64.
The rest of the family
reader reads and writes the format ·
ops is the verbs and the pdfops command ·
render turns a page into pixels ·
pdffont is what a document says about a font ·
extract reads a page back ·
coedit is a PDF several people edit at once ·
app is the browser workbench.
Documentation for all of it: https://go-pdfkit.github.io/docs/
License
BSD-3-Clause — see LICENSE. Copyright the go-pdfkit/reader authors.
Documentation
¶
Overview ¶
Package reader takes existing PDF bytes apart: the object syntax, the cross-reference machinery, stream filters, and the document structure built on top of them. It is the parsing counterpart of github.com/go-pdfkit/pdfkit, which writes PDF.
Nothing outside the Go standard library is used, so the package builds for js/wasm and for every architecture the fleet targets.
Index ¶
- Variables
- func AppendObject(dst []byte, o Object) []byte
- func FormatObject(o Object) []byte
- func ImageFilter(n Name) bool
- func ParseIndirectObject(b []byte, resolve Resolver) (Ref, Object, int, error)
- func ToBool(o Object) (bool, bool)
- func ToFloat(o Object) (float64, bool)
- func ToInt(o Object) (int64, bool)
- func ToString(o Object) ([]byte, bool)
- type Array
- type Bool
- type ContentScanner
- type Decoded
- type Dict
- type Document
- func (d *Document) Catalog() (Dict, error)
- func (d *Document) DecodeStream(s *Stream) ([]byte, Name, error)
- func (d *Document) DecodeStreamRecovering(s *Stream) Decoded
- func (d *Document) Encrypted() bool
- func (d *Document) Get(ref Ref) (Object, error)
- func (d *Document) GetDict(from Dict, key Name) (Dict, bool)
- func (d *Document) Page(i int) (Dict, error)
- func (d *Document) PageContent(i int) ([]byte, error)
- func (d *Document) PageContentDecoded(i int) (Decoded, error)
- func (d *Document) PageCount() int
- func (d *Document) PageOperations(i int) ([]Operation, error)
- func (d *Document) PageRef(i int) (Ref, bool)
- func (d *Document) Protection() (Protection, bool)
- func (d *Document) Repaired() bool
- func (d *Document) Resolve(o Object) (Object, error)
- func (d *Document) Resolver() Resolver
- func (d *Document) Trailer() Dict
- func (d *Document) Version() string
- type Encryption
- type InlineImage
- type Integer
- type Kind
- type Name
- type Null
- type Object
- type Operation
- type Permissions
- type Protection
- type Real
- type Ref
- type Resolver
- type Stream
- type String
- type SyntaxError
- type Writer
Constants ¶
This section is empty.
Variables ¶
var ErrUnsupportedEncryption = errors.New("reader: unsupported security handler")
ErrUnsupportedEncryption is returned for a security handler this package does not implement — anything other than the standard one.
var ErrWrongPassword = errors.New("reader: the password does not open this file")
ErrWrongPassword is returned when a file is encrypted and neither the password given nor the empty one opens it.
Functions ¶
func AppendObject ¶
AppendObject writes an object in PDF syntax, appending to dst. Dictionary keys are written in order, so the same object always produces the same bytes — which is what makes a rewritten file comparable with the one it came from.
func FormatObject ¶
FormatObject renders an object in PDF syntax.
func ImageFilter ¶
ImageFilter reports whether a filter yields an encoded image rather than a byte stream. Decode stops at one of these and hands the caller the still encoded bytes, because decoding them is an image decoder's job.
/CCITTFaxDecode was on this list and is not any more. It does not carry an image with its own idea of how many components it has and how deep they are, the way DCT and JPX do: it produces bilevel samples, one bit a pixel, and the stream dictionary says what they mean. That is a byte stream, so it is a filter — and decoding it here means every caller gets it rather than each writing its own. See ccitt.go.
func ParseIndirectObject ¶
ParseIndirectObject parses "N G obj … endobj" from the start of b, returning the reference it defines, the object itself, and how many bytes it consumed. A stream's /Length may be an indirect reference; resolve supplies it, and a nil or unhelpful resolve falls back to scanning for the endstream keyword.
Types ¶
type Array ¶
type Array []Object
Array is a PDF array.
type ContentScanner ¶
type ContentScanner struct {
// contains filtered or unexported fields
}
A ContentScanner walks a content stream one operation at a time. A stream with rubbish in it yields the operations around the rubbish rather than nothing at all, which is what a renderer needs; ContentScanner.Err reports the first thing that did not parse.
func NewContentScanner ¶
func NewContentScanner(data []byte) *ContentScanner
NewContentScanner reads operations from a decoded content stream.
func (*ContentScanner) Err ¶
func (s *ContentScanner) Err() error
Err reports the first malformed token the scan stepped over, if any.
func (*ContentScanner) Next ¶
func (s *ContentScanner) Next() (Operation, bool)
Next returns the next operation, and false once the stream is exhausted.
type Decoded ¶ added in v0.6.0
type Decoded struct {
// Data is what the chain decoded. A recovered decode leaves fewer bytes
// here than the stream meant to carry, but they are bytes of the kind it
// meant to carry: Data never holds bytes that no filter has decoded.
//
// The one case where Data is still encoded is the deliberate one: a chain
// that stopped at an image filter, which Image names and Recovered does
// not flag, because stopping there is the contract.
Data []byte
// Undecoded holds the bytes the chain could not get past, still in the
// encoding Filter names. It is set instead of Data — never beside it —
// when the filter that failed produced nothing at all.
//
// This is where a compressed content stream, a truncated fax, or a filter
// nobody implements ends up. A caller that wants the stream as it lies has
// to ask for it by a name that says what it is; a caller that reads Data
// cannot be handed it by accident. Painting undecoded bytes as a one-bit
// image looks like a page with something on it, which is why nothing about
// this is left to a flag.
Undecoded []byte
// Image names the image filter the chain stopped at, when it stopped at
// one; Data is then the bytes still encoded in it.
Image Name
// Recovered says the chain could not be run to the end. A caller that must
// not act on damaged data stops here — or calls [Decode], which refuses
// outright.
Recovered bool
// Cause says why the chain stopped, and is set exactly when Recovered is.
Cause error
// Filter names the filter that could not be applied, when one is to blame:
// a chain whose /Filter entry itself is unreadable blames nothing.
Filter Name
}
A Decoded is the outcome of applying a stream's filter chain, including the outcome of a chain that could not be finished.
The point of the type is that a caller can tell the three outcomes apart without having to be careful. Bytes a filter decoded and bytes no filter decoded arrive in different fields, so the second kind cannot be painted as samples or tokenised as content by a caller that forgot to check a flag.
func DecodeRecovering ¶ added in v0.6.0
DecodeRecovering applies a stream dictionary's filter chain to raw and never fails. A filter that cannot be applied — corrupt Flate data, a stream that stops in the middle, a filter name nobody implements — ends the chain, and what the filters before it produced is returned with Decoded.Recovered set. That is deliberately what every other reader does: refusing the stream loses a page the file can still show.
The salvage is always as far down the chain as the filters got, never the bytes as they arrived: a damaged Flate stream yields the prefix it did inflate, in Decoded.Data. A filter that produced nothing at all leaves the bytes in Decoded.Undecoded instead, still in its encoding, because that is what they are.
func DecodeStreamRecovering ¶ added in v0.6.0
DecodeStreamRecovering is DecodeRecovering for a parsed stream.
type Dict ¶
Dict is a PDF dictionary.
func ToDict ¶
ToDict reports the dictionary of a dictionary object, or a stream's own dictionary — a stream is a dictionary everywhere a dictionary is expected.
type Document ¶
type Document struct {
// contains filtered or unexported fields
}
A Document is a parsed PDF file: its cross-reference information, its trailer, and lazy access to every object in it.
func Open ¶
Open parses the cross-reference information of a PDF file held in memory, using the empty password. A file whose tables are damaged is rebuilt by scanning it, so Open fails only when there is no usable document structure at all, or when the file is encrypted and the empty password does not open it — OpenWithPassword then does.
func OpenWithPassword ¶
OpenWithPassword is Open with a password to try. The password is tried both as the user password and as the owner password, the empty one as well.
func (*Document) DecodeStream ¶
DecodeStream applies a stream's filter chain, resolving any indirect decode parameters against this document. A chain that cannot be run to the end is an error; Document.DecodeStreamRecovering salvages instead.
func (*Document) DecodeStreamRecovering ¶ added in v0.6.0
DecodeStreamRecovering applies a stream's filter chain, salvaging what it can from a chain that cannot be run to the end and saying that it did so.
func (*Document) Get ¶
Get returns the object an indirect reference names, or Null when the file does not define it.
func (*Document) Page ¶
Page returns the i'th page's dictionary, counting from one, with the attributes it inherits from its ancestors filled in.
func (*Document) PageContent ¶
PageContent returns the decoded content stream of the i'th page, counting from one. A page whose /Contents is an array has its streams joined, as the specification requires, with a newline between them.
A content stream whose filters cannot be applied contributes what could be salvaged from it rather than failing the page, which is what a renderer needs; the error is for a page that is not there at all. Document.PageContentDecoded says whether any salvaging happened.
func (*Document) PageContentDecoded ¶ added in v0.6.0
PageContentDecoded is Document.PageContent with the outcome of the decode attached: Decoded.Recovered reports that at least one of the page's content streams could not be decoded cleanly, and Decoded.Cause says why.
func (*Document) PageOperations ¶
PageOperations tokenises the i'th page's content stream, counting from one.
func (*Document) Protection ¶ added in v0.4.0
func (d *Document) Protection() (Protection, bool)
Protection reports how the file is protected, and false when it is not protected at all. A document that opened has already been decrypted; this says what it said about itself on the way.
func (*Document) Repaired ¶
Repaired reports whether the document's structure was rebuilt by scanning the file rather than read from its cross-reference tables.
func (*Document) Resolver ¶
Resolver returns the function that turns this document's indirect references into objects, for Resolve and Decode.
type Encryption ¶ added in v0.3.0
type Encryption struct {
UserPassword string
OwnerPassword string
Permissions Permissions
// AES128 asks for the older method, which readers before 2008 understand.
// The default is AES-256, which is what a file should use today.
AES128 bool
}
An Encryption says how a file is to be protected.
Two people can open it: whoever knows the user password, subject to the permissions, and whoever knows the owner password, subject to nothing. An empty user password means the file opens without one and the permissions are all it says.
type InlineImage ¶
An InlineImage is the dictionary and the still-encoded bytes of a BI … ID … EI sequence. Its dictionary uses the abbreviated keys inline images are written with; InlineImage.Expanded gives the ordinary spelling.
func (*InlineImage) Expanded ¶
func (im *InlineImage) Expanded() Dict
Expanded returns the image's dictionary with the abbreviated keys and colour space names written out, so it can be read like any image XObject.
A dictionary may say the same thing twice and disagree with itself: /W 20 beside /Width 10, /CS /RGB beside /ColorSpace /3chanRGB. The specification permits both spellings in an inline image and says nothing about which wins, and the public implementations disagree — pdf.js asks for the abbreviation and falls back to the written-out name, MuPDF asks the other way round.
The abbreviation wins here. Two reasons: it is the spelling Table 93 gives for an inline image, the written-out name being the tolerated alias; and safedocs' own fixture for this case marks the abbreviation as the line to remove "to see the effect", which is a statement about what the reference viewer does.
What is not defensible is the answer changing between runs. Expanding in one pass over the map made the winner depend on Go's map iteration order, which is deliberately random: the same file drew differently each time it was opened, with no way for a caller to notice. Two passes settle it — the written-out names first, then the abbreviations over them — and no order within either pass can matter, because the names in each are distinct.
type Integer ¶
type Integer int64
Integer is a PDF integer. Numbers too large for an int64 are parsed as Real, which is what every other reader does.
type Kind ¶
type Kind uint8
Kind names the eight basic PDF object types plus the two the file format adds on top of them: streams and indirect references.
const ( KindNull Kind = iota KindBool KindInteger KindReal KindString KindName KindArray KindDict KindStream KindRef )
The object kinds. Object.Kind reports one of these.
type Name ¶
type Name string
Name is a PDF name with its #xx escapes resolved and the leading slash dropped, so /Type is Name("Type").
func Decode ¶
Decode applies a stream dictionary's filter chain to raw. It returns the decoded bytes and, when the chain ends in an image filter, that filter's name together with the bytes still encoded in it.
Decode is the strict reading: a chain that cannot be run to the end is an error and yields no bytes. DecodeRecovering is the lenient one, and says which of the two it gave you.
func DecodeStream ¶
DecodeStream is Decode for a parsed stream.
type Null ¶
type Null struct{}
Null is the PDF null object. A missing dictionary entry resolves to Null rather than to a nil Object, so callers never have to nil-check.
type Object ¶
type Object interface {
Kind() Kind
}
An Object is any value the PDF object syntax can express. The concrete types are Null, Bool, Integer, Real, String, Name, Array, Dict, Ref and *Stream; Kind tells them apart without a type switch.
func ParseObject ¶
ParseObject parses a single direct object from the start of b and reports how many bytes it consumed.
type Operation ¶
type Operation struct {
Operator string
Operands []Object
Image *InlineImage
}
An Operation is one step of a content stream: an operator and the operands that precede it. An inline image is reported as the operator "BI" with its dictionary and data attached, since the three keywords it spans are one thing.
func Operations ¶
Operations tokenises a whole content stream. The error it returns describes what did not parse; the operations it returns are still worth having.
type Permissions ¶ added in v0.3.0
type Permissions uint32
Permissions say what a reader may do with a file it can open with the user password. They are advisory — nothing enforces them but the reader's own good manners — and a file opened with the owner password ignores them entirely.
const ( PermPrint Permissions = 1 << 2 PermModify Permissions = 1 << 3 PermCopy Permissions = 1 << 4 PermAnnotate Permissions = 1 << 5 PermFillForms Permissions = 1 << 8 PermExtract Permissions = 1 << 9 // for accessibility PermAssemble Permissions = 1 << 10 PermPrintFaithful Permissions = 1 << 11 // at full resolution // AllPermissions grants every one of them. AllPermissions = PermPrint | PermModify | PermCopy | PermAnnotate | PermFillForms | PermExtract | PermAssemble | PermPrintFaithful )
The permissions a standard security handler can express. Anything not granted is refused; AllPermissions grants everything.
func (Permissions) Allows ¶ added in v0.4.0
func (p Permissions) Allows(want Permissions) bool
Allows reports whether every one of the given permissions is granted.
func (Permissions) String ¶ added in v0.4.0
func (p Permissions) String() string
String lists what is granted, in words.
type Protection ¶ added in v0.4.0
type Protection struct {
// Method names the algorithm the way a person would say it: "RC4-40",
// "RC4-128", "AES-128", "AES-256", or "none" for a file that declares an
// /Encrypt dictionary and then encrypts nothing with it.
Method string
// Revision is the standard security handler's revision: 2, 3, 4, 5 or 6.
Revision int
// Permissions is what the file grants whoever opened it with the user
// password.
Permissions Permissions
// Owner is true when the password the file was opened with was the
// owner's, in which case the permissions do not apply to this reader.
Owner bool
}
A Protection is what a file's security handler says about it: how it is encrypted, and what a reader that opened it with the user password may do.
type Ref ¶
Ref is an indirect reference: the "12 0 R" that stands in for an object stored elsewhere in the file.
type Resolver ¶
A Resolver fetches the object an indirect reference names. A document supplies one; a nil Resolver makes every reference resolve to Null, which is what a caller parsing an isolated fragment wants.
type Stream ¶
Stream is a PDF stream: a dictionary plus the raw, still-encoded bytes between the stream and endstream keywords. Decode applies the filter chain.
type String ¶
type String []byte
String is a PDF string, already unescaped: the bytes a literal or hex string denotes, with no interpretation of their text encoding.
type SyntaxError ¶
A SyntaxError reports malformed PDF syntax and where it was found, so a repair pass can restart from a known offset instead of giving up.
func (*SyntaxError) Error ¶
func (e *SyntaxError) Error() string
Error implements the error interface.
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
A Writer builds a PDF file out of objects. Numbers are handed out by Writer.Reserve, objects are written with Writer.Put, and Writer.Finish adds the cross-reference table and the trailer.
func NewPackedWriter ¶ added in v0.3.0
NewPackedWriter starts a file that puts what it can into compressed object streams and ends in a cross-reference stream. That is smaller — usually by a lot, since a file is mostly small dictionaries — and has been readable since PDF 1.5, which is 2003; the version in the header is raised to 1.5 if it is lower.
func NewWriter ¶
NewWriter starts a file with the given version in its header, "1.7" when the version is empty. Objects are written one after another and the file ends in a cross-reference table, which every reader that has ever existed can read.
func (*Writer) Copy ¶
Copy writes an object from a document into this file, following every indirect reference it reaches and renumbering as it goes, so that objects from several documents can live side by side. An object already copied from the same document keeps the number it was given.
func (*Writer) Encrypt ¶ added in v0.3.0
func (w *Writer) Encrypt(e Encryption)
Encrypt protects everything written from here on. It must be called before anything is written, and the file it produces is not byte-for-byte reproducible: encryption needs randomness, by design.
func (*Writer) Finish ¶
Finish writes the cross-reference table and the trailer and returns the file. /Size is filled in; the caller supplies /Root and whatever else the trailer needs.
Source Files