Documentation
¶
Overview ¶
Package yenc implements the yEnc binary encoding used for Usenet binary posts (http://www.yenc.org/yenc-draft.1.3.txt): a fast streaming decoder with =ybegin/=ypart/=yend parsing and CRC32 verification, and an encoder used by tests and the fake NNTP server.
yEnc maps every byte b to (b+42) mod 256. The "critical" output characters NUL, LF, CR and '=' are escaped as '=' followed by (b+42+64) mod 256; encoders also escape TAB and SPACE at line edges and '.' at the start of a line (to survive NNTP dot-stuffing done badly by some servers).
Index ¶
- Constants
- Variables
- func AppendEncode(dst, data []byte, o EncodeOptions) ([]byte, error)
- func DecodeLine(dst, src []byte, escaped bool) (int, bool)
- func Encode(w io.Writer, data []byte, o EncodeOptions) error
- func EncodeBytes(data []byte, o EncodeOptions) ([]byte, error)
- func ParsePart(line []byte, h *Header) error
- type DecodeOptions
- type EncodeOptions
- type Header
- type LineReader
- type Part
- type Trailer
Constants ¶
const DefaultLineLength = 128
DefaultLineLength is the conventional encoded line length.
const DefaultMaxSize = 64 << 20
DefaultMaxSize bounds the buffer a header can make the decoder allocate. Usenet parts are typically 384 KiB-1 MiB; 64 MiB leaves ample headroom while stopping a hostile header from requesting gigabytes.
const MaxLineSize = 1 << 20
MaxLineSize limits each encoded line, including headers. Normal posts use 128 or 256 bytes; this allows unusually long lines without letting malformed input allocate an unbounded assembly or scratch buffer.
const MaxPreambleSize = 64 << 10
MaxPreambleSize bounds text skipped before the yEnc header.
Variables ¶
var ( // ErrNoHeader means no =ybegin line was found: the article is not yEnc. ErrNoHeader = errors.New("yenc: no =ybegin header") // ErrTruncated means the input ended before the =yend trailer. ErrTruncated = errors.New("yenc: data truncated (no =yend)") // ErrCRCMismatch means a pcrc32 or crc32 check failed. ErrCRCMismatch = errors.New("yenc: crc32 mismatch") // ErrSizeMismatch means the decoded length disagrees with the headers. ErrSizeMismatch = errors.New("yenc: size mismatch") // ErrMalformed covers unparseable keyword lines. ErrMalformed = errors.New("yenc: malformed header") // ErrTooLarge means the headers announce a part larger than the limit. ErrTooLarge = errors.New("yenc: part exceeds size limit") )
Errors returned by the decoder. Decode errors are wrapped with context; use errors.Is to test for them.
Functions ¶
func AppendEncode ¶
func AppendEncode(dst, data []byte, o EncodeOptions) ([]byte, error)
AppendEncode appends the yEnc encoding of data (header, body lines and trailer, CRLF-terminated) to dst.
func DecodeLine ¶
DecodeLine decodes one encoded line (without line terminator) into dst and returns the number of bytes written. dst must have room for len(src) bytes. escaped carries a dangling '=' from the end of the previous line, and the returned flag reports a dangling '=' at the end of this one.
This is the hot loop. Each native word both detects '=' and subtracts 42 from eight independent byte lanes. Fusing detection with decoding avoids scanning each escaped run twice and calling a search routine per escape. The first set lane in the zero-byte mask is the first real escape even when subtraction borrows into a following lane. The short tail is scalar. No allocation, lookup table, unsafe access or architecture-specific ISA requirement is involved. Exact in-place decoding is supported.
func Encode ¶
func Encode(w io.Writer, data []byte, o EncodeOptions) error
Encode writes the yEnc encoding of data to w.
func EncodeBytes ¶
func EncodeBytes(data []byte, o EncodeOptions) ([]byte, error)
EncodeBytes returns the yEnc encoding of data.
Types ¶
type DecodeOptions ¶
type DecodeOptions struct {
// Buffer is reused for the decoded data when it has enough capacity.
Buffer []byte
// MaxSize caps the decoded part size; 0 means DefaultMaxSize.
MaxSize int64
// SkipCRC disables CRC32 verification (sizes are still checked).
SkipCRC bool
}
DecodeOptions tunes decoding. The zero value is valid.
type EncodeOptions ¶
type EncodeOptions struct {
Name string
LineLength int // encoded characters per line; 0 = DefaultLineLength
// Part > 0 produces a multipart post with a =ypart line. Total is the
// number of parts, Begin the 1-based offset of data in the file and
// FileSize the size of the whole file.
Part int
Total int
Begin int64
FileSize int64
// FileCRC, when HasFileCRC is set, is written as crc32= in the =yend of
// a multipart post (the CRC of the whole file).
FileCRC uint32
HasFileCRC bool
}
EncodeOptions describes the part being encoded.
type Header ¶
type Header struct {
Name string // file name
Line int // nominal encoded line length
Size int64 // size of the whole file
Part int // 1-based part number, 0 for single-part posts
Total int // number of parts, 0 if not given
// Begin and End are the 1-based inclusive byte offsets of this part in
// the file. For single-part posts Begin is 1 and End is Size.
Begin int64
End int64
// Multipart is true when the post carried a =ypart line.
Multipart bool
}
Header holds the =ybegin and (for multipart posts) =ypart fields.
func ParseBegin ¶
ParseBegin parses a "=ybegin ..." line.
type LineReader ¶
LineReader yields one line at a time without its line terminator (LF or CRLF). The returned slice is only valid until the next call. At the end of input it returns io.EOF. The NNTP client implements it directly over its connection buffer so article bodies are decoded without an extra copy.
func NewLineReader ¶
func NewLineReader(r io.Reader) LineReader
NewLineReader adapts an io.Reader to a LineReader. Lines longer than the internal buffer are assembled in a side buffer up to MaxLineSize bytes. An oversized line returns ErrTooLarge; the reader must then be discarded.
type Part ¶
Part is one decoded article.
func DecodeLines ¶
func DecodeLines(lr LineReader, opt *DecodeOptions) (*Part, error)
DecodeLines decodes one yEnc part from lr. Lines before =ybegin are skipped. Decoding stops after the =yend line; any remaining lines are left unread so the caller can drain its transport.
When the data decodes but fails verification (ErrCRCMismatch or ErrSizeMismatch) the returned Part is non-nil and holds the data, so callers may choose to use it anyway.
func DecodeReader ¶
func DecodeReader(r io.Reader, opt *DecodeOptions) (*Part, error)
DecodeReader decodes one part from r.
Source Files