Documentation
¶
Overview ¶
Package compress wraps zstd/lz4 block compression and defines the composable codec-chain type (preprocessor → general compressor) used to reach 0.4–0.8 bytes/point (DESIGN.md §3.3, §14 M0).
A [Chain] is a sequence of a column-specific preprocessor (from [chunk]) followed by a general compressor (ZSTD or LZ4). Cold-tier recompression is just a different chain over the same parts. The wrappers are append-style (Compress(dst, src) []byte / Decompress(dst, src) []byte) and use pooled encoders/ decoders to avoid per-call allocation.
Index ¶
Constants ¶
const ( // FlagRaw means the payload is stored uncompressed. FlagRaw byte = 0x00 // FlagCompressed means the payload is algorithm-compressed. FlagCompressed byte = 0x01 )
Flag bytes for the compressed-block format.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Compressor ¶
type Compressor struct {
// contains filtered or unexported fields
}
Compressor is a pooled, append-style compressor. It is safe for concurrent use (each Compress/Decompress call borrows a pooled encoder/decoder).
func NewCompressor ¶
func NewCompressor(alg Algorithm, level Level) *Compressor
NewCompressor returns a Compressor for the given algorithm and level. Level is only meaningful for ZSTD; LZ4 ignores it.
func (*Compressor) Algorithm ¶
func (c *Compressor) Algorithm() Algorithm
Algorithm returns the compressor's algorithm.
func (*Compressor) Compress ¶
func (c *Compressor) Compress(dst, src []byte) []byte
Compress appends the compressed form of src to dst and returns the extended slice. If compression does not reduce the size (rare for small inputs), the raw bytes are stored with a 1-byte prefix FlagRaw.
func (*Compressor) Decompress ¶
func (c *Compressor) Decompress(dst, src []byte) ([]byte, error)
Decompress appends the decompressed form of src to dst and returns the extended slice. src must start with a flag byte (FlagRaw or FlagCompressed).
func (*Compressor) Level ¶ added in v0.37.0
func (c *Compressor) Level() Level
Level returns the compressor's level (0 ⇒ the algorithm default).
type Level ¶
type Level uint8
Level is a compression level. ZSTD levels: 1–22 (default 3, higher = better ratio but slower). LZ4 levels are not exposed (LZ4 is always fastest).
type ZSTDFramer ¶ added in v0.46.0
type ZSTDFramer struct {
// contains filtered or unexported fields
}
ZSTDFramer is a pooled codec for bare zstd frames, without the FlagRaw/FlagCompressed prefix Compressor puts in front of a block. Two properties follow from that and neither is available from Compressor:
- the bytes are a standard zstd frame, so any zstd implementation reads them — an HTTP `Content-Encoding: zstd` body, for one;
- ZSTDFramer.Reader decompresses as a stream, so a caller can bound the *decompressed* size. A whole-buffer decode cannot: it has already allocated the output by the time the size is known, which is exactly what a decompression bomb exploits.
A ZSTDFramer is safe for concurrent use.
func NewZSTDFramer ¶ added in v0.46.0
func NewZSTDFramer(level Level) *ZSTDFramer
NewZSTDFramer returns a ZSTDFramer compressing at the given level.
func (*ZSTDFramer) Compress ¶ added in v0.46.0
func (f *ZSTDFramer) Compress(dst, src []byte) []byte
Compress appends the zstd frame of src to dst and returns the extended slice. Unlike Compressor.Compress it always compresses, even when that grows the input; a caller that cares compares the lengths itself.
func (*ZSTDFramer) Reader ¶ added in v0.46.0
func (f *ZSTDFramer) Reader(r io.Reader) (io.ReadCloser, error)
Reader returns a streaming decompressor over the zstd frame in r. Close returns the decoder to the pool and must be called; it does not close r.
Source Files