transform

transform

import "github.com/kitsunium/sdk/pkg/v1/data/transform"

Package transform compresses and decompresses bytes with the SDK's compressors, without the codec package (ADR 0014). Importing it registers three, which the standard library implements:

box, err := transform.Compress(transform.Gzip, nil, payload)
plain, err := transform.DecompressBounded(transform.Gzip, nil, box, 1<<20)

Algorithms

An Algorithm names a Compressor in a registry of its own — compression is a transform of bytes, never a codec Format. Gzip is RFC 1952, Flate the raw DEFLATE stream of RFC 1951, and Zlib the RFC 1950 envelope HTTP misnames "deflate": three wire formats, not aliases of one another. A compressor another package registers — the module github.com/kitsunium/sdk/third-party/transform adds zstd and s2 — is reached by its Algorithm the same way once that package is imported; Available lists what is registered.

Every function appends to dst, which may be nil, and src must not overlap dst's spare capacity: a scheme writes there while it still reads src.

Bounds

Decompress is bounded by the scheme's own backstop — 256 MiB for the three registered here — so a bomb cannot exhaust memory, but it is the scheme's number, not the caller's. DecompressBounded takes the caller's ceiling: a scheme implementing BoundedDecompressor stops at it, so the ceiling bounds the work and not only the verdict; a scheme that does not is decompressed under its own backstop and judged afterwards. Either way no more than limit bytes are returned, and a stream that would exceed it is refused with DecompressedTooLarge rather than cut short — a truncated success would turn a bomb into a parsing bug further up.

Errors

Match them with errs.HasCode, or the sentinels with errors.Is:

CodeUnknownCompressor       0.2.5.1   no compressor is registered under the Algorithm
CodeCompressedFrameInvalid  0.2.5.4   the codec package's compressed frame is malformed, or its bomb guard tripped
CodeDecompressedTooLarge    0.2.5.6   the plaintext would exceed DecompressBounded's ceiling
CodeGzipFailed              0.3.26.1  compress/gzip refused the stream, compressing or decompressing
CodeFlateFailed             0.3.26.2  compress/flate refused the stream
CodeZlibFailed              0.3.26.3  compress/zlib refused the stream

A malformed stream is the scheme's own refusal, with the scheme's code: GzipFailed, FlateFailed or ZlibFailed for the three registered here, the stdlib's own error kept in the chain beneath it.

Index

• Constants
• Variables
• func Compress(algo Algorithm, dst, src []byte) (encoded []byte, err error)
• func Decompress(algo Algorithm, dst, src []byte) (decoded []byte, err error)
• func DecompressBounded(algo Algorithm, dst, src []byte, limit int64) (decoded []byte, err error)
• type Algorithm
  • func Available() []Algorithm
• type BoundedDecompressor
• type Compressor
  • func Lookup(algo Algorithm) (c Compressor, ok bool)

Constants

The error codes, ranges 0.2.5.* and 0.3.26.*, declared in the core.

const (
    // CodeUnknownCompressor identifies an Algorithm no imported package has
    // registered (0.2.5.1).
    CodeUnknownCompressor errs.Code = coretransform.CodeUnknownCompressor
    // CodeCompressedFrameInvalid identifies a malformed compressed frame of the
    // codec package, or a frame its decompression-bomb guard refused (0.2.5.4).
    CodeCompressedFrameInvalid errs.Code = coretransform.CodeCompressedFrameInvalid
    // CodeDecompressedTooLarge identifies a stream whose plaintext would exceed
    // the ceiling its caller gave (0.2.5.6).
    CodeDecompressedTooLarge errs.Code = coretransform.CodeDecompressedTooLarge
    // CodeGzipFailed identifies a stream compress/gzip refused, compressing or
    // decompressing (0.3.26.1).
    CodeGzipFailed errs.Code = coretransform.CodeGzipFailed
    // CodeFlateFailed identifies a stream compress/flate refused (0.3.26.2).
    CodeFlateFailed errs.Code = coretransform.CodeFlateFailed
    // CodeZlibFailed identifies a stream compress/zlib refused (0.3.26.3): the
    // RFC 1950 envelope, told apart from raw DEFLATE because a stream one of
    // them refuses is often valid under the other.
    CodeZlibFailed errs.Code = coretransform.CodeZlibFailed
)

Variables

The sentinels, for errors.Is: each carries the code of the same name.

var (
    // UnknownCompressor is the sentinel of [CodeUnknownCompressor], returned
    // as is.
    UnknownCompressor = coretransform.UnknownCompressor
    // CompressedFrameInvalid is the sentinel of [CodeCompressedFrameInvalid].
    CompressedFrameInvalid = coretransform.CompressedFrameInvalid
    // DecompressedTooLarge is the sentinel of [CodeDecompressedTooLarge],
    // returned as is.
    DecompressedTooLarge = coretransform.DecompressedTooLarge
    // GzipFailed is the sentinel of [CodeGzipFailed]; the stdlib's error is
    // kept beneath it.
    GzipFailed = coretransform.GzipFailed
    // FlateFailed is the sentinel of [CodeFlateFailed].
    FlateFailed = coretransform.FlateFailed
    // ZlibFailed is the sentinel of [CodeZlibFailed].
    ZlibFailed = coretransform.ZlibFailed
)

func Compress

func Compress(algo Algorithm, dst, src []byte) (encoded []byte, err error)

Compress compresses src with the scheme registered under algo and appends the result to dst. An unregistered algo is refused with UnknownCompressor and dst comes back as it was.

func Decompress

func Decompress(algo Algorithm, dst, src []byte) (decoded []byte, err error)

Decompress decompresses src with the scheme registered under algo and appends the result to dst, bounded by the scheme's own backstop. To bound it by a number of your own, call DecompressBounded. An unregistered algo is refused with UnknownCompressor and dst comes back as it was.

func DecompressBounded

func DecompressBounded(algo Algorithm, dst, src []byte, limit int64) (decoded []byte, err error)

DecompressBounded decompresses src with the scheme registered under algo and appends the result to dst, producing at most limit bytes of plaintext: a stream that would produce more is refused with DecompressedTooLarge and dst comes back as it was. A limit of zero admits an empty stream and nothing else, and a negative limit is read as zero rather than given a meaning of its own.

A scheme implementing BoundedDecompressor stops at limit, so the work is bounded too. A scheme that does not — a consumer's own, or one of the third-party module's — is decompressed under its own backstop and judged afterwards: declining to tighten the work can never change the verdict, only the bytes touched reaching it.

type Algorithm

Algorithm is the name a Compressor registers under. The zero value is reserved as invalid.

type Algorithm = coretransform.Algorithm

The Algorithms registered by importing this package.

const (
    // Gzip is RFC 1952: a DEFLATE stream in a gzip envelope with a CRC-32.
    Gzip Algorithm = "gzip"
    // Flate is the raw DEFLATE stream of RFC 1951, with no envelope.
    Flate Algorithm = "flate"
    // Zlib is the RFC 1950 envelope with an Adler-32 — what HTTP calls
    // "deflate".
    Zlib Algorithm = "zlib"
)

func Available

func Available() []Algorithm

Available returns the registered Algorithms, sorted.

type BoundedDecompressor

BoundedDecompressor is the optional extension of a Compressor that can be told how much plaintext it may produce; DecompressBounded uses it when the scheme has it.

type BoundedDecompressor = coretransform.BoundedDecompressor

type Compressor

Compressor is one compression scheme: Compress and Decompress append to dst. Implementations are safe for concurrent use.

type Compressor = coretransform.Compressor

func Lookup

func Lookup(algo Algorithm) (c Compressor, ok bool)

Lookup returns the Compressor registered under algo.

Generated by gomarkdoc