transform

package
v0.19.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 5, 2026 License: MIT Imports: 3 Imported by: 0

README

transform

import "github.com/kitsunium/sdk/pkg/v1/data/transform"

Package transform compresses and decompresses bytes with the SDK's compressors, without the codec package (ADR 0014). Importing it registers three, which the standard library implements:

box, err := transform.Compress(transform.Gzip, nil, payload)
plain, err := transform.DecompressBounded(transform.Gzip, nil, box, 1<<20)
Algorithms

An Algorithm names a Compressor in a registry of its own — compression is a transform of bytes, never a codec Format. Gzip is RFC 1952, Flate the raw DEFLATE stream of RFC 1951, and Zlib the RFC 1950 envelope HTTP misnames "deflate": three wire formats, not aliases of one another. A compressor another package registers — the module github.com/kitsunium/sdk/third-party/transform adds zstd and s2 — is reached by its Algorithm the same way once that package is imported; Available lists what is registered.

Every function appends to dst, which may be nil, and src must not overlap dst's spare capacity: a scheme writes there while it still reads src.

Bounds

Decompress is bounded by the scheme's own backstop — 256 MiB for the three registered here — so a bomb cannot exhaust memory, but it is the scheme's number, not the caller's. DecompressBounded takes the caller's ceiling: a scheme implementing BoundedDecompressor stops at it, so the ceiling bounds the work and not only the verdict; a scheme that does not is decompressed under its own backstop and judged afterwards. Either way no more than limit bytes are returned, and a stream that would exceed it is refused with DecompressedTooLarge rather than cut short — a truncated success would turn a bomb into a parsing bug further up.

Errors

Match them with errs.HasCode, or the sentinels with errors.Is:

CodeUnknownCompressor       0.2.5.1   no compressor is registered under the Algorithm
CodeCompressedFrameInvalid  0.2.5.4   the codec package's compressed frame is malformed, or its bomb guard tripped
CodeDecompressedTooLarge    0.2.5.6   the plaintext would exceed DecompressBounded's ceiling
CodeGzipFailed              0.3.26.1  compress/gzip refused the stream, compressing or decompressing
CodeFlateFailed             0.3.26.2  compress/flate refused the stream
CodeZlibFailed              0.3.26.3  compress/zlib refused the stream

A malformed stream is the scheme's own refusal, with the scheme's code: GzipFailed, FlateFailed or ZlibFailed for the three registered here, the stdlib's own error kept in the chain beneath it.

Index

Constants

const CodeCompressedFrameInvalid errs.Code = 132356

CodeCompressedFrameInvalid identifies a malformed compressed frame of the codec package, or a frame its decompression-bomb guard refused (0.2.5.4).

Initialised with github.com/kitsunium/sdk/internal/core/data/transform.CodeCompressedFrameInvalid.

Code 0.2.5.4

const CodeDecompressedTooLarge errs.Code = 132358

CodeDecompressedTooLarge identifies a stream whose plaintext would exceed the ceiling its caller gave (0.2.5.6).

Initialised with github.com/kitsunium/sdk/internal/core/data/transform.CodeDecompressedTooLarge.

Code 0.2.5.6

const CodeFlateFailed errs.Code = 203266

CodeFlateFailed identifies a stream compress/flate refused (0.3.26.2).

Initialised with github.com/kitsunium/sdk/internal/core/data/transform.CodeFlateFailed.

Code 0.3.26.2

const CodeGzipFailed errs.Code = 203265

CodeGzipFailed identifies a stream compress/gzip refused, compressing or decompressing (0.3.26.1).

Initialised with github.com/kitsunium/sdk/internal/core/data/transform.CodeGzipFailed.

Code 0.3.26.1

const CodeUnknownCompressor errs.Code = 132353

CodeUnknownCompressor identifies an Algorithm no imported package has registered (0.2.5.1).

Initialised with github.com/kitsunium/sdk/internal/core/data/transform.CodeUnknownCompressor.

Code 0.2.5.1

const CodeZlibFailed errs.Code = 203267

CodeZlibFailed identifies a stream compress/zlib refused (0.3.26.3): the RFC 1950 envelope, told apart from raw DEFLATE because a stream one of them refuses is often valid under the other.

Initialised with github.com/kitsunium/sdk/internal/core/data/transform.CodeZlibFailed.

Code 0.3.26.3

const Flate Algorithm = "flate"

Flate is the raw DEFLATE stream of RFC 1951, with no envelope.

const Gzip Algorithm = "gzip"

Gzip is RFC 1952: a DEFLATE stream in a gzip envelope with a CRC-32.

const Zlib Algorithm = "zlib"

Zlib is the RFC 1950 envelope with an Adler-32 — what HTTP calls "deflate".

Variables

var CompressedFrameInvalid *errs.Error

CompressedFrameInvalid is the sentinel of CodeCompressedFrameInvalid.

Initialised with github.com/kitsunium/sdk/internal/core/data/transform.CompressedFrameInvalid.

Code 0.2.5.4 · reason COMPRESSED_FRAME_INVALID · public "Compressed frame is malformed or exceeds the safety bound"

var DecompressedTooLarge *errs.Error

DecompressedTooLarge is the sentinel of CodeDecompressedTooLarge, returned as is.

Initialised with github.com/kitsunium/sdk/internal/core/data/transform.DecompressedTooLarge.

Code 0.2.5.6 · reason DECOMPRESSED_TOO_LARGE · public "Decompressed payload exceeds the caller's ceiling"

var FlateFailed *errs.Error

FlateFailed is the sentinel of CodeFlateFailed.

Initialised with github.com/kitsunium/sdk/internal/core/data/transform.FlateFailed.

Code 0.3.26.2 · reason FLATE_FAILED · public "flate transform failed"

var GzipFailed *errs.Error

GzipFailed is the sentinel of CodeGzipFailed; the stdlib's error is kept beneath it.

Initialised with github.com/kitsunium/sdk/internal/core/data/transform.GzipFailed.

Code 0.3.26.1 · reason GZIP_FAILED · public "gzip transform failed"

var UnknownCompressor *errs.Error

UnknownCompressor is the sentinel of CodeUnknownCompressor, returned as is.

Initialised with github.com/kitsunium/sdk/internal/core/data/transform.UnknownCompressor.

Code 0.2.5.1 · reason UNKNOWN_COMPRESSOR · public "No compressor is registered under that algorithm"

var ZlibFailed *errs.Error

ZlibFailed is the sentinel of CodeZlibFailed.

Initialised with github.com/kitsunium/sdk/internal/core/data/transform.ZlibFailed.

Code 0.3.26.3 · reason ZLIB_FAILED · public "zlib transform failed"

func Available

func Available() []Algorithm

Available returns the registered Algorithms, sorted.

func Compress

func Compress(algo Algorithm, dst []byte, src []byte) (encoded []byte, err error)

Compress compresses src with the scheme registered under algo and appends the result to dst. An unregistered algo is refused with UnknownCompressor and dst comes back as it was.

func Decompress

func Decompress(algo Algorithm, dst []byte, src []byte) (decoded []byte, err error)

Decompress decompresses src with the scheme registered under algo and appends the result to dst, bounded by the scheme's own backstop. To bound it by a number of your own, call DecompressBounded. An unregistered algo is refused with UnknownCompressor and dst comes back as it was.

func DecompressBounded

func DecompressBounded(algo Algorithm, dst []byte, src []byte, limit int64) (decoded []byte, err error)

DecompressBounded decompresses src with the scheme registered under algo and appends the result to dst, producing at most limit bytes of plaintext: a stream that would produce more is refused with DecompressedTooLarge and dst comes back as it was. A limit of zero admits an empty stream and nothing else, and a negative limit is read as zero rather than given a meaning of its own.

A scheme implementing BoundedDecompressor stops at limit, so the work is bounded too. A scheme that does not — a consumer's own, or one of the third-party module's — is decompressed under its own backstop and judged afterwards: declining to tighten the work can never change the verdict, only the bytes touched reaching it.

func Lookup

func Lookup(algo Algorithm) (c Compressor, ok bool)

Lookup returns the Compressor registered under algo.

type Algorithm

Algorithm is the name a Compressor registers under. The zero value is reserved as invalid.

type Algorithm = coretransform.Algorithm

Algorithm is an alias of github.com/kitsunium/sdk/internal/core/data/transform.Algorithm, declared as:

type Algorithm string

func (Algorithm) Known
func (Algorithm) Known() bool

Known reports whether a has been registered in the Compressor registry.

func (Algorithm) String
func (Algorithm) String() string

String implements fmt.Stringer and returns the raw identifier.

type BoundedDecompressor

BoundedDecompressor is the optional extension of a Compressor that can be told how much plaintext it may produce; DecompressBounded uses it when the scheme has it.

type BoundedDecompressor = coretransform.BoundedDecompressor

BoundedDecompressor is an alias of github.com/kitsunium/sdk/internal/core/data/transform.BoundedDecompressor, declared as:

type BoundedDecompressor interface {
	DecompressBounded(dst []byte, src []byte, max int64) (out []byte, err error)
	Compressor
}

func (BoundedDecompressor) DecompressBounded
func (BoundedDecompressor) DecompressBounded(dst []byte, src []byte, max int64) (out []byte, err error)

type Compressor

Compressor is one compression scheme: Compress and Decompress append to dst. Implementations are safe for concurrent use.

type Compressor = coretransform.Compressor

Compressor is an alias of github.com/kitsunium/sdk/internal/core/data/transform.Compressor, declared as:

type Compressor interface {
	Algorithm() Algorithm
	Compress(dst []byte, src []byte) (out []byte, err error)
	Decompress(dst []byte, src []byte) (out []byte, err error)
}

func (Compressor) Algorithm
func (Compressor) Algorithm() Algorithm

func (Compressor) Compress
func (Compressor) Compress(dst []byte, src []byte) (out []byte, err error)

func (Compressor) Decompress
func (Compressor) Decompress(dst []byte, src []byte) (out []byte, err error)

Generated from docs/api by tools/genindex.

Documentation

Overview

Package transform compresses and decompresses bytes with the SDK's compressors, without the codec package (ADR 0014). Importing it registers three, which the standard library implements: 

box, err := transform.Compress(transform.Gzip, nil, payload)
plain, err := transform.DecompressBounded(transform.Gzip, nil, box, 1<<20)

Algorithms

An Algorithm names a Compressor in a registry of its own — compression is a transform of bytes, never a codec Format. Gzip is RFC 1952, Flate the raw DEFLATE stream of RFC 1951, and Zlib the RFC 1950 envelope HTTP misnames "deflate": three wire formats, not aliases of one another. A compressor another package registers — the module github.com/kitsunium/sdk/third-party/transform adds zstd and s2 — is reached by its Algorithm the same way once that package is imported; Available lists what is registered.

Every function appends to dst, which may be nil, and src must not overlap dst's spare capacity: a scheme writes there while it still reads src.

Bounds

Decompress is bounded by the scheme's own backstop — 256 MiB for the three registered here — so a bomb cannot exhaust memory, but it is the scheme's number, not the caller's. DecompressBounded takes the caller's ceiling: a scheme implementing BoundedDecompressor stops at it, so the ceiling bounds the work and not only the verdict; a scheme that does not is decompressed under its own backstop and judged afterwards. Either way no more than limit bytes are returned, and a stream that would exceed it is refused with DecompressedTooLarge rather than cut short — a truncated success would turn a bomb into a parsing bug further up.

Errors

Match them with errs.HasCode, or the sentinels with errors.Is: 

CodeUnknownCompressor       0.2.5.1   no compressor is registered under the Algorithm
CodeCompressedFrameInvalid  0.2.5.4   the codec package's compressed frame is malformed, or its bomb guard tripped
CodeDecompressedTooLarge    0.2.5.6   the plaintext would exceed DecompressBounded's ceiling
CodeGzipFailed              0.3.26.1  compress/gzip refused the stream, compressing or decompressing
CodeFlateFailed             0.3.26.2  compress/flate refused the stream
CodeZlibFailed              0.3.26.3  compress/zlib refused the stream

A malformed stream is the scheme's own refusal, with the scheme's code: GzipFailed, FlateFailed or ZlibFailed for the three registered here, the stdlib's own error kept in the chain beneath it.

Index

Constants

View Source 
const (
	// CodeUnknownCompressor identifies an Algorithm no imported package has
	// registered (0.2.5.1).
	CodeUnknownCompressor errs.Code = coretransform.CodeUnknownCompressor
	// CodeCompressedFrameInvalid identifies a malformed compressed frame of the
	// codec package, or a frame its decompression-bomb guard refused (0.2.5.4).
	CodeCompressedFrameInvalid errs.Code = coretransform.CodeCompressedFrameInvalid
	// CodeDecompressedTooLarge identifies a stream whose plaintext would exceed
	// the ceiling its caller gave (0.2.5.6).
	CodeDecompressedTooLarge errs.Code = coretransform.CodeDecompressedTooLarge
	// CodeGzipFailed identifies a stream compress/gzip refused, compressing or
	// decompressing (0.3.26.1).
	CodeGzipFailed errs.Code = coretransform.CodeGzipFailed
	// CodeFlateFailed identifies a stream compress/flate refused (0.3.26.2).
	CodeFlateFailed errs.Code = coretransform.CodeFlateFailed
	// CodeZlibFailed identifies a stream compress/zlib refused (0.3.26.3): the
	// RFC 1950 envelope, told apart from raw DEFLATE because a stream one of
	// them refuses is often valid under the other.
	CodeZlibFailed errs.Code = coretransform.CodeZlibFailed
)

The error codes, ranges 0.2.5.* and 0.3.26.*, declared in the core.

Variables

View Source 
var (
	// UnknownCompressor is the sentinel of [CodeUnknownCompressor], returned
	// as is.
	UnknownCompressor = coretransform.UnknownCompressor
	// CompressedFrameInvalid is the sentinel of [CodeCompressedFrameInvalid].
	CompressedFrameInvalid = coretransform.CompressedFrameInvalid
	// DecompressedTooLarge is the sentinel of [CodeDecompressedTooLarge],
	// returned as is.
	DecompressedTooLarge = coretransform.DecompressedTooLarge
	// GzipFailed is the sentinel of [CodeGzipFailed]; the stdlib's error is
	// kept beneath it.
	GzipFailed = coretransform.GzipFailed
	// FlateFailed is the sentinel of [CodeFlateFailed].
	FlateFailed = coretransform.FlateFailed
	// ZlibFailed is the sentinel of [CodeZlibFailed].
	ZlibFailed = coretransform.ZlibFailed
)

The sentinels, for errors.Is: each carries the code of the same name.

Functions

func Compress 

func Compress(algo Algorithm, dst, src []byte) (encoded []byte, err error)

Compress compresses src with the scheme registered under algo and appends the result to dst. An unregistered algo is refused with UnknownCompressor and dst comes back as it was.

func Decompress 

func Decompress(algo Algorithm, dst, src []byte) (decoded []byte, err error)

Decompress decompresses src with the scheme registered under algo and appends the result to dst, bounded by the scheme's own backstop. To bound it by a number of your own, call DecompressBounded. An unregistered algo is refused with UnknownCompressor and dst comes back as it was.

func DecompressBounded 

func DecompressBounded(algo Algorithm, dst, src []byte, limit int64) (decoded []byte, err error)

DecompressBounded decompresses src with the scheme registered under algo and appends the result to dst, producing at most limit bytes of plaintext: a stream that would produce more is refused with DecompressedTooLarge and dst comes back as it was. A limit of zero admits an empty stream and nothing else, and a negative limit is read as zero rather than given a meaning of its own.

A scheme implementing BoundedDecompressor stops at limit, so the work is bounded too. A scheme that does not — a consumer's own, or one of the third-party module's — is decompressed under its own backstop and judged afterwards: declining to tighten the work can never change the verdict, only the bytes touched reaching it.

Types

type Algorithm 

type Algorithm = coretransform.Algorithm

Algorithm is the name a Compressor registers under. The zero value is reserved as invalid. 

const (
	// Gzip is RFC 1952: a DEFLATE stream in a gzip envelope with a CRC-32.
	Gzip Algorithm = "gzip"
	// Flate is the raw DEFLATE stream of RFC 1951, with no envelope.
	Flate Algorithm = "flate"
	// Zlib is the RFC 1950 envelope with an Adler-32 — what HTTP calls
	// "deflate".
	Zlib Algorithm = "zlib"
)

The Algorithms registered by importing this package.

func Available 

func Available() []Algorithm

Available returns the registered Algorithms, sorted.

type BoundedDecompressor 

type BoundedDecompressor = coretransform.BoundedDecompressor

BoundedDecompressor is the optional extension of a Compressor that can be told how much plaintext it may produce; DecompressBounded uses it when the scheme has it.

type Compressor 

type Compressor = coretransform.Compressor

Compressor is one compression scheme: Compress and Decompress append to dst. Implementations are safe for concurrent use.

func Lookup 

func Lookup(algo Algorithm) (c Compressor, ok bool)

Lookup returns the Compressor registered under algo.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.