compression

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 14, 2025 License: MIT Imports: 18 Imported by: 0

Documentation

Overview

Package compression provides high-performance compression support for Nebula

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Algorithm

type Algorithm string

Algorithm represents a compression algorithm

const (
	// None represents no compression
	None Algorithm = "none"
	// Gzip represents gzip compression
	Gzip Algorithm = "gzip"
	// Snappy represents snappy compression
	Snappy Algorithm = "snappy"
	// LZ4 represents lz4 compression
	LZ4 Algorithm = "lz4"
	// Zstd represents zstandard compression
	Zstd Algorithm = "zstd"
	// S2 represents s2 compression (Snappy compatible)
	S2 Algorithm = "s2"
	// Deflate represents deflate compression
	Deflate Algorithm = "deflate"
)

type CompressedChunk

type CompressedChunk struct {
	ID         int
	Original   []byte
	Compressed []byte
	Error      error
}

CompressedChunk represents a compressed data chunk

type Compressor

type Compressor interface {
	// Compress compresses data
	Compress(data []byte) ([]byte, error)
	// Decompress decompresses data
	Decompress(data []byte) ([]byte, error)
	// CompressStream compresses from reader to writer
	CompressStream(dst io.Writer, src io.Reader) error
	// DecompressStream decompresses from reader to writer
	DecompressStream(dst io.Writer, src io.Reader) error
	// Algorithm returns the compression algorithm
	Algorithm() Algorithm
	// Level returns the compression level
	Level() Level
}

Compressor provides compression and decompression functionality

func NewCompressor

func NewCompressor(config *Config) (Compressor, error)

NewCompressor creates a new compressor based on configuration

type CompressorPool

type CompressorPool struct {
	// contains filtered or unexported fields
}

CompressorPool provides pooled compressors for better performance

func NewCompressorPool

func NewCompressorPool(config *Config) *CompressorPool

NewCompressorPool creates a new compressor pool

func (*CompressorPool) Compress

func (cp *CompressorPool) Compress(data []byte) ([]byte, error)

Compress compresses data using a pooled compressor

func (*CompressorPool) Decompress

func (cp *CompressorPool) Decompress(data []byte) ([]byte, error)

Decompress decompresses data using a pooled compressor

func (*CompressorPool) Get

func (cp *CompressorPool) Get() Compressor

Get gets a compressor from pool

func (*CompressorPool) Put

func (cp *CompressorPool) Put(c Compressor)

Put returns compressor to pool

type Config

type Config struct {
	Algorithm   Algorithm
	Level       Level
	BufferSize  int
	Concurrency int
}

Config represents compressor configuration

func DefaultConfig

func DefaultConfig() *Config

DefaultConfig returns default compression configuration

type Level

type Level int

Level represents compression level

const (
	// Fastest prioritizes speed over compression ratio
	Fastest Level = 1
	// Default balances speed and compression
	Default Level = 5
	// Better improves compression at cost of speed
	Better Level = 7
	// Best maximizes compression ratio
	Best Level = 9
)

type ParallelCompressor

type ParallelCompressor struct {
	// contains filtered or unexported fields
}

ParallelCompressor implements parallel compression for large data

func NewParallelCompressor

func NewParallelCompressor(config ParallelConfig, logger *zap.Logger) *ParallelCompressor

NewParallelCompressor creates a new parallel compressor

func (*ParallelCompressor) CompressData

func (pc *ParallelCompressor) CompressData(data []byte) ([]byte, error)

CompressData compresses data in parallel

func (*ParallelCompressor) DecompressData

func (pc *ParallelCompressor) DecompressData(data []byte) ([]byte, error)

DecompressData decompresses data in parallel

func (*ParallelCompressor) GetMetrics

func (pc *ParallelCompressor) GetMetrics() (bytesProcessed, chunksProcessed int64)

GetMetrics returns compression metrics

func (*ParallelCompressor) Stop

func (pc *ParallelCompressor) Stop()

Stop stops the parallel compressor

type ParallelConfig

type ParallelConfig struct {
	Algorithm  Algorithm
	Level      Level
	NumWorkers int // 0 = auto (NumCPU)
	ChunkSize  int // Size of each chunk in bytes
}

ParallelConfig configures parallel compression

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL