chunker

package
v0.9.3-rc2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 1, 2026 License: MIT Imports: 7 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type CDCChunker

type CDCChunker struct {
	// contains filtered or unexported fields
}

CDCChunker implements FastCDC algorithm.

func NewCDCChunker

func NewCDCChunker(minSize, avgSize, maxSize uint32) (*CDCChunker, error)

NewCDCChunker returns a new CDCChunker.

func (*CDCChunker) Chunk

func (c *CDCChunker) Chunk(ctx context.Context, r io.Reader) (<-chan *Chunk, <-chan error)

Chunk splits the reader into content-defined chunks.

type Chunk

type Chunk struct {
	Hash           string // BLAKE3 hash of chunk content
	Offset         int64  // Offset in original stream
	Size           uint32 // Chunk size in bytes
	CompressedSize uint32 // Compressed chunk size in bytes
	Data           []byte // Chunk data
	// contains filtered or unexported fields
}

Chunk represents a single content-defined chunk.

func (*Chunk) Free

func (c *Chunk) Free()

Free returns the chunk data to the pool. The caller is responsible for calling Free when the chunk is no longer needed. Failure to do so will result in leaking memory buffers from the pool. This method is safe to call multiple times.

type Chunker

type Chunker interface {
	// Chunk splits the reader into content-defined chunks.
	// Returns two channels: one for yielding chunks and one for yielding errors.
	Chunk(ctx context.Context, r io.Reader) (<-chan *Chunk, <-chan error)
}

Chunker interface for content-defined chunking.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL