Documentation
¶
Overview ¶
Package chunker implements content-defined chunking (CDC) using the FastCDC algorithm with gear hashing.
Index ¶
Constants ¶
const ( DefaultMinChunk = 256 * 1024 // 256 KiB DefaultAvgChunk = 1024 * 1024 // 1 MiB DefaultMaxChunk = 4 * 1024 * 1024 // 4 MiB )
Default chunk size parameters, the format v1 defaults. Min/Avg/Max define the FastCDC boundaries in bytes.
The average decides how much a single edit costs on the next backup: one changed byte rewrites one chunk. A 1 MiB average is the same order restic and borg use. The maximum is the ceiling a restore cache has to budget for per worker, so it stays at four times the average rather than growing on its own.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Chunk ¶
type Chunk struct {
Start uint64 // Byte offset of this chunk in the original stream
Data []byte // Raw chunk data
Length int // Same as len(Data)
Hash string // HMAC-SHA256 of Data under Config.IDKey, hex
}
Chunk represents a single chunk produced by the chunker.
type Chunker ¶
type Chunker struct {
// contains filtered or unexported fields
}
Chunker splits an input stream into variable-size chunks using FastCDC. It reads from an io.Reader and produces chunks via Next().
type Config ¶
type Config struct {
MinSize int // Minimum chunk size in bytes
AvgSize int // Target average chunk size in bytes
MaxSize int // Maximum chunk size in bytes
Seed [32]byte // keys the gear table
IDKey [32]byte // keys the HMAC-SHA256 that names each chunk
}
Config configures the FastCDC chunker.
Seed and IDKey are secrets of the repository, so cut points and chunk names of a known file cannot be predicted without them. A repository fills them in from its master key; the zero values are valid and only fit tests.
func DefaultConfig ¶
func DefaultConfig() Config
DefaultConfig returns the default chunker configuration.