Documentation
¶
Overview ¶
Package chunker implements the buzhash content-defined chunker used by Proxmox Backup Server (a port of pbs-datastore's chunker): a 64-byte rolling window over the casync hash table, cutting chunks between min (avg/4) and max (avg*4) with a 4 MiB default average.
Index ¶
Constants ¶
const DefaultAvg = 4 << 20
DefaultAvg is the chunk size target Proxmox Backup Server uses.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Chunk ¶
type Chunk struct {
Data []byte // owned by the consumer
Offset uint64 // start offset within the stream
}
Chunk is one content-defined chunk of a stream.
type Chunker ¶
type Chunker struct {
// contains filtered or unexported fields
}
Chunker is the buzhash content-defined chunker used by Proxmox Backup Server (a port of pbs-datastore's chunker.rs). Boundaries depend only on content, so a changed byte resynchronizes within roughly one window.
Chunks are cut between min (avg/4) and max (avg*4): below min no boundary is considered, above max one is forced, in between a boundary falls where hash & (2*avg-1) lands in the top three values — matching the reference implementation bit for bit.
func New ¶
New returns a Chunker with the given average chunk size; 0 means DefaultAvg (4 MiB). The average must be a power of two of at least 4×the window (the break mask arithmetic depends on it).
Source Files