Documentation
¶
Overview ¶
Package chunkers provides the two content-defined chunkers the fstree uses: a byte chunker (ultracdc) for file content, and an item chunker for the index/entry streams (architecture/fstree.md, "Content-defined chunking").
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func SplitBytes ¶
SplitBytes runs the ultracdc content-defined chunker over r and calls fn once per chunk, in order. Each chunk passed to fn is a fresh copy, so fn may retain it (the underlying chunker reuses its read buffer between calls). A nil opts uses ultracdc's default sizes (min 2 KiB, normal 10 KiB, max 64 KiB). An empty reader yields zero chunks.
Types ¶
type ByteOpts ¶
type ByteOpts = gocdc.ChunkerOpts
ByteOpts configures the ultracdc byte chunker (re-exported so callers need not import the upstream package).
type ItemChunker ¶
ItemChunker decides chunk boundaries between whole items (a child key, a directory entry, or a [sepName, childKey] pair). A boundary ends the current run when the low k bits of BLAKE3(item encoding) are zero (average run 2^k), bounded by MinRun and MaxRun so an item is never split and variance is capped.
func NewItemChunker ¶
func NewItemChunker(bits int) ItemChunker
NewItemChunker returns an item chunker with average run 2^bits and derived bounds MinRun = 2^bits / 4 (at least 2) and MaxRun = 2^bits * 4.
func (ItemChunker) IsBoundary ¶
func (c ItemChunker) IsBoundary(enc []byte, runLen int) bool
IsBoundary reports whether the run should end after the item whose canonical encoding is enc, given the current run length (including this item).
Source Files