Documentation
¶
Index ¶
- Variables
- type CompactableSync
- type Compactor
- type CompactorType
- type Option
- func WithC1ZOptions(opts ...dotc1z.C1ZOption) Option
- func WithCompactorType(compactorType CompactorType) Optiondeprecated
- func WithEngine(engine dotc1z.Engine) Option
- func WithOverlayBufferFactor(f float64) Option
- func WithOverlayGateFraction(f float64) Option
- func WithOverlayRecordChunkSize(n int) Option
- func WithOverlaySeenKeyLimit(n int64) Option
- func WithPebbleCompactorMode(mode PebbleCompactorMode) Option
- func WithRunDuration(runDuration time.Duration) Option
- func WithSkipGrantExpansion() Option
- func WithSyncLimit(limit int) Option
- func WithTmpDir(tempDir string) Option
- type PebbleCompactorMode
Constants ¶
This section is empty.
Variables ¶
var ErrNotEnoughFilesToCompact = errors.New("must provide two or more files to compact")
Functions ¶
This section is empty.
Types ¶
type CompactableSync ¶
type Compactor ¶
type Compactor struct {
// contains filtered or unexported fields
}
func NewCompactor ¶
type CompactorType ¶ added in v0.3.36
type CompactorType string
const (
CompactorTypeAttached CompactorType = "attached"
)
type Option ¶ added in v0.3.5
type Option func(*Compactor)
func WithC1ZOptions ¶ added in v0.7.22
WithC1ZOptions sets the C1Z options to use for the compactor. This allows tweaking C1Z opts such as encoder/decoder parallelism.
func WithCompactorType
deprecated
added in
v0.3.36
func WithCompactorType(compactorType CompactorType) Option
Deprecated: There is now only one compactor type, so this option is no longer needed.
func WithEngine ¶ added in v0.12.5
WithEngine selects the storage engine for the compacted output. The default (unset) is sqlite, which is byte-identical to the historical compactor. EnginePebble produces a v3 Pebble c1z via a native record merge whose strategy (overlay / fold / kway) is resolved per run by resolvePebbleMode.
This is the only supported way to choose the engine; an engine passed through WithC1ZOptions does not select the compaction strategy and is overridden.
func WithOverlayBufferFactor ¶ added in v0.13.5
WithOverlayBufferFactor overrides the overlay merge's soft→hard seen-set limit multiplier (default 1.25). A source that crosses the soft limit mid-scan may finish inside the buffer and resume via the K-way path at the source boundary; crossing the hard limit mid-source restarts the bucket through the blind K-way path. Zero (the default) uses the merge's built-in production value.
func WithOverlayGateFraction ¶ added in v0.13.5
WithOverlayGateFraction overrides the overlay merge's statless pre-source gate, expressed as a fraction of the soft seen-set limit (default 0.9): when a source has no cached stats and the seen set is already past the gate, the bucket degrades to the K-way path before scanning the source. Zero (the default) uses the merge's built-in production value.
func WithOverlayRecordChunkSize ¶ added in v0.13.2
WithOverlayRecordChunkSize overrides how many winner records the overlay merge buffers per raw write batch. Zero (the default) uses the merge's built-in production value.
func WithOverlaySeenKeyLimit ¶ added in v0.13.2
WithOverlaySeenKeyLimit overrides the overlay merge's per-bucket seen-set cap: buckets whose estimated key count exceeds it route to the K-way run-file fallback. Zero (the default) uses the merge's built-in production value. Intended for tests and benchmarks; the cap bounds merge memory (~40B per seen key).
func WithPebbleCompactorMode ¶ added in v0.13.2
func WithPebbleCompactorMode(mode PebbleCompactorMode) Option
WithPebbleCompactorMode overrides the Pebble merge strategy. The default (PebbleCompactorModeAuto) chooses for you; passing an explicit mode is intended for benchmarks and debugging, not normal operation. No-op for the SQLite engine.
func WithRunDuration ¶ added in v0.5.13
func WithSkipGrantExpansion ¶ added in v0.8.2
func WithSkipGrantExpansion() Option
WithSkipGrantExpansion skips grant expansion after compaction. This is useful when expansion will be handled separately (e.g. by incremental expansion).
func WithSyncLimit ¶ added in v0.7.0
WithSyncLimit sets the number of syncs to keep after compaction cleanup.
func WithTmpDir ¶ added in v0.3.5
WithTmpDir sets the working directory where files will be created and edited during compaction. If not provided, the temporary directory will be used.
type PebbleCompactorMode ¶ added in v0.13.2
type PebbleCompactorMode string
PebbleCompactorMode selects the record-merge strategy for Pebble (v3) compaction.
const ( // PebbleCompactorModeAuto (the zero value) picks the best strategy. // Currently that is overlay: it won or tied every measured shape // (same-size and skewed inputs at 10/50/500 sources) and degrades // gracefully — buckets whose estimated keys exceed its in-memory // seen-set bound are routed to the kway path inside the merge. PebbleCompactorModeAuto PebbleCompactorMode = "" // PebbleCompactorModeKWay forces the bounded-fan-in external merge // sort. Never the fastest on its own, but it is the machinery // overlay falls back to for oversized buckets; forcing it is for // debugging and benchmarks. PebbleCompactorModeKWay PebbleCompactorMode = "kway" // PebbleCompactorModeOverlay forces the seen-set overlay merge. PebbleCompactorModeOverlay PebbleCompactorMode = "overlay" // PebbleCompactorModeFold forces in-place fold into the base sync. // Note: the output ADOPTS the base sync's id (no fresh sync id), // which C1's compaction bookkeeping currently mishandles — auto // mode never selects fold for that reason; forcing it is on the // caller to know their consumer tolerates a reused sync id. PebbleCompactorModeFold PebbleCompactorMode = "fold" )
Source Files
Directories