Documentation
¶
Index ¶
- Variables
- type ApplyOptions
- type ApplyResult
- type ApplyWorkerPool
- type LeafPageBatchLog
- type LeafPageConcurrentAppendLog
- type LeafPageLog
- type LeafPageLogLaneProvider
- type LeafPagePreparedAppendLog
- type LeafPagePreparedBatchAppendLog
- type LeafPagePreparedBatchLog
- type LeafPagePreparedChildRefBatchLog
- type LeafPagePreparedLog
- type LeafPageReader
- type PageAllocator
- type ParallelMergePressureLevel
- type ParallelMergePressureSource
- type ReadOnlyLeafSpan
- type ReadOnlyLeafSpanSummary
- type ReadOnlyLeafSpanWorkerRange
- type ReadOnlyLeafSpanWorkerRangeSummary
- type ReadOnlyPrepareOptions
- type ReadOnlyPrepareResult
- func (r ReadOnlyPrepareResult) AppendLeafSpanWorkUnitRanges(dst []ReadOnlyLeafSpanWorkerRange, workers int) []ReadOnlyLeafSpanWorkerRange
- func (r ReadOnlyPrepareResult) AppendLeafSpanWorkerRanges(dst []ReadOnlyLeafSpanWorkerRange, workers int) []ReadOnlyLeafSpanWorkerRange
- func (r ReadOnlyPrepareResult) LeafSpanSummary() ReadOnlyLeafSpanSummary
- func (r ReadOnlyPrepareResult) LeafSpanWorkerRangeSummary(workers int) ReadOnlyLeafSpanWorkerRangeSummary
- func (r *ReadOnlyPrepareResult) ResetForReuse()
- func (r ReadOnlyPrepareResult) ReuseOptions() ReadOnlyPrepareOptions
- func (r ReadOnlyPrepareResult) ValidateLeafSpans() error
- type Split
- type Zipper
- func (z *Zipper) Apply(rootID uint64, b *batch.Batch) (newRootID uint64, retiredPages []uint64, metrics adaptive.Metrics, err error)
- func (z *Zipper) ApplyWithOptions(rootID uint64, b *batch.Batch, opts ApplyOptions) (ApplyResult, error)
- func (z *Zipper) CloneWithAllocator(a PageAllocator) *Zipper
- func (z *Zipper) PrepareReadOnly(rootID uint64, b *batch.Batch, opts ReadOnlyPrepareOptions) (ReadOnlyPrepareResult, error)
- func (z *Zipper) PrepareReadOnlyPlan(rootID uint64, ops []batch.Entry, ranges []batch.DeleteRange, ...) (ReadOnlyPrepareResult, error)
- func (z *Zipper) SetAdaptiveLeafEncoding(enabled bool)
- func (z *Zipper) SetFillTargets(leafPPM, internalPPM uint32)
- func (z *Zipper) SetIndexColumnarLeaves(enabled bool)
- func (z *Zipper) SetIndexInternalBaseDelta(enabled bool)
- func (z *Zipper) SetIndexPackedValuePtr(enabled bool)
- func (z *Zipper) SetLeafPageLog(log LeafPageLog)
- func (z *Zipper) SetLeafPageReader(reader LeafPageReader)
- func (z *Zipper) SetLeafPrefixCompression(enabled bool)
- func (z *Zipper) SetMaintenanceOpsPerCoalesce(opsPerCoalesce int)
- func (z *Zipper) SetOuterLeavesInValueLog(enabled bool)
- func (z *Zipper) SetParallelMergePressureSource(src ParallelMergePressureSource)
- func (z *Zipper) SetPiggybackCompaction(enabled bool)
Constants ¶
This section is empty.
Variables ¶
var ( // ErrSpanNativeOutputOwnership marks failures where a span-native worker // produced durable output whose child-ref kind/ownership cannot be accepted by // the reducer for the configured output mode. ErrSpanNativeOutputOwnership = errors.New("zipper: span-native output ownership failure") // ErrSpanNativeReducerValidation marks fail-closed reducer validation failures // before publishing a reconstructed root. ErrSpanNativeReducerValidation = errors.New("zipper: span-native reducer validation failed") )
Functions ¶
This section is empty.
Types ¶
type ApplyOptions ¶ added in v0.6.0
type ApplyOptions struct {
// PrepareReadOnly asks ApplyWithOptions to run the read-only preparation
// pass before applying the delta. This is opt-in because it traverses the
// captured root in addition to the apply pass. It does not publish roots,
// allocate output pages, append leaf/value-log records, retire old pages, or
// transfer durable output ownership.
PrepareReadOnly bool
// ReadOnlyPrepare reuses buffers for the optional read-only preparation
// pass. It is ignored unless PrepareReadOnly is true or ParallelApplyConcurrency
// requests opt-in parallel apply gating.
ReadOnlyPrepare ReadOnlyPrepareOptions
// ReadOnlyPrepareWorkers requests deterministic leaf-span worker ranges for
// reporting. Values <=0 use ParallelApplyConcurrency when opt-in parallel
// apply is requested.
ReadOnlyPrepareWorkers int
// ParallelApplyConcurrency enables the M2 opt-in COW apply worker-pool path.
// Values <=1 leave the existing apply path unchanged. The effective worker
// count is also bounded by GOMAXPROCS, planned leaf spans, span bytes, and the
// minimum work thresholds below.
ParallelApplyConcurrency int
// ParallelApplyWorkerPool supplies reusable workers for opt-in parallel apply.
// If nil, ApplyWithOptions falls back to the existing per-call worker path.
ParallelApplyWorkerPool *ApplyWorkerPool
// ParallelApplyMinSpans gates opt-in worker-pool apply by planned leaf spans.
// Values <=0 disable the span-count gate.
ParallelApplyMinSpans int
// ParallelApplyMinSpanOps gates opt-in worker-pool apply by span-local work.
// Values <=0 disable the op-count gate.
ParallelApplyMinSpanOps int
// ParallelApplyMinSpanBytes gates opt-in worker-pool apply by estimated bytes.
// Values <=0 disable the byte-count gate.
ParallelApplyMinSpanBytes int
// SpanNativeApply enables the M10 opt-in span-native apply candidate path.
// The initial gate is deliberately narrower than M2 parallel COW apply: exact
// point-only leaf spans, no cold build, no maintenance rewrite, and bounded
// workers. Unsupported runs fail closed to the recursive apply fallback.
SpanNativeApply bool
// SpanNativeAllowMaintenancePointOps lets trusted ordered-root callers use
// span-native apply for point-delete maintenance plans whose tombstone/edge
// semantics are intentionally preserved. Raw DB apply must leave this false so
// delete maintenance keeps using the recursive path that prunes and rebalances
// empty or underfull pages.
SpanNativeAllowMaintenancePointOps bool
// SpanNativeForceFallbackReason forces a prepared span-native candidate to use
// the safe recursive/parallel fallback and report this stable reason string.
// It is used by checkpoint/close drains and emergency gates that must not
// produce span-native durable output.
SpanNativeForceFallbackReason string
}
ApplyOptions configures a root apply attempt.
type ApplyResult ¶ added in v0.6.0
type ApplyResult struct {
RootID uint64
PendingRetiredPages []uint64
Metrics adaptive.Metrics
// ReadOnlyPrepare is populated only when ApplyOptions.PrepareReadOnly is
// true or opt-in parallel apply needs the M1 span plan for gating. It is
// planning metadata only; it owns no pager or leaf-log output.
ReadOnlyPrepare ReadOnlyPrepareResult
// ReadOnlyPrepareNs is the time spent in the optional read-only preparation
// pass. It is zero when no read-only preparation ran.
ReadOnlyPrepareNs uint64
// ReadOnlyPrepareRequested reports that read-only preparation was requested
// explicitly or by the opt-in parallel apply gate.
ReadOnlyPrepareRequested bool
// ReadOnlyPrepareValidationFailed distinguishes validation failures from
// read/decode failures for stats.
ReadOnlyPrepareValidationFailed bool
// ReadOnlyPrepareFailed distinguishes read/decode failures from later apply
// failures for callers that need stable fallback labels.
ReadOnlyPrepareFailed bool
// ReadOnlyPrepareWorkerSummary is the deterministic worker-range aggregate for
// the requested read-only worker target.
ReadOnlyPrepareWorkerSummary ReadOnlyLeafSpanWorkerRangeSummary
// ParallelApplyWorkers is the effective opt-in worker count selected after
// applying all gates. ParallelApplyUsed is true when that count was >1.
ParallelApplyWorkers int
ParallelApplyUsed bool
// SpanNativeEligible reports that the M10 opt-in span-native gate accepted the
// read-only plan. SpanNativeWorkers is the bounded worker count selected for
// future span jobs. SpanNativeUsed remains false until the real span-native
// execution engine replaces recursive fallback for eligible runs.
SpanNativeEligible bool
SpanNativeWorkers int
SpanNativeUsed bool
// SpanNativeFallbackReason is a stable fallback reason string when the
// span-native candidate failed closed before, during, or after execution. DB
// callers map it onto their append-only fallback reason enum for stats.
SpanNativeFallbackReason string
}
ApplyResult is the complete in-memory result of a root apply attempt. The retired page list is pending until the caller's install guard succeeds and the new root is committed.
type ApplyWorkerPool ¶ added in v0.6.0
type ApplyWorkerPool struct {
// contains filtered or unexported fields
}
ApplyWorkerPool is a small reusable worker pool for opt-in flush/apply COW work. It runs caller-supplied jobs to completion before Run returns; it does not own durable output, publish roots, or cancel in-flight work on first error. Callers are responsible for collecting worker-local errors/stats and for discarding unpublished output on guarded-publish failure.
func NewApplyWorkerPool ¶ added in v0.6.0
func NewApplyWorkerPool(workers int) *ApplyWorkerPool
NewApplyWorkerPool starts workers reusable across apply attempts. Values <=0 create a nil-equivalent pool with one worker slot; callers normally avoid constructing a pool unless configured concurrency is >1.
func (*ApplyWorkerPool) Close ¶ added in v0.6.0
func (p *ApplyWorkerPool) Close()
Close stops idle workers after all tasks already submitted through Run have completed. Concurrent Run after Close returns errApplyWorkerPoolClosed.
func (*ApplyWorkerPool) Run ¶ added in v0.6.0
func (p *ApplyWorkerPool) Run(workers, count int, fn func(workerID, job int)) error
Run schedules up to workers reusable workers over count jobs. It blocks until all scheduled workers finish. A closed pool returns errApplyWorkerPoolClosed.
func (*ApplyWorkerPool) RunSeeded ¶ added in v0.6.0
func (p *ApplyWorkerPool) RunSeeded(workers, count int, fn func(workerID, job int)) error
RunSeeded schedules one initial job per workerID before dynamic work stealing. This preserves worker-owned resources such as selected leaf-log lanes while retaining load balancing for the remaining jobs.
type LeafPageBatchLog ¶ added in v0.6.0
type LeafPageBatchLog interface {
AppendLeafPages(leafPages [][]byte) ([]page.LeafLogPtr, error)
}
type LeafPageConcurrentAppendLog ¶ added in v0.6.0
type LeafPageConcurrentAppendLog interface {
ConcurrentLeafPageAppends() bool
}
type LeafPageLog ¶ added in v0.4.0
type LeafPageLog interface {
AppendLeafPage(leafPage []byte) (page.LeafLogPtr, error)
}
type LeafPageLogLaneProvider ¶ added in v0.6.0
type LeafPageLogLaneProvider interface {
LeafPageLogLane(workerIndex int) (LeafPageLog, bool)
}
type LeafPagePreparedAppendLog ¶ added in v0.6.0
type LeafPagePreparedAppendLog interface {
PreparedLeafPageAppends() bool
}
type LeafPagePreparedBatchAppendLog ¶ added in v0.6.0
type LeafPagePreparedBatchAppendLog interface {
PreparedLeafPageBatchAppends() bool
}
type LeafPagePreparedBatchLog ¶ added in v0.6.0
type LeafPagePreparedBatchLog interface {
AppendPreparedLeafPages(leafPages [][]byte, preparedPayloads [][]byte) ([]page.LeafLogPtr, error)
}
type LeafPagePreparedChildRefBatchLog ¶ added in v0.6.0
type LeafPagePreparedLog ¶ added in v0.6.0
type LeafPagePreparedLog interface {
AppendPreparedLeafPage(leafPage []byte, preparedPayload []byte) (page.LeafLogPtr, error)
}
type LeafPageReader ¶ added in v0.4.0
type PageAllocator ¶
type ParallelMergePressureLevel ¶ added in v0.4.0
type ParallelMergePressureLevel uint8
const ( ParallelMergePressureNormal ParallelMergePressureLevel = iota ParallelMergePressureHigh ParallelMergePressureCritical )
type ParallelMergePressureSource ¶ added in v0.4.0
type ParallelMergePressureSource func() ParallelMergePressureLevel
type ReadOnlyLeafSpan ¶ added in v0.6.0
type ReadOnlyLeafSpan struct {
// Ref identifies the existing leaf that owns this span. When ColdBuild is
// true there is no existing leaf; Ref is the zero ChildRef and is not
// actionable.
Ref page.ChildRef
// LowKey is the inclusive lower bound for the leaf span. HighKey is the
// exclusive upper bound. A nil bound is open-ended. Non-nil bound slices and
// op-key slices are owned by ReadOnlyPrepareResult until ReuseOptions is used.
LowKey []byte
HighKey []byte
// FirstOpKey and LastOpKey describe the point operations assigned to this
// span. They are nil for range-only spans.
FirstOpKey []byte
LastOpKey []byte
// OpCount is the span-local work count: PointOpCount plus DeleteRangeCount.
// Delete ranges that overlap multiple leaves are counted once per touched
// span, because future workers need each overlapping range in every touched
// leaf span.
OpCount int
PointOpCount int
DeleteRangeCount int
ByteCount int
// PointOpStart/PointOpEnd index the canonical point-op slice returned by the
// batch apply plan. DeleteRangeStart/DeleteRangeEnd index the canonical merged
// delete-range slice. The ranges are half-open and intended for future worker
// construction; they do not keep the originating batch alive.
PointOpStart int
PointOpEnd int
DeleteRangeStart int
DeleteRangeEnd int
}
ReadOnlyLeafSpan describes one existing leaf range touched by a canonical sorted apply plan. It contains only in-memory planning metadata; it does not own prepared pager pages, leaf-log records, value-log records, or pending retired pages.
type ReadOnlyLeafSpanSummary ¶ added in v0.6.0
type ReadOnlyLeafSpanSummary struct {
// Ops is the logical root apply operation count: canonical point ops plus
// canonical delete ranges.
Ops int
PointOps int
DeleteRanges int
// SpanOps and SpanBytes are summed over spans. SpanOps may exceed Ops when a
// delete range overlaps multiple leaf spans.
SpanOps int
SpanBytes int
Spans int
ExactLeafSpans bool
ColdBuild bool
Maintenance bool
// MinSpanOps/MaxSpanOps and MinSpanBytes/MaxSpanBytes are zero when Spans is
// zero.
MinSpanOps int
MaxSpanOps int
MinSpanBytes int
MaxSpanBytes int
SingleOpSpans int
OpenLowSpans int
OpenHighSpans int
}
ReadOnlyLeafSpanSummary is a compact, allocation-free summary of a read-only leaf-span plan. It is intended for callers and benchmarks that need to report span distribution without requiring each caller to walk or retain the span slice.
type ReadOnlyLeafSpanWorkerRange ¶ added in v0.6.0
ReadOnlyLeafSpanWorkerRange assigns a contiguous range of read-only leaf spans to one future worker. The range is a planning primitive only; it does not imply parallel execution or prepared output ownership.
type ReadOnlyLeafSpanWorkerRangeSummary ¶ added in v0.6.0
type ReadOnlyLeafSpanWorkerRangeSummary struct {
TargetWorkers int
Ranges int
Ops int
Bytes int
MinRangeOps int
MaxRangeOps int
MinRangeBytes int
MaxRangeBytes int
SingleSpanRanges int
}
ReadOnlyLeafSpanWorkerRangeSummary is an allocation-free aggregate of the deterministic worker ranges derived from a read-only leaf-span plan.
type ReadOnlyPrepareOptions ¶ added in v0.6.0
type ReadOnlyPrepareOptions struct {
// OmitKeys skips copying span boundary and first/last op key bytes for
// point-only plans. It is for hot-path aggregate/chunk planning that only
// needs op indexes/counts; callers that consume exact span bounds for future
// execution should leave it false. Plans containing delete ranges retain keys
// because range-overlap planning needs exact span bounds.
OmitKeys bool
// DiscardLeafSpans streams spans to LeafSpanCallback without retaining them in
// the result. It is only for aggregate/chunk planning callers that do not need
// ValidateLeafSpans or exact span ownership after PrepareReadOnlyPlan returns.
DiscardLeafSpans bool
LeafSpanCallback func(ReadOnlyLeafSpan)
// AllowMaintenancePointLeafSpans marks point-delete maintenance plans as exact
// prepared-output ownership for ordered-root span-native apply. Normal raw DB
// maintenance leaves this false so the prepared spans remain planning hints.
AllowMaintenancePointLeafSpans bool
// contains filtered or unexported fields
}
ReadOnlyPrepareOptions configures a read-only root preparation pass. The zero value is the normal caller-constructed form. Non-zero buffer reuse options are produced by ReadOnlyPrepareResult.ReuseOptions.
type ReadOnlyPrepareResult ¶ added in v0.6.0
type ReadOnlyPrepareResult struct {
RootID uint64
Ops int
PointOps int
DeleteRanges int
ColdBuild bool
Maintenance bool
OmitKeys bool
// ExactLeafSpans is true when LeafSpans fully describe the existing leaves
// touched by the delta. Delete-containing maintenance can merge/rebalance
// adjacent leaves, so its direct key spans are useful planning hints but are
// not complete prepared-output ownership. Range deletes do not enable
// maintenance in the current apply path, so their overlapping leaf spans are
// exact for planning purposes.
ExactLeafSpans bool
LeafSpans []ReadOnlyLeafSpan
Metrics adaptive.Metrics
// contains filtered or unexported fields
}
ReadOnlyPrepareResult is the read-only portion of a root apply attempt. It is safe to discard on root mismatch because it has not allocated or persisted output pages.
func (ReadOnlyPrepareResult) AppendLeafSpanWorkUnitRanges ¶ added in v0.6.0
func (r ReadOnlyPrepareResult) AppendLeafSpanWorkUnitRanges(dst []ReadOnlyLeafSpanWorkerRange, workers int) []ReadOnlyLeafSpanWorkerRange
AppendLeafSpanWorkUnitRanges appends one contiguous span-native work unit per active worker to dst. The active worker count is clamped by the non-empty prepared leaf spans, ranges preserve span order, and no empty range is emitted. Outputs still publish by original span index.
func (ReadOnlyPrepareResult) AppendLeafSpanWorkerRanges ¶ added in v0.6.0
func (r ReadOnlyPrepareResult) AppendLeafSpanWorkerRanges(dst []ReadOnlyLeafSpanWorkerRange, workers int) []ReadOnlyLeafSpanWorkerRange
AppendLeafSpanWorkerRanges appends deterministic contiguous span partitions to dst. It creates at most workers ranges and never creates empty ranges. The returned ranges preserve span order and are suitable for future parallel preparation steps that still need serial output append and assembly order. No-op inputs return dst unchanged.
func (ReadOnlyPrepareResult) LeafSpanSummary ¶ added in v0.6.0
func (r ReadOnlyPrepareResult) LeafSpanSummary() ReadOnlyLeafSpanSummary
LeafSpanSummary returns an allocation-free aggregate view of r's leaf spans.
func (ReadOnlyPrepareResult) LeafSpanWorkerRangeSummary ¶ added in v0.6.0
func (r ReadOnlyPrepareResult) LeafSpanWorkerRangeSummary(workers int) ReadOnlyLeafSpanWorkerRangeSummary
LeafSpanWorkerRangeSummary returns an aggregate of the deterministic worker ranges for workers without retaining the ranges themselves.
func (*ReadOnlyPrepareResult) ResetForReuse ¶ added in v0.6.0
func (r *ReadOnlyPrepareResult) ResetForReuse()
ResetForReuse clears result metadata while retaining bounded reusable leaf-span and key-arena buffers for a later ReuseOptions call. Oversized buffers are dropped so temporary prepare pools do not retain one-off large batch state indefinitely.
func (ReadOnlyPrepareResult) ReuseOptions ¶ added in v0.6.0
func (r ReadOnlyPrepareResult) ReuseOptions() ReadOnlyPrepareOptions
ReuseOptions returns buffers from r for a later read-only preparation pass. The returned options must not be used while r's LeafSpans are still needed.
func (ReadOnlyPrepareResult) ValidateLeafSpans ¶ added in v0.6.0
func (r ReadOnlyPrepareResult) ValidateLeafSpans() error
ValidateLeafSpans checks the deterministic planning invariants for r's read-only leaf-span view. It is intended for tests and future prepared-output callers that want to assert a plan before using it. ApplyWithOptions and DB wrappers validate opt-in plans before apply/publish so invalid plans fail closed before durable output is produced.
type Zipper ¶
type Zipper struct {
// contains filtered or unexported fields
}
func (*Zipper) Apply ¶
func (z *Zipper) Apply(rootID uint64, b *batch.Batch) (newRootID uint64, retiredPages []uint64, metrics adaptive.Metrics, err error)
Apply applies the batch to the tree rooted at rootID. Returns the new root page ID, list of retired pages, and commit metrics.
func (*Zipper) ApplyWithOptions ¶ added in v0.6.0
func (z *Zipper) ApplyWithOptions(rootID uint64, b *batch.Batch, opts ApplyOptions) (ApplyResult, error)
ApplyWithOptions applies the batch to the tree rooted at rootID and returns a result object suitable for guarded install paths. When opts.PrepareReadOnly is true, or when opt-in parallel apply needs span gating, it first runs PrepareReadOnly and validates the plan before applying the delta. If read-only preparation or validation fails, no root apply runs and no durable output ownership is produced.
func (*Zipper) CloneWithAllocator ¶
func (z *Zipper) CloneWithAllocator(a PageAllocator) *Zipper
CloneWithAllocator returns a zipper that shares config/pager with z but uses the provided allocator.
func (*Zipper) PrepareReadOnly ¶ added in v0.6.0
func (z *Zipper) PrepareReadOnly(rootID uint64, b *batch.Batch, opts ReadOnlyPrepareOptions) (ReadOnlyPrepareResult, error)
PrepareReadOnly discovers the existing leaf spans touched by b without allocating or writing pager/leaf-log output. It is the safe preparation phase that future prepared-output paths can run before the final install section.
func (*Zipper) PrepareReadOnlyPlan ¶ added in v0.6.0
func (z *Zipper) PrepareReadOnlyPlan(rootID uint64, ops []batch.Entry, ranges []batch.DeleteRange, opts ReadOnlyPrepareOptions) (ReadOnlyPrepareResult, error)
PrepareReadOnlyPlan discovers the existing leaf spans touched by an already canonical sorted apply plan without first materializing a batch. Point ops must be sorted in apply order, and ranges must be the canonical merged range slice that batch.ApplyPlan would have produced. It is side-effect-free like PrepareReadOnly and exists for hot planning paths that already own canonical op slices.
func (*Zipper) SetAdaptiveLeafEncoding ¶ added in v0.4.0
func (*Zipper) SetFillTargets ¶
SetFillTargets configures soft-full thresholds for newly written pages. Targets are in parts-per-million where 1_000_000 means "allow full pages".
func (*Zipper) SetIndexColumnarLeaves ¶ added in v0.2.0
func (*Zipper) SetIndexInternalBaseDelta ¶ added in v0.2.0
func (*Zipper) SetIndexPackedValuePtr ¶ added in v0.3.0
func (*Zipper) SetLeafPageLog ¶ added in v0.4.0
func (z *Zipper) SetLeafPageLog(log LeafPageLog)
func (*Zipper) SetLeafPageReader ¶ added in v0.4.0
func (z *Zipper) SetLeafPageReader(reader LeafPageReader)
func (*Zipper) SetLeafPrefixCompression ¶
func (*Zipper) SetMaintenanceOpsPerCoalesce ¶ added in v0.3.0
SetMaintenanceOpsPerCoalesce sets the approximate ops-per-maintenance ratio. Values <= 0 disable maintenance budgeting (full coalesce behavior).
func (*Zipper) SetOuterLeavesInValueLog ¶ added in v0.4.0
func (*Zipper) SetParallelMergePressureSource ¶ added in v0.4.0
func (z *Zipper) SetParallelMergePressureSource(src ParallelMergePressureSource)
SetParallelMergePressureSource configures an optional pressure signal for the internal-node merge fan-out gate. Nil preserves the baseline thresholds.