Documentation
¶
Overview ¶
Package diskio bounds the page-cache impact of the worker's large sequential file writes (operator spill files, local cache downloads, stage-output files).
Mechanism: windowed asynchronous writeback (PostgreSQL's checkpoint_flush_after, RocksDB's bytes_per_sync — their DEFAULT, non-strict shapes). As each window of windowBytes completes, asynchronous writeback is started for it (sync_file_range(SYNC_FILE_RANGE_WRITE)), so dirty pages reach the device within one window of being written instead of waiting on the ~30s kernel flusher. Steady-state dirty footprint per writer ≈ write-rate × device-latency. There is deliberately no per-window blocking wait — the strict variant (RocksDB's off-by-default strict_bytes_per_sync) serialized multi-GB cache downloads behind device writeback at SF100 and regressed exchange-heavy queries 50-110% (run 20260610-203304); see Flusher.wrote.
Why dirty pages specifically: they cannot be reclaimed until written back, so a multi-GB write flood (cache downloads + spill) forces kernel reclaim to evict whatever IS cheaply reclaimable — the clean mmap'd cache-file pages concurrent tasks are still walking. That eviction is the major-fault mechanism behind the PR #112 regression (project_mmap_selfrelief_postmortem_2026-06-10); bounding the write side attacks the cause without any reader coordination.
Class picks the fate of written-back (now clean) windows:
- Spill: read back at most once, much later — each clean window is FADV_DONTNEED'd immediately, so spill bytes never compete with the mmap'd cache pages at all. Read-back streams from NVMe.
- KeepResident: mmap-walked or uploaded right after the write — the windows stay resident (only the dirty bound applies), so the imminent readers take minor faults, not majors.
The write mechanism is gated on --bounded-dirty-writes (default true since the SF100 suite-neutral validation; =false restores kernel-writeback-only), with Spill-class writers additionally kept active by the drop-behind default below. When fully disabled, NewWriter returns the file itself and a nil Flusher, so the only cost is one atomic load at writer construction.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func AddReadDropBytes ¶
func AddReadDropBytes(n int64)
AddReadDropBytes records n bytes dropped behind a single-pass read by a caller that issues its own advise calls (the worker's shuffle mmap walk).
func DropBehindEnabled ¶
func DropBehindEnabled() bool
DropBehindEnabled reports whether single-pass drop-behind is active.
func DropBehindStats ¶
func DropBehindStats() (write, read int64)
DropBehindStats returns cumulative dropped bytes on the write and read sides since process start.
func NewDropBehindReader ¶
NewDropBehindReader wraps f for drop-behind reading. Returns f itself when drop-behind is disabled, so call sites need no branches. The caller keeps ownership of f (Close as usual).
func SetDropBehindEnabled ¶
func SetDropBehindEnabled(on bool)
SetDropBehindEnabled toggles single-pass drop-behind; tests only.
func SetEnabled ¶
func SetEnabled(on bool)
SetEnabled activates or deactivates windowed writeback for writers created afterwards. Called once from worker startup (the --bounded-dirty-writes flag); tests may toggle it.
Types ¶
type Class ¶
type Class int
Class describes how a file's pages are treated once written back.
const ( // Spill marks files written now and read back at most once, later: // sort/window runs, join build/probe partitions, aggregate partial // state, raw-row spill. Spill Class = iota // KeepResident marks files whose pages are wanted immediately after // the write: local cache downloads that are mmap'd and walked, and // stage outputs that are uploaded (and possibly adopted into the // LocalStageCache and mmap'd by a same-worker consumer) right after // Finalize. KeepResident )
type DropBehindCursor ¶
type DropBehindCursor struct {
// contains filtered or unexported fields
}
DropBehindCursor is offset-driven drop-behind for readers that consume a single-pass file via out-of-band preads delivered in ascending order (the WSHF extent-index decode path — docs/design/shuffle-extent-index.md): the reads happen on worker goroutines, but delivery is strictly in order, so the delivery cursor is the safe drop watermark. Same window shape, counters, and advisory-failure latch as DropBehindReader.
func NewDropBehindCursor ¶
func NewDropBehindCursor(f *os.File) *DropBehindCursor
NewDropBehindCursor returns a cursor for f, or nil when drop-behind is disabled. A nil cursor's Advance is a no-op, so call sites need no branches.
func (*DropBehindCursor) Advance ¶
func (c *DropBehindCursor) Advance(off int64)
Advance marks everything below off consumed and drops completed windows behind it, keeping the window containing off resident.
type DropBehindReader ¶
type DropBehindReader struct {
// contains filtered or unexported fields
}
DropBehindReader reads a single-pass file sequentially and drops consumed pages from the page cache one window behind the read cursor, so multi-GB spill read-backs never displace reusable pages. On EOF the whole file is dropped (covers the tail window). Advisory only: FADV errors disable further calls but never affect the data read.
type Flusher ¶
type Flusher struct {
// contains filtered or unexported fields
}
Flusher applies the two-window writeback pattern to one sequentially written file. Not goroutine-safe: every write site already serializes writes per file (bufio behind a mutex, or a single goroutine).
func NewWriter ¶
NewWriter wraps f so windowed writeback tracks every Write. All sequential content writes must go through the returned writer; non-sequential header patches may go to f directly (Finish covers them with a whole-file pass). When the mechanism is disabled it returns f itself and a nil Flusher (whose Finish is a no-op), so call sites need no branches.
Spill-class writers are additionally active whenever drop-behind is on (the default), independent of --bounded-dirty-writes: a Spill file is single-pass by definition, so dropping its written-back windows is the steady-regime fix, not a memory-tight tuning knob. KeepResident writers (imminently re-read files) stay gated on the flag.
func (*Flusher) Finish ¶
func (fl *Flusher) Finish()
Finish completes the page-cache treatment for the file. Call it after the final content write — including any header patch made directly on the file — and before the file is closed. For Spill files it writes back whatever is still dirty (the tail plus any patched header page) and drops the entire file from cache; KeepResident files keep their pages, so it is a no-op. Errors are swallowed: the machinery is advisory and never affects file contents. nil-safe.
Source Files