Documentation
¶
Overview ¶
Package execopt carries the execution-time knobs that are not planning decisions: how much memory one query may hold, and where it is allowed to spill.
Why a ledger rather than a counter ¶
Seven operators in ursus buffer, and every one of them retains the SOURCE's batches rather than copies of them. Columns share payloads on top of that — Rename shares one, so does a sliced String column's character buffer. So a counter that adds up what each operator was handed double-counts, and a budget that double-counts fires early and unpredictably, which is worse for a user than having no budget at all.
The ledger below therefore counts ALLOCATIONS, keyed by data.BufferID, with one reference count per allocation across every account. An allocation two sinks hold is one entry, and it is released when the last account lets go.
What it deliberately does not do ¶
It does not hook the allocator. arrowx pins memory.NewGoAllocator, whose Free is a no-op and which reports nothing, so there is no allocation-site seam to hook — and design/physical.md's proposed kernel.Ctx, which would have carried a reservation into every kernel, was never built. Accounting therefore happens where memory is RETAINED, not where it is allocated. That is exactly right for the buffering operators, which are the only things that hold memory across batches, and it is silent about a single kernel's transient output.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Account ¶
type Account struct {
// contains filtered or unexported fields
}
Account is one operator's share of a budget.
It holds each allocation at most once — retaining the same buffer twice does not hold more memory — so Retain is idempotent per allocation and Release is exact.
A nil *Account is a working no-op, which is what lets an operator written against it work unchanged when no budget was configured.
func (*Account) Check ¶
Check returns a resource error naming this operator when the query is over budget. An operator that can spill calls Over instead and spills.
func (*Account) Over ¶
Over reports whether the QUERY is over its ceiling.
Deliberately query-wide rather than per-account: the budget is shared, so an operator that can spill should do so when the query is under pressure, not only when its own share happens to be the large one.
func (*Account) RetainBytes ¶
RetainBytes records plain-Go state that has no BufferID.
This is not a rounding detail. Three of the largest retentions in the engine are not batches at all — windowSink's one int32 per row PER DISTINCT PARTITIONING, joinBuildSink's one int32 per build row, and quantileAcc's []float64 per group — so a walker over batches alone would report a window sink at half its true size.
type Budget ¶
type Budget struct {
// contains filtered or unexported fields
}
Budget is one query's memory ceiling and spill location.
A nil *Budget means unlimited and untracked; every method is nil-safe so that operators need no branch of their own.
func NewBudget ¶
NewBudget returns a budget with the given limit in bytes; a limit <= 0 means unlimited, and the ledger still tracks the peak so a caller can measure what a query actually held.
func (*Budget) Account ¶
Account opens an operator's view of the budget.
op is the user-facing operator name — "sort", "group_by", "join" — because the only useful thing a memory error can say is which operator could not fit.
func (*Budget) Over ¶
Over reports whether the query is above its ceiling. Always false when unlimited.
func (*Budget) Peak ¶
Peak returns the highest retained total the query reached.
This is the number that demonstrates bounded memory. A test that asserts a query completed proves only that it did not run out; asserting the peak proves it stayed under the ceiling.
func (*Budget) SpillDir ¶
SpillDir returns the directory spill files are created under, or "" for the system temporary directory.
Source Files