Documentation
¶
Overview ¶
Package memlimit reports the memory budget the process actually has, so a size derived from it (the merge cap) tracks the container it runs in rather than a constant that is right at exactly one deployment size.
It is deliberately dependency-free: the answer comes from GOMEMLIMIT, the cgroup, or the host's total memory, in that order of authority.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Bytes ¶
func Bytes() int64
Bytes returns the memory the process may use, or 0 when nothing reports one.
GOMEMLIMIT wins when set: it is what the embedder promised the Go heap, and an embedder that applies a cgroup limit to it (automemlimit and friends) has already reserved its own headroom. Otherwise the cgroup limit is used, since a container's limit — not the host's RAM — is what the kernel kills against, and the host total is the last resort.
func MergeBudget ¶ added in v0.48.0
MergeBudget is how many bytes every concurrent merge may hold together: the caller's configured allowance, or a share of the detected process budget (Bytes) when it is 0. A negative value opts out, returning math.MaxInt64 so the caller's other bounds decide alone.
func MergeConcurrency ¶ added in v0.48.0
MergeConcurrency is how many merges the budget supports: enough that each gets at least [minMergeBytes], never more than cpuLimit, never fewer than one.
Deriving it from memory rather than from cores is the point. A merge's *allowance* is the budget divided by this number, so taking the divisor from the core count prices a memory quantity in CPUs: a 16-core pod with a 4 GiB limit would give each merge 32 MiB and produce a great many merges too small to keep up with ingest. cpuLimit still caps it from above — merging is CPU-bound work, and there is nothing to gain from more merges than there are cores to run them.
func MergeShare ¶
MergeShare is how many bytes one merge may hold: MergeBudget divided across the merges that may run at once and by amplification, the peak resident per byte of the bound (a merge that holds its output twice over passes 2).
func QueryShare ¶ added in v0.43.0
QueryShare is how many bytes one query may read before it is refused, derived from the process budget the same way MergeShare derives the merge allowance.
configured is the caller's cap: 0 takes a share of the detected process budget (Bytes), and a negative value opts out. Opting out returns 0 rather than math.MaxInt64, because the consumer spells "unbounded" as "install no limiter at all" — an effectively-infinite ceiling would still pay to count every byte it will never reject.
Types ¶
type Pool ¶ added in v0.48.0
type Pool struct {
// contains filtered or unexported fields
}
Pool admits work by the bytes it intends to hold, so a budget divided across concurrent merges is one they actually have.
Without it the per-merge share is a promise nobody keeps: the facade fans merges out on its own schedule, every one of them sizes itself against "the budget divided by the concurrency", and nothing stops more than that many from running. The division then understates what a merge may hold *and* overstates how many may hold it — wrong in both directions at once.
A zero or negative total, and the nil Pool, admit everything: an embedder that opts out of the memory bound gets no gate rather than a gate of size zero.
func (*Pool) Acquire ¶ added in v0.48.0
Acquire blocks until n bytes are free, then returns the function that hands them back. The release is idempotent, so `defer release()` is the correct use.
A request larger than the whole pool is clamped to it: such a merge runs alone rather than never, because a bound that can deadlock its own work is worse than one that is occasionally exceeded.
Waiters are served strictly in arrival order, so a queued large request delays smaller ones behind it rather than being starved by them. Requests are near-uniform — each caller asks for its share of one budget — but not identical: the share is divided by a concurrency that rises as engines appear, so a waiter queued when the process held one engine asks for more than one queued later.
func (*Pool) Total ¶ added in v0.48.0
Total reports the pool's size, 0 for a pool that admits everything.
func (*Pool) TryAcquire ¶ added in v0.48.0
TryAcquire takes n bytes if they are free right now, reporting whether it got them. It never queues, so a caller on a shared goroutine — the maintenance loop, which also services flush pressure — can decline the work and come back next cycle instead of parking everything behind it.
Source Files