Documentation
¶
Overview ¶
Package pool provides cross-cutting sync.Pool helpers, capacity-bucketed byte pools, and arena allocation for same-lifetime batches (DESIGN.md §10). Not yet implemented.
Index ¶
- func DefaultCapacity(floor int) int
- type ByteIntMap
- func (m *ByteIntMap) Delete(b []byte) bool
- func (m *ByteIntMap) ForEach(fn func(key []byte, value int))
- func (m *ByteIntMap) Get(b []byte) (int, bool)
- func (m *ByteIntMap) Len() int
- func (m *ByteIntMap) Put(b []byte, v int) (int, bool)
- func (m *ByteIntMap) PutBack()
- func (m *ByteIntMap) PutOrGet(b []byte, v int) (int, bool)
- func (m *ByteIntMap) PutRaw(b []byte, v int, h uint64)
- func (m *ByteIntMap) Reset()
- type FreeList
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DefaultCapacity ¶ added in v0.16.0
DefaultCapacity returns a sensible FreeList capacity for a path bounded by per-fetch parallelism: the greater of runtime.GOMAXPROCS(0) and floor. It covers the expected peak of in-flight buffers (one per concurrent decode) without over-retaining.
Types ¶
type ByteIntMap ¶
type ByteIntMap struct {
// contains filtered or unexported fields
}
ByteIntMap is an open-addressing hash map from []byte keys to int values, using xxh3 for hashing and linear probing. It is designed for the dictionary-encoding hot path ([chunk.EncodeBytes]) where Go's built-in map[string]int is bottlenecked by string-key overhead and a slower hash function.
The zero value is not usable; create one with NewByteIntMap. [Reset] clears it for reuse without freeing the backing arrays (so a pooled instance amortizes allocation). Not safe for concurrent use; callers own synchronization or pool per-goroutine.
Capacity is rounded up to a power of two; the load factor is ≤ 0.75 (probed slots) before [grow] is triggered. Keys are compared with bytes.Equal (no string conversion).
func NewByteIntMap ¶
func NewByteIntMap() *ByteIntMap
NewByteIntMap returns a pooled, ready-to-use map. A reused instance is cleared via ByteIntMap.Reset so its backing arrays are retained; a fresh one allocates at the initial capacity.
func (*ByteIntMap) Delete ¶
func (m *ByteIntMap) Delete(b []byte) bool
Delete removes key b. Returns true if it was present.
func (*ByteIntMap) ForEach ¶
func (m *ByteIntMap) ForEach(fn func(key []byte, value int))
ForEach calls fn for each (key, value) pair. Iteration order is unspecified.
func (*ByteIntMap) Get ¶
func (m *ByteIntMap) Get(b []byte) (int, bool)
Get returns the value for key b and true if present.
func (*ByteIntMap) Len ¶
func (m *ByteIntMap) Len() int
Len returns the number of live entries in the map.
func (*ByteIntMap) Put ¶
func (m *ByteIntMap) Put(b []byte, v int) (int, bool)
Put inserts or updates key b → v. Returns the old value and true if the key existed, or (v, false) for a new insertion.
func (*ByteIntMap) PutBack ¶
func (m *ByteIntMap) PutBack()
PutBack returns m to the pool for reuse. After this, m must not be used.
func (*ByteIntMap) PutOrGet ¶
func (m *ByteIntMap) PutOrGet(b []byte, v int) (int, bool)
PutOrGet inserts b → v if b is absent; otherwise returns the existing value and false. This is the single-lookup dedup path for dictionary building: one probe chain either finds the existing id or inserts a new one.
func (*ByteIntMap) PutRaw ¶
func (m *ByteIntMap) PutRaw(b []byte, v int, h uint64)
PutRaw inserts with a precomputed hash (internal, for re-insertion after Delete).
func (*ByteIntMap) Reset ¶
func (m *ByteIntMap) Reset()
Reset clears the map for reuse without freeing the backing arrays.
type FreeList ¶ added in v0.16.0
type FreeList[T any] struct { // contains filtered or unexported fields }
FreeList is a GC-stable, bounded recycler for pointers of type T. It is the zero-allocation replacement for sync.Pool in paths where buffers must survive allocation-driven GC bursts.
Problem it solves: sync.Pool is cleared by the runtime at the start of every GC (its per-P caches are dropped, victim caches rotated out over the next cycle). Under sustained allocation pressure — concurrent fetches plus other large allocators — a sync.Pool stays drained, so every Get returns a fresh object and the capacity of its backing slices is lost. The decode-buffer path pays for that with a full chunk.resize reallocation on every fetch (the disk_io profile showed ~38 GB/35 s of resize churn, >50% of CPU in GC).
FreeList holds its entries as ordinary rooted references (a guarded slice), so they are NOT collectable and NOT cleared at GC: a buffer Put back here keeps its capacity across any number of collections until the next Get reclaims it. Capacity bounds resident memory; a Put past capacity drops the pointer (it becomes collectable normally), so the list cannot grow unbounded.
Get returns a recycled pointer or nil when empty — callers allocate on a nil return. The mutex critical section is a slice head/tail op (no allocation), so contention is minimal; for very high fan-in a sharded list can be layered on top.
func NewFreeList ¶ added in v0.16.0
NewFreeList returns a FreeList holding up to capacity recycled pointers. A capacity below 1 disables recycling (Put is a no-op, Get always returns nil) — useful for tests and size-0 fast paths. Size the capacity to the peak number of concurrently live buffers: in-flight buffers that don't fit are dropped and GC'd, keeping memory bounded to roughly capacity × buffer-size.
func (*FreeList[T]) Get ¶ added in v0.16.0
func (p *FreeList[T]) Get() *T
Get returns a recycled *T, or nil when the list is empty. A nil return means the caller should allocate (there is no New constructor — keeping the type literal avoids hiding allocations behind a pool that promises "no allocation").
Source Files