Documentation
¶
Overview ¶
Package compute provides vectorized numeric kernels used by gobi's compute layer. Kernels dispatch to a SIMD implementation when built with `GOEXPERIMENT=simd` on a supported architecture (arm64 NEON, amd64 AVX2/AVX-512), and to a portable scalar implementation everywhere else.
Positioning vs arrow-go/arrow/compute ¶
arrow-go ships its own compute package (a "native-go Acero-like engine") at `arrow.compute`. Its Datum-based function-registry API is much more general than gobi's typed-slice kernels — it handles chunked arrays, arbitrary arrow types, promotion rules, null propagation, and overflow modes uniformly. That generality costs per-row overhead.
Measured on arm64 (Apple M3 Pro, arrow-go v18.7.0, Go 1.27rc1) via `benchmarks/arrow_compute/main.go`:
Float64 Add, n=10M rows gobi Series.Add: 755 ps/row (1324 Mrows/s) arrow compute.Add: 1570 ps/row ( 637 Mrows/s) 2.1× slower Float64 → Int64 Cast, n=10M rows gobi Cast: 3838 ps/row ( 261 Mrows/s) arrow CastArray: 275 ps/row (3629 Mrows/s) 13.9× faster
Two entirely different stories:
For Add (and any arithmetic), the 2× gap holds at 100K, 1M, and 10M rows. That's per-row dispatch cost inside arrow-go's `exec.ArrayKernelExec` + `ExecSpan` traversal, not a flat setup tax that amortizes at large N. On amd64, arrow-go's hand-written AVX2 SIMD kernels close some of that gap on arithmetic, but a 4-lane float64 SIMD win (theoretical 4×) vs a 2× per-row dispatch penalty nets out to a wash — not a transformative win.
For Cast, arrow-go's hand-written NEON SIMD kernel (`internal/kernels/cast_numeric_neon_arm64.s`) crushes gobi's scalar builder loop 14× on arm64. The batch nature of casts amortizes the dispatch overhead; the SIMD kernel dominates the runtime.
Overlap surface (measured, not aspirational):
category arrow-go/compute gobi
------------------------ --------------------- --------------------
Add / Sub / Mul / Div amd64 SIMD (.s files); keep gobi's — arrow-go's
arm64 scalar per-row dispatch is 2× overhead
Type casts amd64 + arm64 SIMD ✅ use arrow-go's — 14× faster
on arm64 (measured)
Constant-factor scalar amd64 SIMD use arrow-go's on amd64
Compare (Eq/Ne/Lt/…) amd64 SIMD (.s files); keep gobi's — measured 1.4×
arm64 scalar slower on arm64 via arrow-go
Fused compare chains absent unique (BBox, Range,
WithinSqDist)
Reductions (Sum/Min/Max) absent unique when landed
arm64 SIMD arithmetic absent landed in v0.4.0
(portable simd, Go 1.27+)
Takeaway ¶
gobi/compute isn't a replacement for arrow-go/compute — different design points. arrow-go/compute is the right choice for one-shot batch operations where the Datum overhead is amortized across the batch AND where the general-arrow-type dispatch actually matters. gobi/compute is the right choice for the tight-inner- loop shapes on the LazyFrame hot path: comparisons, fused compare chains, and (eventually) reductions — the operations that recur per-row in filter/groupby/aggregate pipelines where 2× per-row dispatch overhead is prohibitive.
Survey done in v0.3.9: measured Add / Ge / Cast against arrow-go/compute at n = 100K, 1M, 10M. Cast was the sole clear win (13.9× on arm64 via `cast_numeric_neon_arm64.s`) and got wired into `Expr.Cast`. Everything else in arrow-go's kernel surface either has SIMD only on amd64 (arithmetic, compare, constant-factor mul — Datum dispatch narrows the SIMD win to a wash) or has no SIMD at all (Boolean ops, string compare, rounding, sort, filter, hash, set lookup, temporal casts — scalar Go behind a 2× dispatch tax). arrow-go has NO aggregate/reduction kernels at all — Sum/Min/Max/Mean/etc. remain gobi's territory.
Surface stability ¶
The compute package is INTERNAL to gobi. Its API is not covered by the top-level gobi module's SemVer guarantees — kernels are added, renamed, or specialized without notice. Callers outside gobi should use the higher-level Series / Frame / Expr surface instead.
Build tags ¶
The SIMD path lives behind `//go:build goexperiment.simd && (arm64 || amd64)`. As of Go 1.27 (August 2026) the stdlib `simd` package shipped but remains behind `GOEXPERIMENT=simd` — code linked without the flag gets the scalar path. This file provides the entry points; specific implementations live in *_simd.go / *_scalar.go pairs alongside. When the experiment eventually graduates the tag simplifies to just the arch check.
Adding a kernel ¶
Every kernel is a plain function with typed slice arguments and a `[]bool` or scalar output. No arrow types, no gobi types — that keeps the SIMD-friendly inner loops free of any indirection the compiler can't lower. Callers convert to/from arrow at the package boundary.
Kernel signature convention:
Cmp<Type><Op>(a []<Type>, b <Type>, out []bool)
element-wise compare against a scalar; writes into out.
Cmp<Type><Op>Vec(a, b []<Type>, out []bool)
element-wise compare against a same-length column.
Sum<Type>(a []<Type>) <Type>
Min<Type>(a []<Type>) (<Type>, bool) — bool = "saw a value"
Max<Type>(a []<Type>) (<Type>, bool)
Before adding an arithmetic-shaped kernel here, check whether arrow-go/compute already covers it — most likely it does with better amd64 SIMD than we could match. Fused shapes and comparisons are our unique territory.
Every kernel is safe to call with zero-length inputs and produces the identity value (0 / (0,false) / no-op writes).
Index ¶
- func AndChainF64BBox(a []float64, aLo, aHi float64, b []float64, bLo, bHi float64, out []bool)
- func AndChainF64Range(a []float64, lo, hi float64, out []bool)
- func BoundsF64(xs, ys []float64) (minX, minY, maxX, maxY float64, ok bool)
- func CmpF64Ge(a []float64, b float64, out []bool)
- func CmpF64Gt(a []float64, b float64, out []bool)
- func CmpF64Le(a []float64, b float64, out []bool)
- func CmpF64Lt(a []float64, b float64, out []bool)
- func CmpI64Ge(a []int64, b int64, out []bool)
- func CmpI64Gt(a []int64, b int64, out []bool)
- func CmpI64Le(a []int64, b int64, out []bool)
- func CmpI64Lt(a []int64, b int64, out []bool)
- func CountTrue(a []bool) int
- func Enabled() bool
- func MaxF64(a []float64) (float64, bool)
- func MaxI64(a []int64) (int64, bool)
- func MinF64(a []float64) (float64, bool)
- func MinI64(a []int64) (int64, bool)
- func PIPCrossingCount(xs, ys []float64, tx, ty float64) bool
- func PolygonCentroidShoelace(xs, ys []float64) (cx, cy float64, ok bool)
- func SumF64(a []float64) float64
- func SumI64(a []int64) int64
- func WithinSqDistF64(lats, lons []float64, refLat, refLon, cosRefLat, sqThreshold float64, ...)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func AndChainF64BBox ¶
AndChainF64BBox writes
out[i] = (aLo <= a[i] <= aHi) && (bLo <= b[i] <= bHi)
in a single pass — the canonical 2D bbox filter shape. Fusing both columns' comparisons into one kernel avoids the intermediate []bool that a "per-column primitive + scalar AND" composition would allocate. Callers that just want a 1D range should use AndChainF64Range instead.
a and b must have equal length; out must have len >= len(a). Panics on length mismatch (invariant guaranteed by callers operating on same-frame columns).
func AndChainF64Range ¶
AndChainF64Range writes out[i] = (lo <= a[i]) && (a[i] <= hi). Fused two-sided range check — the shape most bbox filters produce. Short-circuits per element on the low side to skip the high compare on out-of-range rows.
func BoundsF64 ¶ added in v0.4.1
BoundsF64 computes the axis-aligned bounding box of the points held in parallel Xs / Ys slices. Returns (minX, minY, maxX, maxY, ok=true) when both slices are non-empty; ok=false when either is empty. Mismatched slice lengths are caller error; the kernel derives bounds from the shorter slice.
Semantics match geometry.BoundsFromXY exactly. This is the scalar back-end; the SIMD version in geom_simd.go uses simd.Float64s.Min / .Max for a ~3-4× throughput win on large input.
func CmpF64Ge ¶
CmpF64Ge writes out[i] = a[i] >= b for i in [0, len(a)). out must have len >= len(a); extra tail is left untouched. Callers are expected to pre-allocate out sized to len(a).
func CmpI64Ge ¶ added in v0.4.1
CmpI64Ge writes out[i] = a[i] >= b. Same shape as CmpF64Ge but on int64 columns. Signatures match cmp_simd.go — SIMD build vectorizes the compare via simd.Int64s.GreaterEqual + mask store.
func CountTrue ¶ added in v0.4.1
CountTrue returns the number of true entries in a. Foundational bool-reduce kernel used by filter mask sizing (`Frame.Filter` can allocate the keep-index slice at exact selectivity instead of over-allocating to full mask length) and any bool-column count/sum reduction.
Scalar loop over `bool` values — Go's `bool` is a distinct type, not a byte to be truth-tested. The SIMD file (cmp_simd.go) shares this implementation today; if a benchmark ever justifies it, a hand-written popcount would go there.
func Enabled ¶
func Enabled() bool
Enabled reports whether this build has SIMD kernels compiled in. True when built with `GOEXPERIMENT=simd` on arm64 or amd64; false otherwise (portable scalar fallback active). Callers don't need to branch on this — kernel dispatch happens at build time — but tests and benchmarks may want to log which path is running.
func MinF64 ¶
MinF64 returns (min, true) when a is non-empty, else (0, false). NaN semantics match Go's built-in `<`: NaN comparisons return false, so a first-seen NaN sticks unless a later non-NaN value is strictly less. Empty groups return (0, false).
func PIPCrossingCount ¶ added in v0.4.1
PIPCrossingCount — scalar back-end. Delegates to the shared pipCrossingCountScalar helper in geom_common.go. SIMD variant with a lane-parallel body lives in geom_simd.go.
Semantics: returns the parity of ray-crossings when casting a horizontal ray from (tx, ty) rightward through the polygon ring defined by parallel Xs / Ys. inside=true when the crossing count is odd; false when even (or when the ring has fewer than 3 points).
The reformulated crossing-count form (running `crossings int` accumulator, `inside = (crossings & 1) == 1` at the tail) breaks the scalar `inside = !inside` dependency chain used in geometry.PIPRingFromXY. Output matches the AoS toggle exactly.
Handles closed and unclosed rings via the same (n-1, 0) closing-edge walk PIPRingFromXY uses.
func PolygonCentroidShoelace ¶ added in v0.4.1
PolygonCentroidShoelace — scalar back-end. Delegates to the shared polygonCentroidShoelaceScalar helper in geom_common.go. SIMD variant with a lane-parallel body lives in geom_simd.go.
Semantics: shoelace area-weighted centroid on the input ring. Returns (cx, cy, ok=true) when the ring has ≥3 points; (0, 0, false) when it has fewer. When areaTwo == 0 (degenerate zero-area ring) the arithmetic-mean-of-segment-starts fallback is returned via (sxFallback, syFallback, true).
Handles both closed and unclosed rings — a closing edge from (last, first) is added iff last != first.
func SumI64 ¶
SumI64 returns the sum of every element in a as int64. Overflow wraps (matches Go's built-in `+` semantics). Empty input returns 0.
func WithinSqDistF64 ¶
func WithinSqDistF64(lats, lons []float64, refLat, refLon, cosRefLat, sqThreshold float64, out []bool)
WithinSqDistF64 writes
out[i] = ((lats[i]-refLat)² + ((lons[i]-refLon)·cosRefLat)²) <= sqThreshold
— the equirectangular-approximation "point within radius r of (refLat, refLon)" filter. Compute-heavy shape: 2 subtractions + 2 squarings + 1 multiply-by-scaling + 1 add + 1 compare per row, all fully vectorizable. Callers precompute cosRefLat once (avoids per-row trig) and pass the squared threshold to save a sqrt in the inner loop.
Accuracy: equirect approximation is fine for distances small relative to Earth's radius (< a few hundred km). For global distances use a proper haversine impl (not SIMD-friendly on the current simd.Float64s surface — no atan2/trig).
lats and lons must have equal length; out must have len >= len(lats).
Types ¶
This section is empty.
Source Files