kernel

package
v0.18.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 24, 2026 License: AGPL-3.0 Imports: 12 Imported by: 0

Documentation

Overview

Package kernel provides type-specialized vectorized operations for the query engine. Generic functions are monomorphized at compile time and dispatch is resolved once at query init time (not per-row), eliminating type-switch overhead from hot loops.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CanonicalFloat32 added in v0.18.1 

func CanonicalFloat32(f float32) float32

CanonicalFloat32 is CanonicalFloat64 for float32.

func CanonicalFloat64 added in v0.18.1 

func CanonicalFloat64(f float64) float64

CanonicalFloat64 / CanonicalFloat32 fold a value onto the one bit pattern the order above treats as canonical for it: every NaN payload onto one NaN, and -0.0 onto +0.0. CompareFloat64/32 call both of those pairs EQUAL, and the standing invariant (ADR-0012 item 8) is that two values the comparator calls equal must also SERIALIZE alike — otherwise a GROUP BY splits one group in two, a hash join misses a pair the comparator matches, or a shuffle routes two equal keys to different partitions and the distributed answer stops agreeing with the single-process one.

func CompareDecimalAt 

func CompareDecimalAt(a *batch.Vector, ai int, b *batch.Vector, bi int) int

CompareDecimalAt orders two DECIMAL values by NUMERIC value, which is what PostgreSQL's `numeric` ordering means and what every other comparator in this file already does for its type. Before this arm existed, DECIMAL fell through the three resolvers' defaults to a comparator that reports every row equal, so `ORDER BY dec_col` was a stable no-op that returned input order, and a sort-merge join on a DECIMAL key matched every row against every row. The other path in the tree — compareAny over Vector.GetValue — compares the FORMATTED string instead, where "10.001" sorts before "2.0002". Same query, three different sequences depending on which path answered (#394).

The comparison is EXACT at every scale. Equal scales compare the unscaled Int128s directly — that is every sort over one column, every sorted run and every k-way merge over runs. Unequal scales, reachable where two separately declared DECIMAL columns meet, rescale the smaller-scale operand by 10^(delta) and compare the unscaled integers; if that product overflows Int128 the two are compared as big.Int rather than approximated.

Exactness is not a nicety here: SortMergeJoin uses this comparator for key EQUALITY (sort_merge_join.go), so an approximate answer is a spurious JOIN MATCH. The float64 rescale this replaced held to 2^53 unscaled units and then started reporting 9007199254740993 and 9007199254740992.0 — which differ by one unscaled unit at the common scale — as the same key.

func CompareDecimalValues added in v0.18.1 

func CompareDecimalValues(av batch.Int128, as int, bv batch.Int128, bs int) int

CompareDecimalValues is CompareDecimalAt on values already read out of their columns — the form the col-col FILTER kernel needs, which reads its two slices once per batch rather than per row. One function so the sort comparator, the sort-merge join key and the filter cannot drift apart.

func CompareFloat32 added in v0.18.1 

func CompareFloat32(a, b float32) int

CompareFloat32 orders two float32 values with NaN greatest and NaN == NaN.

Native float32 comparisons, not a widen-to-float64-and-delegate: widening every element cost ZZSortFloat32NoNulls +2.03% (benchmarked against CompareFloat64(float64(a), float64(b)), the form this replaced) for a rule that needs nothing float64 offers — float32's `<`/`>`/`==`/self-inequality already carry the same PostgreSQL order this function documents for float64.

func CompareFloat64 added in v0.18.1 

func CompareFloat64(a, b float64) int

CompareFloat64 orders two float64 values with NaN greatest and NaN == NaN.

func CompareValuesAt added in v0.18.1 

func CompareValuesAt(a *batch.Vector, ai int, b *batch.Vector, bi int) int

CompareValuesAt orders two values of the same type, WITHOUT consulting either row's null bit. Callers that can see a NULL use compareElemAt (for a container's elements) or one of the resolvers (for a column).

The type comes from a: two vectors reaching one comparator always carry the same type — the sort compares one column against itself and the join's planner gate requires identical key types.

func DecimalConstText added in v0.18.1 

func DecimalConstText(v any) (string, bool)

DecimalConstText renders a comparison constant as the decimal text a DECIMAL column's domain is reached through, and reports whether the constant IS a number.

The second result is not decoration. A constant nobody can read used to resolve to the value ZERO and match every stored zero (#463) — the worst shape a failure can take, because it neither errors nor returns nothing. PostgreSQL refuses the query instead ("invalid input syntax for type numeric"), and ADR-0012 makes PostgreSQL the authority on error-versus-not, so a false here is a query error at the caller, not a value.

Exponent form is passed through untouched: batch.DecimalTextAt folds the exponent into the scaling exactly, where expanding it through a float64 first is what lost 1e400 entirely.

func FloatCompareOp added in v0.18.1 

func FloatCompareOp[T FloatOrdered](a, b T, op CompareOp) bool

FloatCompareOp applies one of the six predicates to a pair. The row-at-a- time paths (exec's ColumnCompare fallback, expr's CmpFloat64) use this so they answer what the vectorized kernel answers; the kernels themselves resolve the operator ONCE and keep the per-row form above.

func FloatEq added in v0.18.1 

func FloatEq[T FloatOrdered](a, b T) bool

FloatEq reports a = b under PostgreSQL's float order (NaN equals NaN).

func FloatGe added in v0.18.1 

func FloatGe[T FloatOrdered](a, b T) bool

FloatGe reports a >= b under PostgreSQL's float order.

func FloatGt added in v0.18.1 

func FloatGt[T FloatOrdered](a, b T) bool

FloatGt reports a > b under PostgreSQL's float order.

func FloatLe added in v0.18.1 

func FloatLe[T FloatOrdered](a, b T) bool

FloatLe reports a <= b under PostgreSQL's float order.

func FloatLt added in v0.18.1 

func FloatLt[T FloatOrdered](a, b T) bool

FloatLt reports a < b under PostgreSQL's float order (NaN is greatest).

func FloatNe added in v0.18.1 

func FloatNe[T FloatOrdered](a, b T) bool

FloatNe reports a <> b under PostgreSQL's float order.

`a == a || b == b` rather than `!(a != a && b != b)`: the two are the same predicate, but this one short-circuits on the FIRST operand for every non-NaN row, which is the row the branch predictor sees.

func KeyFloat32Bits added in v0.18.1 

func KeyFloat32Bits(f float32) uint32

KeyFloat32Bits is KeyFloat64Bits for float32.

func KeyFloat64Bits added in v0.18.1 

func KeyFloat64Bits(f float64) uint64

KeyFloat64Bits / KeyFloat32Bits are Float64bits/Float32bits over the canonical value — the bits any KEY, hash or partition router should use for a float.

func StatsDomainValue added in v0.18.1 

func StatsDomainValue(typ batch.TypeID, scale int, v any) (any, bool)

StatsDomainValue converts a SQL literal into the representation a column's parquet STATISTICS and DICTIONARY entries are in, and reports whether the conversion exists.

It is the producer half of the rule the prune layer cannot enforce for itself: `scan.CanPruneRowGroup` compares two `any` values by their Go kind and has no idea what either MEANS, so a raw file bound and an engine literal that both land in the same kind get compared as if they agreed. Three columns did exactly that (#442, and #438 which is the same defect seen through a DECIMAL): 

DECIMAL(18,4)  stats hold the UNSCALED integer (1500.15 -> 15001500)
               and the literal arrives as float64(1500.15), so every row
               group whose unscaled bound exceeds the literal is pruned.
IPV6, UUID     stats hold the RAW 16 bytes and the literal arrives as
               text, and '2' (0x32) sorts above every byte of a
               2001:db8:: address, so every row group is pruned.

The engine's own order for those columns is the stored one — the filter kernel converts the LITERAL (parseIPv6ToRawString, and decimalLiteralAt against the vector's scale) rather than rendering the column — so this function is that same conversion, hoisted to where the planner still knows the column's type and scale. Rendering the bounds the other way would be wrong for IPv6: text order is not address order ('2001:db8::10' sorts below '2001:db8::5').

A false second result means "no conversion" and the caller must WITHHOLD the predicate from the prune layer entirely. Every type is listed explicitly and there is no pass-through default, because a new type that silently inherited "compare it raw" is precisely how this class arrives.

func UUIDLiteralToRaw 

func UUIDLiteralToRaw(s string) string

UUIDLiteralToRaw is parseUUIDToRawString for the row-at-a-time predicate in package exec, so the two comparison paths convert the literal identically.

Types

type Accumulator 

type Accumulator struct {
	SumI64    int64
	SumF64    float64
	SumDec    batch.Int128
	Count     int64
	MinI64    int64
	MaxI64    int64
	MinF64    float64
	MaxF64    float64
	MinDec    batch.Int128
	MaxDec    batch.Int128
	MinStr    string
	MaxStr    string
	HasMin    bool
	HasMax    bool
	IsFloat   bool // true when the source column is a float type (or AVG over int64, which accumulates in float64 to avoid int64 sum wraparound)
	IsDecimal bool // true when the source column is DECIMAL
	IsString  bool // true when the source column is byte-backed (MIN/MAX): STRING, BYTES, IPV6, CIDR, UUID
	IsBool    bool // true when the source column is BOOL (MIN/MAX); the value rides in MinI64/MaxI64 as 0/1
	DecScale  int  // scale for DECIMAL columns
	// DecOverflow marks a DECIMAL SUM that left the 128-bit range. SumDec
	// then holds the WRAPPED value — a different number — so the emit path
	// turns this into a query error instead of writing it out (#455). It
	// rides the accumulator rather than a per-operator flag because every
	// merge, spill and clone path already carries the accumulator.
	DecOverflow bool
	// StrType is the SOURCE column type behind MinStr/MaxStr. The five
	// byte-backed types share one accumulator slot but not one boxed shape:
	// IPV6 and UUID store raw 16-byte values that only round-trip into their
	// own vector as []byte, while STRING and CIDR store their own text. Boxing
	// them all as a Go string handed the IPV6 output vector 16 arbitrary bytes
	// as an ADDRESS TO PARSE, which fails and writes NULL (#417).
	StrType batch.TypeID
}

Accumulator holds aggregate state with typed precision. Int64 sums stay int64 (no float64 precision loss); float sums use float64. Decimal sums use Int128 for exact fixed-point arithmetic.

func (*Accumulator) DecimalAvg added in v0.18.1 

func (a *Accumulator) DecimalAvg() (batch.Int128, bool)

DecimalAvg is FinalAvg's exact half for a DECIMAL accumulator: the unscaled quotient at batch.AvgScale(DecScale). ok=false means the exact answer does not fit an Int128 — a query error, not a rounding.

func (*Accumulator) FinalAvg 

func (a *Accumulator) FinalAvg() any

FinalAvg returns the accumulated average.

Over a DECIMAL it is exact numeric division at scale+AvgScaleIncrement (batch.AvgScale), rounded half away from zero — see that constant for why the increment is fixed rather than PostgreSQL's significant-digit rule.

nil for a quotient with no Int128 is NOT the answer to that case: callers run exec.decAggErr first, which fails the query the way a SUM overflow does. Returning nil here would be a NULL the client cannot tell from "no rows".

func (*Accumulator) FinalMax 

func (a *Accumulator) FinalMax() any

FinalMax returns the accumulated maximum.

func (*Accumulator) FinalMin 

func (a *Accumulator) FinalMin() any

FinalMin returns the accumulated minimum.

func (*Accumulator) FinalSum 

func (a *Accumulator) FinalSum() any

FinalSum returns the accumulated sum as the appropriate type.

A DECIMAL sum is EXACT: Int128 at the column's own scale, so SUM over a DECIMAL is a DECIMAL and not the float64 that dropped every digit past the 16th (#455). An overflowed sum is not returned at all — the caller checks DecOverflow first and fails the query, since the wrapped value is a different number wearing the right type.

func (*Accumulator) Merge 

func (a *Accumulator) Merge(other *Accumulator)

Merge combines another accumulator's state into this one. Used for parallel aggregation: each worker builds partial state, then merges.

type BatchAggKernel 

type BatchAggKernel func(acc *Accumulator, vec *batch.Vector, sel []uint32, vecLen int)

BatchAggKernel processes an entire column (or selection) into an accumulator. Used for non-grouped aggregation or pre-aggregated groups.

func ResolveBatchAvg 

func ResolveBatchAvg(typ batch.TypeID) BatchAggKernel

ResolveBatchAvg returns a batch-level kernel for AVG. Differs from ResolveBatchSum only for int64-class inputs (float64 accumulation).

func ResolveBatchCount 

func ResolveBatchCount() BatchAggKernel

ResolveBatchCount returns a batch-level count kernel.

func ResolveBatchMax 

func ResolveBatchMax(typ batch.TypeID) BatchAggKernel

ResolveBatchMax returns a batch-level max kernel for the given column type.

func ResolveBatchMin 

func ResolveBatchMin(typ batch.TypeID) BatchAggKernel

ResolveBatchMin returns a batch-level min kernel for the given column type.

func ResolveBatchSum 

func ResolveBatchSum(typ batch.TypeID) BatchAggKernel

ResolveBatchSum returns a batch-level sum kernel for the given column type.

type ColColFilterKernel 

type ColColFilterKernel func(left, right *batch.Vector, sel []uint32, vecLen int, outSel []uint32) []uint32

ColColFilterKernel compares two columns element-wise, returning matching row indices.

func ResolveColColFilterKernel 

func ResolveColColFilterKernel(typ batch.TypeID, op CompareOp) ColColFilterKernel

ResolveColColFilterKernel creates a ColColFilterKernel for comparing two columns of the given type. Returns nil if the type is not supported.

type CompareOp 

type CompareOp int

CompareOp represents a comparison operation. 

const (
	OpEq CompareOp = iota
	OpNe
	OpLt
	OpLe
	OpGt
	OpGe
)

type DecimalLiteral added in v0.18.1 

type DecimalLiteral struct {
	// contains filtered or unexported fields
}

DecimalLiteral is a numeric literal held as the EXACT text it was written with, ready to be compared against a DECIMAL column in that column's own domain.

It exists because a literal is not a float64. `compileLit` used to turn every numeric literal that is not an int64 into one, and a float64 carries ~15-16 significant decimal digits where a DECIMAL(38,10) carries 38: the literal a user typed was silently replaced by the nearest double before it ever met the column, so `= 493827160549382.7160549350` matched nothing and `>` gained a row (#452). Text is the only lossless carrier the whole way from the parser to a kernel, which is why the filter kernels already take their DECIMAL constant that way (compareFilterDecimal).

The resolution — text, at the column's scale, plus the residual of any digits the scale cannot hold — is the SAME one compareFilterDecimal performs, through the same decimalLiteralAt: one comparison rule for one predicate, per #394. What this type adds is the cache, for the row-at-a-time paths that would otherwise re-parse per row.

Safe for concurrent use: a resolved literal is published whole, through an atomic pointer, and a losing racer merely re-resolves to the same value.

func NewDecimalLiteral added in v0.18.1 

func NewDecimalLiteral(text string) *DecimalLiteral

NewDecimalLiteral binds literal text — plain or exponent form — for comparison against DECIMAL columns. The text is kept VERBATIM: the exponent is folded into the scaling exactly when the literal is resolved at a column's scale, never expanded through a float64 first (#463).

func (*DecimalLiteral) Compare added in v0.18.1 

func (d *DecimalLiteral) Compare(vec *batch.Vector, row int, op CompareOp) bool

Compare answers `vec[row] <op> literal`.

func (*DecimalLiteral) Numeric added in v0.18.1 

func (d *DecimalLiteral) Numeric() bool

Numeric reports whether the literal's text names a number at all. A false here is a query error at the comparison — PostgreSQL raises "invalid input syntax for type numeric" rather than reading the text as zero (#463).

func (*DecimalLiteral) Order added in v0.18.1 

func (d *DecimalLiteral) Order(vec *batch.Vector, row int) int

Order returns -1, 0 or +1 as vec[row] is less than, equal to, or greater than the literal — exactly, including for a literal with more fractional digits than the column's scale (which equals no stored value but still has a place in the order) and for one wider than the carrier itself (which orders above or below every value the column can hold).

The caller owns the null check: a NULL row has no value to order.

func (*DecimalLiteral) OrderAt added in v0.18.1 

func (d *DecimalLiteral) OrderAt(cell batch.Int128, scale int) int

OrderAt is Order against a value already read out of a column at `scale`.

func (*DecimalLiteral) Text added in v0.18.1 

func (d *DecimalLiteral) Text() string

Text is the literal's source text, verbatim.

type FilterKernel 

type FilterKernel func(vec *batch.Vector, sel []uint32, vecLen int, outSel []uint32) []uint32

FilterKernel evaluates a column against a pre-resolved constant for all rows, returning the indices of matching rows.

func ResolveFilterKernel 

func ResolveFilterKernel(typ batch.TypeID, op CompareOp, value any) FilterKernel

ResolveFilterKernel creates a FilterKernel for comparing a column of the given type against a constant value. The type dispatch happens once here; the returned function has no type switches in its inner loop.

func ResolveInFilterKernel 

func ResolveInFilterKernel(typ batch.TypeID, values []any, negate bool) FilterKernel

ResolveInFilterKernel creates a FilterKernel that checks set membership. The set is built once; the inner loop does a hash lookup per element.

func ResolveLikeFilterKernel 

func ResolveLikeFilterKernel(pattern string, negate bool) FilterKernel

ResolveLikeFilterKernel creates a FilterKernel for SQL LIKE pattern matching. Converts SQL LIKE patterns (% and _) to optimized matching functions.

type FloatOrdered added in v0.18.1 

type FloatOrdered interface{ ~float32 | ~float64 }

FloatOrdered is the float element type the predicates below are written for.

type Numeric 

type Numeric interface {
	~int32 | ~int64 | ~float32 | ~float64
}

Numeric constrains types that support arithmetic operations.

type Ordered 

type Ordered interface {
	~int32 | ~int64 | ~float32 | ~float64 | ~string
}

Ordered constrains types that support comparison.

type RowAggUpdater 

type RowAggUpdater func(acc *Accumulator, vec *batch.Vector, row int)

RowAggUpdater updates an accumulator for a single row (used in grouped aggregation). The type dispatch is resolved once; the function body has no type switches.

func ResolveRowAvg 

func ResolveRowAvg(typ batch.TypeID) RowAggUpdater

ResolveRowAvg returns a row-level updater for AVG. Differs from ResolveRowSum only for int64-class inputs (float64 accumulation).

func ResolveRowAvgNoNulls 

func ResolveRowAvgNoNulls(typ batch.TypeID) RowAggUpdater

ResolveRowAvgNoNulls is the no-null-check variant of ResolveRowAvg.

func ResolveRowCount 

func ResolveRowCount(countStar bool) RowAggUpdater

ResolveRowCount returns a row-level count updater. If countStar is true, counts all rows (including nulls).

func ResolveRowMax 

func ResolveRowMax(typ batch.TypeID) RowAggUpdater

ResolveRowMax returns a row-level max updater for the given column type.

func ResolveRowMaxNoNulls 

func ResolveRowMaxNoNulls(typ batch.TypeID) RowAggUpdater

ResolveRowMaxNoNulls returns a no-null-check max updater.

func ResolveRowMin 

func ResolveRowMin(typ batch.TypeID) RowAggUpdater

ResolveRowMin returns a row-level min updater for the given column type.

func ResolveRowMinNoNulls 

func ResolveRowMinNoNulls(typ batch.TypeID) RowAggUpdater

ResolveRowMinNoNulls returns a no-null-check min updater.

func ResolveRowSum 

func ResolveRowSum(typ batch.TypeID) RowAggUpdater

ResolveRowSum returns a row-level sum updater for the given column type.

func ResolveRowSumNoNulls 

func ResolveRowSumNoNulls(typ batch.TypeID) RowAggUpdater

ResolveRowSumNoNulls returns a no-null-check sum updater.

type SortCompareKernel 

type SortCompareKernel func(a *batch.Vector, ai int, b *batch.Vector, bi int) int

SortCompareKernel compares one row from vector a against one row from vector b. Returns -1, 0, or 1. Null handling is included.

func ResolveSortCompare 

func ResolveSortCompare(typ batch.TypeID) SortCompareKernel

ResolveSortCompare returns a comparison function for the given column type. The returned function has no type switch — the type is baked into the closure.

A nil return means "this resolver cannot order that type". Callers must treat nil as a refusal, not as a tie: SortMergeJoin uses these kernels for key EQUALITY, so a comparator that reports every pair equal is not a degraded sort, it is a cross product presented as an inner join. Until #415 the default arm returned exactly such a closure and ARRAY, ROW, MAP and VECTOR all fell into it — `ORDER BY arr_col` was a silent no-op and the `cmp == nil` guard in sort_merge_join.go was dead code. All 22 types are enumerated now; nil is reserved for a type the engine does not have.

func ResolveSortCompareNoNulls 

func ResolveSortCompareNoNulls(typ batch.TypeID) SortCompareKernel

ResolveSortCompareNoNulls returns a sort compare function that skips null checks.

func ResolveSortCompareNullsLast 

func ResolveSortCompareNullsLast(typ batch.TypeID) SortCompareKernel

ResolveSortCompareNullsLast returns a sort compare function with NULLS LAST ordering.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.