Documentation
¶
Overview ¶
Package expr provides a typed expression engine for evaluating SQL expressions against record batches. It replaces the string-based expression parsing with a compiled expression tree built from the SQL parser AST.
Index ¶
- Variables
- func RegisterFunc(name string, fn ScalarFunc)
- func ToFloat64(v any) float64
- func ToInt64(v any) int64
- type And
- type ArrayLitExpr
- type Between
- type BinOp
- type BinOpFloat64
- type BinOpInt64
- type BoolExpr
- type Case
- type CaseWhen
- type Cast
- type Cmp
- type CmpFloat64
- type CmpInt64
- type CmpOp
- type CmpTemporalLit
- type Coalesce
- type ColRef
- func (e *ColRef) Eval(b *batch.RecordBatch, row int) any
- func (e *ColRef) EvalFloat64(b *batch.RecordBatch, row int) (float64, bool)
- func (e *ColRef) EvalFloat64Vec(b *batch.RecordBatch, dst []float64, n int) bool
- func (e *ColRef) EvalInt64(b *batch.RecordBatch, row int) (int64, bool)
- func (e *ColRef) EvalString(b *batch.RecordBatch, row int) (string, bool)
- type CorrelatedExistsSubquery
- type CorrelatedInSubquery
- type CorrelatedScalarSubquery
- type ExistsSubquery
- type Expr
- func Compile(node plansql.Node) (Expr, error)
- func CompileSelectExpr(expr plansql.Node, alias string) (Expr, string, error)
- func CompileWithFullScope(node plansql.Node, runner SubqueryRunner, outerTables map[string]bool, ...) (Expr, error)
- func CompileWithRunner(node plansql.Node, runner SubqueryRunner) (Expr, error)
- func CompileWithScope(node plansql.Node, runner SubqueryRunner, outerTables map[string]bool) (Expr, error)
- type Float64Expr
- type FuncCall
- type FuncRegistry
- func (r *FuncRegistry) Has(name string) bool
- func (r *FuncRegistry) Lookup(name string) ScalarFunc
- func (r *FuncRegistry) LookupVec(name string) VecScalarFunc
- func (r *FuncRegistry) Names() []string
- func (r *FuncRegistry) Register(name string, fn ScalarFunc)
- func (r *FuncRegistry) RegisterVec(name string, fn VecScalarFunc)
- func (r *FuncRegistry) RegisterVecReturn(name string, dimFn func() int)
- func (r *FuncRegistry) Unregister(name string) bool
- func (r *FuncRegistry) VecReturnDim(name string) (dim int, ok bool)
- type In
- type InSubquery
- type Int64Expr
- type IntervalValue
- type IsNull
- type Like
- type Lit
- type Not
- type Or
- type ParamRef
- type ScalarFunc
- type ScalarSubquery
- type SubqueryRunner
- type UDFCall
- type UDFDef
- type UDFPersister
- type UDFStore
- func (s *UDFStore) CompileUDFCall(name string, argExprs []Expr) (Expr, error)
- func (s *UDFStore) Get(name string) (UDFDef, bool)
- func (s *UDFStore) List() []UDFDef
- func (s *UDFStore) LoadDefs(defs []UDFDef) int
- func (s *UDFStore) Register(def UDFDef, isAdmin bool) error
- func (s *UDFStore) SetPersister(p UDFPersister)
- func (s *UDFStore) Unregister(name, caller string, isAdmin bool) error
- type UnaryOp
- type VecExpr
- type VecFloat64Expr
- type VecScalarFunc
Constants ¶
This section is empty.
Variables ¶
var DefaultRegistry = NewFuncRegistry()
DefaultRegistry is the global function registry used by the expression engine.
var DefaultUDFs = NewUDFStore()
DefaultUDFs is the global UDF store.
Functions ¶
func RegisterFunc ¶
func RegisterFunc(name string, fn ScalarFunc)
RegisterFunc registers a custom scalar function in the default registry.
Types ¶
type ArrayLitExpr ¶
type ArrayLitExpr struct {
Elements []Expr
}
ArrayLitExpr evaluates to a []any containing the evaluated elements.
func (*ArrayLitExpr) Eval ¶
func (e *ArrayLitExpr) Eval(b *batch.RecordBatch, row int) any
type BinOpFloat64 ¶
type BinOpFloat64 struct {
Left, Right Float64Expr
Op string
// contains filtered or unexported fields
}
BinOpFloat64 is a typed binary op that operates on float64 without boxing. Uses a pre-resolved arithOp opcode for the hot EvalFloat64 path to avoid per-row string comparison on the Op field. The opcode is resolved lazily via opOnce so external callers can construct BinOpFloat64 directly with only Op populated; concurrent pipeline workers see the same opCode after the first call returns thanks to sync.Once's happens-before guarantee.
func (*BinOpFloat64) CloneVec ¶
func (e *BinOpFloat64) CloneVec() *BinOpFloat64
CloneVec creates a deep copy of the BinOpFloat64 tree with fresh scratch buffers. Required for parallel pipeline execution where multiple workers must not share mutable vecBuf state. Stateless leaf nodes (ColRef, Literal) are shared; only BinOpFloat64 nodes (which own vecBuf) are cloned.
func (*BinOpFloat64) Eval ¶
func (e *BinOpFloat64) Eval(b *batch.RecordBatch, row int) any
func (*BinOpFloat64) EvalFloat64 ¶
func (e *BinOpFloat64) EvalFloat64(b *batch.RecordBatch, row int) (float64, bool)
func (*BinOpFloat64) EvalFloat64Vec ¶
func (e *BinOpFloat64) EvalFloat64Vec(b *batch.RecordBatch, dst []float64, n int) bool
EvalFloat64Vec evaluates left and right operands in bulk, then applies the arithmetic op in a tight loop. Eliminates ~5 function calls per row.
type BinOpInt64 ¶
type BinOpInt64 struct {
Left, Right Int64Expr
Op string
// contains filtered or unexported fields
}
BinOpInt64 is a typed binary op that operates on int64 without boxing. opCode is resolved lazily via opOnce so external construction with only Op populated stays safe; see BinOpFloat64 for the same pattern.
func (*BinOpInt64) Eval ¶
func (e *BinOpInt64) Eval(b *batch.RecordBatch, row int) any
func (*BinOpInt64) EvalFloat64 ¶
func (e *BinOpInt64) EvalFloat64(b *batch.RecordBatch, row int) (float64, bool)
EvalFloat64 allows BinOpInt64 to be used as Float64Expr (int→float promotion).
func (*BinOpInt64) EvalInt64 ¶
func (e *BinOpInt64) EvalInt64(b *batch.RecordBatch, row int) (int64, bool)
type BoolExpr ¶
type BoolExpr interface {
EvalBool(b *batch.RecordBatch, row int) bool
}
BoolExpr evaluates a boolean expression (used for WHERE/HAVING/JOIN conditions).
type Case ¶
type Case struct {
Operand Expr // optional: CASE <operand> WHEN ...
Whens []CaseWhen // WHEN condition THEN result
Else Expr // optional ELSE clause
}
Case is a CASE WHEN ... THEN ... ELSE ... END expression.
type CaseWhen ¶
type CaseWhen struct {
Cond Expr // the condition (or value to compare against operand)
Result Expr
}
CaseWhen is a single WHEN clause in a CASE expression.
type CmpFloat64 ¶
type CmpFloat64 struct {
Left, Right Float64Expr
Op CmpOp
}
CmpFloat64 is a typed comparison that operates on float64 without boxing.
func (*CmpFloat64) Eval ¶
func (e *CmpFloat64) Eval(b *batch.RecordBatch, row int) any
func (*CmpFloat64) EvalBool ¶
func (e *CmpFloat64) EvalBool(b *batch.RecordBatch, row int) bool
type CmpTemporalLit ¶
type CmpTemporalLit struct {
Col *ColRef
Lit string // original literal text (generic-fallback operand)
Op CmpOp
Flip bool // literal was the LEFT operand: evaluate as (lit OP col)
// contains filtered or unexported fields
}
CmpTemporalLit compares a bare column against a string literal that parses as a date/timestamp, without per-row parsing, cache lookups, or boxing — the generic path spent 3.2% of SF100 worker CPU inside the date-parse memo's sync.Map.Load (interface-key hashing dominated; 2026-07-25 re-rank). The literal is parsed once into BOTH temporal units at compile time; the unit is chosen from the column's resolved type per batch. Every non-fast sub-case (non-temporal column, the epoch-zero literal guard) delegates to the generic compare() with the original operand order, keeping semantics bit-identical with Cmp.
func (*CmpTemporalLit) Eval ¶
func (e *CmpTemporalLit) Eval(b *batch.RecordBatch, row int) any
func (*CmpTemporalLit) EvalBool ¶
func (e *CmpTemporalLit) EvalBool(b *batch.RecordBatch, row int) bool
type ColRef ¶
type ColRef struct {
Name string
// contains filtered or unexported fields
}
ColRef reads a column value from the batch. Caches the column index and type after first resolution for zero-allocation reads on numeric types. The cache is guarded by sync.Once so concurrent callers (parallel pipeline workers sharing this *ColRef via captured expression closures) don't race on the resolution writes.
func (*ColRef) EvalFloat64 ¶
EvalFloat64 reads the column value as float64 without any boxing. Returns (0, false) if null or column not found. Uses cached column type to dispatch directly to the typed data slice, avoiding the extra function call and redundant type switch in GetNumericFloat64.
func (*ColRef) EvalFloat64Vec ¶
EvalFloat64Vec evaluates the column for all rows [0, n) into dst.
func (*ColRef) EvalString ¶
EvalString reads the column value as string without boxing.
type CorrelatedExistsSubquery ¶
type CorrelatedExistsSubquery struct {
}
CorrelatedExistsSubquery evaluates a correlated EXISTS subquery per-row.
func (*CorrelatedExistsSubquery) Eval ¶
func (e *CorrelatedExistsSubquery) Eval(b *batch.RecordBatch, row int) any
func (*CorrelatedExistsSubquery) EvalBool ¶
func (e *CorrelatedExistsSubquery) EvalBool(b *batch.RecordBatch, row int) bool
type CorrelatedInSubquery ¶
type CorrelatedInSubquery struct {
}
CorrelatedInSubquery checks if a value is in the result set of a correlated subquery.
func (*CorrelatedInSubquery) Eval ¶
func (e *CorrelatedInSubquery) Eval(b *batch.RecordBatch, row int) any
func (*CorrelatedInSubquery) EvalBool ¶
func (e *CorrelatedInSubquery) EvalBool(b *batch.RecordBatch, row int) bool
type CorrelatedScalarSubquery ¶
type CorrelatedScalarSubquery struct {
}
CorrelatedScalarSubquery evaluates a correlated scalar subquery per-row. Unlike ScalarSubquery, it cannot cache the result because the inner query depends on values from the outer row.
func (*CorrelatedScalarSubquery) Eval ¶
func (e *CorrelatedScalarSubquery) Eval(b *batch.RecordBatch, row int) any
type ExistsSubquery ¶
type ExistsSubquery struct {
SQL string
Runner SubqueryRunner
Not bool
// contains filtered or unexported fields
}
ExistsSubquery evaluates to true if a subquery returns any rows. Example: WHERE EXISTS (SELECT 1 FROM orders WHERE orders.user_id = users.id) Uncorrelated: executed once and result cached.
func (*ExistsSubquery) Eval ¶
func (e *ExistsSubquery) Eval(b *batch.RecordBatch, row int) any
func (*ExistsSubquery) EvalBool ¶
func (e *ExistsSubquery) EvalBool(_ *batch.RecordBatch, _ int) bool
type Expr ¶
type Expr interface {
Eval(b *batch.RecordBatch, row int) any
}
Expr evaluates an expression against a record batch row, returning a typed value.
func CompileSelectExpr ¶
CompileSelectExpr compiles a SELECT column expression from our AST. Returns the compiled expression and the output column name.
func CompileWithFullScope ¶
func CompileWithFullScope(node plansql.Node, runner SubqueryRunner, outerTables map[string]bool, outerCols map[string]string) (Expr, error)
CompileWithFullScope is like CompileWithScope but also accepts a column-to-table mapping for resolving unqualified column references in correlated subqueries.
func CompileWithRunner ¶
func CompileWithRunner(node plansql.Node, runner SubqueryRunner) (Expr, error)
CompileWithRunner converts our AST Node into an Expr tree, with support for subquery expressions (scalar subqueries, IN subquery, EXISTS).
func CompileWithScope ¶
func CompileWithScope(node plansql.Node, runner SubqueryRunner, outerTables map[string]bool) (Expr, error)
CompileWithScope converts our AST Node into an Expr tree with full scope information, enabling correlated subquery detection and per-row execution. outerTables contains the table names and aliases from the outer query.
type Float64Expr ¶
type Float64Expr interface {
EvalFloat64(b *batch.RecordBatch, row int) (float64, bool)
}
Float64Expr evaluates to float64 without boxing.
type FuncCall ¶
FuncCall represents a scalar function call.
Note: this struct holds NO per-call mutable state. A previous version cached an args buffer on the receiver to avoid per-call allocation, but that was unsafe under parallel pipeline execution: aggPreProject closures (and other wrapped-expression paths) capture the same *FuncCall by pointer rather than cloning it per worker, so concurrent goroutines stomped on the shared args buffer and produced non-deterministic Q02 row counts at SF0.01 (and worse at SF100). The fn / vecFn lookup caches are guarded by sync.Once so concurrent first-time lookups don't race either.
type FuncRegistry ¶
type FuncRegistry struct {
// contains filtered or unexported fields
}
FuncRegistry is a concurrent-safe registry of scalar functions.
func NewFuncRegistry ¶
func NewFuncRegistry() *FuncRegistry
NewFuncRegistry creates a new empty function registry.
func (*FuncRegistry) Has ¶
func (r *FuncRegistry) Has(name string) bool
Has returns true if a function with the given name exists.
func (*FuncRegistry) Lookup ¶
func (r *FuncRegistry) Lookup(name string) ScalarFunc
Lookup returns the function with the given name, or nil if not found.
func (*FuncRegistry) LookupVec ¶
func (r *FuncRegistry) LookupVec(name string) VecScalarFunc
LookupVec returns the vectorized function with the given name, or nil if not found.
func (*FuncRegistry) Names ¶
func (r *FuncRegistry) Names() []string
Names returns all registered function names.
func (*FuncRegistry) Register ¶
func (r *FuncRegistry) Register(name string, fn ScalarFunc)
Register adds or replaces a scalar function.
func (*FuncRegistry) RegisterVec ¶
func (r *FuncRegistry) RegisterVec(name string, fn VecScalarFunc)
RegisterVec adds a vectorized implementation for a scalar function.
func (*FuncRegistry) RegisterVecReturn ¶
func (r *FuncRegistry) RegisterVecReturn(name string, dimFn func() int)
RegisterVecReturn marks a function as returning a VECTOR. dimFn is evaluated lazily (at plan time) to obtain the output dimensionality — embed(), for example, derives it from the configured embedding provider.
func (*FuncRegistry) Unregister ¶
func (r *FuncRegistry) Unregister(name string) bool
Unregister removes a scalar function. Returns true if it existed.
func (*FuncRegistry) VecReturnDim ¶
func (r *FuncRegistry) VecReturnDim(name string) (dim int, ok bool)
VecReturnDim reports whether the named function returns a VECTOR and, if so, its current output dimension. ok is false for non-vector-returning functions.
type InSubquery ¶
type InSubquery struct {
Expr Expr
SQL string
Runner SubqueryRunner
Not bool
// contains filtered or unexported fields
}
InSubquery checks if a value is in the result set of a subquery. Example: WHERE user_id IN (SELECT user_id FROM active_users) Uncorrelated: executed once and result set cached in a hash set for O(1) lookup.
func (*InSubquery) Eval ¶
func (e *InSubquery) Eval(b *batch.RecordBatch, row int) any
func (*InSubquery) EvalBool ¶
func (e *InSubquery) EvalBool(b *batch.RecordBatch, row int) bool
type Int64Expr ¶
type Int64Expr interface {
EvalInt64(b *batch.RecordBatch, row int) (int64, bool)
}
Int64Expr evaluates to int64 without boxing.
type IntervalValue ¶
IntervalValue represents a SQL INTERVAL (e.g., INTERVAL '30' DAY).
type Lit ¶
type Lit struct {
Val any
}
Lit returns a constant value.
func (*Lit) EvalFloat64 ¶
func (*Lit) EvalFloat64Vec ¶
EvalFloat64Vec fills dst[0:n] with the literal value.
type ParamRef ¶
type ParamRef struct {
Index int
// contains filtered or unexported fields
}
ParamRef is an expression node that references a UDF parameter by index.
type ScalarFunc ¶
ScalarFunc is a scalar function implementation.
type ScalarSubquery ¶
type ScalarSubquery struct {
SQL string
Runner SubqueryRunner
// contains filtered or unexported fields
}
ScalarSubquery evaluates a subquery that returns a single scalar value. Example: WHERE price > (SELECT AVG(price) FROM products) Uncorrelated: executed once and result cached.
func (*ScalarSubquery) Eval ¶
func (e *ScalarSubquery) Eval(_ *batch.RecordBatch, _ int) any
type SubqueryRunner ¶
SubqueryRunner executes a SQL subquery and returns its result rows. Each row is a map of column name to value.
type UDFCall ¶
type UDFCall struct {
Name string
ArgExprs []Expr // caller-supplied argument expressions
Body Expr // compiled UDF body with ParamRef nodes
// contains filtered or unexported fields
}
UDFCall evaluates a user-defined function by binding arguments, then evaluating the compiled body expression.
type UDFDef ¶
type UDFDef struct {
Name string // function name (lowercase)
Params []string // parameter names (lowercase)
Body string // SQL expression body (e.g. "param1 * 2 + param2")
Owner string // who created this function (empty = system/unowned)
Locked bool // if true, only the owner (or admin) can modify/drop
}
UDFDef defines a user-defined function.
type UDFPersister ¶
UDFPersister is called after UDF register/unregister to persist the current state.
type UDFStore ¶
type UDFStore struct {
// contains filtered or unexported fields
}
UDFStore holds compiled UDF definitions for use by the expression engine. Thread-safe for concurrent reads and writes.
func (*UDFStore) CompileUDFCall ¶
CompileUDFCall creates a UDFCall expression node.
func (*UDFStore) LoadDefs ¶
LoadDefs registers pre-existing UDF definitions (e.g., from KV on startup). Skips compilation errors silently so one bad UDF doesn't block startup.
func (*UDFStore) SetPersister ¶
func (s *UDFStore) SetPersister(p UDFPersister)
SetPersister sets the function called after UDF mutations to persist state.
type VecExpr ¶
type VecExpr interface {
EvalVec(b *batch.RecordBatch, out *batch.Vector, n int)
}
VecExpr evaluates an expression for an entire batch at once, writing results directly to the output vector. This avoids per-row interface dispatch and boxing.
type VecFloat64Expr ¶
type VecFloat64Expr interface {
EvalFloat64Vec(b *batch.RecordBatch, dst []float64, n int) bool
}
VecFloat64Expr evaluates an expression for all rows [0, n) at once, writing results to dst. Returns true if any output is null. Eliminates per-row function call overhead (~5 calls/row/expression).