engine

package
v0.5.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 19, 2025 License: AGPL-3.0 Imports: 28 Imported by: 0

Documentation

Overview

Package engine provides SQL parsing, planning, and execution for tinySQL.

This file focuses on the query compilation cache:

  • What: A lightweight in-memory cache that stores parsed/compiled representations of SQL statements (CompiledQuery).
  • How: Queries are keyed by their exact SQL string. The cache holds a Statement AST plus metadata (ParsedAt) and returns it to callers to avoid re-parsing. A simple FIFO eviction based on oldest ParsedAt keeps the cache within a fixed size.
  • Why: Parsing is comparatively expensive and often repeated in loops or hot paths. Caching reduces parse overhead, improves latency, and keeps the execution path predictable while remaining simple and thread-safe.

Package engine implements the tinySQL execution engine.

What: This module evaluates parsed SQL statements (AST) against the storage layer and produces ResultSets. It covers DDL/DML/SELECT, joins, grouping, aggregates, expression evaluation, simple functions (JSON_*, DATEDIFF, etc.), and a minimal tri-state boolean logic (true/false/unknown).

How: The executor converts tables to row maps with both qualified and unqualified column keys, applies WHERE/GROUP/HAVING/ORDER/LIMIT/OFFSET, and optionally combines results with UNION/EXCEPT/INTERSECT. Expression evaluation is recursive over a small algebra of literals, variables, unary/ binary ops, IS NULL, and function calls. Aggregate evaluation runs per- group with reusable helpers shared with the scalar evaluator.

Why: Keeping execution self-contained and data-structure driven (Row maps and simple slices) makes the engine easy to reason about and to extend with new functions, operators, and clauses without introducing heavy planners.

Package engine contains the SQL lexer used by the parser.

What: A minimal, whitespace- and comment-aware tokenizer that recognizes identifiers, keywords, numeric and string literals, and symbols. How: Single-pass rune-based scanner supporting -- and /* */ comments, uppercasing keywords, and preserving identifier case. Keywords are a fixed allow-list tailored to tinySQL features. Why: A compact lexer reduces parser complexity and keeps error messages local and actionable without external dependencies.

Package engine provides a hand-written SQL parser for tinySQL.

What: It parses a practical subset of SQL into an AST (statements and expressions) used by the execution engine. Supported features include DDL, DML, SELECT with JOIN/GROUP/HAVING/ORDER/LIMIT/OFFSET, and set ops. How: A straightforward recursive-descent parser over a small token stream from the lexer. It favors clarity and precise error messages. Ident-like parsing accepts keywords as identifiers to keep the grammar practical for common column names. Why: A small, readable parser is easy to extend and reason about, enabling rapid iteration on language features without a complex generator toolchain.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func RegisterTableFunc added in v0.5.1

func RegisterTableFunc(fn TableFunction)

RegisterTableFunc registers a table-valued function globally

Types

type AlterJob added in v0.5.1

type AlterJob struct {
	Name   string
	Enable *bool // nil means no-op
}

AlterJob represents ALTER JOB ... ENABLE/DISABLE

type AlterTable added in v0.3.0

type AlterTable struct {
	Table     string
	AddColumn *storage.Column // For ADD COLUMN

}

AlterTable represents an ALTER TABLE statement.

type Binary

type Binary struct {
	Op          string
	Left, Right Expr
}

Binary represents binary operators (+,-,*,/, comparisons, AND/OR).

type CSVTableFunc added in v0.5.1

type CSVTableFunc struct{}

CSVTableFunc implements table_from_csv(source, options)

func (*CSVTableFunc) Execute added in v0.5.1

func (f *CSVTableFunc) Execute(ctx context.Context, args []Expr, env ExecEnv, row Row) (*ResultSet, error)

func (*CSVTableFunc) Name added in v0.5.1

func (f *CSVTableFunc) Name() string

func (*CSVTableFunc) ValidateArgs added in v0.5.1

func (f *CSVTableFunc) ValidateArgs(args []Expr) error

type CTE

type CTE struct {
	Name   string
	Select *Select
	// Recursive indicates this CTE was declared under WITH RECURSIVE
	Recursive bool
}

CTE represents a Common Table Expression (WITH clause)

type CaseExpr added in v0.3.0

type CaseExpr struct {
	Operand Expr
	Whens   []CaseWhen
	Else    Expr
}

CaseExpr represents a CASE ... WHEN ... THEN ... [ELSE ...] END expression.

type CaseWhen added in v0.3.0

type CaseWhen struct {
	When Expr
	Then Expr
}

CaseWhen pairs WHEN condition and THEN result expressions.

type ColumnIndex added in v0.3.1

type ColumnIndex struct {
	// contains filtered or unexported fields
}

ColumnIndex provides fast column lookups for better performance

func NewColumnIndex added in v0.3.1

func NewColumnIndex(rows []Row, columns []string) *ColumnIndex

NewColumnIndex creates an index for the given rows and columns

func (*ColumnIndex) GetRows added in v0.3.1

func (idx *ColumnIndex) GetRows(indices []int) []Row

GetRows returns the rows at the given indices

func (*ColumnIndex) Lookup added in v0.3.1

func (idx *ColumnIndex) Lookup(column string, value any) []int

Lookup returns row indices that match the given column value

type CompiledQuery

type CompiledQuery struct {
	SQL       string
	Statement Statement
	ParsedAt  time.Time
}

CompiledQuery represents a pre-parsed and cached SQL query.

func (*CompiledQuery) Execute

func (cq *CompiledQuery) Execute(ctx context.Context, db *storage.DB, tenant string) (*ResultSet, error)

Execute runs a compiled query against the database.

type CreateIndex added in v0.3.0

type CreateIndex struct {
	Name        string
	Table       string
	Columns     []string
	Unique      bool
	IfNotExists bool
}

CreateIndex represents a CREATE INDEX statement.

type CreateJob added in v0.5.1

type CreateJob struct {
	Name         string
	ScheduleType string // CRON, INTERVAL, ONCE
	CronExpr     string
	IntervalMs   int64
	RunAt        *time.Time
	Timezone     string
	MaxRuntimeMs int64
	NoOverlap    bool
	CatchUp      bool
	Enabled      bool
	SQLText      string
}

CreateJob represents a CREATE JOB statement.

type CreateTable

type CreateTable struct {
	Name        string
	Cols        []storage.Column
	IsTemp      bool
	AsSelect    *Select
	IfNotExists bool // IF NOT EXISTS clause
}

CreateTable represents a CREATE TABLE statement.

type CreateView added in v0.3.0

type CreateView struct {
	Name        string
	Select      *Select
	IfNotExists bool
	OrReplace   bool
}

CreateView represents a CREATE VIEW statement.

type Delete

type Delete struct {
	Table string
	Where Expr
}

Delete represents a DELETE statement.

type DropIndex added in v0.3.0

type DropIndex struct {
	Name     string
	Table    string // Optional: some DBs require table name
	IfExists bool
}

DropIndex represents a DROP INDEX statement.

type DropJob added in v0.5.1

type DropJob struct {
	Name string
}

DropJob represents DROP JOB <name>

type DropTable

type DropTable struct {
	Name     string
	IfExists bool // IF EXISTS clause
}

DropTable represents a DROP TABLE statement.

type DropView added in v0.3.0

type DropView struct {
	Name     string
	IfExists bool
}

DropView represents a DROP VIEW statement.

type ExecEnv

type ExecEnv struct {
	// contains filtered or unexported fields
}

type Expr

type Expr interface{}

type ExpressionCache added in v0.3.1

type ExpressionCache struct {
	// contains filtered or unexported fields
}

ExpressionCache provides caching for frequently evaluated expressions

func NewExpressionCache added in v0.3.1

func NewExpressionCache() *ExpressionCache

NewExpressionCache creates a new expression cache

func (*ExpressionCache) Clear added in v0.3.1

func (ec *ExpressionCache) Clear()

Clear empties the cache

func (*ExpressionCache) Get added in v0.3.1

func (ec *ExpressionCache) Get(exprStr string, row Row) (any, bool)

Get retrieves a cached result for the given expression and row

func (*ExpressionCache) Set added in v0.3.1

func (ec *ExpressionCache) Set(exprStr string, row Row, result any)

Set stores a result in the cache

func (*ExpressionCache) Stats added in v0.3.1

func (ec *ExpressionCache) Stats() (hits, misses int, hitRate float64)

Stats returns cache performance statistics

type FromItem

type FromItem struct {
	Table     string         // Tabellenname (wenn echte Tabelle)
	Alias     string         // Alias für Tabelle oder Subselect
	Subquery  *Select        // Falls abgeleitete Tabelle: das Select-Statement
	TableFunc *TableFuncCall // Wenn FROM eine table-valued function ist
}

FromItem kann eine echte Tabelle oder ein Subselect (Derived Table) sein.

type FuncCall

type FuncCall struct {
	Name     string
	Args     []Expr
	Star     bool
	Distinct bool        // For COUNT(DISTINCT col)
	Over     *OverClause // For window functions
}

FuncCall represents a function call, optionally with a star (COUNT(*)).

type HashJoinOptimizer added in v0.3.1

type HashJoinOptimizer struct {
	// contains filtered or unexported fields
}

HashJoinOptimizer provides hash-based join optimizations

func (*HashJoinOptimizer) ProcessOptimizedJoin added in v0.3.1

func (h *HashJoinOptimizer) ProcessOptimizedJoin(leftRows, rightRows []Row, condition Expr, joinType OptimizedJoinType) ([]Row, error)

OptimizedJoinProcessor handles optimized join operations

type InExpr added in v0.3.0

type InExpr struct {
	Expr   Expr
	Values []Expr
	Negate bool // For NOT IN
}

InExpr represents "expr IN (val1, val2, ...)"

type Insert

type Insert struct {
	Table string
	Cols  []string
	Rows  [][]Expr
}

Insert represents an INSERT statement.

type IsNull

type IsNull struct {
	Expr   Expr
	Negate bool
}

IsNull represents IS [NOT] NULL predicate.

type JSONLinesTableFunc added in v0.5.1

type JSONLinesTableFunc struct{}

JSONLinesTableFunc implements table_from_json_lines(source)

func (*JSONLinesTableFunc) Execute added in v0.5.1

func (f *JSONLinesTableFunc) Execute(ctx context.Context, args []Expr, env ExecEnv, row Row) (*ResultSet, error)

func (*JSONLinesTableFunc) Name added in v0.5.1

func (f *JSONLinesTableFunc) Name() string

func (*JSONLinesTableFunc) ValidateArgs added in v0.5.1

func (f *JSONLinesTableFunc) ValidateArgs(args []Expr) error

type JSONTableFunc added in v0.5.1

type JSONTableFunc struct{}

JSONTableFunc implements table_from_json(source, spec)

func (*JSONTableFunc) Execute added in v0.5.1

func (f *JSONTableFunc) Execute(ctx context.Context, args []Expr, env ExecEnv, row Row) (*ResultSet, error)

func (*JSONTableFunc) Name added in v0.5.1

func (f *JSONTableFunc) Name() string

func (*JSONTableFunc) ValidateArgs added in v0.5.1

func (f *JSONTableFunc) ValidateArgs(args []Expr) error

type JoinClause

type JoinClause struct {
	Type  JoinType
	Right FromItem
	On    Expr
}

JoinClause holds a JOIN type with the right side and join condition.

type JoinCondition added in v0.3.1

type JoinCondition struct {
	LeftColumn  string
	RightColumn string
	Operator    string
	IsEquiJoin  bool
}

JoinCondition represents a parsed join condition

type JoinType

type JoinType int
const (
	// JoinInner represents INNER JOIN.
	JoinInner JoinType = iota
	// JoinLeft represents LEFT (OUTER) JOIN.
	JoinLeft
	// JoinRight represents RIGHT (OUTER) JOIN.
	JoinRight
)

type LikeExpr added in v0.3.0

type LikeExpr struct {
	Expr    Expr
	Pattern Expr
	Escape  Expr // Optional ESCAPE character
	Negate  bool // For NOT LIKE
}

LikeExpr represents "expr LIKE pattern [ESCAPE char]"

type Literal

type Literal struct{ Val any }

Literal holds a constant value (number, string, bool, NULL).

type OptimizedJoinType added in v0.3.1

type OptimizedJoinType int

OptimizedJoinType defines the type of join operation for optimizations

const (
	OptimizedJoinTypeInner OptimizedJoinType = iota
	OptimizedJoinTypeLeft
	OptimizedJoinTypeRight
)

type OrderItem

type OrderItem struct {
	Col  string
	Desc bool
}

OrderItem specifies ordering column and direction.

type OverClause added in v0.5.0

type OverClause struct {
	PartitionBy []Expr       // PARTITION BY expressions
	OrderBy     []OrderItem  // ORDER BY items
	Frame       *WindowFrame // ROWS/RANGE frame specification
}

OverClause represents the OVER clause for window functions.

type Parser

type Parser struct {
	// contains filtered or unexported fields
}

Parser holds the lexer and current/peek tokens for recursive-descent parsing.

func NewParser

func NewParser(sql string) *Parser

NewParser creates a new SQL parser for the provided input string.

func (*Parser) ParseStatement

func (p *Parser) ParseStatement() (Statement, error)

ParseStatement parses a single SQL statement into an AST.

type QueryCache

type QueryCache struct {
	// contains filtered or unexported fields
}

QueryCache manages compiled queries.

func NewQueryCache

func NewQueryCache(maxSize int) *QueryCache

NewQueryCache creates a new query cache with the specified maximum size.

func (*QueryCache) Clear

func (qc *QueryCache) Clear()

Clear removes all cached queries.

func (*QueryCache) Compile

func (qc *QueryCache) Compile(sql string) (*CompiledQuery, error)

Compile parses and caches a SQL query for reuse.

func (*QueryCache) MustCompile

func (qc *QueryCache) MustCompile(sql string) *CompiledQuery

MustCompile is like Compile but panics on error (similar to regexp.MustCompile).

func (*QueryCache) Size

func (qc *QueryCache) Size() int

Size returns the number of cached queries.

func (*QueryCache) Stats

func (qc *QueryCache) Stats() map[string]interface{}

Stats returns cache statistics.

type ResultSet

type ResultSet struct {
	Cols []string
	Rows []Row
}

ResultSet holds the column order and the returned rows from a query. Cols preserve the display order; Rows store values in a case-insensitive map.

func Execute

func Execute(ctx context.Context, db *storage.DB, tenant string, stmt Statement) (*ResultSet, error)

Execute runs a parsed SQL statement against the given storage DB and tenant. It dispatches to handlers per statement kind and returns a ResultSet for SELECT (nil for DDL/DML). The context is checked at safe points to support cancellation.

type Row

type Row map[string]any

Row represents a single result row mapped by lower-cased column name. Keys include both qualified (table.column) and unqualified (column) names to simplify expression evaluation and projection.

type Select

type Select struct {
	Distinct   bool
	DistinctOn []Expr
	From       FromItem
	Joins      []JoinClause
	Projs      []SelectItem
	Where      Expr
	GroupBy    []Expr
	Having     Expr
	OrderBy    []OrderItem
	Limit      *int
	Offset     *int
	Union      *UnionClause // For UNION operations
	CTEs       []CTE        // Common Table Expressions
}

Select represents a SELECT query and its clauses.

type SelectItem

type SelectItem struct {
	Expr  Expr
	Alias string
	Star  bool
}

SelectItem represents a projection item, optionally with alias or *.

type Statement

type Statement interface{}

Statement is the root interface for all parsed SQL statements.

type SubqueryExpr added in v0.3.0

type SubqueryExpr struct {
	Select *Select
}

SubqueryExpr wraps a SELECT used as an expression (scalar subquery).

type TableFuncCall added in v0.5.1

type TableFuncCall struct {
	Name  string
	Args  []Expr
	Alias string // Optional table alias
}

TableFuncCall represents a table function call in a FROM clause

type TableFunction added in v0.5.1

type TableFunction interface {
	// Name returns the function name (e.g., "table_from_json")
	Name() string

	// Execute evaluates the TVF and returns a result set (columns + rows)
	Execute(ctx context.Context, args []Expr, env ExecEnv, row Row) (*ResultSet, error)

	// ValidateArgs checks if the provided arguments are valid
	ValidateArgs(args []Expr) error
}

TableFunction represents a table-valued function that can be used in FROM clauses Example: SELECT * FROM table_from_json(http('...'), spec)

func GetTableFunc added in v0.5.1

func GetTableFunc(name string) (TableFunction, bool)

GetTableFunc retrieves a registered table function by name

type Unary

type Unary struct {
	Op   string
	Expr Expr
}

Unary represents unary operators like +, -, NOT.

type UnionClause

type UnionClause struct {
	Type  UnionType
	Right *Select
	Next  *UnionClause // For chaining multiple UNIONs
}

UnionClause represents a set operation chaining RIGHT select with current one.

type UnionType

type UnionType int
const (
	// UnionDistinct corresponds to UNION (distinct).
	UnionDistinct UnionType = iota
	// UnionAll corresponds to UNION ALL.
	UnionAll
	// Except corresponds to EXCEPT.
	Except
	// Intersect corresponds to INTERSECT.
	Intersect
)

type Update

type Update struct {
	Table string
	Sets  map[string]Expr
	Where Expr
}

Update represents an UPDATE statement.

type VarRef

type VarRef struct{ Name string }

VarRef refers to a column (qualified or unqualified) in expressions.

type WindowFrame added in v0.5.0

type WindowFrame struct {
	Mode       string // "ROWS" or "RANGE"
	StartType  string // "UNBOUNDED", "CURRENT", or "OFFSET"
	StartValue int    // Offset value for PRECEDING/FOLLOWING
	EndType    string // "UNBOUNDED", "CURRENT", or "OFFSET"
	EndValue   int    // Offset value for PRECEDING/FOLLOWING
}

WindowFrame represents ROWS/RANGE BETWEEN frame specification.

type XMLTableFunc added in v0.5.1

type XMLTableFunc struct{}

XMLTableFunc implements table_from_xml(source, record_name)

func (*XMLTableFunc) Execute added in v0.5.1

func (f *XMLTableFunc) Execute(ctx context.Context, args []Expr, env ExecEnv, row Row) (*ResultSet, error)

func (*XMLTableFunc) Name added in v0.5.1

func (f *XMLTableFunc) Name() string

func (*XMLTableFunc) ValidateArgs added in v0.5.1

func (f *XMLTableFunc) ValidateArgs(args []Expr) error

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL