Documentation
¶
Overview ¶
Package engine provides SQL parsing, planning, and execution for tinySQL.
This file focuses on the query compilation cache:
- What: A lightweight in-memory cache that stores parsed/compiled representations of SQL statements (CompiledQuery).
- How: Queries are keyed by their exact SQL string. The cache holds a Statement AST plus metadata (ParsedAt) and returns it to callers to avoid re-parsing. A simple FIFO eviction based on oldest ParsedAt keeps the cache within a fixed size.
- Why: Parsing is comparatively expensive and often repeated in loops or hot paths. Caching reduces parse overhead, improves latency, and keeps the execution path predictable while remaining simple and thread-safe.
Package engine implements the tinySQL execution engine.
What: This module evaluates parsed SQL statements (AST) against the storage layer and produces ResultSets. It covers DDL/DML/SELECT, joins, grouping, aggregates, expression evaluation, simple functions (JSON_*, DATEDIFF, etc.), and a minimal tri-state boolean logic (true/false/unknown).
How: The executor converts tables to row maps with both qualified and unqualified column keys, applies WHERE/GROUP/HAVING/ORDER/LIMIT/OFFSET, and optionally combines results with UNION/EXCEPT/INTERSECT. Expression evaluation is recursive over a small algebra of literals, variables, unary/ binary ops, IS NULL, and function calls. Aggregate evaluation runs per- group with reusable helpers shared with the scalar evaluator.
Why: Keeping execution self-contained and data-structure driven (Row maps and simple slices) makes the engine easy to reason about and to extend with new functions, operators, and clauses without introducing heavy planners.
Package engine contains the SQL lexer used by the parser.
What: A minimal, whitespace- and comment-aware tokenizer that recognizes identifiers, keywords, numeric and string literals, and symbols. How: Single-pass rune-based scanner supporting -- and /* */ comments, uppercasing keywords, and preserving identifier case. Keywords are a fixed allow-list tailored to tinySQL features. Why: A compact lexer reduces parser complexity and keeps error messages local and actionable without external dependencies.
Package engine provides a hand-written SQL parser for tinySQL.
What: It parses a practical subset of SQL into an AST (statements and expressions) used by the execution engine. Supported features include DDL, DML, SELECT with JOIN/GROUP/HAVING/ORDER/LIMIT/OFFSET, and set ops. How: A straightforward recursive-descent parser over a small token stream from the lexer. It favors clarity and precise error messages. Ident-like parsing accepts keywords as identifiers to keep the grammar practical for common column names. Why: A small, readable parser is easy to extend and reason about, enabling rapid iteration on language features without a complex generator toolchain.
Index ¶
- func RegisterTableFunc(fn TableFunction)
- type AlterJob
- type AlterTable
- type Binary
- type CSVTableFunc
- type CTE
- type CaseExpr
- type CaseWhen
- type ColumnIndex
- type CompiledQuery
- type CreateIndex
- type CreateJob
- type CreateTable
- type CreateView
- type Delete
- type DropIndex
- type DropJob
- type DropTable
- type DropView
- type ExecEnv
- type Expr
- type ExpressionCache
- type FromItem
- type FuncCall
- type HashJoinOptimizer
- type InExpr
- type Insert
- type IsNull
- type JSONLinesTableFunc
- type JSONTableFunc
- type JoinClause
- type JoinCondition
- type JoinType
- type LikeExpr
- type Literal
- type OptimizedJoinType
- type OrderItem
- type OverClause
- type Parser
- type QueryCache
- type ResultSet
- type Row
- type Select
- type SelectItem
- type Statement
- type SubqueryExpr
- type TableFuncCall
- type TableFunction
- type Unary
- type UnionClause
- type UnionType
- type Update
- type VarRef
- type WindowFrame
- type XMLTableFunc
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func RegisterTableFunc ¶ added in v0.5.1
func RegisterTableFunc(fn TableFunction)
RegisterTableFunc registers a table-valued function globally
Types ¶
type AlterTable ¶ added in v0.3.0
AlterTable represents an ALTER TABLE statement.
type CSVTableFunc ¶ added in v0.5.1
type CSVTableFunc struct{}
CSVTableFunc implements table_from_csv(source, options)
func (*CSVTableFunc) Name ¶ added in v0.5.1
func (f *CSVTableFunc) Name() string
func (*CSVTableFunc) ValidateArgs ¶ added in v0.5.1
func (f *CSVTableFunc) ValidateArgs(args []Expr) error
type CTE ¶
type CTE struct {
Name string
Select *Select
// Recursive indicates this CTE was declared under WITH RECURSIVE
Recursive bool
}
CTE represents a Common Table Expression (WITH clause)
type CaseExpr ¶ added in v0.3.0
CaseExpr represents a CASE ... WHEN ... THEN ... [ELSE ...] END expression.
type ColumnIndex ¶ added in v0.3.1
type ColumnIndex struct {
// contains filtered or unexported fields
}
ColumnIndex provides fast column lookups for better performance
func NewColumnIndex ¶ added in v0.3.1
func NewColumnIndex(rows []Row, columns []string) *ColumnIndex
NewColumnIndex creates an index for the given rows and columns
func (*ColumnIndex) GetRows ¶ added in v0.3.1
func (idx *ColumnIndex) GetRows(indices []int) []Row
GetRows returns the rows at the given indices
type CompiledQuery ¶
CompiledQuery represents a pre-parsed and cached SQL query.
type CreateIndex ¶ added in v0.3.0
CreateIndex represents a CREATE INDEX statement.
type CreateJob ¶ added in v0.5.1
type CreateJob struct {
Name string
ScheduleType string // CRON, INTERVAL, ONCE
CronExpr string
IntervalMs int64
RunAt *time.Time
Timezone string
MaxRuntimeMs int64
NoOverlap bool
CatchUp bool
Enabled bool
SQLText string
}
CreateJob represents a CREATE JOB statement.
type CreateTable ¶
type CreateTable struct {
Name string
Cols []storage.Column
IsTemp bool
AsSelect *Select
IfNotExists bool // IF NOT EXISTS clause
}
CreateTable represents a CREATE TABLE statement.
type CreateView ¶ added in v0.3.0
CreateView represents a CREATE VIEW statement.
type DropIndex ¶ added in v0.3.0
type DropIndex struct {
Name string
Table string // Optional: some DBs require table name
IfExists bool
}
DropIndex represents a DROP INDEX statement.
type DropJob ¶ added in v0.5.1
type DropJob struct {
Name string
}
DropJob represents DROP JOB <name>
type ExpressionCache ¶ added in v0.3.1
type ExpressionCache struct {
// contains filtered or unexported fields
}
ExpressionCache provides caching for frequently evaluated expressions
func NewExpressionCache ¶ added in v0.3.1
func NewExpressionCache() *ExpressionCache
NewExpressionCache creates a new expression cache
func (*ExpressionCache) Clear ¶ added in v0.3.1
func (ec *ExpressionCache) Clear()
Clear empties the cache
func (*ExpressionCache) Get ¶ added in v0.3.1
func (ec *ExpressionCache) Get(exprStr string, row Row) (any, bool)
Get retrieves a cached result for the given expression and row
func (*ExpressionCache) Set ¶ added in v0.3.1
func (ec *ExpressionCache) Set(exprStr string, row Row, result any)
Set stores a result in the cache
func (*ExpressionCache) Stats ¶ added in v0.3.1
func (ec *ExpressionCache) Stats() (hits, misses int, hitRate float64)
Stats returns cache performance statistics
type FromItem ¶
type FromItem struct {
Table string // Tabellenname (wenn echte Tabelle)
Alias string // Alias für Tabelle oder Subselect
Subquery *Select // Falls abgeleitete Tabelle: das Select-Statement
TableFunc *TableFuncCall // Wenn FROM eine table-valued function ist
}
FromItem kann eine echte Tabelle oder ein Subselect (Derived Table) sein.
type FuncCall ¶
type FuncCall struct {
Name string
Args []Expr
Star bool
Distinct bool // For COUNT(DISTINCT col)
Over *OverClause // For window functions
}
FuncCall represents a function call, optionally with a star (COUNT(*)).
type HashJoinOptimizer ¶ added in v0.3.1
type HashJoinOptimizer struct {
// contains filtered or unexported fields
}
HashJoinOptimizer provides hash-based join optimizations
func (*HashJoinOptimizer) ProcessOptimizedJoin ¶ added in v0.3.1
func (h *HashJoinOptimizer) ProcessOptimizedJoin(leftRows, rightRows []Row, condition Expr, joinType OptimizedJoinType) ([]Row, error)
OptimizedJoinProcessor handles optimized join operations
type JSONLinesTableFunc ¶ added in v0.5.1
type JSONLinesTableFunc struct{}
JSONLinesTableFunc implements table_from_json_lines(source)
func (*JSONLinesTableFunc) Name ¶ added in v0.5.1
func (f *JSONLinesTableFunc) Name() string
func (*JSONLinesTableFunc) ValidateArgs ¶ added in v0.5.1
func (f *JSONLinesTableFunc) ValidateArgs(args []Expr) error
type JSONTableFunc ¶ added in v0.5.1
type JSONTableFunc struct{}
JSONTableFunc implements table_from_json(source, spec)
func (*JSONTableFunc) Name ¶ added in v0.5.1
func (f *JSONTableFunc) Name() string
func (*JSONTableFunc) ValidateArgs ¶ added in v0.5.1
func (f *JSONTableFunc) ValidateArgs(args []Expr) error
type JoinClause ¶
JoinClause holds a JOIN type with the right side and join condition.
type JoinCondition ¶ added in v0.3.1
JoinCondition represents a parsed join condition
type LikeExpr ¶ added in v0.3.0
type LikeExpr struct {
Expr Expr
Pattern Expr
Escape Expr // Optional ESCAPE character
Negate bool // For NOT LIKE
}
LikeExpr represents "expr LIKE pattern [ESCAPE char]"
type Literal ¶
type Literal struct{ Val any }
Literal holds a constant value (number, string, bool, NULL).
type OptimizedJoinType ¶ added in v0.3.1
type OptimizedJoinType int
OptimizedJoinType defines the type of join operation for optimizations
const ( OptimizedJoinTypeInner OptimizedJoinType = iota OptimizedJoinTypeLeft OptimizedJoinTypeRight )
type OverClause ¶ added in v0.5.0
type OverClause struct {
PartitionBy []Expr // PARTITION BY expressions
OrderBy []OrderItem // ORDER BY items
Frame *WindowFrame // ROWS/RANGE frame specification
}
OverClause represents the OVER clause for window functions.
type Parser ¶
type Parser struct {
// contains filtered or unexported fields
}
Parser holds the lexer and current/peek tokens for recursive-descent parsing.
func (*Parser) ParseStatement ¶
ParseStatement parses a single SQL statement into an AST.
type QueryCache ¶
type QueryCache struct {
// contains filtered or unexported fields
}
QueryCache manages compiled queries.
func NewQueryCache ¶
func NewQueryCache(maxSize int) *QueryCache
NewQueryCache creates a new query cache with the specified maximum size.
func (*QueryCache) Compile ¶
func (qc *QueryCache) Compile(sql string) (*CompiledQuery, error)
Compile parses and caches a SQL query for reuse.
func (*QueryCache) MustCompile ¶
func (qc *QueryCache) MustCompile(sql string) *CompiledQuery
MustCompile is like Compile but panics on error (similar to regexp.MustCompile).
func (*QueryCache) Size ¶
func (qc *QueryCache) Size() int
Size returns the number of cached queries.
func (*QueryCache) Stats ¶
func (qc *QueryCache) Stats() map[string]interface{}
Stats returns cache statistics.
type ResultSet ¶
ResultSet holds the column order and the returned rows from a query. Cols preserve the display order; Rows store values in a case-insensitive map.
func Execute ¶
func Execute(ctx context.Context, db *storage.DB, tenant string, stmt Statement) (*ResultSet, error)
Execute runs a parsed SQL statement against the given storage DB and tenant. It dispatches to handlers per statement kind and returns a ResultSet for SELECT (nil for DDL/DML). The context is checked at safe points to support cancellation.
type Row ¶
Row represents a single result row mapped by lower-cased column name. Keys include both qualified (table.column) and unqualified (column) names to simplify expression evaluation and projection.
type Select ¶
type Select struct {
Distinct bool
DistinctOn []Expr
From FromItem
Joins []JoinClause
Projs []SelectItem
Where Expr
GroupBy []Expr
Having Expr
OrderBy []OrderItem
Limit *int
Offset *int
Union *UnionClause // For UNION operations
CTEs []CTE // Common Table Expressions
}
Select represents a SELECT query and its clauses.
type SelectItem ¶
SelectItem represents a projection item, optionally with alias or *.
type Statement ¶
type Statement interface{}
Statement is the root interface for all parsed SQL statements.
type SubqueryExpr ¶ added in v0.3.0
type SubqueryExpr struct {
Select *Select
}
SubqueryExpr wraps a SELECT used as an expression (scalar subquery).
type TableFuncCall ¶ added in v0.5.1
TableFuncCall represents a table function call in a FROM clause
type TableFunction ¶ added in v0.5.1
type TableFunction interface {
// Name returns the function name (e.g., "table_from_json")
Name() string
// Execute evaluates the TVF and returns a result set (columns + rows)
Execute(ctx context.Context, args []Expr, env ExecEnv, row Row) (*ResultSet, error)
// ValidateArgs checks if the provided arguments are valid
ValidateArgs(args []Expr) error
}
TableFunction represents a table-valued function that can be used in FROM clauses Example: SELECT * FROM table_from_json(http('...'), spec)
func GetTableFunc ¶ added in v0.5.1
func GetTableFunc(name string) (TableFunction, bool)
GetTableFunc retrieves a registered table function by name
type UnionClause ¶
type UnionClause struct {
Type UnionType
Right *Select
Next *UnionClause // For chaining multiple UNIONs
}
UnionClause represents a set operation chaining RIGHT select with current one.
type VarRef ¶
type VarRef struct{ Name string }
VarRef refers to a column (qualified or unqualified) in expressions.
type WindowFrame ¶ added in v0.5.0
type WindowFrame struct {
Mode string // "ROWS" or "RANGE"
StartType string // "UNBOUNDED", "CURRENT", or "OFFSET"
StartValue int // Offset value for PRECEDING/FOLLOWING
EndType string // "UNBOUNDED", "CURRENT", or "OFFSET"
EndValue int // Offset value for PRECEDING/FOLLOWING
}
WindowFrame represents ROWS/RANGE BETWEEN frame specification.
type XMLTableFunc ¶ added in v0.5.1
type XMLTableFunc struct{}
XMLTableFunc implements table_from_xml(source, record_name)
func (*XMLTableFunc) Name ¶ added in v0.5.1
func (f *XMLTableFunc) Name() string
func (*XMLTableFunc) ValidateArgs ¶ added in v0.5.1
func (f *XMLTableFunc) ValidateArgs(args []Expr) error