Documentation
¶
Overview ¶
Package formula holds the formula language: lexer, parser, function catalog, binder, bytecode, the Go virtual machine and the SQL generator with its SQLite dialect; the DuckDB dialect follows as Phase 1 continues. One compiler, three targets; the browser VM in web/vm executes the same bytecode.
Parse turns text into a tree, Bind turns the tree into a Program by the static typing rules of ADR-0019, VM.Run executes a Program over the values of its slots, and GenerateSQL turns a Program into a value and an error expression for a Dialect. The grammar and semantics are fixed by ADR-0019, the query functions by ADR-0020, and the suite under conformance/ is the specification. Composite values and the query functions are not implemented yet.
Index ¶
- Constants
- Variables
- func Classes() map[Class]string
- func Equal(a, b Value) bool
- func NearlyEqual(a, b Value, rel float64) bool
- func ParseCaseDate(s string) (int64, error)
- func Sexpr(n Node) string
- type Array
- type Binary
- type BindOptions
- type BindingSuite
- type BlankLit
- type BoolLit
- type Call
- type Case
- type CaseContext
- type Class
- type Column
- type Context
- type Dialect
- type DuckDB
- func (d DuckDB) Call(name string, args []SQLExpr, params *Params) (SQLExpr, error)
- func (c DuckDB) Cond(e SQLExpr) string
- func (DuckDB) ContextArgs(ctx *Context) []any
- func (DuckDB) False() string
- func (d DuckDB) Limits() Limits
- func (DuckDB) Load(column string, k Kind) string
- func (DuckDB) Name() string
- func (DuckDB) NotFinite(val string) string
- func (c DuckDB) Num(e SQLExpr) string
- func (DuckDB) NumberLit(f float64) string
- func (DuckDB) Param(name string) string
- func (DuckDB) Pow(x, y string) string
- func (d DuckDB) Render(e SQLExpr) string
- func (DuckDB) Setup() []string
- func (c DuckDB) Text(e SQLExpr) string
- func (DuckDB) TextLit(s string) string
- func (DuckDB) True() string
- func (d DuckDB) WithLimits(l Limits) Dialect
- type Error
- type EvalError
- type Function
- type Ident
- type If
- type Instr
- type Kind
- type Limits
- type Member
- type Mode
- type Node
- type NumberLit
- type Op
- type Param
- type Params
- type Program
- type QuerySlot
- type RegisteredFunc
- type RejectCase
- type Rows
- type SQL
- type SQLExpr
- type SQLResult
- type SQLite
- func (d SQLite) Call(name string, args []SQLExpr, params *Params) (SQLExpr, error)
- func (c SQLite) Cond(e SQLExpr) string
- func (SQLite) ContextArgs(ctx *Context) []any
- func (SQLite) False() string
- func (d SQLite) Limits() Limits
- func (SQLite) Load(column string, k Kind) string
- func (SQLite) Name() string
- func (SQLite) NotFinite(val string) string
- func (c SQLite) Num(e SQLExpr) string
- func (SQLite) NumberLit(f float64) string
- func (SQLite) Param(name string) string
- func (SQLite) Pow(x, y string) string
- func (d SQLite) Render(e SQLExpr) string
- func (SQLite) Setup() []string
- func (c SQLite) Text(e SQLExpr) string
- func (SQLite) TextLit(s string) string
- func (SQLite) True() string
- func (d SQLite) WithLimits(l Limits) Dialect
- type Scope
- type Slot
- type Suite
- type SuiteLimits
- type SyntaxSuite
- type TextLit
- type ThisRef
- type Token
- type TokenKind
- type Type
- type Unary
- type VM
- type Value
- type Variable
Constants ¶
const ( ErrDivByZero = "div_by_zero" ErrNotANumber = "not_a_number" ErrBadText = "bad_text" ErrOutOfRange = "out_of_range" ErrLengthMismatch = "length_mismatch" ErrTooManyElements = "too_many_elements" ErrQueryFailed = "query_failed" )
Error codes (ADR-0019, ADR-0020).
const DefaultMaxText = 1 << 16
DefaultMaxText is the text limit of a new database: 65,536 code points, room for a long note, and small enough that a vector of 2,048 of them on the analytical engine stays under a gigabyte.
const MinMaxText = 64
MinMaxText is the smallest text limit a database may set: every rendering of a number or a date fits in it.
Variables ¶
var ErrorCodes = map[string]bool{ "div_by_zero": true, "not_a_number": true, "bad_text": true, "out_of_range": true, "length_mismatch": true, "too_many_elements": true, "unknown_source": true, "unknown_column": true, "query_failed": true, }
ErrorCodes is the closed list of evaluation error codes (ADR-0019, ADR-0020). A case may only expect one of these.
Functions ¶
func Equal ¶
Equal reports whether two values are the same value of the same kind. It is the identity used by tests, not the = operator of the language.
func NearlyEqual ¶
NearlyEqual compares two values exactly, except numbers, which may differ by the relative tolerance: the allowance of ADR-0019 for floating-point aggregates across targets.
func ParseCaseDate ¶
ParseCaseDate reads a date as the suite writes it: YYYY-MM-DD or YYYY-MM-DD HH:MM:SS, UTC.
Types ¶
type Array ¶
Array is an array<scalar>, stored column-wise: numbers, dates and booleans in Nums, text, refs and json in Strs, with Nulls marking blank elements. Composite operations arrive with a later change.
type Binary ¶
Binary is an infix operation. Op is one of the arithmetic, text, comparison and logical operator kinds.
type BindingSuite ¶
type BindingSuite struct {
Suite string `json:"suite"`
Description string `json:"description,omitempty"`
Variables map[string]string `json:"variables,omitempty"`
// Mode is "any" (the default) or "auto" (ADR-0015).
Mode string `json:"mode,omitempty"`
Accept []string `json:"accept"`
Reject []RejectCase `json:"reject"`
File string `json:"-"`
}
BindingSuite is one file of definition-time cases for the binder: formulas that must bind and formulas it must refuse, in a mode.
func LoadBindingSuites ¶
func LoadBindingSuites(dir string) ([]*BindingSuite, error)
LoadBindingSuites loads every binding suite from dir, sorted by file name.
type Call ¶
Call is a function call by canonical name. IF, AND, OR and NOT never appear here: the parser turns them into If, Binary and Unary nodes.
type Case ¶
type Case struct {
Name string `json:"name,omitempty"`
Formula string `json:"formula"`
// Inputs are values for declared variables; an absent variable is blank.
// Dates are written as YYYY-MM-DD or YYYY-MM-DD HH:MM:SS.
Inputs map[string]json.RawMessage `json:"inputs,omitempty"`
// Context sets the context functions: user, groups, now (a date).
Context *CaseContext `json:"context,omitempty"`
// Type is the static type the formula must have, when the case pins it.
// A date result must declare it so that Expected can be written as text.
Type string `json:"type,omitempty"`
// Expected is the value, in the same notation as Inputs. Exactly one of
// Expected and Error is set; Expected may be null for a blank result.
Expected json.RawMessage `json:"expected,omitempty"`
// Error is the error code the evaluation must stop with.
Error string `json:"error,omitempty"`
}
Case is one evaluation case: a formula, its inputs and context, and the value or the error it must produce on every target.
type CaseContext ¶
type CaseContext struct {
User string `json:"user,omitempty"`
Groups []string `json:"groups,omitempty"`
Now string `json:"now,omitempty"`
}
CaseContext is the context a case runs in.
func (*CaseContext) Context ¶
func (c *CaseContext) Context() (*Context, error)
Context converts the case context to a VM context; nil for none.
type Class ¶
type Class string
Class is the evaluation class of a function (ADR-0015, ADR-0016). It decides where a formula using the function may appear.
type Context ¶
type Context struct {
// User is the id of the user the formula runs for; empty for a system
// recompute.
User string
// Groups are the groups the user belongs to.
Groups []string
// Now is the current time in seconds since the epoch, UTC. Zero means
// the wall clock.
Now int64
}
Context is what the context functions see (ADR-0015).
type Dialect ¶
type Dialect interface {
Name() string
NumberLit(f float64) string
TextLit(s string) string
True() string
False() string
// Load references a column holding a variable of the kind, cast to the
// SQL type the dialect computes with.
Load(column string, k Kind) string
// Num, Cond and Text read an expression as a number, a condition or
// text with the blank rules of ADR-0005: a blank is 0, FALSE or "".
Num(e SQLExpr) string
Cond(e SQLExpr) string
Text(e SQLExpr) string
// NotFinite is the condition that a numeric value is not a finite
// number, the overflow check of ADR-0019.
NotFinite(val string) string
// Pow spells x to the power y as the VM computes it.
Pow(x, y string) string
// Render applies the TEXT rule of ADR-0019 to a scalar.
Render(e SQLExpr) string
// Call translates a builtin by canonical name. Args are translated
// already; the dialect reads their kinds.
Call(name string, args []SQLExpr, params *Params) (SQLExpr, error)
// Param references a context parameter by name.
Param(name string) string
// ContextArgs binds the context parameters for database/sql, in the
// form the dialect's SQL expects.
ContextArgs(ctx *Context) []any
// Setup lists statements to run on a connection before the generated
// SQL can be used: the DuckDB macros, nothing for SQLite.
Setup() []string
// Limits are the size limits the generated SQL enforces (ADR-0021).
Limits() Limits
// WithLimits returns the dialect under other limits.
WithLimits(l Limits) Dialect
}
Dialect spells one SQL engine.
type DuckDB ¶
type DuckDB struct {
// contains filtered or unexported fields
}
DuckDB is the dialect for DuckDB through github.com/duckdb/duckdb-go.
func NewDuckDB ¶
func NewDuckDB() DuckDB
NewDuckDB makes the DuckDB dialect under the default limits.
func (DuckDB) ContextArgs ¶
ContextArgs implements Dialect: groups bind as a list of text.
func (DuckDB) Pow ¶
Pow implements Dialect: DuckDB's pow is the C library's, which the VM's double-double power matches.
func (DuckDB) WithLimits ¶
WithLimits implements Dialect.
type Error ¶
type Error struct {
// Pos is the byte offset of the error in the formula.
Pos int
// Col is the 1-based column in characters, for people.
Col int
Msg string
}
Error is a lexical or syntax error at a position in a formula.
type EvalError ¶
EvalError is an evaluation error: a code from the closed list, the byte offset of the failing operation in the source when known, and a message.
type Function ¶
type Function struct {
Name string `json:"name"`
Aliases []string `json:"aliases,omitempty"`
Class Class `json:"class"`
Category string `json:"category"`
Params []Param `json:"params"`
Returns string `json:"returns"`
Summary string `json:"summary"`
}
Function is one catalog entry.
func Functions ¶
func Functions() []*Function
Functions returns every catalog entry in catalog order.
func LookupFunction ¶
LookupFunction returns the catalog entry for a function name or alias, case-insensitively, or nil when there is none.
type Kind ¶
type Kind uint8
Kind is the kind of a value, static or at run time (ADR-0013).
const ( KindNull Kind = iota KindBool KindNumber KindText KindDate KindRef KindJSON KindArray KindRows )
The kinds. KindNull is the blank: the static type of the BLANK literal and the run-time kind of a missing value.
type Limits ¶
type Limits struct {
// MaxText is the longest text a formula may produce or a document may
// store, in code points. A longer result is the error out_of_range,
// decided before the text is built.
MaxText int
}
Limits are the size limits a database evaluates under (ADR-0021). They are a property of the database, chosen when it is created and kept with its metadata, so that a formula gives the same result wherever the database is opened and a commit recomputes as it was committed. A server's resource budgets, memory, threads and timeouts, are not limits: they decide whether a recompute runs, never what it returns.
type Mode ¶
type Mode uint8
Mode says where a formula will run, which decides the function classes it may use (ADR-0015).
type Node ¶
type Node interface {
// Pos is the byte offset of the node in the formula: the operator for
// operators, the start otherwise.
Pos() int
}
Node is a node of a parsed formula. The parser produces it; the binder turns it into bytecode. Function names in a Call are canonical upper-case catalog names; variable names are kept as written and resolved case-insensitively by the binder.
func Parse ¶
Parse turns a formula into a syntax tree. It checks the grammar of ADR-0019 and every call against the catalog: the function must exist and the number of arguments must fit. Names, types and function classes are the binder's job.
Precedence from highest to lowest: postfix %, unary -, ^ (right- associative), * / \, + -, &, comparisons, NOT, AND, OR. IF c THEN a ELSE b is an expression whose ELSE branch extends as far right as it can.
type Op ¶
type Op uint8
Op is an opcode. The numbering is part of the format: append only.
const ( OpNop Op = iota OpConstNum // push Nums[arg] OpConstText // push Texts[arg] OpTrue // push TRUE OpFalse // push FALSE OpBlank // push BLANK OpLoad // push the variable in slot arg OpNeg // -x OpPercent // x% OpNot // NOT x OpAdd OpSub OpMul OpDiv // / : by zero yields 0 OpDivStrict // \ : by zero is an error OpPow OpConcat OpEq OpNe OpLt OpLe OpGt OpGe OpJump // pc = arg OpJumpIfFalse // pop; when FALSE or blank, pc = arg OpAndJump // when the top is FALSE or blank, replace it by FALSE and pc = arg; else pop OpOrJump // when the top is TRUE, pc = arg; else pop OpToBool // replace a blank top by FALSE OpCall // call builtin arg>>8 with arg&0xff arguments from the stack OpLookup // reserved: LOOKUP through query slot arg (ADR-0020) OpQuery // reserved: QUERY through query slot arg (ADR-0020) OpReturn // the top is the result )
The opcodes. Arg is the operand of the instruction.
type Param ¶
type Param struct {
Name string `json:"name"`
// Type names the argument's static type, or a shape such as "numbers"
// (a number or an array of numbers), "source" or "condition" for the
// query functions.
Type string `json:"type"`
// Optional: this argument and every one after it may be left out.
Optional bool `json:"optional,omitempty"`
// Repeat: the argument may be given any number of times.
Repeat bool `json:"repeat,omitempty"`
}
Param is one argument of a function.
type Params ¶
type Params struct {
// contains filtered or unexported fields
}
Params records the context parameters a statement uses.
type Program ¶
type Program struct {
Nums []float64
Texts []string
Slots []Slot
Code []Instr
Pos []int32 // byte offset in the source per instruction, for messages
Result Type
Classes []Class // function classes used, in catalog order
Queries []QuerySlot
// MaxStack is the stack depth the program needs.
MaxStack int
}
Program is a compiled formula.
func Bind ¶
func Bind(src string, scope *Scope, opts BindOptions) (*Program, error)
Bind parses and binds a formula against a scope.
func BindNode ¶
BindNode binds a parsed formula. src is the source the tree came from, for positions in messages.
func (*Program) MarshalJSON ¶
MarshalJSON renders the debug form: one line per instruction with the opcode name, the operand, and what the operand refers to.
type QuerySlot ¶
QuerySlot is a compiled LOOKUP or QUERY call (ADR-0020). Reserved: the binder does not emit them yet.
type RegisteredFunc ¶
type RegisteredFunc struct {
// Name is the SQL function name.
Name string
// NArgs is the argument count, the limit included when Limited, or -1
// for variadic.
NArgs int
// Kinds gives the kind expected at each position; the last repeats.
Kinds []Kind
// Fn is the implementation, the VM's own.
Fn func(vm *VM, args []Value) (Value, *EvalError)
// Limited: the last argument is the text limit, and Fn reads it from
// the VM it is called with.
Limited bool
}
RegisteredFunc describes an engine function a SQL dialect calls back into Go (ADR-0019).
func SQLiteFunctions ¶
func SQLiteFunctions() []RegisteredFunc
SQLiteFunctions lists the engine functions the SQLite dialect expects to be registered, as fado_<name> and fado_<name>_err. The sqlitefn package registers them from this list.
func (RegisteredFunc) Kind ¶
func (f RegisteredFunc) Kind(i int) Kind
Kind returns the kind expected at argument position i.
type RejectCase ¶
RejectCase is a formula that must be refused, with the reason for people.
type SQL ¶
type SQL struct {
// Value is the value expression.
Value string
// Error is the error expression, or "" when the formula cannot fail.
Error string
// Kind is the kind of the value expression.
Kind Kind
// Params lists the context parameters the expressions reference, by
// name: now (seconds since the epoch, always bound, never NULL), user
// (the user id or NULL) and groups (the user's groups or NULL, in the
// form the dialect's ContextArgs binds).
Params []string
}
SQL is a generated statement fragment.
func GenerateSQL ¶
GenerateSQL translates a program for a dialect. column gives the column expression that holds each slot's value.
func (SQL) ErrorOrNull ¶
ErrorOrNull returns the error expression, or NULL when there is none.
type SQLExpr ¶
type SQLExpr struct {
// Val is the value expression.
Val string
// Err is the error expression, or "" when the expression cannot fail.
Err string
// Kind is the static kind of Val.
Kind Kind
// Sure reports that Val is never NULL, so no COALESCE is needed.
Sure bool
}
SQLExpr is one translated expression with its static kind.
type SQLResult ¶
type SQLResult struct {
// Query is the statement that ran, for messages.
Query string
// Value is the value column as a Value of the program's result kind.
Value Value
// Code is the error column, or "" when NULL.
Code string
}
SQLResult is what RunSQL got back from the engine.
func RunSQL ¶
RunSQL executes one program through database/sql on a dialect: it generates the SQL, binds vars, one value per program slot in slot order, as named parameters over a one-row subselect, binds the context, and reads the value and the error columns. It is the shared half of every SQL conformance and property runner; comparing with the VM is the runner's job.
type SQLite ¶
type SQLite struct {
// contains filtered or unexported fields
}
SQLite is the dialect for SQLite through modernc.org/sqlite.
func NewSQLite ¶
func NewSQLite() SQLite
NewSQLite makes the SQLite dialect under the default limits.
func (SQLite) ContextArgs ¶
ContextArgs implements Dialect: now is always bound, user is text or NULL, groups is a JSON array or NULL.
func (SQLite) NotFinite ¶
NotFinite implements Dialect. SQLite yields NULL for NaN and keeps infinities, so both are checked.
func (SQLite) Setup ¶
Setup implements Dialect: the engine functions come from the sqlitefn package's registration, not from statements.
func (SQLite) WithLimits ¶
WithLimits implements Dialect.
type Scope ¶
type Scope struct {
// contains filtered or unexported fields
}
Scope is the set of variables a formula may read. Names resolve case-insensitively (ADR-0013).
type Suite ¶
type Suite struct {
// Suite is the short name, unique across files.
Suite string `json:"suite"`
Description string `json:"description,omitempty"`
// Variables declares the input variables the cases may set, by type:
// bool, number, text, date, ref, json, array<scalar>, rows.
Variables map[string]string `json:"variables,omitempty"`
// Limits sets the size limits the cases evaluate under (ADR-0021);
// absent, the defaults. A suite at a small limit reaches the boundary
// with short texts.
Limits *SuiteLimits `json:"limits,omitempty"`
Cases []Case `json:"cases"`
// File is where the suite was loaded from, for messages.
File string `json:"-"`
}
Suite is one file of evaluation cases.
func LoadSuites ¶
LoadSuites loads every evaluation suite from dir, sorted by file name.
func (*Suite) EvalLimits ¶
EvalLimits are the limits the suite's cases evaluate under.
type SuiteLimits ¶
type SuiteLimits struct {
MaxText int `json:"max_text"`
}
SuiteLimits is the limits field of a suite.
type SyntaxSuite ¶
type SyntaxSuite struct {
Suite string `json:"suite"`
Description string `json:"description,omitempty"`
Variables map[string]string `json:"variables,omitempty"`
Accept []string `json:"accept"`
Reject []RejectCase `json:"reject"`
File string `json:"-"`
}
SyntaxSuite is one file of definition-time cases: formulas the parser must accept and formulas it must reject.
func LoadSyntaxSuites ¶
func LoadSyntaxSuites(dir string) ([]*SyntaxSuite, error)
LoadSyntaxSuites loads every syntax suite from dir, sorted by file name.
type Token ¶
type Token struct {
Kind TokenKind
// Text is the name as written for a name, the unescaped value for a
// text literal, and the source text otherwise.
Text string
// Num is the value of a number literal.
Num float64
// Pos is the byte offset of the token in the formula.
Pos int
}
Token is one lexical token with its position in the formula.
type TokenKind ¶
type TokenKind uint8
TokenKind is the kind of a lexical token.
const ( TokEOF TokenKind = iota TokNumber TokText TokIdent TokAnd TokOr TokNot TokIf TokThen TokElse TokTrue TokFalse TokBlank TokThis TokLParen TokRParen TokComma TokDot TokPlus TokMinus TokStar TokSlash TokBackslash TokCaret TokPercent TokAmp TokEq TokNe TokLt TokLe TokGt TokGe )
Token kinds. Keywords are case-insensitive and reserved (ADR-0019).
type Type ¶
type Type struct {
Kind Kind
// Elem is the element kind of an array.
Elem Kind
// Cols is the schema of rows; nil when unknown.
Cols []Column
}
Type is a static type: a scalar kind, an array of a scalar kind, or rows with a schema.
func ParseType ¶
ParseType reads a type as written in class definitions and in the conformance suite: a scalar name, array<scalar>, or rows. It also reads "blank", the static type of a formula that is BLANK, so that every type String writes decodes again; a variable is never declared blank, which is the declaring side's check.
type VM ¶
type VM struct {
// Limits are the size limits of the database the programs belong to
// (ADR-0021). NewVM sets the defaults; set others before Run.
Limits Limits
// contains filtered or unexported fields
}
VM executes programs. A VM is not safe for concurrent use; make one per goroutine. Its stack is reused across runs.
type Value ¶
type Value struct {
// contains filtered or unexported fields
}
Value is a run-time value. Scalars live in the struct and never allocate: bool, number and date in num, text, ref and json in str. Composites (ADR-0013) will hang off pointers added with the composites change; the scalar layout does not move. The zero Value is blank.
func ParseCaseValue ¶
func ParseCaseValue(raw json.RawMessage, t Type) (Value, error)
ParseCaseValue reads a case value written in the suite notation as a value of the given type.
func ValueFromDriver ¶
ValueFromDriver converts a value as a database driver delivers it (nil, int64, float64, string or []byte) to a Value of the expected kind. KindNull means "whatever it is": numbers become numbers and text text.
func (Value) DriverValue ¶
DriverValue converts a Value to what a database driver takes: nil, float64, int64 or string. Dates are REAL seconds, booleans 0 and 1.
func (Value) Num ¶
Num returns the number of a number, the 1 or 0 of a bool, the seconds of a date, and 0 for a blank: the zero of ADR-0005.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package fuzz generates random, well-typed formulas with inputs and runs each one on the Go VM and on a SQL dialect, reporting any divergence.
|
Package fuzz generates random, well-typed formulas with inputs and runs each one on the Go VM and on a SQL dialect, reporting any divergence. |
|
Package sqlitefn registers the formula engine's Go functions into the pure-Go SQLite driver, so that the SQLite dialect of the SQL generator agrees with the Go VM by construction (ADR-0019).
|
Package sqlitefn registers the formula engine's Go functions into the pure-Go SQLite driver, so that the SQLite dialect of the SQL generator agrees with the Go VM by construction (ADR-0019). |