Documentation
¶
Overview ¶
Package sqlast parses the read statements an agent may run (SELECT, WITH ... SELECT and EXPLAIN SELECT) into a syntax tree, per dialect, and analyses it: it resolves every column to the base columns it comes from (through aliases, functions, expressions, subqueries, CTEs, set operations and joins), checks functions against an allowlist and decides how PII columns may be used.
ParseWrite reads, for the same analysis, the writes whose values come back (RETURNING, a SELECT over data-modifying CTEs) or that store a query's rows (INSERT ... SELECT).
The grammar is a deliberately small subset of SQL. Anything outside it is refused: a statement the parser cannot read with certainty never runs.
Index ¶
- func HasPlaceholder(d sqlclass.Dialect, sql string) bool
- func KnownFunction(name string) bool
- func NeedsWriteAnalysis(d sqlclass.Dialect, sql string) bool
- func StringValue(d sqlclass.Dialect, text string) (string, bool)
- type Analysis
- type Assignment
- type Between
- type Binary
- type Body
- type CTE
- type Case
- type Cast
- type Catalog
- type Collate
- type ColumnRef
- type Derived
- type Env
- type Exists
- type Expr
- type FuncCall
- type In
- type IsTest
- type Join
- type KCheck
- type Kind
- type Like
- type Limit
- type LitKind
- type Literal
- type OrderItem
- type Output
- type Paren
- type Prov
- type Query
- type Replacement
- type Select
- type SelectItem
- type SetOp
- type Source
- type Span
- type Statement
- type Subquery
- type Table
- type TableExpr
- type TableName
- type Tuple
- type Unary
- type Use
- type ValueKind
- type ValueUse
- type When
- type Window
- type With
- type Write
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func HasPlaceholder ¶ added in v0.1.2
HasPlaceholder reports whether sql holds a placeholder: a plain string literal '${...}', or "${" in any other string form. A statement that cannot be lexed is reported as holding one.
func KnownFunction ¶ added in v0.5.0
KnownFunction reports whether name (folded) is in the allowlist of any dialect.
func NeedsWriteAnalysis ¶ added in v0.5.0
NeedsWriteAnalysis reports whether a write statement must pass the write analysis (ParseWrite, then Analyze) before it runs while mask rules exist: its rows come back (a RETURNING list, or a statement led by WITH whose main statement is a query), or it is an INSERT or REPLACE that holds a query (its source, a CTE or a subquery in VALUES). A statement that cannot be lexed reports true: the parser refuses it.
func StringValue ¶ added in v0.1.2
StringValue returns the value of a string-literal token ('...', "..." in MySQL, E'...', N'...', $tag$...$tag$), or false for any other text. Unlike the substitution path it does not refuse backslashes: it serves warnings.
Types ¶
type Analysis ¶
type Analysis struct {
Outputs []Output
// Relations are the base relations read, "db.table".
Relations []string
// Uses are the PII columns the statement touches.
Uses []Use
// KChecks must all pass (k rows or more) before the statement runs.
KChecks []KCheck
// PIIFilter is set when a PII column is compared with a constant: row
// estimates must not reach the agent.
PIIFilter bool
// LitFilter is set when a PII column is compared for equality with a
// literal the agent wrote (not a placeholder, not IS NULL), anywhere in
// the statement. Its results carry no cell references: the agent chose
// the value behind them, and the rows it selected may be linked to the
// literal through any join, subquery or set operation.
LitFilter bool
Replacements []Replacement
// Values are the placeholders compared with PII columns.
Values []ValueUse
// KeyFilters are the non-PII columns compared with = and a literal in
// WHERE (the console checks whether one is a unique key).
KeyFilters []Source
}
Analysis is what Analyze finds.
func Analyze ¶
Analyze resolves the provenance of every output column of st, checks the functions against the allowlist and, when env.Masking is set, applies the PII usage rules. Errors are *sqlclass.Refusal values.
type Assignment ¶ added in v0.5.0
Assignment is one "column = value" of a SET list. Value is nil for DEFAULT. Qualifier is the MySQL "t." of a qualified column.
type Body ¶
type Body interface {
// contains filtered or unexported methods
}
Body is a *Select, a *SetOp or a parenthesised *Query.
type CTE ¶
CTE is one common table expression: a query, or (PostgreSQL, from ParseWrite) a data-modifying statement whose RETURNING list gives its columns.
type Catalog ¶
type Catalog interface {
// Lookup returns the relations a table name (folded parts, as written)
// may denote: none when it is unknown, several when it is ambiguous.
Lookup(parts []string) ([]Table, error)
}
Catalog resolves table names.
type Env ¶
type Env struct {
Catalog Catalog
// Rule returns the mask mode of a base column, and whether a rule
// covers it.
Rule func(Source) (mode string, masked bool)
// Masking is false for an unmask plan: provenance and functions are
// still checked, but not the PII usage rules.
Masking bool
// Value resolves a placeholder: the value the human typed for a name, or
// the value of a referenced cell. ok is false when the console does not
// know it (yet, for a typed name).
Value func(kind ValueKind, name string) (value string, ok bool)
}
Env is what Analyze needs besides the statement.
type FuncCall ¶
type FuncCall struct {
// Name is the folded, dot-joined function name.
Name string
Args []Expr
Star bool // COUNT(*)
Distinct bool
// OrderBy is an ordered-set argument: string_agg(x, ',' ORDER BY y),
// GROUP_CONCAT(x ORDER BY y).
OrderBy []*OrderItem
// Separator is the SEPARATOR string of a MySQL GROUP_CONCAT.
Separator *Literal
Filter Expr
Over *Window
Sp Span
}
FuncCall is a function call, aggregate or window function.
type IsTest ¶
type IsTest struct {
X Expr
Not bool
What string // NULL, TRUE, FALSE, UNKNOWN, DISTINCT
Y Expr // DISTINCT FROM y
Sp Span
}
IsTest is x IS [NOT] NULL|TRUE|FALSE|UNKNOWN or IS [NOT] DISTINCT FROM y.
type Join ¶
type Join struct {
Kind string // INNER, LEFT, RIGHT, FULL, CROSS
Natural bool
Left, Right TableExpr
On Expr
Using []string
Sp Span
}
Join joins two table expressions.
type KCheck ¶
type KCheck struct {
SQL string
// Grouped checks the smallest group (its result may be NULL when no
// group remains); otherwise it counts the rows.
Grouped bool
}
KCheck is a row-count query the console runs before the statement, to check that a PII filter, grouping or aggregate covers at least k rows.
type Kind ¶
type Kind int
Kind says how a value derives from its sources.
const ( // KindConst involves no column. KindConst Kind = iota // KindCount is a row count (COUNT): it reveals no value of its sources. KindCount // KindIdentity is a value of one of its sources: a plain reference, // through aliases, subqueries, CTEs and set operations, or MIN/MAX. KindIdentity // KindAggregate is another aggregate of its sources (SUM, AVG, ...). KindAggregate // KindExpr is any other expression of its sources. KindExpr )
Kinds, from the least to the most transformed.
type Limit ¶
type Limit struct {
Count, Offset int64
}
Limit is a LIMIT/OFFSET or FETCH FIRST clause; both are literal integers.
type Literal ¶
type Literal struct {
Kind LitKind
// Text is the source text of the literal (with its quotes).
Text string
Sp Span
}
Literal is a constant.
type Output ¶
type Output struct {
// Label is the column label the engine must report (an alias or a
// plain column name), "" when the engine chooses it.
Label string
Prov Prov
// Mask is the mask mode of the column, "" for none.
Mask string
}
Output is one result column.
type Prov ¶
type Prov struct {
Sources []Source
Kind Kind
// Sensitive is set when a source is under a mask rule and the value
// reveals source values (Kind Identity, Aggregate or Expr).
Sensitive bool
// Modes are the mask modes of the sensitive sources.
Modes []string
// Lit is set when some values may be literals of the statement (a
// UNION of a column and a constant, for instance).
Lit bool
// contains filtered or unexported fields
}
Prov is the provenance of a value.
type Query ¶
Query is a full query expression: an optional WITH, a body and the ORDER BY and LIMIT that apply to the whole body.
type Replacement ¶
Replacement substitutes a token literal with the value it stands for in the statement that runs.
type Select ¶
type Select struct {
Distinct bool
DistinctOn []Expr
Items []*SelectItem
From []TableExpr
Where Expr
GroupBy []Expr
Having Expr
Sp Span
// Spans of the FROM list, the WHERE condition, the GROUP BY list and
// the HAVING condition (without their keywords); zero when absent.
FromSp, WhereSp, GroupSp, HavingSp Span
}
Select is one SELECT core.
type SelectItem ¶
SelectItem is one select-list entry: an expression with an optional alias, or a star (optionally qualified: t.*).
type Source ¶
type Source struct {
DB, Table, Column string
// View marks a column of a view (or of a relation whose base columns
// are unknown): rules match it by column name.
View bool
}
Source is a base column, as the catalog names it. For PostgreSQL DB is the schema.
type Statement ¶
type Statement struct {
// Explain is set for EXPLAIN SELECT: the console runs the EXPLAIN
// itself, without ANALYZE.
Explain bool
Query *Query
// Write is set instead of Query for an INSERT, REPLACE, UPDATE or
// DELETE.
Write *Write
// SQL is the statement text the spans refer to.
SQL string
Dialect sqlclass.Dialect
}
Statement is one parsed statement: a read (Query) or, from ParseWrite, a write (Write).
func Parse ¶
Parse parses one read statement: SELECT, WITH ... SELECT or EXPLAIN followed by one of them. Anything else, or any construct outside the supported grammar, is refused with a *sqlclass.Refusal. A trailing ';' and surrounding blanks are dropped, as sqlclass.Classify does.
func ParseWrite ¶ added in v0.5.0
ParseWrite parses one data-modifying statement for the write analysis (see NeedsWriteAnalysis): an INSERT, REPLACE, UPDATE or DELETE, possibly led by a WITH, or (PostgreSQL) a SELECT led by a WITH whose CTEs modify data. The grammar is the read grammar plus single-table writes:
- INSERT and REPLACE with a column list, VALUES rows (DEFAULT allowed), DEFAULT VALUES, a query, or a MySQL SET list; PostgreSQL and SQLite ON CONFLICT ... DO NOTHING | DO UPDATE SET, MySQL ON DUPLICATE KEY UPDATE;
- UPDATE with a SET list (PostgreSQL (a, b) = (x, y) lists of values), FROM (PostgreSQL, SQLite) and WHERE;
- DELETE FROM with USING (PostgreSQL) and WHERE;
- MySQL ORDER BY and LIMIT on UPDATE and DELETE;
- a RETURNING list, read as a select list.
Anything else (MERGE, multi-table MySQL writes, ONLY, WHERE CURRENT OF, subfield or subscript targets, a subquery assigned to a column list, PostgreSQL 18 RETURNING WITH, ...) is refused, as is a statement that modifies nothing.
type TableExpr ¶
type TableExpr interface {
// contains filtered or unexported methods
}
TableExpr is a *TableName, a *Derived or a *Join.
type TableName ¶
type TableName struct {
// Parts are the folded name parts: [table], [db, table] or
// [catalog, schema, table].
Parts []string
Alias string
ColAliases []string
Sp Span
}
TableName is a named relation: a table, a view or a CTE.
type ValueKind ¶ added in v0.1.2
type ValueKind int
ValueKind tells a value typed by the human from a cell reference.
type ValueUse ¶ added in v0.1.2
type ValueUse struct {
Kind ValueKind
// Name is "email" for '${email}', "r3.1.2" for '${r3.1.2}'.
Name string
// Column is the PII column the placeholder is compared with.
Column Source
// InList is set inside IN (...).
InList bool
Span Span
}
ValueUse is a placeholder literal compared with a PII column.
type Write ¶ added in v0.5.0
type Write struct {
With *With
Verb string // INSERT, REPLACE, UPDATE or DELETE
// Target is the relation written.
Target *TableName
// Columns is the column list of an INSERT, nil when absent.
Columns []string
// Rows are the VALUES rows of an INSERT; a nil item is DEFAULT.
Rows [][]Expr
// Source is the query an INSERT stores.
Source *Query
// Set is the SET list of an UPDATE or of a MySQL INSERT ... SET.
Set []*Assignment
// From is the FROM list of an UPDATE or the USING list of a DELETE.
From []TableExpr
Where Expr
// OrderBy and Limit are the MySQL and SQLite ORDER BY and LIMIT of an
// UPDATE or a DELETE.
OrderBy []*OrderItem
Limit *Limit
// Conflict is the conflict target of an ON CONFLICT clause (index
// expressions and their WHERE); Upsert and UpsertWhere are its DO
// UPDATE SET list and condition, or the ON DUPLICATE KEY UPDATE list.
Conflict []Expr
ConflictWhere Expr
Upsert []*Assignment
UpsertWhere Expr
// Returning is the RETURNING list, nil when absent.
Returning []*SelectItem
Sp Span
}
Write is a data-modifying statement: INSERT, REPLACE, UPDATE or DELETE.