README
¶
Generated Documentation and Embedded Help Texts
Many parts of the parser package include special consideration for generation
or other production of user-facing documentation. This includes interactive help
messages, generated documentation of the set of available functions, or diagrams
of the various expressions.
Generated documentation is produced and maintained at compile time, while the interactive, contextual help is returned at runtime.
We equip the generated parser with the ability to report contextual help in two circumstances:
- when the user explicitly requests help with the HELPTOKEN (current syntax: standalone "
??") - when the user makes a grammatical mistake (e.g.
INSERT sometable INTO(x, y) ...)
We use the docgen tool to produce the generated documentation files that are
then included in the broader (handwritten) published documentation.
Help texts embedded in the grammar
The help is embedded in the grammar using special markers in yacc comments, for example:
// %Help: HELPKEY - shortdescription
// %Category: SomeCat
// %Text: whatever until next %marker at start of line, or non-comment.
// %SeeAlso: whatever until next %marker at start of line, or non-comment.
// %End (optional)
The "HELPKEY" becomes the map key in the generated Go map.
These texts are extracted automatically by help.awk and converted
into a Go data structure in help_messages.go.
Support in the parser
Primary mechanism - LALR error recovery
The primary mechanism is leveraging error recovery in LALR parsers
using the special error token [1] [2]: when an unexpected token is
encountered, the LALR parser will pop tokens on the stack until the
prefix matches a grammar rule with the special "error" token (if
any). If such a rule exists, its action is used to reduce and the
erroneous tokens are discarded.
This mechanism is used both when the user makes a mistake, and when the user inserts the HELPTOKEN in the middle of a statement. When present in the middle of a statement, HELPTOKEN is considered an error and triggers the error recovery.
We use this for contextual help by providing error rules that
generate a contextual help text during LALR error recovery.
For example:
backup_stmt:
BACKUP targets TO string_or_placeholder opt_as_of_clause opt_incremental opt_with_options
{
$$.val = &Backup{Targets: $2.targetList(), To: $4.expr(), IncrementalFrom: $6.exprs(), AsOf: $5.asOfClause(), Options: $7.kvOptions()}
}
| BACKUP error { return helpWith(sqllex, `BACKUP`) }
In this example, the grammar specifies that if the BACKUP keyword is
followed by some input tokens such that the first (valid) grammar rule
doesn't apply, the parser will "recover from the error" by
backtracking up until the point it only sees BACKUP on the stack
followed by non-parsable tokens, at which points it takes the error
rule and executes its action.
The action is return helpWith(...). What this does is:
- halts parsing (the generated parser executes all actions
in a big loop; a
returninterrupts this loop); - makes the parser return with an error (the
helpWithfunction returns non-zero); - extends the parsing error message with a help text; this help text can subsequently be exploited in a client to display the help message in a friendly manner.
Code generation
Since the pattern "{ return helpWith(sqllex, ...) }" is common, we also implement
a shorthand syntax based on comments, for example:
backup_stmt:
...
| BACKUP error // SHOW HELP: BACKUP
The special comment syntax "SHOW HELP: XXXX" is substituted by means
of an auxiliary script (replace_help_rules.awk) into the form
explained above.
Secondary mechanism - explicit help token
The mechanism described above works both when the user make a grammatical error and when they place the HELPTOKEN in the middle of a statement, rendering it invalid.
However for contextual help this is not sufficient: what happens if the user requests HELPTOKEN at a position in the grammar where everything before is a complete, valid SQL input?
For example: DELETE FROM foo ?
When encountering this input, the LALR parser will see DELETE FROM foo first, then reduce using the DELETE action because everything
up to this point is a valid DELETE statement. When the HELPTOKEN is
encountered, the statement has already been completed and the LALR
parser doesn't 'know' any more that it was in the context of a DELETE
statement.
If we try to place an error-based recovery rule at the top-level:
stmt:
alter_stmt
| backup_stmt
| ...
| delete_stmt
| ...
| error { ??? }
This wouldn't work: the code inside the error action cannot
"observe" the tokens observed so far and there would be no way to know
whether the error should be about DELETE, or instead about ALTER,
BACKUP, etc.
So in order to handle HELPTOKEN after a valid statement, we must place it in a rule where the context is still available, that is before the statement's grammar rule is reduced.
Where would that be? Suppose we had a simple statement rule:
somesimplestmt:
SIMPLE DO SOMETHING { $$ = reduce(...) }
| SIMPLE error { help ... }
We could extend with:
somesimplestmt:
SIMPLE DO SOMETHING { $$ = reduce(...) }
| SIMPLE DO SOMETHING HELPTOKEN { help ... }
| SIMPLE error { help ... }
(the alternative also works:
somesimplestmt:
SIMPLE DO SOMETHING { $$ = reduce(...) }
| SIMPLE DO SOMETHING error { help ... }
| SIMPLE error { help ... }
)
That is all fine and dandy, but in SQL we have statements with many alternate forms, for example:
alter_rename_table_stmt:
ALTER TABLE relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE relation_expr RENAME opt_column name TO name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME opt_column name TO name { ... }
To add complementary handling of the help token at the end of valid statements we could, but would hate to, duplicate all the rules:
alter_rename_table_stmt:
ALTER TABLE relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE relation_expr RENAME TO qualified_name HELPTOKEN { help ... }
| ALTER TABLE IF EXISTS relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME TO qualified_name HELPTOKEN { help ... }
| ALTER TABLE relation_expr RENAME opt_column name TO name { ... }
| ALTER TABLE relation_expr RENAME opt_column name TO name HELPTOKEN { help ... }
| ALTER TABLE IF EXISTS relation_expr RENAME opt_column name TO name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME opt_column name TO name HELPTOKEN { help ... }
This duplication is horrendous (not to mention hard to maintain), so instead we should attempt to factor the help token in a context where it is still known that we are dealing just with that statement.
The following works:
alter_rename_table_stmt:
real_alter_rename_table_stmt { $$ = $1 }
| real_alter_rename_table_stmt HELPTOKEN { help ... }
real_alter_rename_table_stmt:
ALTER TABLE relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE relation_expr RENAME opt_column name TO name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME opt_column name TO name { ... }
Or does it? Without anything else, yacc complains with a "shift/reduce
conflict". The reason is coming from the ambiguity: when the parsing
stack contains everything sufficient to match a
real_alter_rename_table_stmt, there is a choice between reducing
the simple form alter_rename_table_stmt: real_alter_rename_table_stmt, or shifting into the more complex
form alter_rename_table_stmt: real_alter_rename_table_stmt HELPTOKEN.
This is another form of the textbook situation when yacc is used to
parse if-else statements in a programming language: the rule stmt: IF cond THEN body | IF cond THEN body ELSE body is ambiguous (and yields
a shift/reduce conflict) for exactly the same reason.
The solution here is also straight out of a textbook: one simply informs yacc of the relative priority between the two candidate rules. In this case, when faced with a neutral choice, we encourage yacc to shift. The particular mechanism is to tell yacc that one rule has a higher priority than another.
It just so happens however that the yacc language only allows us to set relative priorities of tokens, not rules. And here we have a problem, of the two rules that need to be prioritized, only one has a token to work with (the one with HELPTOKEN). Which token should we prioritize for the other?
Conveniently yacc knows about this trouble and offers us an awkward,
but working solution: we can tell it "use for this rule the same
priority level as an existing token, even though the token is not part
of the rule". The syntax for this is rule %prec TOKEN.
We can then use this as follows:
alter_rename_table_stmt:
real_alter_rename_table_stmt %prec LOWTOKEN { $$ = $1 }
| real_alter_rename_table_stmt HELPTOKEN %prec HIGHTOKEN { help ... }
We could create two new pseudo-tokens for this (called LOWTOKEN and
HIGHTOKEN) however conveniently we can also reuse otherwise valid
tokens that have known relative priorities. We settled in our case on
VALUES (low priority) and UMINUS (high priority).
Code generation
With the latter mechanism presented above the pattern
rule:
somerule %prec VALUES
| somerule HELPTOKEN %prec UMINUS { help ...}`
becomes super common, so we automate it with the following special syntax:
rule:
somerule // EXTEND WITH HELP: XXX
And the code replacement in replace_help_rules.awk expands this to
the form above automatically.
Generated Documentation
Documentation of the SQL functions and operators is generated by the docgen
utility, using make generate PKG=./docs/.... The markdown-formatted files are
kept in docs/generated/sql and should be re-generated whenever the
functions/operators they document change, and indeed if regenerating produces a
diff, a CI failure is expected.
References
Documentation
¶
Index ¶
- Constants
- Variables
- func HasMultipleStatements(sql string) bool
- func LastLexicalToken(sql string) (lastTok int, ok bool)
- func ParseExpr(sql string) (tree.Expr, error)
- func ParseExprs(sql []string) (tree.Exprs, error)
- func ParseQualifiedTableName(sql string) (*tree.TableName, error)
- func ParseTableName(sql string) (*tree.UnresolvedObjectName, error)
- func ParseTableNameWithQualifiedNames(sql string) (*tree.UnresolvedObjectName, error)
- func ParseType(sql string) (tree.ResolvableTypeReference, error)
- func RunShowSyntax(ctx context.Context, stmt string, ...)
- func SplitFirstStatement(sql string) (pos int, ok bool)
- type HelpMessage
- type HelpMessageBody
- type Parser
- type Statement
- type Statements
- type TokenString
Constants ¶
const ABORT = lex.ABORT
const ACCESS = lex.ACCESS
const ACTION = lex.ACTION
const ADD = lex.ADD
const ADMIN = lex.ADMIN
const AFTER = lex.AFTER
const AGGREGATE = lex.AGGREGATE
const ALIGNMENT = lex.ALIGNMENT
const ALL = lex.ALL
const ALLOW_CONNECTIONS = lex.ALLOW_CONNECTIONS
const ALTER = lex.ALTER
const ALWAYS = lex.ALWAYS
const ANALYSE = lex.ANALYSE
const ANALYZE = lex.ANALYZE
const AND = lex.AND
const AND_AND = lex.AND_AND
const ANNOTATE_TYPE = lex.ANNOTATE_TYPE
const ANY = lex.ANY
const ARRAY = lex.ARRAY
const AS = lex.AS
const ASC = lex.ASC
const ASSIGNMENT = lex.ASSIGNMENT
const ASYMMETRIC = lex.ASYMMETRIC
const AS_LA = lex.AS_LA
const AT = lex.AT
const ATOMIC = lex.ATOMIC
const ATTACH = lex.ATTACH
const ATTRIBUTE = lex.ATTRIBUTE
const AUTHORIZATION = lex.AUTHORIZATION
const AUTO = lex.AUTO
const AUTOMATIC = lex.AUTOMATIC
const BACKUP = lex.BACKUP
const BACKUPS = lex.BACKUPS
const BASETYPE = lex.BASETYPE
const BCONST = lex.BCONST
const BEFORE = lex.BEFORE
const BEGIN = lex.BEGIN
const BETWEEN = lex.BETWEEN
const BIGINT = lex.BIGINT
const BIGSERIAL = lex.BIGSERIAL
const BINARY = lex.BINARY
const BIT = lex.BIT
const BITCONST = lex.BITCONST
const BLOCK_COMMENT = lex.BLOCK_COMMENT
const BOOLEAN = lex.BOOLEAN
const BOTH = lex.BOTH
const BOX2D = lex.BOX2D
const BPCHAR = lex.BPCHAR
const BUCKET_COUNT = lex.BUCKET_COUNT
const BUFFER_USAGE_LIMIT = lex.BUFFER_USAGE_LIMIT
const BUNDLE = lex.BUNDLE
const BY = lex.BY
const BYPASSRLS = lex.BYPASSRLS
const CACHE = lex.CACHE
const CALL = lex.CALL
const CALLED = lex.CALLED
const CANCEL = lex.CANCEL
const CANCELQUERY = lex.CANCELQUERY
const CANONICAL = lex.CANONICAL
const CASCADE = lex.CASCADE
const CASCADED = lex.CASCADED
const CASE = lex.CASE
const CAST = lex.CAST
const CATEGORY = lex.CATEGORY
const CBRT = lex.CBRT
const CHAIN = lex.CHAIN
const CHANGEFEED = lex.CHANGEFEED
const CHAR = lex.CHAR
const CHARACTER = lex.CHARACTER
const CHARACTERISTICS = lex.CHARACTERISTICS
const CHECK = lex.CHECK
const CHECK_OPTION = lex.CHECK_OPTION
const CLASS = lex.CLASS
const CLOSE = lex.CLOSE
const CLUSTER = lex.CLUSTER
const COALESCE = lex.COALESCE
const COLLATABLE = lex.COLLATABLE
const COLLATE = lex.COLLATE
const COLLATION = lex.COLLATION
const COLLATION_VERSION = lex.COLLATION_VERSION
const COLUMN = lex.COLUMN
const COLUMNS = lex.COLUMNS
const COMBINEFUNC = lex.COMBINEFUNC
const COMMENT = lex.COMMENT
const COMMENTS = lex.COMMENTS
const COMMIT = lex.COMMIT
const COMMITTED = lex.COMMITTED
const COMMUTATOR = lex.COMMUTATOR
const COMPACT = lex.COMPACT
const COMPLETE = lex.COMPLETE
const COMPRESSION = lex.COMPRESSION
const CONCAT = lex.CONCAT
const CONCURRENTLY = lex.CONCURRENTLY
const CONFIGURATION = lex.CONFIGURATION
const CONFIGURATIONS = lex.CONFIGURATIONS
const CONFIGURE = lex.CONFIGURE
const CONFLICT = lex.CONFLICT
const CONNECT = lex.CONNECT
const CONNECTION = lex.CONNECTION
const CONSTRAINT = lex.CONSTRAINT
const CONSTRAINTS = lex.CONSTRAINTS
const CONTAINED_BY = lex.CONTAINED_BY
const CONTAINS = lex.CONTAINS
const CONTROLCHANGEFEED = lex.CONTROLCHANGEFEED
const CONTROLJOB = lex.CONTROLJOB
const CONVERSION = lex.CONVERSION
const CONVERT = lex.CONVERT
const COPY = lex.COPY
const COSINE_DISTANCE = lex.COSINE_DISTANCE
const COST = lex.COST
const CREATE = lex.CREATE
const CREATEDB = lex.CREATEDB
const CREATELOGIN = lex.CREATELOGIN
const CREATEROLE = lex.CREATEROLE
const CROSS = lex.CROSS
const CSV = lex.CSV
const CUBE = lex.CUBE
const CURRENT = lex.CURRENT
const CURRENT_CATALOG = lex.CURRENT_CATALOG
const CURRENT_DATE = lex.CURRENT_DATE
const CURRENT_ROLE = lex.CURRENT_ROLE
const CURRENT_SCHEMA = lex.CURRENT_SCHEMA
const CURRENT_TIME = lex.CURRENT_TIME
const CURRENT_TIMESTAMP = lex.CURRENT_TIMESTAMP
const CURRENT_USER = lex.CURRENT_USER
const CYCLE = lex.CYCLE
const DATA = lex.DATA
const DATABASE = lex.DATABASE
const DATABASES = lex.DATABASES
const DATE = lex.DATE
const DAY = lex.DAY
const DEALLOCATE = lex.DEALLOCATE
const DEC = lex.DEC
const DECIMAL = lex.DECIMAL
const DECLARE = lex.DECLARE
const DEFAULT = lex.DEFAULT
const DEFAULTS = lex.DEFAULTS
const DEFERRABLE = lex.DEFERRABLE
const DEFERRED = lex.DEFERRED
const DEFINER = lex.DEFINER
const DELETE = lex.DELETE
const DELIMITER = lex.DELIMITER
const DEPENDS = lex.DEPENDS
const DESC = lex.DESC
const DESCRIBE = lex.DESCRIBE
const DESERIALFUNC = lex.DESERIALFUNC
const DESTINATION = lex.DESTINATION
const DETACH = lex.DETACH
const DETACHED = lex.DETACHED
const DICTIONARY = lex.DICTIONARY
const DISABLE = lex.DISABLE
const DISABLE_PAGE_SKIPPING = lex.DISABLE_PAGE_SKIPPING
const DISCARD = lex.DISCARD
const DISTINCT = lex.DISTINCT
const DO = lex.DO
const DOMAIN = lex.DOMAIN
const DOT_DOT = lex.DOT_DOT
const DOUBLE = lex.DOUBLE
const DROP = lex.DROP
const EACH = lex.EACH
const ELEMENT = lex.ELEMENT
const ELSE = lex.ELSE
const ENABLE = lex.ENABLE
const ENCODING = lex.ENCODING
const ENCRYPTED = lex.ENCRYPTED
const ENCRYPTION_PASSPHRASE = lex.ENCRYPTION_PASSPHRASE
const END = lex.END
const ENUM = lex.ENUM
const ENUMS = lex.ENUMS
const ERROR = lex.ERROR
const ESCAPE = lex.ESCAPE
const EVENT = lex.EVENT
const EXCEPT = lex.EXCEPT
const EXCLUDE = lex.EXCLUDE
const EXCLUDING = lex.EXCLUDING
const EXECUTE = lex.EXECUTE
const EXECUTION = lex.EXECUTION
const EXISTS = lex.EXISTS
const EXPERIMENTAL = lex.EXPERIMENTAL
const EXPERIMENTAL_AUDIT = lex.EXPERIMENTAL_AUDIT
const EXPERIMENTAL_FINGERPRINTS = lex.EXPERIMENTAL_FINGERPRINTS
const EXPERIMENTAL_REPLICA = lex.EXPERIMENTAL_REPLICA
const EXPIRATION = lex.EXPIRATION
const EXPLAIN = lex.EXPLAIN
const EXPORT = lex.EXPORT
const EXPRESSION = lex.EXPRESSION
const EXTENDED = lex.EXTENDED
const EXTENSION = lex.EXTENSION
const EXTERNAL = lex.EXTERNAL
const EXTRACT = lex.EXTRACT
const EXTRACT_DURATION = lex.EXTRACT_DURATION
const FALSE = lex.FALSE
const FAMILY = lex.FAMILY
const FCONST = lex.FCONST
const FETCH = lex.FETCH
const FETCHTEXT = lex.FETCHTEXT
const FETCHTEXT_PATH = lex.FETCHTEXT_PATH
const FETCHVAL = lex.FETCHVAL
const FETCHVAL_PATH = lex.FETCHVAL_PATH
const FILES = lex.FILES
const FILTER = lex.FILTER
const FINALFUNC = lex.FINALFUNC
const FINALFUNC_EXTRA = lex.FINALFUNC_EXTRA
const FINALFUNC_MODIFY = lex.FINALFUNC_MODIFY
const FINALIZE = lex.FINALIZE
const FIRST = lex.FIRST
const FLOAT = lex.FLOAT
const FLOAT4 = lex.FLOAT4
const FLOAT8 = lex.FLOAT8
const FLOORDIV = lex.FLOORDIV
const FOLLOWING = lex.FOLLOWING
const FOR = lex.FOR
const FORCE = lex.FORCE
const FORCE_INDEX = lex.FORCE_INDEX
const FOREIGN = lex.FOREIGN
const FORMAT = lex.FORMAT
const FREEZE = lex.FREEZE
const FROM = lex.FROM
const FULL = lex.FULL
const FUNCTION = lex.FUNCTION
const FUNCTIONS = lex.FUNCTIONS
const GENERATED = lex.GENERATED
const GENERATED_ALWAYS = lex.GENERATED_ALWAYS
const GEOGRAPHY = lex.GEOGRAPHY
const GEOMETRY = lex.GEOMETRY
const GEOMETRYCOLLECTION = lex.GEOMETRYCOLLECTION
const GEOMETRYCOLLECTIONM = lex.GEOMETRYCOLLECTIONM
const GEOMETRYCOLLECTIONZ = lex.GEOMETRYCOLLECTIONZ
const GEOMETRYCOLLECTIONZM = lex.GEOMETRYCOLLECTIONZM
const GEOMETRYM = lex.GEOMETRYM
const GEOMETRYZ = lex.GEOMETRYZ
const GEOMETRYZM = lex.GEOMETRYZM
const GLOBAL = lex.GLOBAL
const GRANT = lex.GRANT
const GRANTED = lex.GRANTED
const GRANTS = lex.GRANTS
const GREATER_EQUALS = lex.GREATER_EQUALS
const GREATEST = lex.GREATEST
const GROUP = lex.GROUP
const GROUPING = lex.GROUPING
const GROUPS = lex.GROUPS
const HAMMING_DISTANCE = lex.HAMMING_DISTANCE
const HANDLER = lex.HANDLER
const HASH = lex.HASH
const HASHES = lex.HASHES
const HAVING = lex.HAVING
const HEADER = lex.HEADER
const HELPTOKEN = lex.HELPTOKEN
const HIGH = lex.HIGH
const HINT = lex.HINT
const HISTOGRAM = lex.HISTOGRAM
const HOUR = lex.HOUR
const HYPOTHETICAL = lex.HYPOTHETICAL
const ICONST = lex.ICONST
const ICU_LOCALE = lex.ICU_LOCALE
const ICU_RULES = lex.ICU_RULES
const IDENT = lex.IDENT
const IDENTITY = lex.IDENTITY
const IF = lex.IF
const IFERROR = lex.IFERROR
const IFNULL = lex.IFNULL
const IGNORE_FOREIGN_KEYS = lex.IGNORE_FOREIGN_KEYS
const ILIKE = lex.ILIKE
const IMMEDIATE = lex.IMMEDIATE
const IMMUTABLE = lex.IMMUTABLE
const IMPLICIT = lex.IMPLICIT
const IMPORT = lex.IMPORT
const IN = lex.IN
const INCLUDE = lex.INCLUDE
const INCLUDING = lex.INCLUDING
const INCREMENT = lex.INCREMENT
const INCREMENTAL = lex.INCREMENTAL
const INDEX = lex.INDEX
const INDEXES = lex.INDEXES
const INDEX_CLEANUP = lex.INDEX_CLEANUP
const INET = lex.INET
const INET_CONTAINED_BY_OR_EQUALS = lex.INET_CONTAINED_BY_OR_EQUALS
const INET_CONTAINS_OR_EQUALS = lex.INET_CONTAINS_OR_EQUALS
const INHERIT = lex.INHERIT
const INHERITS = lex.INHERITS
const INITCOND = lex.INITCOND
const INITIALLY = lex.INITIALLY
const INJECT = lex.INJECT
const INLINE = lex.INLINE
const INNER = lex.INNER
const INOUT = lex.INOUT
const INPUT = lex.INPUT
const INSERT = lex.INSERT
const INSTEAD = lex.INSTEAD
const INT = lex.INT
const INTEGER = lex.INTEGER
const INTERLEAVE = lex.INTERLEAVE
const INTERNALLENGTH = lex.INTERNALLENGTH
const INTERSECT = lex.INTERSECT
const INTERVAL = lex.INTERVAL
const INTO = lex.INTO
const INTO_DB = lex.INTO_DB
const INVERTED = lex.INVERTED
const INVOKER = lex.INVOKER
const IS = lex.IS
const ISERROR = lex.ISERROR
const ISNULL = lex.ISNULL
const ISOLATION = lex.ISOLATION
const IS_TEMPLATE = lex.IS_TEMPLATE
const JACCARD_DISTANCE = lex.JACCARD_DISTANCE
const JOB = lex.JOB
const JOBS = lex.JOBS
const JOIN = lex.JOIN
const JSON = lex.JSON
const JSONB = lex.JSONB
const JSON_ALL_EXISTS = lex.JSON_ALL_EXISTS
const JSON_SOME_EXISTS = lex.JSON_SOME_EXISTS
const KEY = lex.KEY
const KEYS = lex.KEYS
const KMS = lex.KMS
const KV = lex.KV
const L1_DISTANCE = lex.L1_DISTANCE
const L2_DISTANCE = lex.L2_DISTANCE
const LANGUAGE = lex.LANGUAGE
const LARGE = lex.LARGE
const LAST = lex.LAST
const LATERAL = lex.LATERAL
const LATEST = lex.LATEST
const LC_COLLATE = lex.LC_COLLATE
const LC_CTYPE = lex.LC_CTYPE
const LEADING = lex.LEADING
const LEAKPROOF = lex.LEAKPROOF
const LEASE = lex.LEASE
const LEAST = lex.LEAST
const LEFT = lex.LEFT
const LEFTARG = lex.LEFTARG
const LESS = lex.LESS
const LESS_EQUALS = lex.LESS_EQUALS
const LEVEL = lex.LEVEL
const LIKE = lex.LIKE
const LIMIT = lex.LIMIT
const LINESTRING = lex.LINESTRING
const LINESTRINGM = lex.LINESTRINGM
const LINESTRINGZ = lex.LINESTRINGZ
const LINESTRINGZM = lex.LINESTRINGZM
const LIST = lex.LIST
const LOCAL = lex.LOCAL
const LOCALE = lex.LOCALE
const LOCALE_PROVIDER = lex.LOCALE_PROVIDER
const LOCALTIME = lex.LOCALTIME
const LOCALTIMESTAMP = lex.LOCALTIMESTAMP
const LOCKED = lex.LOCKED
const LOGGED = lex.LOGGED
const LOGIN = lex.LOGIN
const LOOKUP = lex.LOOKUP
const LOW = lex.LOW
const LSHIFT = lex.LSHIFT
const MAIN = lex.MAIN
const MATCH = lex.MATCH
const MATERIALIZED = lex.MATERIALIZED
const MAXVALUE = lex.MAXVALUE
const MERGE = lex.MERGE
const MERGES = lex.MERGES
const METHOD = lex.METHOD
const MFINALFUNC = lex.MFINALFUNC
const MFINALFUNC_EXTRA = lex.MFINALFUNC_EXTRA
const MFINALFUNC_MODIFY = lex.MFINALFUNC_MODIFY
const MINITCOND = lex.MINITCOND
const MINUTE = lex.MINUTE
const MINVALUE = lex.MINVALUE
const MINVFUNC = lex.MINVFUNC
const MODIFYCLUSTERSETTING = lex.MODIFYCLUSTERSETTING
const MODULUS = lex.MODULUS
const MONTH = lex.MONTH
const MSFUNC = lex.MSFUNC
const MSPACE = lex.MSPACE
const MSSPACE = lex.MSSPACE
const MSTYPE = lex.MSTYPE
const MULTILINESTRING = lex.MULTILINESTRING
const MULTILINESTRINGM = lex.MULTILINESTRINGM
const MULTILINESTRINGZ = lex.MULTILINESTRINGZ
const MULTILINESTRINGZM = lex.MULTILINESTRINGZM
const MULTIPOINT = lex.MULTIPOINT
const MULTIPOINTM = lex.MULTIPOINTM
const MULTIPOINTZ = lex.MULTIPOINTZ
const MULTIPOINTZM = lex.MULTIPOINTZM
const MULTIPOLYGON = lex.MULTIPOLYGON
const MULTIPOLYGONM = lex.MULTIPOLYGONM
const MULTIPOLYGONZ = lex.MULTIPOLYGONZ
const MULTIPOLYGONZM = lex.MULTIPOLYGONZM
const MULTIRANGE_TYPE_NAME = lex.MULTIRANGE_TYPE_NAME
const MaxInt = int(MaxUint >> 1)
const MaxUint = ^uint(0)
const NAME = lex.NAME
const NAMES = lex.NAMES
const NAN = lex.NAN
const NATURAL = lex.NATURAL
const NEGATOR = lex.NEGATOR
const NEG_INNER_PRODUCT = lex.NEG_INNER_PRODUCT
const NEVER = lex.NEVER
const NEW = lex.NEW
const NEXT = lex.NEXT
const NO = lex.NO
const NOBYPASSRLS = lex.NOBYPASSRLS
const NOCANCELQUERY = lex.NOCANCELQUERY
const NOCONTROLCHANGEFEED = lex.NOCONTROLCHANGEFEED
const NOCONTROLJOB = lex.NOCONTROLJOB
const NOCREATEDB = lex.NOCREATEDB
const NOCREATELOGIN = lex.NOCREATELOGIN
const NOCREATEROLE = lex.NOCREATEROLE
const NOINHERIT = lex.NOINHERIT
const NOLOGIN = lex.NOLOGIN
const NOMODIFYCLUSTERSETTING = lex.NOMODIFYCLUSTERSETTING
const NONE = lex.NONE
const NOREPLICATION = lex.NOREPLICATION
const NORMAL = lex.NORMAL
const NOSUPERUSER = lex.NOSUPERUSER
const NOT = lex.NOT
const NOTHING = lex.NOTHING
const NOTNULL = lex.NOTNULL
const NOT_EQUALS = lex.NOT_EQUALS
const NOT_LA = lex.NOT_LA
const NOT_REGIMATCH = lex.NOT_REGIMATCH
const NOT_REGMATCH = lex.NOT_REGMATCH
const NOVIEWACTIVITY = lex.NOVIEWACTIVITY
const NOWAIT = lex.NOWAIT
const NO_INDEX_JOIN = lex.NO_INDEX_JOIN
const NULL = lex.NULL
const NULLIF = lex.NULLIF
const NULLS = lex.NULLS
const NUMERIC = lex.NUMERIC
const OBJECT = lex.OBJECT
const OF = lex.OF
const OFF = lex.OFF
const OFFSET = lex.OFFSET
const OID = lex.OID
const OIDS = lex.OIDS
const OIDVECTOR = lex.OIDVECTOR
const OLD = lex.OLD
const ON = lex.ON
const ONLY = lex.ONLY
const ONLY_DATABASE_STATS = lex.ONLY_DATABASE_STATS
const OPERATOR = lex.OPERATOR
const OPT = lex.OPT
const OPTION = lex.OPTION
const OPTIONS = lex.OPTIONS
const OR = lex.OR
const ORDER = lex.ORDER
const ORDINALITY = lex.ORDINALITY
const OTHERS = lex.OTHERS
const OUT = lex.OUT
const OUTER = lex.OUTER
const OUTPUT = lex.OUTPUT
const OVER = lex.OVER
const OVERLAPS = lex.OVERLAPS
const OVERLAY = lex.OVERLAY
const OWNED = lex.OWNED
const OWNER = lex.OWNER
const PARALLEL = lex.PARALLEL
const PARAMETER = lex.PARAMETER
const PARENT = lex.PARENT
const PARSER = lex.PARSER
const PARTIAL = lex.PARTIAL
const PARTITION = lex.PARTITION
const PARTITIONS = lex.PARTITIONS
const PASSEDBYVALUE = lex.PASSEDBYVALUE
const PASSWORD = lex.PASSWORD
const PAUSE = lex.PAUSE
const PAUSED = lex.PAUSED
const PHYSICAL = lex.PHYSICAL
const PLACEHOLDER = lex.PLACEHOLDER
const PLACING = lex.PLACING
const PLAIN = lex.PLAIN
const PLAN = lex.PLAN
const PLANS = lex.PLANS
const POINT = lex.POINT
const POINTM = lex.POINTM
const POINTZ = lex.POINTZ
const POINTZM = lex.POINTZM
const POLICY = lex.POLICY
const POLYGON = lex.POLYGON
const POLYGONM = lex.POLYGONM
const POLYGONZ = lex.POLYGONZ
const POLYGONZM = lex.POLYGONZM
const POSITION = lex.POSITION
const POSTFIXOP = lex.POSTFIXOP
const PRECEDING = lex.PRECEDING
const PRECISION = lex.PRECISION
const PREFERRED = lex.PREFERRED
const PREPARE = lex.PREPARE
const PRESERVE = lex.PRESERVE
const PRIMARY = lex.PRIMARY
const PRIORITY = lex.PRIORITY
const PRIVILEGES = lex.PRIVILEGES
const PROCEDURAL = lex.PROCEDURAL
const PROCEDURE = lex.PROCEDURE
const PROCEDURES = lex.PROCEDURES
const PROCESS_MAIN = lex.PROCESS_MAIN
const PROCESS_TOAST = lex.PROCESS_TOAST
const PUBLIC = lex.PUBLIC
const PUBLICATION = lex.PUBLICATION
const QUERIES = lex.QUERIES
const QUERY = lex.QUERY
const RANGE = lex.RANGE
const RANGES = lex.RANGES
const READ = lex.READ
const READ_ONLY = lex.READ_ONLY
const READ_WRITE = lex.READ_WRITE
const REAL = lex.REAL
const RECEIVE = lex.RECEIVE
const RECURRING = lex.RECURRING
const RECURSIVE = lex.RECURSIVE
const REF = lex.REF
const REFERENCES = lex.REFERENCES
const REFERENCING = lex.REFERENCING
const REFRESH = lex.REFRESH
const REGCLASS = lex.REGCLASS
const REGIMATCH = lex.REGIMATCH
const REGNAMESPACE = lex.REGNAMESPACE
const REGPROC = lex.REGPROC
const REGPROCEDURE = lex.REGPROCEDURE
const REGTYPE = lex.REGTYPE
const REINDEX = lex.REINDEX
const RELEASE = lex.RELEASE
const REMAINDER = lex.REMAINDER
const REMOVE_PATH = lex.REMOVE_PATH
const RENAME = lex.RENAME
const REPEATABLE = lex.REPEATABLE
const REPLACE = lex.REPLACE
const REPLICA = lex.REPLICA
const REPLICATION = lex.REPLICATION
const RESET = lex.RESET
const RESTART = lex.RESTART
const RESTORE = lex.RESTORE
const RESTRICT = lex.RESTRICT
const RESTRICTED = lex.RESTRICTED
const RESUME = lex.RESUME
const RETRY = lex.RETRY
const RETURN = lex.RETURN
const RETURNING = lex.RETURNING
const RETURNS = lex.RETURNS
const REVISION_HISTORY = lex.REVISION_HISTORY
const REVOKE = lex.REVOKE
const RIGHT = lex.RIGHT
const RIGHTARG = lex.RIGHTARG
const ROLE = lex.ROLE
const ROLES = lex.ROLES
const ROLLBACK = lex.ROLLBACK
const ROLLUP = lex.ROLLUP
const ROUTINE = lex.ROUTINE
const ROUTINES = lex.ROUTINES
const ROW = lex.ROW
const ROWS = lex.ROWS
const RSHIFT = lex.RSHIFT
const RULE = lex.RULE
const RUNNING = lex.RUNNING
const SAFE = lex.SAFE
const SAVEPOINT = lex.SAVEPOINT
const SCATTER = lex.SCATTER
const SCHEDULE = lex.SCHEDULE
const SCHEDULES = lex.SCHEDULES
const SCHEMA = lex.SCHEMA
const SCHEMAS = lex.SCHEMAS
const SCONST = lex.SCONST
const SCRUB = lex.SCRUB
const SEARCH = lex.SEARCH
const SECOND = lex.SECOND
const SECURITY = lex.SECURITY
const SECURITY_BARRIER = lex.SECURITY_BARRIER
const SECURITY_INVOKER = lex.SECURITY_INVOKER
const SEED = lex.SEED
const SELECT = lex.SELECT
const SEND = lex.SEND
const SEQUENCE = lex.SEQUENCE
const SEQUENCES = lex.SEQUENCES
const SERIALFUNC = lex.SERIALFUNC
const SERIALIZABLE = lex.SERIALIZABLE
const SERVER = lex.SERVER
const SESSION = lex.SESSION
const SESSIONS = lex.SESSIONS
const SESSION_USER = lex.SESSION_USER
const SET = lex.SET
const SETOF = lex.SETOF
const SETTING = lex.SETTING
const SETTINGS = lex.SETTINGS
const SFUNC = lex.SFUNC
const SHARE = lex.SHARE
const SHAREABLE = lex.SHAREABLE
const SHOW = lex.SHOW
const SIMILAR = lex.SIMILAR
const SIMPLE = lex.SIMPLE
const SKIP = lex.SKIP
const SKIP_DATABASE_STATS = lex.SKIP_DATABASE_STATS
const SKIP_LOCKED = lex.SKIP_LOCKED
const SKIP_MISSING_FOREIGN_KEYS = lex.SKIP_MISSING_FOREIGN_KEYS
const SKIP_MISSING_SEQUENCES = lex.SKIP_MISSING_SEQUENCES
const SKIP_MISSING_SEQUENCE_OWNERS = lex.SKIP_MISSING_SEQUENCE_OWNERS
const SKIP_MISSING_VIEWS = lex.SKIP_MISSING_VIEWS
const SMALLINT = lex.SMALLINT
const SMALLSERIAL = lex.SMALLSERIAL
const SNAPSHOT = lex.SNAPSHOT
const SOME = lex.SOME
const SORTOP = lex.SORTOP
const SPLIT = lex.SPLIT
const SQL = lex.SQL
const SQRT = lex.SQRT
const SSPACE = lex.SSPACE
const STABLE = lex.STABLE
const START = lex.START
const STATEMENT = lex.STATEMENT
const STATISTICS = lex.STATISTICS
const STATUS = lex.STATUS
const STDIN = lex.STDIN
const STDOUT = lex.STDOUT
const STORAGE = lex.STORAGE
const STORE = lex.STORE
const STORED = lex.STORED
const STRATEGY = lex.STRATEGY
const STRICT = lex.STRICT
const STRING = lex.STRING
const STYPE = lex.STYPE
const SUBSCRIPT = lex.SUBSCRIPT
const SUBSCRIPTION = lex.SUBSCRIPTION
const SUBSTRING = lex.SUBSTRING
const SUBTYPE = lex.SUBTYPE
const SUBTYPE_DIFF = lex.SUBTYPE_DIFF
const SUBTYPE_OPCLASS = lex.SUBTYPE_OPCLASS
const SUPERUSER = lex.SUPERUSER
const SUPPORT = lex.SUPPORT
const SYMMETRIC = lex.SYMMETRIC
const SYNTAX = lex.SYNTAX
const SYSID = lex.SYSID
const SYSTEM = lex.SYSTEM
const TABLE = lex.TABLE
const TABLES = lex.TABLES
const TABLESPACE = lex.TABLESPACE
const TEMP = lex.TEMP
const TEMPLATE = lex.TEMPLATE
const TEMPORARY = lex.TEMPORARY
const TEXT = lex.TEXT
const TEXTSEARCHMATCH = lex.TEXTSEARCHMATCH
const THEN = lex.THEN
const THROTTLING = lex.THROTTLING
const TIES = lex.TIES
const TIME = lex.TIME
const TIMESTAMP = lex.TIMESTAMP
const TIMESTAMPTZ = lex.TIMESTAMPTZ
const TIMETZ = lex.TIMETZ
const TO = lex.TO
const TRACE = lex.TRACE
const TRACING = lex.TRACING
const TRAILING = lex.TRAILING
const TRANSACTION = lex.TRANSACTION
const TRANSACTIONS = lex.TRANSACTIONS
const TRANSFORM = lex.TRANSFORM
const TREAT = lex.TREAT
const TRIGGER = lex.TRIGGER
const TRIM = lex.TRIM
const TRUE = lex.TRUE
const TRUNCATE = lex.TRUNCATE
const TRUSTED = lex.TRUSTED
const TYPE = lex.TYPE
const TYPEANNOTATE = lex.TYPEANNOTATE
const TYPECAST = lex.TYPECAST
const TYPES = lex.TYPES
const TYPMOD_IN = lex.TYPMOD_IN
const TYPMOD_OUT = lex.TYPMOD_OUT
const UMINUS = lex.UMINUS
const UNBOUNDED = lex.UNBOUNDED
const UNCOMMITTED = lex.UNCOMMITTED
const UNION = lex.UNION
const UNIQUE = lex.UNIQUE
const UNKNOWN = lex.UNKNOWN
const UNLOGGED = lex.UNLOGGED
const UNSAFE = lex.UNSAFE
const UNSPLIT = lex.UNSPLIT
const UNTIL = lex.UNTIL
const UPDATE = lex.UPDATE
const UPSERT = lex.UPSERT
const USAGE = lex.USAGE
const USE = lex.USE
const USER = lex.USER
const USERS = lex.USERS
const USING = lex.USING
const UUID = lex.UUID
const VACUUM = lex.VACUUM
const VALID = lex.VALID
const VALIDATE = lex.VALIDATE
const VALIDATOR = lex.VALIDATOR
const VALUE = lex.VALUE
const VALUES = lex.VALUES
const VARBIT = lex.VARBIT
const VARCHAR = lex.VARCHAR
const VARIABLE = lex.VARIABLE
const VARIADIC = lex.VARIADIC
const VARYING = lex.VARYING
const VERBOSE = lex.VERBOSE
const VERSION = lex.VERSION
const VIEW = lex.VIEW
const VIEWACTIVITY = lex.VIEWACTIVITY
const VIRTUAL = lex.VIRTUAL
const VOLATILE = lex.VOLATILE
const WHEN = lex.WHEN
const WHERE = lex.WHERE
const WINDOW = lex.WINDOW
const WITH = lex.WITH
const WITHIN = lex.WITHIN
const WITHOUT = lex.WITHOUT
const WITH_LA = lex.WITH_LA
const WORK = lex.WORK
const WRAPPER = lex.WRAPPER
const WRITE = lex.WRITE
const XML = lex.XML
const YAML = lex.YAML
const YEAR = lex.YEAR
const YES = lex.YES
const ZONE = lex.ZONE
Variables ¶
var AllHelp = func(h map[string]HelpMessageBody) string { cmds := make(map[string][]string) for c, details := range h { if details.Category == "" { continue } cmds[details.Category] = append(cmds[details.Category], c) } // Ensure the result is deterministic. var categories []string for c, l := range cmds { categories = append(categories, c) sort.Strings(l) } sort.Strings(categories) // Compile the final help index. var buf bytes.Buffer w := tabwriter.NewWriter(&buf, 0, 0, 1, ' ', 0) for _, cat := range categories { fmt.Fprintf(w, "%s:\n", cases.Title(language.English).String(cat)) for _, item := range cmds[cat] { fmt.Fprintf(w, "\t\t%s\t%s\n", item, h[item].ShortDescription) } fmt.Fprintln(w) } _ = w.Flush() return buf.String() }(helpMessages)
AllHelp contains an overview of all statements with help messages. For example, displayed in the CLI shell with \h without additional parameters.
var HelpMessages = func(h map[string]HelpMessageBody) map[string]HelpMessageBody { appendSeeAlso := func(newItem, prevItems string) string { if prevItems != "" { return newItem + "\n " + prevItems } return newItem } reformatSeeAlso := func(seeAlso string) string { return strings.Replace(seeAlso, ", ", "\n ", -1) } srcMsg := h["<SOURCE>"] srcMsg.SeeAlso = reformatSeeAlso(strings.TrimSpace(srcMsg.SeeAlso)) selectMsg := h["<SELECTCLAUSE>"] selectMsg.SeeAlso = reformatSeeAlso(strings.TrimSpace(selectMsg.SeeAlso)) for k, m := range h { m = h[k] m.ShortDescription = strings.TrimSpace(m.ShortDescription) m.Text = strings.TrimSpace(m.Text) m.SeeAlso = strings.TrimSpace(m.SeeAlso) if strings.Contains(m.Text, "<source>") && k != "<SOURCE>" { m.Text = strings.TrimSpace(m.Text) + "\n\n" + strings.TrimSpace(srcMsg.Text) m.SeeAlso = appendSeeAlso(srcMsg.SeeAlso, m.SeeAlso) } if strings.Contains(m.Text, "<selectclause>") && k != "<SELECTCLAUSE>" { m.Text = strings.TrimSpace(m.Text) + "\n\n" + strings.TrimSpace(selectMsg.Text) m.SeeAlso = appendSeeAlso(selectMsg.SeeAlso, m.SeeAlso) } if strings.Contains(m.Text, "<tablename>") { m.SeeAlso = appendSeeAlso("SHOW TABLES", m.SeeAlso) } m.SeeAlso = reformatSeeAlso(m.SeeAlso) h[k] = m } return h }(helpMessages)
HelpMessages is the registry of all help messages, keyed by the top-level statement that they document. The key is intended for use via the \h client-side command.
Functions ¶
func HasMultipleStatements ¶
HasMultipleStatements returns true if the sql string contains more than one statements.
func LastLexicalToken ¶
LastLexicalToken returns the last lexical token. If the string has no lexical tokens, returns 0 and ok=false.
func ParseExprs ¶
ParseExprs is a short-hand for parseExprs(sql)
func ParseQualifiedTableName ¶
ParseQualifiedTableName parses a SQL string of the form `[ database_name . ] [ schema_name . ] table_name`.
func ParseTableName ¶
func ParseTableName(sql string) (*tree.UnresolvedObjectName, error)
ParseTableName parses a table name.
func ParseTableNameWithQualifiedNames ¶
func ParseTableNameWithQualifiedNames(sql string) (*tree.UnresolvedObjectName, error)
ParseTableNameWithQualifiedNames can be used to parse an input table name that might be prefixed with an unquoted qualified name. The standard ParseTableName cannot do this due to limitations with our parser. In particular, the parser can't parse different productions individually -- it must parse them as part of a top level statement. This causes qualified names that contain keywords to require quotes, which are not required in some cases due to Postgres compatibility (in particular, as arguments to pg_dump). This function gets around this limitation by parsing the input table name as a column name with a fake non-keyword prefix, and then shifting the result down into an UnresolvedObjectName.
func ParseType ¶
func ParseType(sql string) (tree.ResolvableTypeReference, error)
ParseType parses a column type.
func RunShowSyntax ¶
func RunShowSyntax( ctx context.Context, stmt string, report func(ctx context.Context, field, msg string), reportErr func(ctx context.Context, err error), )
RunShowSyntax analyzes the syntax and reports its structure as data for the client. Even an error is reported as data.
Since errors won't propagate to the client as an error, but as a result, the usual code path to capture and record errors will not be triggered. Instead, the caller can pass a reportErr closure to capture errors instead. May be nil.
func SplitFirstStatement ¶
SplitFirstStatement returns the length of the prefix of the string up to and including the first semicolon that separates statements. If there is no semicolon, returns ok=false.
Types ¶
type HelpMessage ¶
type HelpMessage struct {
// Command is set if the message is about a statement.
Command string
// Function is set if the message is about a built-in function.
Function string
// HelpMessageBody contains the details of the message.
HelpMessageBody
}
HelpMessage describes a contextual help message.
func (*HelpMessage) Format ¶
func (h *HelpMessage) Format(w io.Writer)
Format prints out details about the message onto the specified output stream.
func (*HelpMessage) String ¶
func (h *HelpMessage) String() string
String implements the fmt.String interface.
type HelpMessageBody ¶
HelpMessageBody defines the body of a help text. The messages are structured to facilitate future help navigation functionality.
type Parser ¶
type Parser struct {
// contains filtered or unexported fields
}
Parser wraps a scanner, parser and other utilities present in the parser package.
func (*Parser) Parse ¶
func (p *Parser) Parse(sql string) (Statements, error)
Parse parses the sql and returns a list of statements.
func (*Parser) ParseWithInt ¶
ParseWithInt parses a sql statement string and returns a list of Statements. The INT token will result in the specified TInt type.
type Statement ¶
type Statement struct {
// AST is the root of the AST tree for the parsed statement.
AST tree.Statement
// SQL is the original SQL from which the statement was parsed. Note that this
// is not appropriate for use in logging, as it may contain passwords and
// other sensitive data.
SQL string
// NumPlaceholders indicates the number of arguments to the statement (which
// are referenced through placeholders). This corresponds to the highest
// argument position (i.e. the x in "$x") that appears in the query.
//
// Note: where there are "gaps" in the placeholder positions, this number is
// based on the highest position encountered. For example, for `SELECT $3`,
// NumPlaceholders is 3. These cases are malformed and will result in a
// type-check error.
NumPlaceholders int
// NumAnnotations indicates the number of annotations in the tree. It is equal
// to the maximum annotation index.
NumAnnotations tree.AnnotationIdx
}
Statement is the result of parsing a single statement. It contains the AST node along with other information.
func ParseOne ¶
ParseOne parses a sql statement string, ensuring that it contains only a single statement, and returns that Statement. ParseOne will always interpret the INT and SERIAL types as 64-bit types, since this is used in various internal-execution paths where we might receive bits of SQL from other nodes. In general,earwe expect that all user-generated SQL has been run through the ParseWithInt() function.
type Statements ¶
type Statements []Statement
Statements is a list of parsed statements.
func Parse ¶
func Parse(sql string) (Statements, error)
Parse parses a sql statement string and returns a list of Statements.
func (Statements) String ¶
func (stmts Statements) String() string
String returns the AST formatted as a string.
func (Statements) StringWithFlags ¶
func (stmts Statements) StringWithFlags(flags tree.FmtFlags) string
StringWithFlags returns the AST formatted as a string (with the given flags).
type TokenString ¶
TokenString is the unit value returned by Tokens.
func Tokens ¶
func Tokens(sql string) (tokens []TokenString, ok bool)
Tokens decomposes the input into lexical tokens.
Source Files
Directories