sqlddl

package
v0.2.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 17, 2026 License: Apache-2.0 Imports: 4 Imported by: 0

Documentation

Overview

Package sqlddl is codefit's hand-written SQL-DDL parser. It implements providers.SchemaParser (the second implementer after the Prisma parser), turning ordered SQL-DDL migrations into the neutral core/db.Schema. It supports THREE dialects today — PostgreSQL (Postgres(), the default New() construction), MySQL (MySQL()), and SQL Server/T-SQL (SQLServer()) — wired via internal/mcp/schemaparser.go, which selects the Dialect descriptor from the project's .codefit.yaml database.type setting; the tokenizer and reducer are dialect-agnostic by design, consuming only the selected Dialect's data.

A Parser is bound to exactly one Dialect at construction time (New(opts ...Option), WithDialect) — see dialect.go for the descriptor shape. Two pieces consume it (ADR 0018, and the per-dialect descriptor of ADR 0022): a statement splitter (split.go) that is aware of the dialect's comment styles, identifier quoting and (for PostgreSQL) dollar-quoting ($$…$$ / $tag$…$tag$) — so a PL/pgSQL DO block's internal semicolons never cut a statement — and an incremental, dialect-FREE reducer (reduce.go) that applies statements IN ORDER over a mutable schema (CREATE TABLE creates, ALTER ADD/DROP COLUMN mutates, CREATE INDEX indexes, ADD CONSTRAINT constrains), consulting the dialect only for type/modifier vocabulary (dialect.TypeMap / dialect.Modifiers). The splitter re-emits every quoted identifier canonicalized to ANSI "..." as it tokenizes, so the reducer's regexes never need to know the source dialect's quoting style.

It parses a DECLARED SUBSET; everything outside (CHECK constraints, ON DELETE actions, partial-index WHERE, CREATE TYPE, ALTER COLUMN, all DML) is skip-and-declared, each locked by a test. View/Procedure/Trigger are populated with Name/Pos (and Table for triggers); their bodies, view columns, and the materialized flag are NOT captured — a declared limit for the future DB-020..041 rules. No SQL parsing library is used (CGO_ENABLED=0).

Known limit (Unit I rework, disclosed not silent): codefit does not model stored-procedure/trigger routine BODIES. For MySQL, a body wrapped by the client-tool "DELIMITER //" ... "DELIMITER ;" convention is handled correctly — split.go's DELIMITER tracking keeps the whole body as ONE statement, so it is captured by the routine/trigger HEAD regex and no body-internal fragment is ever reduced as its own top-level statement. For T-SQL, a GO-batched routine/trigger body has no such protection (its internal ';'s terminate normally, each becoming its own statement); a body fragment that happens to be CREATE-TABLE-shaped MAY therefore surface as a spurious top-level table. An earlier "inRoutineBody" guard tried to speculatively suppress this by matching BEGIN/END as raw text, but that guard was itself unsound — it matched BEGIN/END inside string literals, was not depth-counted (a nested BEGIN...END closed it early), and was never reset between files (a stuck-open guard from one file could swallow a later file's real tables) — real regressions on VALID input. It has been removed; the T-SQL routine-body case is now a documented, rare limit rather than a fragile guard. Separately, the "GO" batch-separator recognition (split.go) requires the ENTIRE trimmed line to be exactly "GO", so it cannot match part of a longer identifier — but it also means a column literally named "go" standing alone on its own line (vanishingly unlikely in real DDL) would collide with batch-separator recognition; this narrow collision is accepted, not guarded against. Relatedly, MySQL client DELIMITER directives are only recognized when their argument is punctuation-only (e.g. "DELIMITER //", "DELIMITER $$") — this is what lets an ordinary "delimiter VARCHAR(1)" column definition parse as a column, not a directive (Unit I rework, C1). A word-based custom delimiter (e.g. "DELIMITER GO") is therefore NOT recognized as a delimiter directive; this is a narrow, accepted limit, not a bug — punctuation-only delimiters cover the overwhelming majority of real-world MySQL dump/migration tooling.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Dialect added in v0.2.1

type Dialect struct {
	// Name identifies the dialect (e.g. "postgresql", "mysql", "sqlserver").
	Name string

	// LineComments lists the line-comment prefixes this dialect recognizes
	// (e.g. {{"--", false}} for PostgreSQL/T-SQL, {{"--", true}, {"#", false}}
	// for MySQL). Block comments (/* ... */) are universal across dialects and
	// not a field.
	LineComments []LineComment

	// IdentQuotes lists the quoted-identifier delimiter pairs this dialect
	// recognizes (e.g. {{'"','"',true}} for PostgreSQL, {{'`','`',true}} for
	// MySQL, {{'[',']',true},{'"','"',true}} for T-SQL). Every quoted
	// identifier is re-emitted as a canonical ANSI "..." identifier
	// regardless of the source delimiter — the seam that keeps the reducer
	// dialect-free.
	IdentQuotes []QuotePair

	// DoubleQuoteIsString is true when this dialect treats a bare " as a
	// STRING literal delimiter rather than an identifier quote (MySQL's
	// default ANSI_QUOTES-off behavior). PostgreSQL and T-SQL: false.
	DoubleQuoteIsString bool

	// DollarQuoting enables PostgreSQL-style $tag$...$tag$ body delimiters
	// (PL/pgSQL DO/function bodies). MySQL and T-SQL: false.
	DollarQuoting bool

	// TypeMap maps a lowercased base type keyword (already stripped of
	// length/precision and array markers by typeBase) to the neutral
	// db.Type. An unmapped keyword falls back to db.TypeUnknown — the
	// honest fallback, never a crash or a silent drop.
	TypeMap map[string]db.Type

	// Modifiers is the set of UPPERCASE column-tail keywords that end the
	// type expression at splitTypeAndMods time — parsed and deliberately
	// ignored (e.g. UNSIGNED, AUTO_INCREMENT, IDENTITY), never corrupting
	// the mapped type.
	Modifiers map[string]bool

	// TriggerHasInlineBody is a per-dialect DATUM (Phase 2.2, Unit A2,
	// architecture/pg-trigger-body-link): does a CREATE TRIGGER statement in
	// this dialect carry its own logic inline, or is it only a WIRE to a
	// separately-defined function/procedure? PostgreSQL triggers are wires —
	// "CREATE TRIGGER x ... EXECUTE FUNCTION fn();" — false: there is no body
	// to capture, so the statement is always Complete and the logic is found
	// via Trigger.ExecutesFunction instead. MySQL and T-SQL triggers embed
	// their logic directly in the statement ("FOR EACH ROW SET ..." / "AS
	// BEGIN ... END") — true: reduce.go's routineBody derivation (the same
	// tokenizer-state formula routines use) still applies.
	//
	// reduce.go's triggerBody consults this DATUM instead of branching on
	// dialect.Name, keeping the one-shared-reducer/per-dialect-data
	// architecture ADR 0022 established — a new dialect only ever adds a
	// Dialect value, never a branch in split.go or reduce.go.
	TriggerHasInlineBody bool
}

Dialect is a per-SQL-dialect DATA descriptor consumed by the shared, dialect-free tokenizer (split.go) and reducer (reduce.go). Adding a new SQL dialect means adding a new Dialect value — never a branch in the tokenizer or the reducer (see docs/decisions for the SQL-dialect-descriptor ADR).

Lexical fields (LineComments, IdentQuotes, DoubleQuoteIsString, DollarQuoting) are consumed ONLY by split.go: the tokenizer canonicalizes every identifier-quoting style to ANSI double-quotes ("...") as it emits statement text, so reduce.go's regexes and normalizeName never need to know the source dialect's quoting style.

Vocabulary fields (TypeMap, Modifiers) are consumed ONLY by reduce.go/types.go: TypeMap maps a lowercased base type keyword to the neutral db.Type; Modifiers is the parse-and-ignore set of column-tail keywords that END the type expression (e.g. UNSIGNED, IDENTITY, AUTO_INCREMENT) without corrupting the mapped type into TypeUnknown.

func MySQL added in v0.2.1

func MySQL() Dialect

MySQL returns the MySQL dialect descriptor: backtick-quoted identifiers, "--" and "#" line comments, and " as a STRING delimiter (MySQL's default ANSI_QUOTES-off behavior) rather than an identifier quote. No dollar quoting. Unlike PostgreSQL's unconditional "--", MySQL's "--" opens a comment ONLY when immediately followed by a boundary (whitespace, a control char, or end-of-line/EOF); "#" stays unconditional in both dialects. TypeMap and Modifiers hold the MySQL type/modifier vocabulary (Unit C, design §3): UNSIGNED/ZEROFILL/AUTO_INCREMENT/CHARACTER SET/COLLATE are parse-and-ignore Modifiers, never corrupting the mapped type.

func Postgres added in v0.2.1

func Postgres() Dialect

Postgres returns the PostgreSQL dialect descriptor. Its values reproduce TODAY's hardcoded parser behavior EXACTLY: this is the identity/no-op transform that keeps every existing caller and golden fixture byte-identical after the descriptor layer lands. It is also the default dialect for New() when no Option is given.

func SQLServer added in v0.2.1

func SQLServer() Dialect

SQLServer returns the T-SQL (SQL Server) dialect descriptor: this is the first dialect with TWO simultaneously-active IdentQuotes pairs — bracket identifiers ([Users], schema-qualified as [dbo].[Users]) AND ANSI double-quote identifiers (the QUOTED_IDENTIFIER ON default). Unlike PG's symmetric '"'/'"' pair and MySQL's symmetric '`'/'`' pair, the bracket pair is ASYMMETRIC (Open='[' != Close=']'); scanIdentQuoted already scans by Close alone so this required no tokenizer change (locked by TestSplit_SQLServerBracketIdentifierCanonicalized / ...BracketDoublingEscape). ']]' doubles to one literal ']', matching the universal Doubling convention. "--" is unconditional here (unlike MySQL's boundary-gated "--"). No dollar quoting. TypeMap and Modifiers hold the T-SQL type/modifier vocabulary (Unit F, design §3): IDENTITY/ROWGUIDCOL are parse-and-ignore Modifiers, never corrupting the mapped type.

type LineComment added in v0.2.1

type LineComment struct {
	// Prefix is the literal comment-opening text (e.g. "--", "#").
	Prefix string

	// RequireBoundaryAfter, when true, means Prefix opens a comment only if
	// the character immediately following it is a boundary (whitespace or a
	// control char) or there is no following character (end-of-line/EOF).
	// When false, Prefix is unconditional — matching as soon as it is found.
	RequireBoundaryAfter bool
}

LineComment is one line-comment prefix a dialect recognizes.

RequireBoundaryAfter encodes a real lexical divergence between dialects: PostgreSQL's "--" opens a line comment unconditionally, but MySQL's "--" opens one ONLY when immediately followed by a boundary — whitespace, a control char, or end-of-line/EOF (https://dev.mysql.com/doc/refman/8.0/en/comments.html). Without this distinction, "SELECT 1--1 FROM t;" under MySQL would be misread as "SELECT 1" with the rest silently swallowed as a comment. MySQL sets this true for "--" and false for "#" (which is unconditional in both dialects); PostgreSQL sets it false for "--".

type Option added in v0.2.1

type Option func(*Parser)

Option configures a Parser at construction time.

func WithDialect added in v0.2.1

func WithDialect(d Dialect) Option

WithDialect binds the parser to the given SQL dialect descriptor. Absent this option, New() defaults to Postgres() — every existing caller is unaffected.

type Parser

type Parser struct {
	// contains filtered or unexported fields
}

Parser is the SQL-DDL schema parser: it reconstructs the neutral db.Schema by applying DDL statements IN ORDER (an incremental reducer over migrations). It implements providers.SchemaParser and nothing else — SQL is a schema source, not a programming language codefit audits for security/surface (ADR 0018).

A Parser is bound to exactly one Dialect at construction time (never threaded through ParseSchema/SchemaParser) — see Option/WithDialect.

func New

func New(opts ...Option) *Parser

New returns a SQL-DDL schema parser. Without options it defaults to the PostgreSQL dialect — today's behavior, unchanged.

func (*Parser) ParseSchema

func (p *Parser) ParseSchema(sources []providers.SourceFile) (*db.Schema, error)

ParseSchema parses ordered SQL-DDL sources (e.g. Flyway migrations, already version-ordered by the caller) into the accumulated final schema. It is filesystem-free: the caller reads and orders the files (ADR 0014). Statements outside the declared subset are skipped, never an error.

type QuotePair added in v0.2.1

type QuotePair struct {
	Open, Close byte
	Doubling    bool
}

QuotePair is one quoted-identifier delimiter pair. Doubling is true when an occurrence of Close inside the identifier is escaped by writing it twice: two double-quotes mean one literal double-quote, two closing brackets mean one literal bracket, two backticks mean one literal backtick — true for every dialect this package supports.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL