sqlddl

package
v0.3.0-alpha.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 23, 2026 License: Apache-2.0 Imports: 4 Imported by: 0

Documentation

Overview

Package sqlddl is codefit's hand-written SQL-DDL parser. It implements providers.SchemaParser (the second implementer after the Prisma parser), turning ordered SQL-DDL migrations into the neutral core/db.Schema. It supports THREE dialects today — PostgreSQL (Postgres(), the default New() construction), MySQL (MySQL()), and SQL Server/T-SQL (SQLServer()) — wired via internal/mcp/schemaparser.go, which selects the Dialect descriptor from the project's .codefit.yaml database.type setting; the tokenizer and reducer are dialect-agnostic by design, consuming only the selected Dialect's data.

A Parser is bound to exactly one Dialect at construction time (New(opts ...Option), WithDialect) — see dialect.go for the descriptor shape. Two pieces consume it (ADR 0018, and the per-dialect descriptor of ADR 0022): a statement splitter (split.go) that is aware of the dialect's comment styles, identifier quoting and (for PostgreSQL) dollar-quoting ($$…$$ / $tag$…$tag$) — so a PL/pgSQL DO block's internal semicolons never cut a statement — and an incremental, dialect-FREE reducer (reduce.go) that applies statements IN ORDER over a mutable schema (CREATE TABLE creates, ALTER ADD/DROP COLUMN mutates, CREATE INDEX indexes, ADD CONSTRAINT constrains), consulting the dialect only for type/modifier vocabulary (dialect.TypeMap / dialect.Modifiers). The splitter re-emits every quoted identifier canonicalized to ANSI "..." as it tokenizes, so the reducer's regexes never need to know the source dialect's quoting style.

Two boundaries inside the reducer are DERIVED rather than lexical, because neither is expressed by a delimiter the splitter can see. A statement run onto a CREATE TABLE's tail with no terminator is cut at that tail (ADR 0041), and a CREATE TABLE body item missing its separating comma before a table-level key constraint is cut at the constraint head (ADR 0042). Both rest on a grammar fact rather than a guess, both leave the host table proven, and both route anything they find but cannot reduce to the honest-abstention floor.

Not every relation the reducer READS is a table, and it remembers the difference (ADR 0045). A CREATE SEQUENCE and a CREATE (MATERIALIZED) VIEW put their NAME in a reducer-internal registry with no model surface of their own, so the ownership statement pg_dump writes for every relation it dumps — "ALTER TABLE public.<name>_id_seq OWNER TO <role>", which PostgreSQL legally accepts for every relation kind — is not mistaken for a table declaration. Without it, a sequence became a zero-column table that DB-050 then asked the agent about. The guard is EVIDENCE-driven, never name- or action-driven: a name nothing in the scanned files declared still materializes with db.ReasonTableNeverDeclared, exactly as before.

It parses a DECLARED SUBSET; everything outside (CHECK constraints, ON DELETE actions, partial-index WHERE, CREATE TYPE, ALTER COLUMN, all DML) is skip-and-declared, each locked by a test. View/Procedure/Trigger are populated with Name/Pos (and Table for triggers) AND with their BODY (db.Body{Text, Complete}), which is what the DB-020/030/031/040/041 rules read; view columns and the materialized flag are still NOT captured — a declared limit. No SQL parsing library is used (CGO_ENABLED=0).

Routine bodies, and how the two dialect families get there (disclosed, not silent): for MySQL, a body wrapped by the client-tool "DELIMITER //" ... "DELIMITER ;" convention is kept as ONE statement by split.go's DELIMITER tracking, so it is captured by the routine/trigger HEAD regex and no body-internal fragment is ever reduced as its own top-level statement. For T-SQL, the body is captured to the "GO" batch separator (or EOF) per ADR 0027, so a CREATE-TABLE-shaped fragment inside a GO-batched routine body is ABSORBED into that body rather than surfacing as a spurious top-level table — which closes the phantom-table limit ADR 0022 had declared. The trade ADR 0027 accepted is the opposite shape: a T-SQL routine with NO trailing GO swallows whatever follows it up to EOF. An earlier "inRoutineBody" guard tried to suppress the phantom speculatively by matching BEGIN/END as raw text, but that guard was itself unsound — it matched BEGIN/END inside string literals, was not depth-counted (a nested BEGIN...END closed it early), and was never reset between files (a stuck-open guard from one file could swallow a later file's real tables) — real regressions on VALID input. It was removed and replaced by the batch-boundary capture above, which is structural rather than speculative. Separately, the "GO" batch-separator recognition (split.go) requires the ENTIRE trimmed line to be exactly "GO", so it cannot match part of a longer identifier — but it also means a column literally named "go" standing alone on its own line (vanishingly unlikely in real DDL) would collide with batch-separator recognition; this narrow collision is accepted, not guarded against. Relatedly, MySQL client DELIMITER directives are only recognized when their argument is punctuation-only (e.g. "DELIMITER //", "DELIMITER $$") — this is what lets an ordinary "delimiter VARCHAR(1)" column definition parse as a column, not a directive (Unit I rework, C1). A word-based custom delimiter (e.g. "DELIMITER GO") is therefore NOT recognized as a delimiter directive; this is a narrow, accepted limit, not a bug — punctuation-only delimiters cover the overwhelming majority of real-world MySQL dump/migration tooling.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Dialect added in v0.2.1

type Dialect struct {
	// Name identifies the dialect (e.g. "postgresql", "mysql", "sqlserver").
	Name string

	// LineComments lists the line-comment prefixes this dialect recognizes
	// (e.g. {{"--", false}} for PostgreSQL/T-SQL, {{"--", true}, {"#", false}}
	// for MySQL). Block comments (/* ... */) are universal across dialects and
	// not a field.
	LineComments []LineComment

	// IdentQuotes lists the quoted-identifier delimiter pairs this dialect
	// recognizes (e.g. {{'"','"',true}} for PostgreSQL, {{'`','`',true}} for
	// MySQL, {{'[',']',true},{'"','"',true}} for T-SQL). Every quoted
	// identifier is re-emitted as a canonical ANSI "..." identifier
	// regardless of the source delimiter — the seam that keeps the reducer
	// dialect-free.
	IdentQuotes []QuotePair

	// DoubleQuoteIsString is true when this dialect treats a bare " as a
	// STRING literal delimiter rather than an identifier quote (MySQL's
	// default ANSI_QUOTES-off behavior). PostgreSQL and T-SQL: false.
	DoubleQuoteIsString bool

	// DollarQuoting enables PostgreSQL-style $tag$...$tag$ body delimiters
	// (PL/pgSQL DO/function bodies). MySQL and T-SQL: false.
	DollarQuoting bool

	// TypeMap maps a lowercased base type keyword (already stripped of
	// length/precision and array markers by typeBase) to the neutral
	// db.Type. An unmapped keyword falls back to db.TypeUnknown — the
	// honest fallback, never a crash or a silent drop.
	TypeMap map[string]db.Type

	// Modifiers is the set of UPPERCASE column-tail keywords that end the
	// type expression at splitTypeAndMods time — parsed and deliberately
	// ignored (e.g. UNSIGNED, AUTO_INCREMENT, IDENTITY), never corrupting
	// the mapped type.
	Modifiers map[string]bool

	// TriggerHasInlineBody is a per-dialect DATUM (Phase 2.2, Unit A2,
	// architecture/pg-trigger-body-link): does a CREATE TRIGGER statement in
	// this dialect carry its own logic inline, or is it only a WIRE to a
	// separately-defined function/procedure? PostgreSQL triggers are wires —
	// "CREATE TRIGGER x ... EXECUTE FUNCTION fn();" — false: there is no body
	// to capture, so the statement is always Complete and the logic is found
	// via Trigger.ExecutesFunction instead. MySQL and T-SQL triggers embed
	// their logic directly in the statement ("FOR EACH ROW SET ..." / "AS
	// BEGIN ... END") — true: reduce.go's routineBody derivation (the same
	// tokenizer-state formula routines use) still applies.
	//
	// reduce.go's triggerBody consults this DATUM instead of branching on
	// dialect.Name, keeping the one-shared-reducer/per-dialect-data
	// architecture ADR 0022 established — a new dialect only ever adds a
	// Dialect value, never a branch in split.go or reduce.go.
	TriggerHasInlineBody bool

	// RoutineBodyEndsAtBatchSeparator is a per-dialect DATUM (ADR 0027,
	// architecture/tsql-body-truncation-limit CLOSED): when true, a CREATE
	// FUNCTION/PROCEDURE/TRIGGER body is delimited by the GO batch separator
	// (or EOF), not by an internal ';'. split() consults this DATUM (via
	// isRoutineHead in reduce.go) to suspend ';'-flushing once it recognizes a
	// routine head, so the whole body is captured whole up to GO/EOF instead
	// of being cut at the body's first internal ';'.
	//
	// true ONLY for SQLServer: T-SQL has neither PostgreSQL's dollar-quoting
	// nor MySQL's DELIMITER directive to protect internal ';'s inside a
	// routine body, so without this DATUM every multi-statement T-SQL body
	// would truncate. PostgreSQL and MySQL set this explicitly false — their
	// existing dollar-quoting/DELIMITER mechanisms already capture the whole
	// body, and GO is not a batch separator in either dialect.
	RoutineBodyEndsAtBatchSeparator bool

	// PartitionSchemeOnClause is a per-dialect DATUM (partition-capture):
	// does this dialect declare TABLE PARTITIONING by naming a partition
	// SCHEME in a trailing "ON <scheme> (<column>)" clause, with the
	// strategy living in a separately declared CREATE PARTITION FUNCTION?
	//
	// true ONLY for SQLServer. PostgreSQL and MySQL both spell partitioning
	// inline as "PARTITION BY <strategy> (<key>)", which reduce.go reads on
	// every dialect without consulting this datum (the grammar is the same
	// word in both, and a dialect that does not have it simply never emits
	// it). T-SQL's form needs the datum because "ON <name>" is ALSO how
	// T-SQL names a FILEGROUP — the vendored AdventureWorksDW corpus ends
	// every CREATE TABLE with ") ON [PRIMARY];" — and only the parenthesized
	// column distinguishes the two. Gating on this datum keeps that read
	// from ever being attempted against a PostgreSQL/MySQL tail, where "ON"
	// belongs to a different grammar entirely (e.g. "ON COMMIT DROP").
	//
	// It also gates whether CREATE PARTITION FUNCTION / CREATE PARTITION
	// SCHEME statements are read at all: they exist only in this dialect.
	PartitionSchemeOnClause bool

	// HashPrefixedTempTables is a per-dialect DATUM (ADR 0043): does this
	// dialect mark a TEMPORARY table by a '#' NAME PREFIX instead of a
	// keyword?
	//
	// true ONLY for SQLServer. T-SQL has no TEMPORARY keyword at all: it
	// writes "CREATE TABLE #Scratch (...)" for a session-local temporary table
	// and "##Scratch" for a global one, so the TEMP/TEMPORARY keyword
	// recognition that covers PostgreSQL and MySQL does nothing for it and a
	// separate recognition is needed.
	//
	// It is a DATUM rather than an unconditional rule because '#' means
	// nothing of the sort in the other two dialects: MySQL treats a bare '#'
	// as a LINE COMMENT (see LineComments above), and a PostgreSQL table can
	// legitimately be named "#weird" with quoting. Withholding either as
	// "session-scoped" would be codefit deciding, on a lexical accident, that
	// a persistent table is not part of the schema — a silent deletion, the
	// exact failure this slice exists to end.
	HashPrefixedTempTables bool
}

Dialect is a per-SQL-dialect DATA descriptor consumed by the shared, dialect-free tokenizer (split.go) and reducer (reduce.go). Adding a new SQL dialect means adding a new Dialect value — never a branch in the tokenizer or the reducer (see docs/decisions for the SQL-dialect-descriptor ADR).

Lexical fields (LineComments, IdentQuotes, DoubleQuoteIsString, DollarQuoting) are consumed ONLY by split.go: the tokenizer canonicalizes every identifier-quoting style to ANSI double-quotes ("...") as it emits statement text, so reduce.go's regexes and normalizeName never need to know the source dialect's quoting style.

Vocabulary fields (TypeMap, Modifiers) are consumed ONLY by reduce.go/types.go: TypeMap maps a lowercased base type keyword to the neutral db.Type; Modifiers is the parse-and-ignore set of column-tail keywords that END the type expression (e.g. UNSIGNED, IDENTITY, AUTO_INCREMENT) without corrupting the mapped type into TypeUnknown.

func MySQL added in v0.2.1

func MySQL() Dialect

MySQL returns the MySQL dialect descriptor: backtick-quoted identifiers, "--" and "#" line comments, and " as a STRING delimiter (MySQL's default ANSI_QUOTES-off behavior) rather than an identifier quote. No dollar quoting. Unlike PostgreSQL's unconditional "--", MySQL's "--" opens a comment ONLY when immediately followed by a boundary (whitespace, a control char, or end-of-line/EOF); "#" stays unconditional in both dialects. TypeMap and Modifiers hold the MySQL type/modifier vocabulary (Unit C, design §3): UNSIGNED/ZEROFILL/AUTO_INCREMENT/CHARACTER SET/COLLATE are parse-and-ignore Modifiers, never corrupting the mapped type.

func Postgres added in v0.2.1

func Postgres() Dialect

Postgres returns the PostgreSQL dialect descriptor. Its values reproduce TODAY's hardcoded parser behavior EXACTLY: this is the identity/no-op transform that keeps every existing caller and golden fixture byte-identical after the descriptor layer lands. It is also the default dialect for New() when no Option is given.

func SQLServer added in v0.2.1

func SQLServer() Dialect

SQLServer returns the T-SQL (SQL Server) dialect descriptor: this is the first dialect with TWO simultaneously-active IdentQuotes pairs — bracket identifiers ([Users], schema-qualified as [dbo].[Users]) AND ANSI double-quote identifiers (the QUOTED_IDENTIFIER ON default). Unlike PG's symmetric '"'/'"' pair and MySQL's symmetric '`'/'`' pair, the bracket pair is ASYMMETRIC (Open='[' != Close=']'); scanIdentQuoted already scans by Close alone so this required no tokenizer change (locked by TestSplit_SQLServerBracketIdentifierCanonicalized / ...BracketDoublingEscape). ']]' doubles to one literal ']', matching the universal Doubling convention. "--" is unconditional here (unlike MySQL's boundary-gated "--"). No dollar quoting. TypeMap and Modifiers hold the T-SQL type/modifier vocabulary (Unit F, design §3): IDENTITY/ROWGUIDCOL are parse-and-ignore Modifiers, never corrupting the mapped type.

type LineComment added in v0.2.1

type LineComment struct {
	// Prefix is the literal comment-opening text (e.g. "--", "#").
	Prefix string

	// RequireBoundaryAfter, when true, means Prefix opens a comment only if
	// the character immediately following it is a boundary (whitespace or a
	// control char) or there is no following character (end-of-line/EOF).
	// When false, Prefix is unconditional — matching as soon as it is found.
	RequireBoundaryAfter bool
}

LineComment is one line-comment prefix a dialect recognizes.

RequireBoundaryAfter encodes a real lexical divergence between dialects: PostgreSQL's "--" opens a line comment unconditionally, but MySQL's "--" opens one ONLY when immediately followed by a boundary — whitespace, a control char, or end-of-line/EOF (https://dev.mysql.com/doc/refman/8.0/en/comments.html). Without this distinction, "SELECT 1--1 FROM t;" under MySQL would be misread as "SELECT 1" with the rest silently swallowed as a comment. MySQL sets this true for "--" and false for "#" (which is unconditional in both dialects); PostgreSQL sets it false for "--".

type Option added in v0.2.1

type Option func(*Parser)

Option configures a Parser at construction time.

func WithDialect added in v0.2.1

func WithDialect(d Dialect) Option

WithDialect binds the parser to the given SQL dialect descriptor. Absent this option, New() defaults to Postgres() — every existing caller is unaffected.

type Parser

type Parser struct {
	// contains filtered or unexported fields
}

Parser is the SQL-DDL schema parser: it reconstructs the neutral db.Schema by applying DDL statements IN ORDER (an incremental reducer over migrations). It implements providers.SchemaParser and nothing else — SQL is a schema source, not a programming language codefit audits for security/surface (ADR 0018).

A Parser is bound to exactly one Dialect at construction time (never threaded through ParseSchema/SchemaParser) — see Option/WithDialect.

func New

func New(opts ...Option) *Parser

New returns a SQL-DDL schema parser. Without options it defaults to the PostgreSQL dialect — today's behavior, unchanged.

func (*Parser) ParseSchema

func (p *Parser) ParseSchema(sources []providers.SourceFile) (*db.Schema, error)

ParseSchema parses ordered SQL-DDL sources (e.g. Flyway migrations, already version-ordered by the caller) into the accumulated final schema. It is filesystem-free: the caller reads and orders the files (ADR 0014). Statements outside the declared subset are skipped, never an error.

type QuotePair added in v0.2.1

type QuotePair struct {
	Open, Close byte
	Doubling    bool
}

QuotePair is one quoted-identifier delimiter pair. Doubling is true when an occurrence of Close inside the identifier is escaped by writing it twice: two double-quotes mean one literal double-quote, two closing brackets mean one literal bracket, two backticks mean one literal backtick — true for every dialect this package supports.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL