README
¶
Omni
SQL toolchain for multiple database engines. Each engine provides a parser, AST, and additional components such as catalog simulation and semantic analysis -- all in pure Go with zero dependencies.
Features
- Zero dependencies -- pure Go, no CGo, no generated code at runtime
- Full AST -- every parsed statement produces a complete abstract syntax tree
- Position tracking -- every AST node carries byte-offset location info
- Beyond parsing -- catalog simulation, DDL semantic analysis, and more per engine
Status
| Engine | Parser | Catalog | Completion |
|---|---|---|---|
| PostgreSQL | ✅ | ✅ | Planned |
| MySQL | 🚧 | Planned | Planned |
| SQL Server | 🚧 | Planned | Planned |
| Oracle | 🚧 | Planned | Planned |
Quick Start
go get github.com/bytebase/omni
PostgreSQL
package main
import (
"fmt"
"github.com/bytebase/omni/pg"
"github.com/bytebase/omni/pg/ast"
)
func main() {
stmts, err := pg.Parse("SELECT 1; CREATE TABLE t (id int);")
if err != nil {
panic(err)
}
for _, s := range stmts {
fmt.Printf("%-20T %s\n", s.AST, s.Text)
}
// *ast.SelectStmt SELECT 1;
// *ast.CreateStmt CREATE TABLE t (id int);
}
Architecture
omni/
├── pg/ PostgreSQL
│ ├── parse.go Public API: Parse(sql) → []Statement
│ ├── ast/ 210+ AST node types
│ ├── parser/ Recursive descent parser (~29K lines)
│ ├── catalog/ In-memory catalog simulation & DDL analysis
│ ├── parsertest/ 746 test cases
│ └── pgregress/ PostgreSQL regression test compatibility
├── mysql/ MySQL
│ ├── ast/ AST node types
│ ├── parser/ Recursive descent parser
│ └── parsertest/ Test cases
├── mssql/ SQL Server (T-SQL)
│ ├── ast/ AST node types
│ ├── parser/ Recursive descent parser
│ └── parsertest/ Test cases
├── oracle/ Oracle
│ ├── ast/ AST node types
│ └── parser/ Recursive descent parser
└── scripts/ Shared build & audit tooling
Development
# Run all tests
make test
# Test a specific engine
make test-pg
make test-mysql
make test-mssql
make test-oracle
# Build everything
make build
License
MIT -- see LICENSE.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package cassandra provides a parser for Apache Cassandra CQL (Cassandra Query Language).
|
Package cassandra provides a parser for Apache Cassandra CQL (Cassandra Query Language). |
|
ast
Package ast defines parse-tree node types for the omni Cassandra CQL parser.
|
Package ast defines parse-tree node types for the omni Cassandra CQL parser. |
|
Package cosmosdb provides a parser for Azure Cosmos DB NoSQL SQL API queries.
|
Package cosmosdb provides a parser for Azure Cosmos DB NoSQL SQL API queries. |
|
analysis
Package analysis extracts field-level information from parsed CosmosDB queries.
|
Package analysis extracts field-level information from parsed CosmosDB queries. |
|
ast
Package ast defines parse tree node types for Azure Cosmos DB NoSQL SQL API.
|
Package ast defines parse tree node types for Azure Cosmos DB NoSQL SQL API. |
|
parser
Package parser implements a hand-written parser for Azure Cosmos DB NoSQL SQL API.
|
Package parser implements a hand-written parser for Azure Cosmos DB NoSQL SQL API. |
|
doris
|
|
|
analysis
Package analysis provides query analysis for Doris SQL — statement classification, field extraction, and table access tracking.
|
Package analysis provides query analysis for Doris SQL — statement classification, field extraction, and table access tracking. |
|
ast
Package ast defines Doris parse-tree node types.
|
Package ast defines Doris parse-tree node types. |
|
parser
Package parser implements a hand-written Doris SQL lexer and recursive-descent parser.
|
Package parser implements a hand-written Doris SQL lexer and recursive-descent parser. |
|
Package elasticsearch provides a parser for Kibana Dev Console-style Elasticsearch REST request blocks.
|
Package elasticsearch provides a parser for Kibana Dev Console-style Elasticsearch REST request blocks. |
|
analysis
Package analysis classifies Elasticsearch REST API requests for masking and extracts field references from request bodies.
|
Package analysis classifies Elasticsearch REST API requests for masking and extracts field references from request bodies. |
|
parser
Package parser implements a hand-written recursive-descent parser for Kibana Dev Console-style Elasticsearch REST request blocks.
|
Package parser implements a hand-written recursive-descent parser for Kibana Dev Console-style Elasticsearch REST request blocks. |
|
googlesql
|
|
|
analysis
Package analysis provides Bytebase-facing query analysis for GoogleSQL — the SQL dialect shared by Google BigQuery and Google Cloud Spanner.
|
Package analysis provides Bytebase-facing query analysis for GoogleSQL — the SQL dialect shared by Google BigQuery and Google Cloud Spanner. |
|
ast
Package ast defines GoogleSQL parse-tree node types.
|
Package ast defines GoogleSQL parse-tree node types. |
|
ast/cmd/genwalker
command
Command genwalker generates walk_generated.go from parsenodes.go and node.go.
|
Command genwalker generates walk_generated.go from parsenodes.go and node.go. |
|
diagnostics
Package diagnostics provides a thin reporting layer over the GoogleSQL SQL parser, shaping raw parse errors into structured diagnostics suitable for editor integration (LSP-style), CI linting, and problem-panel display.
|
Package diagnostics provides a thin reporting layer over the GoogleSQL SQL parser, shaping raw parse errors into structured diagnostics suitable for editor integration (LSP-style), CI linting, and problem-panel display. |
|
parser
Package parser implements a hand-written GoogleSQL lexer and (in future DAG nodes) a recursive-descent parser.
|
Package parser implements a hand-written GoogleSQL lexer and (in future DAG nodes) a recursive-descent parser. |
|
parsertest
Package parsertest provides corpus-extraction helpers shared by the googlesql parser's corpus-closure gates (legacy_corpus_test.go and official_corpus_test.go).
|
Package parsertest provides corpus-extraction helpers shared by the googlesql parser's corpus-closure gates (legacy_corpus_test.go and official_corpus_test.go). |
|
mariadb
|
|
|
ast
Package ast defines MySQL parse tree node types.
|
Package ast defines MySQL parse tree node types. |
|
ast/cmd/genwalker
command
Command genwalker generates walk_generated.go from parsenodes.go and node.go.
|
Command genwalker generates walk_generated.go from parsenodes.go and node.go. |
|
catalog
Package catalog — define.go
|
Package catalog — define.go |
|
completion
Package completion provides parser-native C3-style SQL completion for MySQL.
|
Package completion provides parser-native C3-style SQL completion for MySQL. |
|
deparse
Package deparse converts MySQL AST nodes back to SQL text, matching MySQL 8.0's SHOW CREATE VIEW formatting.
|
Package deparse converts MySQL AST nodes back to SQL text, matching MySQL 8.0's SHOW CREATE VIEW formatting. |
|
parser
Package parser implements a recursive descent SQL parser for MySQL.
|
Package parser implements a recursive descent SQL parser for MySQL. |
|
scope
Package scope provides a shared, dependency-free scope data structure for tracking visible table references and resolving column names.
|
Package scope provides a shared, dependency-free scope data structure for tracking visible table references and resolving column names. |
|
validate
Package validate performs static semantic checks on a MySQL AST produced by mysql/parser.
|
Package validate performs static semantic checks on a MySQL AST produced by mysql/parser. |
|
Package mongo provides a parser for MongoDB shell (mongosh) commands.
|
Package mongo provides a parser for MongoDB shell (mongosh) commands. |
|
analysis
Package analysis extracts structural information from parsed MongoDB statements.
|
Package analysis extracts structural information from parsed MongoDB statements. |
|
ast
Package ast defines MongoDB (mongosh) parse tree node types.
|
Package ast defines MongoDB (mongosh) parse tree node types. |
|
catalog
Package catalog provides metadata context for MongoDB auto-completion.
|
Package catalog provides metadata context for MongoDB auto-completion. |
|
completion
Package completion provides auto-complete for MongoDB shell (mongosh) commands.
|
Package completion provides auto-complete for MongoDB shell (mongosh) commands. |
|
ast
Package ast defines T-SQL parse tree node types.
|
Package ast defines T-SQL parse tree node types. |
|
ast/cmd/genwalker
command
Command genwalker generates walk_generated.go from parsenodes.go and node.go.
|
Command genwalker generates walk_generated.go from parsenodes.go and node.go. |
|
completion
Package completion provides parser-native C3-style SQL completion for MSSQL.
|
Package completion provides parser-native C3-style SQL completion for MSSQL. |
|
parser
Package parser - alter_objects.go implements T-SQL ALTER DATABASE and ALTER INDEX parsing.
|
Package parser - alter_objects.go implements T-SQL ALTER DATABASE and ALTER INDEX parsing. |
|
mysql
|
|
|
ast
Package ast defines MySQL parse tree node types.
|
Package ast defines MySQL parse tree node types. |
|
ast/cmd/genwalker
command
Command genwalker generates walk_generated.go from parsenodes.go and node.go.
|
Command genwalker generates walk_generated.go from parsenodes.go and node.go. |
|
catalog
Package catalog — define.go
|
Package catalog — define.go |
|
completion
Package completion provides parser-native C3-style SQL completion for MySQL.
|
Package completion provides parser-native C3-style SQL completion for MySQL. |
|
deparse
Package deparse converts MySQL AST nodes back to SQL text, matching MySQL 8.0's SHOW CREATE VIEW formatting.
|
Package deparse converts MySQL AST nodes back to SQL text, matching MySQL 8.0's SHOW CREATE VIEW formatting. |
|
parser
Package parser implements a recursive descent SQL parser for MySQL.
|
Package parser implements a recursive descent SQL parser for MySQL. |
|
scope
Package scope provides a shared, dependency-free scope data structure for tracking visible table references and resolving column names.
|
Package scope provides a shared, dependency-free scope data structure for tracking visible table references and resolving column names. |
|
validate
Package validate performs static semantic checks on a MySQL AST produced by mysql/parser.
|
Package validate performs static semantic checks on a MySQL AST produced by mysql/parser. |
|
ast
Package ast defines Oracle PL/SQL parse tree node types.
|
Package ast defines Oracle PL/SQL parse tree node types. |
|
ast/cmd/genwalker
command
Command genwalker generates walk_generated.go from parsenodes.go and node.go.
|
Command genwalker generates walk_generated.go from parsenodes.go and node.go. |
|
parser
Package parser implements an Oracle PL/SQL lexer and recursive descent parser.
|
Package parser implements an Oracle PL/SQL lexer and recursive descent parser. |
|
parser/cmd/compatreport
command
|
|
|
Package partiql provides a hand-written parser for the PartiQL query language (AWS DynamoDB, Azure Cosmos DB).
|
Package partiql provides a hand-written parser for the PartiQL query language (AWS DynamoDB, Azure Cosmos DB). |
|
analysis
Package analysis provides query analysis for PartiQL statements.
|
Package analysis provides query analysis for PartiQL statements. |
|
ast
Package ast defines parse-tree node types for the omni PartiQL parser.
|
Package ast defines parse-tree node types for the omni PartiQL parser. |
|
catalog
Package catalog provides metadata context for PartiQL auto-completion and semantic analysis.
|
Package catalog provides metadata context for PartiQL auto-completion and semantic analysis. |
|
completion
Package completion provides auto-complete candidates for PartiQL queries.
|
Package completion provides auto-complete candidates for PartiQL queries. |
|
parser
Package parser — aggregate function-call parsing (DAG node 14, parser-aggregates).
|
Package parser — aggregate function-call parsing (DAG node 14, parser-aggregates). |
|
ast
Package nodes defines PostgreSQL parse tree node types.
|
Package nodes defines PostgreSQL parse tree node types. |
|
ast/cmd/genwalker
command
Command genwalker generates walk_generated.go from parsenodes.go.
|
Command genwalker generates walk_generated.go from parsenodes.go. |
|
completion
Package completion provides parser-native C3-style SQL completion for PostgreSQL.
|
Package completion provides parser-native C3-style SQL completion for PostgreSQL. |
|
parser
Package parser implements a PostgreSQL-compatible SQL lexer and parser.
|
Package parser implements a PostgreSQL-compatible SQL lexer and parser. |
|
pgregress
Package pgregress provides tools for running PostgreSQL regression test SQL files against the pgparser parser to verify parse compatibility.
|
Package pgregress provides tools for running PostgreSQL regression test SQL files against the pgparser parser to verify parse compatibility. |
|
plpgsql/ast
Package ast defines PL/pgSQL AST node types.
|
Package ast defines PL/pgSQL AST node types. |
|
plpgsql/parser
Package parser implements a recursive descent PL/pgSQL parser.
|
Package parser implements a recursive descent PL/pgSQL parser. |
|
ast
Package nodes defines PostgreSQL parse tree node types.
|
Package nodes defines PostgreSQL parse tree node types. |
|
ast/cmd/genwalker
command
Command genwalker generates walk_generated.go from parsenodes.go.
|
Command genwalker generates walk_generated.go from parsenodes.go. |
|
completion
Package completion provides parser-native C3-style SQL completion for PostgreSQL.
|
Package completion provides parser-native C3-style SQL completion for PostgreSQL. |
|
parser
Package parser implements a PostgreSQL-compatible SQL lexer and parser.
|
Package parser implements a PostgreSQL-compatible SQL lexer and parser. |
|
pgregress
Package pgregress provides tools for running PostgreSQL regression test SQL files against the pgparser parser to verify parse compatibility.
|
Package pgregress provides tools for running PostgreSQL regression test SQL files against the pgparser parser to verify parse compatibility. |
|
plpgsql/ast
Package ast defines PL/pgSQL AST node types.
|
Package ast defines PL/pgSQL AST node types. |
|
plpgsql/parser
Package parser implements a recursive descent PL/pgSQL parser.
|
Package parser implements a recursive descent PL/pgSQL parser. |
|
Dashboard serves a web-based monitoring UI for all parser engine pipelines.
|
Dashboard serves a web-based monitoring UI for all parser engine pipelines. |
|
snowflake
|
|
|
advisor
Package advisor implements the lint-rule framework for Snowflake SQL.
|
Package advisor implements the lint-rule framework for Snowflake SQL. |
|
advisor/rules
Package rules provides the production lint rules for the Snowflake advisor.
|
Package rules provides the production lint rules for the Snowflake advisor. |
|
analysis
Package analysis provides utilities for classifying and analyzing parsed Snowflake SQL AST nodes.
|
Package analysis provides utilities for classifying and analyzing parsed Snowflake SQL AST nodes. |
|
ast
Package ast defines snowflake parse-tree node types.
|
Package ast defines snowflake parse-tree node types. |
|
ast/cmd/genwalker
command
Command genwalker generates walk_generated.go from parsenodes.go and node.go.
|
Command genwalker generates walk_generated.go from parsenodes.go and node.go. |
|
deparse
Package deparse converts Snowflake AST nodes back to SQL strings.
|
Package deparse converts Snowflake AST nodes back to SQL strings. |
|
diagnostics
Package diagnostics provides a thin reporting layer over the Snowflake SQL parser, shaping raw parse errors into structured diagnostics suitable for editor integration (LSP-style), CI linting, and problem-panel display.
|
Package diagnostics provides a thin reporting layer over the Snowflake SQL parser, shaping raw parse errors into structured diagnostics suitable for editor integration (LSP-style), CI linting, and problem-panel display. |
|
parser
Package parser implements a hand-written Snowflake SQL lexer and (in future DAG nodes) a recursive-descent parser.
|
Package parser implements a hand-written Snowflake SQL lexer and (in future DAG nodes) a recursive-descent parser. |
|
starrocks
|
|
|
analysis
Package analysis provides query analysis for Doris SQL — statement classification, field extraction, and table access tracking.
|
Package analysis provides query analysis for Doris SQL — statement classification, field extraction, and table access tracking. |
|
ast
Package ast defines Doris parse-tree node types.
|
Package ast defines Doris parse-tree node types. |
|
parser
Package parser implements a hand-written Doris SQL lexer and recursive-descent parser.
|
Package parser implements a hand-written Doris SQL lexer and recursive-descent parser. |
|
tidb
|
|
|
ast
Package ast defines MySQL parse tree node types.
|
Package ast defines MySQL parse tree node types. |
|
ast/cmd/genwalker
command
Command genwalker generates walk_generated.go from parsenodes.go and node.go.
|
Command genwalker generates walk_generated.go from parsenodes.go and node.go. |
|
catalog
Package catalog — define.go
|
Package catalog — define.go |
|
completion
Package completion provides parser-native C3-style SQL completion for TiDB.
|
Package completion provides parser-native C3-style SQL completion for TiDB. |
|
deparse
Package deparse converts MySQL AST nodes back to SQL text, matching MySQL 8.0's SHOW CREATE VIEW formatting.
|
Package deparse converts MySQL AST nodes back to SQL text, matching MySQL 8.0's SHOW CREATE VIEW formatting. |
|
parser
Package parser implements a recursive descent SQL parser for MySQL.
|
Package parser implements a recursive descent SQL parser for MySQL. |
|
scope
Package scope provides a shared, dependency-free scope data structure for tracking visible table references and resolving column names.
|
Package scope provides a shared, dependency-free scope data structure for tracking visible table references and resolving column names. |
|
trino
|
|
|
analysis
Package analysis provides Bytebase-facing query analysis for Trino SQL: statement classification (the query_type / validateQuery contract) and query span extraction (table/column lineage for masking and access tracking).
|
Package analysis provides Bytebase-facing query analysis for Trino SQL: statement classification (the query_type / validateQuery contract) and query span extraction (table/column lineage for masking and access tracking). |
|
ast
Package ast defines Trino parse-tree node types.
|
Package ast defines Trino parse-tree node types. |
|
catalog
Package catalog provides in-memory metadata context for Trino auto-completion and query-span/lineage analysis.
|
Package catalog provides in-memory metadata context for Trino auto-completion and query-span/lineage analysis. |
|
completion
Package completion provides parser-native auto-complete candidates for Trino SQL, the omni counterpart of bytebase's plugin/parser/trino Completion.
|
Package completion provides parser-native auto-complete candidates for Trino SQL, the omni counterpart of bytebase's plugin/parser/trino Completion. |
|
deparse
Package deparse renders Trino schema metadata back into SQL data-definition language (SDL).
|
Package deparse renders Trino schema metadata back into SQL data-definition language (SDL). |
|
internal/trinooracle
Package trinooracle is the differential-testing oracle for the omni Trino parser.
|
Package trinooracle is the differential-testing oracle for the omni Trino parser. |
|
parser
Package parser implements a hand-written recursive-descent parser for Trino SQL (Trino 481 grammar, derived from the official SqlBase.g4).
|
Package parser implements a hand-written recursive-descent parser for Trino SQL (Trino 481 grammar, derived from the official SqlBase.g4). |
Click to show internal directories.
Click to hide internal directories.