gosqlx

package
v1.15.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 30, 2026 License: Apache-2.0 Imports: 24 Imported by: 2

README

gosqlx - High-Level SQL Parsing API

Go Reference

The gosqlx package provides a convenient, high-level API for SQL parsing in GoSQLX. It wraps the lower-level tokenizer and parser APIs to provide a simple, ergonomic interface for common operations with automatic object pool management.

Quick Start

import "github.com/ajitpratap0/GoSQLX/pkg/gosqlx"

// Parse SQL in one line
ast, err := gosqlx.Parse("SELECT * FROM users WHERE active = true")
if err != nil {
    log.Fatal(err)
}

// Or just validate
if err := gosqlx.Validate("SELECT * FROM users"); err != nil {
    log.Fatal("Invalid SQL:", err)
}

Features

  • Simple API: Parse SQL with a single function call
  • Automatic Resource Management: Object pools handled internally
  • Batch Processing: Efficient parsing of multiple queries
  • Validation: Quick syntax validation without full AST construction
  • Format Support: Basic SQL formatting (with plans for advanced formatting)
  • Production Ready: Built on top of GoSQLX's production-tested parser

Installation

go get github.com/ajitpratap0/GoSQLX

API Reference

Core Functions
Parse

Parse SQL string into an Abstract Syntax Tree (AST):

func Parse(sql string) (*ast.AST, error)

Example:

ast, err := gosqlx.Parse("SELECT * FROM users")
if err != nil {
    return err
}

// Access parsed statements
for _, stmt := range ast.Statements {
    // Process each statement
}
ParseBytes

Parse SQL from a byte slice (useful for file I/O):

func ParseBytes(sql []byte) (*ast.AST, error)

Example:

sqlBytes := []byte("SELECT * FROM users")
ast, err := gosqlx.ParseBytes(sqlBytes)
Validate

Validate SQL syntax without building the full AST:

func Validate(sql string) error

Example:

if err := gosqlx.Validate(userInput); err != nil {
    return fmt.Errorf("invalid SQL: %w", err)
}
MustParse

Parse SQL or panic (useful for testing and initialization):

func MustParse(sql string) *ast.AST

Example:

// In tests or init functions
ast := gosqlx.MustParse("SELECT 1")
Batch Processing Functions
ParseMultiple

Parse multiple SQL statements efficiently by reusing parser resources:

func ParseMultiple(queries []string) ([]*ast.AST, error)

Example:

queries := []string{
    "SELECT * FROM users",
    "SELECT * FROM orders",
    "SELECT * FROM products",
}

asts, err := gosqlx.ParseMultiple(queries)
if err != nil {
    return err
}

for i, ast := range asts {
    fmt.Printf("Query %d: %d statements\n", i, len(ast.Statements))
}
ValidateMultiple

Validate multiple SQL statements:

func ValidateMultiple(queries []string) error

Example:

queries := []string{
    "SELECT * FROM users",
    "INSERT INTO logs (msg) VALUES ('test')",
}

if err := gosqlx.ValidateMultiple(queries); err != nil {
    return fmt.Errorf("validation failed: %w", err)
}
Formatting Functions
Format

Format SQL according to specified options:

func Format(sql string, options FormatOptions) (string, error)

Example:

opts := gosqlx.DefaultFormatOptions()
opts.AddSemicolon = true
opts.UppercaseKeywords = true

formatted, err := gosqlx.Format("select * from users", opts)
// Returns validated SQL with semicolon added
FormatOptions

Control formatting behavior:

type FormatOptions struct {
    IndentSize        int  // Number of spaces for indentation (default: 2)
    UppercaseKeywords bool // Uppercase SQL keywords (default: false)
    AddSemicolon      bool // Add semicolon if missing (default: false)
    SingleLineLimit   int  // Max line length before breaking (default: 80)
}

Note: Full AST-based formatting is planned for a future release. Current implementation validates SQL and applies basic formatting options.

Use Cases

1. Input Validation
func validateUserQuery(userSQL string) error {
    // Quick validation before executing
    if err := gosqlx.Validate(userSQL); err != nil {
        return fmt.Errorf("invalid SQL: %w", err)
    }
    return nil
}
2. SQL Analysis Tool
func analyzeSQL(sql string) error {
    ast, err := gosqlx.Parse(sql)
    if err != nil {
        return err
    }

    // Analyze the AST
    for _, stmt := range ast.Statements {
        switch s := stmt.(type) {
        case *ast.SelectStatement:
            fmt.Println("Found SELECT statement")
        case *ast.InsertStatement:
            fmt.Println("Found INSERT statement")
        // Handle other statement types
        }
    }

    return nil
}
3. Batch Query Processing
func processMigrationFiles(files []string) error {
    var queries []string

    // Read all SQL files
    for _, file := range files {
        content, err := os.ReadFile(file)
        if err != nil {
            return err
        }
        queries = append(queries, string(content))
    }

    // Validate all at once
    if err := gosqlx.ValidateMultiple(queries); err != nil {
        return fmt.Errorf("migration validation failed: %w", err)
    }

    // Parse all efficiently
    asts, err := gosqlx.ParseMultiple(queries)
    if err != nil {
        return err
    }

    // Process each migration
    for i, ast := range asts {
        fmt.Printf("Migration %s: %d statements\n", files[i], len(ast.Statements))
    }

    return nil
}
4. SQL Formatting Service
func formatSQLEndpoint(w http.ResponseWriter, r *http.Request) {
    sql := r.FormValue("sql")

    opts := gosqlx.DefaultFormatOptions()
    opts.AddSemicolon = true
    opts.IndentSize = 4

    formatted, err := gosqlx.Format(sql, opts)
    if err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    w.Write([]byte(formatted))
}

Migration from Low-Level API

If you're currently using the low-level tokenizer and parser APIs directly, migrating to gosqlx is simple:

Before (Low-Level API)
import (
    "github.com/ajitpratap0/GoSQLX/pkg/sql/tokenizer"
    "github.com/ajitpratap0/GoSQLX/pkg/sql/parser"
)

func parseSQL(sql string) (*ast.AST, error) {
    // Get tokenizer from pool
    tkz := tokenizer.GetTokenizer()
    defer tokenizer.PutTokenizer(tkz)

    // Tokenize
    tokens, err := tkz.Tokenize([]byte(sql))
    if err != nil {
        return nil, err
    }

    // Convert tokens
    converter := parser.NewTokenConverter()
    result, err := converter.Convert(tokens)
    if err != nil {
        return nil, err
    }

    // Parse
    p := parser.NewParser()
    defer p.Release()

    return p.Parse(result.Tokens)
}
After (High-Level API)
import "github.com/ajitpratap0/GoSQLX/pkg/gosqlx"

func parseSQL(sql string) (*ast.AST, error) {
    return gosqlx.Parse(sql)
}

Benefits:

  • Less boilerplate code
  • Automatic resource management
  • Simpler error handling
  • Same performance characteristics

Best Practices

1. Use Validate for Quick Checks

If you only need to check syntax validity, use Validate instead of Parse:

// Good - faster for validation only
if err := gosqlx.Validate(sql); err != nil {
    return err
}

// Less efficient - builds full AST
if _, err := gosqlx.Parse(sql); err != nil {
    return err
}
2. Batch Processing

For multiple queries, use batch functions to reuse parser resources:

// Good - reuses resources
asts, err := gosqlx.ParseMultiple(queries)

// Less efficient - recreates resources for each query
for _, sql := range queries {
    ast, err := gosqlx.Parse(sql)
    // ...
}
3. Use MustParse Only for Literals

Use MustParse only with SQL you control (tests, constants):

// Good - SQL literal in code
var testQuery = gosqlx.MustParse("SELECT 1")

// Bad - user input could panic
ast := gosqlx.MustParse(userInput) // Don't do this!
4. Performance-Critical Code

For performance-critical applications, consider using the low-level API directly:

// High-level API - simpler but slight overhead
ast, err := gosqlx.Parse(sql)

// Low-level API - more control, slightly faster
tkz := tokenizer.GetTokenizer()
defer tokenizer.PutTokenizer(tkz)
// ... manual token/parse operations

Error Handling

All functions return descriptive errors with context:

ast, err := gosqlx.Parse("SELECT * FROM")
if err != nil {
    // Error includes position and context
    fmt.Printf("Parse error: %v\n", err)
    // Output: Parse error: parsing failed: unexpected EOF at line 1, column 14
}

Batch functions include query index in error messages:

err := gosqlx.ValidateMultiple(queries)
if err != nil {
    // Error includes query number
    fmt.Printf("Error: %v\n", err)
    // Output: Error: query 2: invalid SQL: unexpected token 'FROM'
}

Supported SQL Features

The gosqlx package supports all SQL features provided by GoSQLX:

  • DML: SELECT, INSERT, UPDATE, DELETE
  • DDL: CREATE, ALTER, DROP
  • JOINs: INNER, LEFT, RIGHT, FULL OUTER, CROSS, NATURAL
  • Subqueries: Scalar, row, table subqueries
  • Window Functions: All SQL-99 window functions with OVER clause
  • CTEs: WITH clause, recursive CTEs
  • Set Operations: UNION, INTERSECT, EXCEPT
  • Advanced Clauses: GROUP BY, HAVING, ORDER BY, LIMIT, OFFSET
  • Multiple Dialects: PostgreSQL, MySQL, SQL Server, Oracle, SQLite

Performance

The high-level API has minimal overhead compared to the low-level API:

BenchmarkParse-8              50000    ~30-50 μs/op    ~15 KB/op
BenchmarkParseMultiple-8      10000   ~150-200 μs/op   ~75 KB/op (5 queries)
BenchmarkValidate-8           50000    ~30-50 μs/op    ~15 KB/op
BenchmarkFormat-8             45000    ~35-55 μs/op    ~16 KB/op

Object pooling keeps allocations low and performance high.

Examples

See comprehensive examples in:

Advanced Usage

For advanced use cases requiring fine-grained control:

  • Use the low-level APIs in pkg/sql/tokenizer and pkg/sql/parser
  • Access AST visitor pattern in pkg/sql/ast/visitor.go
  • Implement custom AST traversal and analysis

Contributing

Contributions are welcome! Please see the main GoSQLX repository for contribution guidelines.

License

This package is part of GoSQLX. See the main repository for license information.

Documentation

Overview

Package gosqlx provides high-level convenience functions for SQL parsing, validation, formatting, and metadata extraction, with automatic object pool management.

GoSQLX is a production-ready, high-performance SQL parsing SDK for Go that supports multiple SQL dialects with comprehensive SQL-99 and SQL:2003 feature support. The primary entry points are Parse, Validate, Format, and ParseWithDialect.

Overview

This package wraps the lower-level tokenizer and parser APIs to provide a simple, ergonomic interface for common SQL operations. All object pool management is handled internally, making it ideal for applications that prioritize ease of use over fine-grained performance control.

Key functions:

  • Parse: tokenizes and parses SQL, returning an *ast.AST
  • Validate: checks whether SQL is syntactically valid without building a full AST
  • Format: formats SQL text according to configurable style options
  • ParseWithDialect: parses SQL using dialect-specific rules (PostgreSQL, MySQL, etc.)
  • ParseMultiple: efficiently parses a batch of SQL statements by reusing pooled objects
  • ParseWithContext: parse with context support for cancellation and timeouts

For performance-critical applications requiring fine-grained control over object lifecycle and pooling, use the lower-level APIs in pkg/sql/tokenizer and pkg/sql/parser directly.

Key Features

  • Blazing Fast: 1.38M+ ops/sec sustained, 1.5M+ peak throughput
  • Memory Efficient: 60-80% reduction through intelligent object pooling
  • Thread-Safe: Race-free, validated with comprehensive concurrent testing
  • Zero-Copy: Direct byte slice operations with <1μs latency
  • Multi-Dialect: PostgreSQL, MySQL, SQL Server, Oracle, SQLite support
  • Production-Ready: ~80-85% SQL-99 compliance, battle-tested

Supported SQL Features (v1.6.0)

SQL Standards Compliance:

  • DML: SELECT, INSERT, UPDATE, DELETE with complex expressions
  • DDL: CREATE TABLE/VIEW/INDEX, ALTER TABLE, DROP statements
  • CTEs: WITH clause, RECURSIVE CTEs with proper termination
  • Set Operations: UNION, EXCEPT, INTERSECT with proper precedence
  • Window Functions: Complete SQL-99 support (ROW_NUMBER, RANK, DENSE_RANK, NTILE, LAG, LEAD, FIRST_VALUE, LAST_VALUE)
  • Window Frames: ROWS/RANGE with BETWEEN clauses and frame bounds
  • JOIN Types: INNER, LEFT, RIGHT, FULL OUTER, CROSS, NATURAL with USING/ON
  • MERGE: SQL:2003 MERGE with WHEN MATCHED/NOT MATCHED clauses
  • Grouping: GROUPING SETS, ROLLUP, CUBE (SQL-99 T431)
  • FETCH: FETCH FIRST/NEXT with ROWS ONLY, WITH TIES, PERCENT (SQL-99 F861)
  • Materialized Views: CREATE, DROP, REFRESH MATERIALIZED VIEW
  • TRUNCATE: TRUNCATE TABLE with CASCADE/RESTRICT, RESTART/CONTINUE IDENTITY
  • Expressions: BETWEEN, IN, LIKE, IS NULL, CASE, CAST, subqueries
  • Ordering: NULLS FIRST/LAST in ORDER BY clauses (SQL-99 F851)

PostgreSQL Extensions (v1.6.0):

  • LATERAL JOIN: Correlated subqueries in FROM clause
  • JSON/JSONB Operators: ->, ->>, #>, #>>, @>, <@, ?, ?|, ?&, #-
  • DISTINCT ON: PostgreSQL-specific row selection
  • FILTER Clause: Conditional aggregation (SQL:2003 T612)
  • RETURNING Clause: Return modified rows from INSERT/UPDATE/DELETE
  • Aggregate ORDER BY: ORDER BY inside aggregate functions

Performance Characteristics

Object Pooling:

  • AST pool: sync.Pool-based AST container reuse
  • Tokenizer pool: Reusable tokenizer instances
  • Statement pools: Individual pools for SELECT, INSERT, UPDATE, DELETE
  • Expression pools: Pooled identifiers, binary expressions, literals
  • Pool efficiency: 95%+ hit rate in production workloads

Benchmarks (v1.6.0):

  • Parse throughput: 1.38M+ operations/second sustained
  • Peak throughput: 1.5M+ operations/second
  • Tokenization: 8M+ tokens/second
  • Latency: <1μs for complex queries with window functions
  • Memory reduction: 60-80% with object pooling
  • Token comparison: 14x faster with Type field (0.28ns vs 4.9ns)
  • Keyword suggestions: 575x faster with caching

Thread Safety

All functions in this package are thread-safe and race-free. The package has been validated through comprehensive concurrent testing with 20,000+ concurrent operations showing zero race conditions.

Object pools are safely managed with sync.Pool, providing lock-free performance while maintaining thread safety guarantees.

Error Handling

All parsing errors are structured with error codes and detailed position information:

  • E1xxx: Tokenization errors (unexpected character, invalid token)
  • E2xxx: Parser errors (syntax error, unexpected token)
  • E3xxx: Semantic errors (undefined reference, type mismatch)

Errors include:

  • Precise line and column information
  • Relevant SQL context excerpt
  • Helpful error messages with suggestions
  • Error recovery hints for common mistakes

Quick Start

Basic SQL parsing:

sql := "SELECT * FROM users WHERE active = true"
ast, err := gosqlx.Parse(sql)
if err != nil {
    log.Fatal(err)
}
fmt.Printf("Parsed: %T\n", ast)

Common Usage Patterns

Parsing with timeout:

ast, err := gosqlx.ParseWithTimeout(sql, 5*time.Second)
if err == context.DeadlineExceeded {
    log.Println("Parsing timed out")
}

Parsing multiple queries efficiently:

queries := []string{
    "SELECT * FROM users",
    "SELECT * FROM orders",
}
asts, err := gosqlx.ParseMultiple(queries)

Validating SQL syntax:

if err := gosqlx.Validate("SELECT * FROM users"); err != nil {
    fmt.Printf("Invalid SQL: %v\n", err)
}

Extracting metadata:

sql := "SELECT u.name, o.total FROM users u JOIN orders o ON u.id = o.user_id"
ast, _ := gosqlx.Parse(sql)
metadata := gosqlx.ExtractMetadata(ast)
fmt.Printf("Tables: %v, Columns: %v\n", metadata.Tables, metadata.Columns)

Memory Management

The gosqlx package automatically manages object pools for optimal performance. When using the convenience functions (Parse, ParseMultiple, etc.), objects are automatically returned to pools after use.

For manual control over object lifecycle, use the lower-level APIs:

// Manual object pool management
tkz := tokenizer.GetTokenizer()
defer tokenizer.PutTokenizer(tkz)

astObj := ast.NewAST()
defer ast.ReleaseAST(astObj)

// Use objects
tokens, err := tkz.Tokenize(sqlBytes)
result, err := parser.Parse(tokens)

IMPORTANT: Always use defer with pool return functions to prevent resource leaks and maintain optimal performance. Object pooling provides 60-80% memory reduction.

PostgreSQL JSON/JSONB Support

Complete support for PostgreSQL JSON operators:

// Field access operators
SELECT data->'name' FROM users;           // Get JSON field as JSON
SELECT data->>'name' FROM users;          // Get JSON field as text

// Path access operators
SELECT data#>'{address,city}' FROM users; // Get nested value as JSON
SELECT data#>>'{address,city}' FROM users; // Get nested value as text

// Containment operators
SELECT * FROM users WHERE data @> '{"status":"active"}';  // Contains
SELECT * FROM users WHERE '{"status":"active"}' <@ data;  // Contained by

// Existence operators
SELECT * FROM users WHERE data ? 'email';          // Has key
SELECT * FROM users WHERE data ?| array['a','b']; // Has any key
SELECT * FROM users WHERE data ?& array['a','b']; // Has all keys

// Delete operator
SELECT data #- '{address,zip}' FROM users; // Delete at path

Window Functions

Full SQL-99 window function support with all frame specifications:

// Ranking functions
SELECT name, salary,
    ROW_NUMBER() OVER (ORDER BY salary DESC) as row_num,
    RANK() OVER (PARTITION BY dept ORDER BY salary DESC) as rank,
    DENSE_RANK() OVER (ORDER BY score) as dense_rank,
    NTILE(4) OVER (ORDER BY score) as quartile
FROM employees;

// Analytic functions with offsets
SELECT date, amount,
    LAG(amount, 1) OVER (ORDER BY date) as prev_amount,
    LEAD(amount, 2, 0) OVER (ORDER BY date) as future_amount
FROM transactions;

// Window frames
SELECT date, amount,
    SUM(amount) OVER (
        ORDER BY date
        ROWS BETWEEN 2 PRECEDING AND CURRENT ROW
    ) as rolling_sum,
    AVG(amount) OVER (
        ORDER BY date
        RANGE UNBOUNDED PRECEDING
    ) as running_avg
FROM transactions;

Advanced SQL Features

MERGE statements (SQL:2003):

MERGE INTO target t
USING source s ON t.id = s.id
WHEN MATCHED THEN
    UPDATE SET t.value = s.value
WHEN NOT MATCHED THEN
    INSERT (id, value) VALUES (s.id, s.value);

GROUPING SETS, ROLLUP, CUBE (SQL-99 T431):

-- Explicit grouping combinations
SELECT region, product, SUM(sales)
FROM orders
GROUP BY GROUPING SETS ((region), (product), (region, product), ());

-- Hierarchical subtotals
SELECT year, quarter, SUM(revenue)
FROM sales
GROUP BY ROLLUP (year, quarter);

-- All possible combinations
SELECT region, product, SUM(amount)
FROM sales
GROUP BY CUBE (region, product);

LATERAL JOIN (PostgreSQL):

SELECT u.name, recent_orders.order_date
FROM users u,
LATERAL (
    SELECT * FROM orders
    WHERE user_id = u.id
    ORDER BY order_date DESC
    LIMIT 3
) recent_orders;

FILTER clause (SQL:2003 T612):

SELECT
    COUNT(*) FILTER (WHERE status = 'active') AS active_count,
    SUM(amount) FILTER (WHERE type = 'credit') AS total_credits
FROM transactions;

RETURNING clause (PostgreSQL):

INSERT INTO users (name, email)
VALUES ('John', 'john@example.com')
RETURNING id, created_at;

UPDATE products
SET price = price * 1.1
WHERE category = 'Electronics'
RETURNING id, price;

Integration Examples

Database query analysis:

func analyzeQuery(query string) error {
    ast, err := gosqlx.Parse(query)
    if err != nil {
        return fmt.Errorf("invalid SQL: %w", err)
    }

    // Extract metadata for query optimization
    tables := gosqlx.ExtractTables(ast)
    columns := gosqlx.ExtractColumns(ast)
    functions := gosqlx.ExtractFunctions(ast)

    fmt.Printf("Query uses %d tables, %d columns, %d functions\n",
        len(tables), len(columns), len(functions))
    return nil
}

SQL security scanning:

import "github.com/ajitpratap0/GoSQLX/pkg/sql/security"

func checkSQLSafety(query string) error {
    scanner := security.NewScanner()
    findings := scanner.Scan(query)

    for _, finding := range findings {
        if finding.Severity == security.SeverityCritical {
            return fmt.Errorf("SQL injection risk: %s", finding.Message)
        }
    }
    return nil
}

Query transformation:

func transformQuery(sql string) (string, error) {
    ast, err := gosqlx.Parse(sql)
    if err != nil {
        return "", err
    }

    // Use visitor pattern to transform AST
    // Then format back to SQL
    opts := gosqlx.DefaultFormatOptions()
    opts.UppercaseKeywords = true
    return gosqlx.Format(sql, opts)
}

Known Limitations

While GoSQLX supports a comprehensive set of SQL features, the following are partially supported or not yet fully implemented:

  1. CASE Expressions: Simple and searched CASE expressions in some contexts
  2. CAST Expressions: Type conversion in complex expressions
  3. IN Expressions: Complex value lists and nested subqueries in some contexts
  4. BETWEEN Expressions: Range comparisons in complex expressions
  5. Schema-Qualified Names: Some 3-part qualified names (db.schema.table)
  6. Complex Recursive CTEs: Recursive CTEs with complex JOIN syntax

These limitations represent areas of ongoing development. For queries using these features, parsing may succeed with partial AST representation, or may fail with descriptive error messages.

CLI Tool Integration

The gosqlx CLI tool provides command-line access to parsing functionality:

# Install CLI
go install github.com/ajitpratap0/GoSQLX/cmd/gosqlx@latest

# Validate SQL
gosqlx validate "SELECT * FROM users WHERE active = true"

# Format SQL
gosqlx format -i query.sql

# Analyze SQL structure
gosqlx analyze "SELECT COUNT(*) FROM orders GROUP BY status"

# Parse to JSON AST
gosqlx parse -f json query.sql

# Start LSP server for IDE integration
gosqlx lsp

LSP Server (v1.6.0)

GoSQLX includes a full Language Server Protocol implementation for IDE integration:

# Start LSP server
gosqlx lsp

# With debug logging
gosqlx lsp --log /tmp/lsp.log

LSP Features:

  • Real-time SQL syntax validation with diagnostics
  • Hover documentation for 60+ SQL keywords and functions
  • Intelligent autocomplete with 100+ keywords and 22 snippets
  • SQL code formatting with customizable options
  • Document symbols for SQL statement navigation
  • Function signature help for 20+ SQL functions
  • Quick fixes (add semicolon, uppercase keywords)

VSCode Extension:

  • Search "GoSQLX" in VSCode marketplace
  • Automatic integration with gosqlx binary
  • Multi-dialect SQL support
  • Customizable formatting preferences

Configuration

GoSQLX can be configured via .gosqlx.yml file:

# .gosqlx.yml
dialect: postgresql
format:
  indent_size: 2
  uppercase_keywords: true
  max_line_length: 100
linter:
  rules:
    L001: error  # Trailing whitespace
    L007: warn   # Keyword case

See docs/CONFIGURATION.md for complete configuration reference.

Documentation

Additional documentation:

  • docs/GETTING_STARTED.md - Quick start guide for new users
  • docs/USAGE_GUIDE.md - Comprehensive usage guide
  • docs/LSP_GUIDE.md - LSP server and IDE integration
  • docs/LINTING_RULES.md - All 10 linting rules (L001-L010)
  • docs/CONFIGURATION.md - Configuration file reference
  • docs/SQL_COMPATIBILITY.md - SQL dialect compatibility matrix

Production Deployment

GoSQLX is production-ready and battle-tested:

  • Race Detection: Zero race conditions (validated with 20,000+ concurrent operations)
  • Performance: 1.5M ops/sec peak, 1.38M+ sustained throughput
  • Unicode Support: Full international compliance (8 languages tested)
  • SQL Compatibility: Multi-dialect with 115+ real-world queries validated
  • Memory Management: Zero leaks detected, stable under extended load
  • Error Handling: Robust recovery with precise position information

Quality Metrics:

  • Thread Safety: 5/5 stars - Race-free codebase confirmed
  • Performance: 5/5 stars - 1.38M+ ops/sec sustained, <1μs latency
  • Reliability: 5/5 stars - 95%+ success rate on real-world SQL
  • Memory Efficiency: 5/5 stars - 60-80% reduction with pooling

Package Structure

The gosqlx package is part of the larger GoSQLX SDK:

pkg/
├── gosqlx/          # High-level convenience API (this package)
├── sql/
│   ├── tokenizer/   # Zero-copy SQL lexer
│   ├── parser/      # Recursive descent parser
│   ├── ast/         # Abstract Syntax Tree nodes
│   ├── keywords/    # SQL keyword definitions
│   └── security/    # SQL injection detection
├── models/          # Core data structures (100% test coverage)
├── errors/          # Structured error handling
├── metrics/         # Performance monitoring
├── linter/          # SQL linting engine (10 rules)
└── lsp/             # Language Server Protocol server

For fine-grained control, use the lower-level packages directly.

Contributing

Contributions are welcome! See the project repository for contribution guidelines.

Repository: https://github.com/ajitpratap0/GoSQLX Issues: https://github.com/ajitpratap0/GoSQLX/issues Discussions: https://github.com/ajitpratap0/GoSQLX/discussions

License

GoSQLX is licensed under the Apache License 2.0. See LICENSE file for details.

This file provides SQL metadata extraction functions for the gosqlx package.

The extraction functions traverse the Abstract Syntax Tree (AST) to collect metadata such as table names, column references, function calls, and qualified identifiers. These functions are useful for query analysis, security scanning, dependency tracking, and query optimization.

Extraction Functions Overview

The gosqlx package provides six main extraction functions:

  • ExtractTables: Simple table names (e.g., "users", "orders")
  • ExtractTablesQualified: Qualified table names (e.g., "public.users")
  • ExtractColumns: Simple column names (e.g., "name", "email")
  • ExtractColumnsQualified: Qualified column names (e.g., "u.name")
  • ExtractFunctions: Function names (e.g., "COUNT", "SUM")
  • ExtractMetadata: All metadata in one call (convenience function)

All extraction functions are thread-safe and can be called concurrently on different AST instances. They return deduplicated results, so each identifier appears only once in the output regardless of how many times it appears in the query.

Performance Characteristics

Extraction functions are optimized for performance:

  • Single AST traversal per extraction call
  • O(N) time complexity where N is the number of AST nodes
  • HashMap-based deduplication for O(1) lookup
  • Minimal memory allocation (reuses visitor pattern)

For large ASTs (1000+ nodes), expect extraction times <100μs on modern hardware.

Supported Expression Types

The extraction functions correctly traverse all standard expression types including: CASE (simple and searched), CAST, IN, BETWEEN, EXTRACT, SUBSTRING, POSITION, subqueries, recursive CTEs with JOINs, window functions, and all arithmetic/logical operators.

If you encounter parsing issues with SQL queries that should be supported, please report them at: https://github.com/ajitpratap0/GoSQLX/issues

Example (AdvancedFeatures)

Example_advancedFeatures demonstrates parsing advanced SQL features.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	// Window functions
	windowSQL := "SELECT name, ROW_NUMBER() OVER (PARTITION BY dept ORDER BY salary DESC) as rank FROM employees"
	ast1, err := gosqlx.Parse(windowSQL)
	if err != nil {
		log.Fatal(err)
	}

	// CTEs
	cteSQL := "WITH active AS (SELECT * FROM users WHERE active = true) SELECT * FROM active"
	ast2, err := gosqlx.Parse(cteSQL)
	if err != nil {
		log.Fatal(err)
	}

	// JOINs
	joinSQL := "SELECT u.name, o.total FROM users u INNER JOIN orders o ON u.id = o.user_id"
	ast3, err := gosqlx.Parse(joinSQL)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Parsed window: %d, CTE: %d, JOIN: %d\n", len(ast1.Statements), len(ast2.Statements), len(ast3.Statements))
}
Output:
Parsed window: 1, CTE: 1, JOIN: 1
Example (AnalyzeQuery)

Example_analyzeQuery demonstrates using extraction for query analysis.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := `SELECT
		u.username,
		COUNT(o.id) as total_orders,
		SUM(o.amount) as total_spent,
		UPPER(u.email) as email_upper
	FROM users u
	LEFT JOIN orders o ON u.id = o.user_id
	WHERE u.active = true AND o.status = 'completed'
	GROUP BY u.id, u.username, u.email
	HAVING COUNT(o.id) > 10
	ORDER BY total_spent DESC`

	ast, err := gosqlx.Parse(sql)
	if err != nil {
		log.Fatal(err)
	}

	metadata := gosqlx.ExtractMetadata(ast)

	fmt.Printf("Query Analysis:\n")
	fmt.Printf("- Joins %d tables\n", len(metadata.Tables))
	fmt.Printf("- References %d columns\n", len(metadata.Columns))
	fmt.Printf("- Uses %d functions\n", len(metadata.Functions))
}
Output:
Query Analysis:
- Joins 2 tables
- References 8 columns
- Uses 3 functions
Example (Batch)

Example_batch demonstrates parsing multiple SQL statements efficiently.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	queries := []string{
		"SELECT * FROM users",
		"SELECT * FROM orders",
		"SELECT * FROM products",
	}

	asts, err := gosqlx.ParseMultiple(queries)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Successfully parsed %d queries\n", len(asts))
}
Output:
Successfully parsed 3 queries
Example (BatchValidation)

Example_batchValidation demonstrates validating multiple queries.

package main

import (
	"fmt"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	queries := []string{
		"SELECT * FROM users",
		"INSERT INTO logs (message) VALUES ('test')",
		"UPDATE users SET active = false WHERE id = 1",
		"DELETE FROM temp_data WHERE created_at < NOW()",
	}

	if err := gosqlx.ValidateMultiple(queries); err != nil {
		fmt.Printf("Validation failed: %v\n", err)
		return
	}

	fmt.Printf("All %d queries are valid\n", len(queries))
}
Output:
All 4 queries are valid
Example (ComplexQuery)

Example_complexQuery demonstrates parsing a complex SQL query.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := `
		SELECT
			u.id,
			u.name,
			COUNT(o.id) as order_count
		FROM users u
		LEFT JOIN orders o ON u.id = o.user_id
		WHERE u.active = true
		GROUP BY u.id, u.name
		ORDER BY order_count DESC
	`

	ast, err := gosqlx.Parse(sql)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Parsed complex query with %d statement(s)\n", len(ast.Statements))
}
Output:
Parsed complex query with 1 statement(s)
Example (Cte)

Example_cte demonstrates parsing Common Table Expressions (CTEs).

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := `
		WITH active_users AS (
			SELECT * FROM users WHERE active = true
		)
		SELECT * FROM active_users
	`

	ast, err := gosqlx.Parse(sql)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Successfully parsed CTE query\n")
	_ = ast
}
Output:
Successfully parsed CTE query
Example (ErrorHandling)

Example_errorHandling demonstrates proper error handling.

package main

import (
	"fmt"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := "SELECT * FROM" // Invalid: missing table name

	ast, err := gosqlx.Parse(sql)
	if err != nil {
		fmt.Println("Parse error occurred")
		// In real code: log detailed error message
		_ = err
		return
	}

	_ = ast
}
Output:
Parse error occurred
Example (ExtractColumns)

Example_extractColumns demonstrates extracting column names from a query.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := "SELECT u.name, u.email FROM users u WHERE u.active = true ORDER BY u.created_at"

	ast, err := gosqlx.Parse(sql)
	if err != nil {
		log.Fatal(err)
	}

	columns := gosqlx.ExtractColumns(ast)
	fmt.Printf("Found %d columns\n", len(columns))
}
Output:
Found 4 columns
Example (ExtractFromCTE)

Example_extractFromCTE demonstrates extracting from queries with CTEs.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := `WITH active_users AS (
		SELECT id, name FROM users WHERE active = true
	)
	SELECT name, COUNT(*) FROM active_users GROUP BY name`

	ast, err := gosqlx.Parse(sql)
	if err != nil {
		log.Fatal(err)
	}

	metadata := gosqlx.ExtractMetadata(ast)
	fmt.Printf("Tables found: %d\n", len(metadata.Tables))
}
Output:
Tables found: 2
Example (ExtractFunctions)

Example_extractFunctions demonstrates extracting function names from a query.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := "SELECT COUNT(*), AVG(salary), UPPER(name) FROM employees"

	ast, err := gosqlx.Parse(sql)
	if err != nil {
		log.Fatal(err)
	}

	functions := gosqlx.ExtractFunctions(ast)
	fmt.Printf("Found %d functions\n", len(functions))
}
Output:
Found 3 functions
Example (ExtractMetadata)

Example_extractMetadata demonstrates extracting comprehensive metadata from a query.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := `SELECT u.name, COUNT(o.id) as order_count
		FROM users u
		LEFT JOIN orders o ON u.id = o.user_id
		WHERE u.active = true
		GROUP BY u.name
		HAVING COUNT(o.id) > 5`

	ast, err := gosqlx.Parse(sql)
	if err != nil {
		log.Fatal(err)
	}

	metadata := gosqlx.ExtractMetadata(ast)
	fmt.Printf("Tables: %d, Columns: %d, Functions: %d\n",
		len(metadata.Tables), len(metadata.Columns), len(metadata.Functions))
}
Output:
Tables: 2, Columns: 4, Functions: 1
Example (ExtractTables)

Example_extractTables demonstrates extracting table names from a query.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := "SELECT * FROM users u JOIN orders o ON u.id = o.user_id"

	ast, err := gosqlx.Parse(sql)
	if err != nil {
		log.Fatal(err)
	}

	tables := gosqlx.ExtractTables(ast)
	fmt.Printf("Found %d tables\n", len(tables))
}
Output:
Found 2 tables
Example (ExtractTablesQualified)

Example_extractTablesQualified demonstrates extracting qualified table names.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := "SELECT * FROM users JOIN orders ON users.id = orders.user_id"

	ast, err := gosqlx.Parse(sql)
	if err != nil {
		log.Fatal(err)
	}

	tables := gosqlx.ExtractTablesQualified(ast)
	fmt.Printf("Found %d tables\n", len(tables))
}
Output:
Found 2 tables
Example (ExtractWindowFunctions)

Example_extractWindowFunctions demonstrates extracting window functions.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := `SELECT
		name,
		salary,
		ROW_NUMBER() OVER (PARTITION BY department ORDER BY salary DESC) as rank,
		AVG(salary) OVER (PARTITION BY department) as dept_avg
	FROM employees`

	ast, err := gosqlx.Parse(sql)
	if err != nil {
		log.Fatal(err)
	}

	functions := gosqlx.ExtractFunctions(ast)
	fmt.Printf("Window functions found: %d\n", len(functions))
}
Output:
Window functions found: 2
Example (Format)

Example_format demonstrates SQL formatting with options.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := "SELECT * FROM users WHERE active = true"

	// Use default formatting options
	opts := gosqlx.DefaultFormatOptions()
	opts.AddSemicolon = true

	formatted, err := gosqlx.Format(sql, opts)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Formatted SQL length: %d\n", len(formatted))
}
Output:
Formatted SQL length: 40
Example (FormatWithOptions)

Example_formatWithOptions demonstrates custom formatting options.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := "SELECT id, name FROM users"

	opts := gosqlx.FormatOptions{
		IndentSize:        4,
		UppercaseKeywords: true,
		AddSemicolon:      true,
		SingleLineLimit:   80,
	}

	formatted, err := gosqlx.Format(sql, opts)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Formatted with custom options: %d chars\n", len(formatted))
}
Output:
Formatted with custom options: 27 chars
Example (MigrationFromLowLevel)

Example_migrationFromLowLevel demonstrates migrating from low-level API.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	// Instead of manually managing tokenizer and parser:
	// tkz := tokenizer.GetTokenizer()
	// defer tokenizer.PutTokenizer(tkz)
	// tokens, err := tkz.Tokenize([]byte(sql))
	// ...
	// p := parser.NewParser()
	// defer p.Release()
	// ast, err := p.Parse(tokens)

	// Simply use:
	ast, err := gosqlx.Parse("SELECT * FROM users")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Migrated to simple API: %d statement(s)\n", len(ast.Statements))
}
Output:
Migrated to simple API: 1 statement(s)
Example (MustParse)

Example_mustParse demonstrates MustParse for SQL literals.

package main

import (
	"fmt"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	// Use MustParse only with SQL literals you control
	// (e.g., in tests or initialization code)
	ast := gosqlx.MustParse("SELECT 1")

	fmt.Printf("Type: %T\n", ast)
}
Output:
Type: *ast.AST
Example (ParseBytes)

Example_parseBytes demonstrates parsing from a byte slice.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	// Useful when SQL is already in byte form (e.g., from file I/O)
	sqlBytes := []byte("SELECT * FROM users")

	ast, err := gosqlx.ParseBytes(sqlBytes)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Parsed from bytes: %d statement(s)\n", len(ast.Statements))
}
Output:
Parsed from bytes: 1 statement(s)
Example (RealWorldUsage)

Example_realWorldUsage demonstrates a realistic use case.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	// Validate user input before executing
	userSQL := "SELECT * FROM users WHERE id = 1"

	// First validate
	if err := gosqlx.Validate(userSQL); err != nil {
		fmt.Println("Invalid SQL from user")
		return
	}

	// Parse to inspect structure
	ast, err := gosqlx.Parse(userSQL)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Valid query with %d statement(s)\n", len(ast.Statements))
}
Output:
Valid query with 1 statement(s)
Example (Simple)

Example_simple demonstrates the simplest way to parse SQL.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := "SELECT * FROM users"

	ast, err := gosqlx.Parse(sql)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Parsed %d statement(s)\n", len(ast.Statements))
}
Output:
Parsed 1 statement(s)
Example (Validate)

Example_validate demonstrates SQL validation.

package main

import (
	"fmt"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	// Valid SQL
	if err := gosqlx.Validate("SELECT * FROM users"); err != nil {
		fmt.Println("Invalid SQL")
	} else {
		fmt.Println("Valid SQL")
	}

	// Invalid SQL
	if err := gosqlx.Validate("INVALID SQL"); err != nil {
		fmt.Println("Invalid SQL detected")
	}

}
Output:
Valid SQL
Invalid SQL detected
Example (ValidateMultiple)

Example_validateMultiple demonstrates validating multiple SQL statements.

package main

import (
	"fmt"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	queries := []string{
		"SELECT * FROM users",
		"INSERT INTO users (name) VALUES ('test')",
		"UPDATE users SET active = true WHERE id = 1",
	}

	if err := gosqlx.ValidateMultiple(queries); err != nil {
		fmt.Printf("Validation failed: %v\n", err)
		return
	}

	fmt.Println("All queries are valid")
}
Output:
All queries are valid
Example (ValidationWithExtraction)

Example_validationWithExtraction demonstrates combining validation with metadata extraction.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	userSQL := "SELECT u.id, u.name FROM users u WHERE u.status = 'active'"

	// First validate
	if err := gosqlx.Validate(userSQL); err != nil {
		fmt.Println("Invalid SQL")
		return
	}

	// Parse and extract metadata
	ast, err := gosqlx.Parse(userSQL)
	if err != nil {
		log.Fatal(err)
	}

	metadata := gosqlx.ExtractMetadata(ast)

	// Check if accessing sensitive tables
	sensitiveTable := "users"
	for _, table := range metadata.Tables {
		if table == sensitiveTable {
			fmt.Printf("Query accesses sensitive table: %s\n", sensitiveTable)
		}
	}
}
Output:
Query accesses sensitive table: users
Example (WindowFunctions)

Example_windowFunctions demonstrates parsing SQL with window functions.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := "SELECT name, salary, ROW_NUMBER() OVER (ORDER BY salary DESC) as rank FROM employees"

	ast, err := gosqlx.Parse(sql)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Successfully parsed window function query\n")
	_ = ast
}
Output:
Successfully parsed window function query

Index

Examples

Constants

View Source
const Version = "1.14.0"

Version is the current GoSQLX library version.

Variables

View Source
var (
	// ErrSyntax indicates a parser-level syntax problem.
	ErrSyntax = errors.New("gosqlx: syntax error")
	// ErrTokenize indicates a tokenizer/lexer problem before parsing began.
	ErrTokenize = errors.New("gosqlx: tokenize error")
	// ErrTimeout indicates parsing was aborted because the context deadline
	// expired. Equivalent to errors.Is(err, context.DeadlineExceeded) for
	// context-originated timeouts; gosqlx wraps both under ErrTimeout so
	// callers can match a single sentinel.
	ErrTimeout = errors.New("gosqlx: parse timeout")
	// ErrUnsupportedDialect indicates the dialect supplied via WithDialect is
	// not recognized by the underlying keywords package.
	ErrUnsupportedDialect = errors.New("gosqlx: unsupported dialect")
	// ErrTooLarge is returned when input exceeds the configured maximum byte
	// size (see WithMaxBytes). Callers can test errors.Is(err, ErrTooLarge) to
	// distinguish cap-enforcement failures from read/parse errors.
	ErrTooLarge = errors.New("gosqlx: input too large")
)

Sentinel error values returned (wrapped) by the new ParseTree / ParseReader entry points. Callers can test specific failure classes with errors.Is:

tree, err := gosqlx.ParseTree(ctx, sql)
switch {
case errors.Is(err, gosqlx.ErrSyntax):
    // surface SQL syntax problem to the user
case errors.Is(err, gosqlx.ErrTokenize):
    // lexical/tokenization problem
case errors.Is(err, gosqlx.ErrTimeout):
    // context deadline exceeded
case errors.Is(err, gosqlx.ErrUnsupportedDialect):
    // caller passed an unknown WithDialect value
}

The underlying *errors.Error is still available via errors.As or via the ErrorCode / ErrorLocation / ErrorHint helpers in this package.

Functions

func ErrorCode added in v1.15.0

func ErrorCode(err error) errors.ErrorCode

ErrorCode extracts the structured error code from an error returned by Parse, Validate, or other gosqlx functions. It unwraps through fmt.Errorf wrapping to find the underlying *errors.Error.

Returns the ErrorCode (e.g., "E2001") if found, or empty string if the error is not a structured GoSQLX error.

Example:

_, err := gosqlx.Parse("SELECT * FORM users")
code := gosqlx.ErrorCode(err)
if code == errors.ErrCodeExpectedToken {
    // handle expected-token error
}

func ErrorHint added in v1.15.0

func ErrorHint(err error) string

ErrorHint extracts the hint/suggestion from an error returned by Parse, Validate, or other gosqlx functions.

Returns the hint string if found, or empty string if the error is not a structured GoSQLX error or has no hint.

Example:

_, err := gosqlx.Parse("SELECT * FORM users")
if hint := gosqlx.ErrorHint(err); hint != "" {
    fmt.Printf("Hint: %s\n", hint)
}

func ErrorLocation added in v1.15.0

func ErrorLocation(err error) *models.Location

ErrorLocation extracts the source location (line/column) from an error returned by Parse, Validate, or other gosqlx functions. It unwraps through fmt.Errorf wrapping to find the underlying *errors.Error.

Returns a pointer to the Location if found, or nil if the error is not a structured GoSQLX error.

Example:

_, err := gosqlx.Parse("SELECT * FORM users")
if loc := gosqlx.ErrorLocation(err); loc != nil {
    fmt.Printf("Error at line %d, column %d\n", loc.Line, loc.Column)
}

func ExtractColumns added in v1.6.0

func ExtractColumns(astNode *ast.AST) []string

ExtractColumns extracts all column references from an AST.

This function traverses the AST and collects column references from:

  • SELECT lists
  • WHERE conditions
  • GROUP BY clauses
  • ORDER BY clauses
  • JOIN conditions
  • HAVING clauses

Returns a deduplicated slice of column names (without table qualifiers).

Example:

sql := "SELECT u.name, u.email FROM users u WHERE u.active = true ORDER BY u.created_at"
ast, _ := gosqlx.Parse(sql)
columns := gosqlx.ExtractColumns(ast)
// columns = ["name", "email", "active", "created_at"]
Example

ExampleExtractColumns demonstrates extracting column names from a query.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := "SELECT u.name, u.email FROM users u WHERE u.active = true ORDER BY u.created_at"

	ast, err := gosqlx.Parse(sql)
	if err != nil {
		log.Fatal(err)
	}

	columns := gosqlx.ExtractColumns(ast)
	fmt.Printf("Found %d columns\n", len(columns))
}
Output:
Found 4 columns

func ExtractFunctions added in v1.6.0

func ExtractFunctions(astNode *ast.AST) []string

ExtractFunctions extracts all function calls from an AST.

This function traverses the AST and collects all function names, including:

  • Aggregate functions (COUNT, SUM, AVG, etc.)
  • Window functions (ROW_NUMBER, RANK, etc.)
  • Scalar functions (UPPER, LOWER, NOW, etc.)

Returns a deduplicated slice of function names.

Example:

sql := "SELECT COUNT(*), UPPER(name) FROM users"
ast, _ := gosqlx.Parse(sql)
functions := gosqlx.ExtractFunctions(ast)
// functions = ["COUNT", "UPPER"]
Example

ExampleExtractFunctions demonstrates extracting function names from a query.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := "SELECT COUNT(*), AVG(salary), UPPER(name) FROM employees"

	ast, err := gosqlx.Parse(sql)
	if err != nil {
		log.Fatal(err)
	}

	functions := gosqlx.ExtractFunctions(ast)
	fmt.Printf("Found %d functions\n", len(functions))
}
Output:
Found 3 functions

func ExtractTables added in v1.6.0

func ExtractTables(astNode *ast.AST) []string

ExtractTables extracts all table names from an AST.

This function traverses the AST and collects all table references from:

  • FROM clauses
  • JOIN clauses
  • Subqueries and CTEs
  • INSERT/UPDATE/DELETE statements

Returns a deduplicated slice of table names.

Example:

sql := "SELECT * FROM users u JOIN orders o ON u.id = o.user_id"
ast, _ := gosqlx.Parse(sql)
tables := gosqlx.ExtractTables(ast)
// tables = ["users", "orders"]
Example

ExampleExtractTables demonstrates extracting table names from a query.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := "SELECT * FROM users u JOIN orders o ON u.id = o.user_id"

	ast, err := gosqlx.Parse(sql)
	if err != nil {
		log.Fatal(err)
	}

	tables := gosqlx.ExtractTables(ast)
	fmt.Printf("Found %d tables\n", len(tables))
}
Output:
Found 2 tables

func Fingerprint added in v1.14.0

func Fingerprint(sql string) (string, error)

Fingerprint returns a stable 64-character SHA-256 hex digest for the given SQL. Structurally identical queries with different literal values produce the same fingerprint, making this suitable for query deduplication, caching, and slow-query grouping.

Example:

fp1, _ := gosqlx.Fingerprint("SELECT * FROM users WHERE id = 1")
fp2, _ := gosqlx.Fingerprint("SELECT * FROM users WHERE id = 999")
// fp1 == fp2

func Format

func Format(sql string, options FormatOptions) (string, error)

Format parses SQL into an AST and renders it back to text using the AST-based formatting engine. The result is syntactically valid, consistently styled SQL controlled by the provided FormatOptions.

Example:

sql := "select * from users where active=true"
opts := gosqlx.DefaultFormatOptions()
opts.UppercaseKeywords = true
formatted, err := gosqlx.Format(sql, opts)

Returns the formatted SQL string or an error if SQL is invalid.

Example

Example demonstrating SQL formatting

sql := "SELECT * FROM users WHERE active = true"
opts := DefaultFormatOptions()
opts.AddSemicolon = true

formatted, err := Format(sql, opts)
if err != nil {
	panic(err)
}

_ = formatted
Example

ExampleFormat demonstrates SQL formatting with options.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := "SELECT * FROM users WHERE active = true"

	opts := gosqlx.DefaultFormatOptions()
	opts.AddSemicolon = true

	formatted, err := gosqlx.Format(sql, opts)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Formatted SQL ends with semicolon: %v\n", formatted[len(formatted)-1] == ';')
}
Output:
Formatted SQL ends with semicolon: true

func FormatAST added in v1.15.0

func FormatAST(a *ast.AST, opts ...FormatOption) string

FormatAST renders a raw *ast.AST back to SQL text. This is the escape-hatch equivalent of FormatTree for callers that hold the underlying AST directly (e.g., from the low-level parser API). Internally it delegates to pkg/formatter.FormatAST — it does not re-parse.

func FormatTree added in v1.15.0

func FormatTree(t *Tree, opts ...FormatOption) string

FormatTree renders a Tree back to SQL text using the AST-based formatter. It does not re-tokenize or re-parse; if you already have a Tree, prefer this over the top-level Format(sql, opts) function.

func LoadSchema added in v1.14.0

func LoadSchema(db *sql.DB, loader dbschema.Loader, schemaName string) (*dbschema.DatabaseSchema, error)

LoadSchema connects to a live database and returns its schema metadata. Pass a dialect-specific loader from pkg/schema/postgres, pkg/schema/mysql, or pkg/schema/sqlite. schemaName may be empty to use the database default.

Example - SQLite:

import (
    _ "modernc.org/sqlite"
    "github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
    sqschema "github.com/ajitpratap0/GoSQLX/pkg/schema/sqlite"
)

db, _ := sql.Open("sqlite", ":memory:")
loader := sqschema.NewLoader()
s, err := gosqlx.LoadSchema(db, loader, "main")

Example - PostgreSQL:

import (
    _ "github.com/lib/pq"
    pgschema "github.com/ajitpratap0/GoSQLX/pkg/schema/postgres"
)

db, _ := sql.Open("postgres", "host=localhost user=app dbname=mydb sslmode=disable")
loader := pgschema.NewLoader()
s, err := gosqlx.LoadSchema(db, loader, "public")

func MustParse

func MustParse(sql string) *ast.AST

MustParse is like Parse but panics on error.

This is useful for parsing SQL literals at startup or in tests where parse errors indicate a programming bug.

Example:

// In test or init()
ast := gosqlx.MustParse("SELECT 1")
Example

ExampleMustParse demonstrates MustParse for SQL literals.

package main

import (
	"fmt"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	// Use MustParse only with SQL literals you control
	ast := gosqlx.MustParse("SELECT 1")

	fmt.Printf("Type: %T\n", ast)
}
Output:
Type: *ast.AST

func Normalize added in v1.14.0

func Normalize(sql string) (string, error)

Normalize parses sql, replaces all literal values (strings, numbers, booleans, NULLs) with "?" placeholders, and returns the re-formatted SQL.

Two queries that differ only in literal values (e.g., WHERE id = 1 vs WHERE id = 42) produce identical output. Existing parameter placeholders ($1, ?, :name) are preserved.

Returns an error if the SQL cannot be parsed.

Example:

norm, err := gosqlx.Normalize("SELECT * FROM users WHERE id = 42")
// norm == "SELECT * FROM users WHERE id = ?"

func Parse

func Parse(sql string) (*ast.AST, error)

Parse tokenizes and parses SQL in one call, returning an Abstract Syntax Tree (AST).

This function handles all object pool management internally, making it ideal for simple use cases. The parser supports comprehensive SQL features including:

SQL Standards (v1.6.0):

  • DML: SELECT, INSERT, UPDATE, DELETE with complex expressions
  • DDL: CREATE TABLE/VIEW/INDEX, ALTER TABLE, DROP statements
  • Window Functions: ROW_NUMBER, RANK, DENSE_RANK, NTILE, LAG, LEAD, etc.
  • CTEs: WITH clause including RECURSIVE support
  • Set Operations: UNION, EXCEPT, INTERSECT with proper precedence
  • JOIN Types: INNER, LEFT, RIGHT, FULL OUTER, CROSS, NATURAL
  • MERGE: WHEN MATCHED/NOT MATCHED clauses (SQL:2003)
  • Grouping: GROUPING SETS, ROLLUP, CUBE (SQL-99 T431)
  • FETCH: FETCH FIRST/NEXT with ROWS ONLY, WITH TIES, PERCENT
  • TRUNCATE: TRUNCATE TABLE with CASCADE/RESTRICT options
  • Materialized Views: CREATE/DROP/REFRESH MATERIALIZED VIEW

PostgreSQL Extensions (v1.6.0):

  • LATERAL JOIN: Correlated subqueries in FROM clause
  • JSON/JSONB Operators: ->, ->>, #>, #>>, @>, <@, ?, ?|, ?&, #-
  • DISTINCT ON: PostgreSQL-specific row selection
  • FILTER Clause: Conditional aggregation (SQL:2003 T612)
  • RETURNING Clause: Return modified rows from INSERT/UPDATE/DELETE
  • Aggregate ORDER BY: ORDER BY inside aggregate functions

Performance: This function achieves 1.38M+ operations/second sustained throughput with <1μs latency through intelligent object pooling.

Thread Safety: This function is thread-safe and can be called concurrently from multiple goroutines. Object pools are managed safely with sync.Pool.

Error Handling: Returns structured errors with error codes (E1xxx for tokenization, E2xxx for parsing, E3xxx for semantic errors). Errors include precise line/column information and helpful suggestions.

Example - Basic parsing:

sql := "SELECT * FROM users WHERE active = true"
ast, err := gosqlx.Parse(sql)
if err != nil {
    log.Fatal(err)
}
fmt.Printf("Parsed: %T\n", ast)

Example - PostgreSQL JSON operators:

sql := "SELECT data->>'name' FROM users WHERE data @> '{\"status\":\"active\"}'"
ast, err := gosqlx.Parse(sql)

Example - Window functions:

sql := `SELECT name, salary,
    RANK() OVER (PARTITION BY dept ORDER BY salary DESC) as rank
    FROM employees`
ast, err := gosqlx.Parse(sql)

Example - LATERAL JOIN:

sql := `SELECT u.name, o.order_date FROM users u,
    LATERAL (SELECT * FROM orders WHERE user_id = u.id LIMIT 3) o`
ast, err := gosqlx.Parse(sql)

For batch processing or performance-critical code, use the lower-level tokenizer and parser APIs directly to reuse objects across multiple queries.

See also: ParseWithContext, ParseWithTimeout, ParseMultiple for specialized use cases.

Example

ExampleParse demonstrates basic SQL parsing.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := "SELECT * FROM users WHERE active = true"
	ast, err := gosqlx.Parse(sql)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Parsed %d statement(s)\n", len(ast.Statements))
}
Output:
Parsed 1 statement(s)
Example

Example demonstrating the simple Parse API

sql := "SELECT * FROM users WHERE active = true"

ast, err := Parse(sql)
if err != nil {
	panic(err)
}

// Use the AST
_ = ast

func ParseBytes

func ParseBytes(sql []byte) (*ast.AST, error)

ParseBytes is like Parse but accepts a byte slice.

This avoids the string-to-byte conversion that Parse performs internally, making it more efficient when you already have SQL as bytes (e.g., from file I/O or network reads).

Example:

sqlBytes := []byte("SELECT * FROM users")
astNode, err := gosqlx.ParseBytes(sqlBytes)
Example

ExampleParseBytes demonstrates parsing from a byte slice.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sqlBytes := []byte("SELECT * FROM users")

	ast, err := gosqlx.ParseBytes(sqlBytes)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Parsed from bytes: %d statement(s)\n", len(ast.Statements))
}
Output:
Parsed from bytes: 1 statement(s)

func ParseMultiple

func ParseMultiple(queries []string) ([]*ast.AST, error)

ParseMultiple parses multiple SQL statements efficiently by reusing pooled objects.

This function is significantly more efficient than calling Parse() repeatedly because it obtains tokenizer and parser objects from the pool once and reuses them for all queries. This provides:

  • 30-40% performance improvement for batch operations
  • Reduced pool contention from fewer get/put operations
  • Lower memory allocation overhead
  • Better CPU cache locality

Thread Safety: This function is thread-safe. However, if processing queries concurrently, use Parse() in parallel goroutines instead for better throughput.

Performance: For N queries, this function has approximately O(N) performance with the overhead of object pool operations amortized across all queries. Benchmarks show:

  • 10 queries: ~40% faster than 10x Parse() calls
  • 100 queries: ~45% faster than 100x Parse() calls
  • 1000 queries: ~50% faster than 1000x Parse() calls

Error Handling: Returns an error for the first query that fails to parse. The error includes the query index (0-based) to identify which query failed. Already-parsed ASTs are not returned on error.

Memory Management: All pooled objects are properly returned to pools via defer, even if an error occurs during parsing.

Example - Batch parsing:

queries := []string{
    "SELECT * FROM users",
    "SELECT * FROM orders",
    "INSERT INTO logs (message) VALUES ('test')",
}
asts, err := gosqlx.ParseMultiple(queries)
if err != nil {
    log.Fatalf("Batch parsing failed: %v", err)
}
fmt.Printf("Parsed %d queries\n", len(asts))

Example - Processing migration scripts:

migrationSQL := []string{
    "CREATE TABLE users (id INT PRIMARY KEY, name VARCHAR(100))",
    "CREATE INDEX idx_users_name ON users(name)",
    "INSERT INTO users VALUES (1, 'admin')",
}
asts, err := gosqlx.ParseMultiple(migrationSQL)

Example - Analyzing query logs:

queryLog := loadQueryLog() // []string of SQL queries
asts, err := gosqlx.ParseMultiple(queryLog)
for i, ast := range asts {
    tables := gosqlx.ExtractTables(ast)
    fmt.Printf("Query %d uses tables: %v\n", i, tables)
}

For concurrent processing of independent queries, use Parse() in parallel:

var wg sync.WaitGroup
for _, sql := range queries {
    wg.Add(1)
    go func(s string) {
        defer wg.Done()
        ast, _ := gosqlx.Parse(s)
        // Process ast
    }(sql)
}
wg.Wait()

See also: ValidateMultiple for validation-only batch processing.

Example

ExampleParseMultiple demonstrates parsing multiple SQL statements efficiently.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	queries := []string{
		"SELECT * FROM users",
		"SELECT * FROM orders",
		"SELECT * FROM products",
	}

	asts, err := gosqlx.ParseMultiple(queries)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Parsed %d queries\n", len(asts))
}
Output:
Parsed 3 queries
Example

Example demonstrating batch parsing

queries := []string{
	"SELECT * FROM users",
	"SELECT * FROM orders",
}

asts, err := ParseMultiple(queries)
if err != nil {
	panic(err)
}

for i, ast := range asts {
	_ = i
	_ = ast
}

func ParseWithContext

func ParseWithContext(ctx context.Context, sql string) (*ast.AST, error)

ParseWithContext tokenizes and parses SQL with context support for cancellation and timeouts.

This function handles all object pool management internally and supports cancellation via the provided context. It's ideal for long-running operations, web servers, or any application that needs to gracefully handle timeouts and cancellation.

The function checks the context before starting and periodically during parsing to ensure responsive cancellation. This makes it suitable for user-facing applications where parsing needs to be interrupted if the user cancels the operation or the request timeout expires.

Thread Safety: This function is thread-safe and can be called concurrently from multiple goroutines. Each call operates on independent pooled objects.

Context Handling:

  • Returns context.Canceled if ctx.Done() is closed during parsing
  • Returns context.DeadlineExceeded if the context timeout expires
  • Checks context state before tokenization and parsing phases
  • Supports context.WithTimeout, context.WithDeadline, context.WithCancel

Performance: Same as Parse() - 1.38M+ ops/sec sustained with minimal context checking overhead (<1% performance impact).

Example - Basic timeout:

ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()

ast, err := gosqlx.ParseWithContext(ctx, sql)
if err == context.DeadlineExceeded {
    log.Println("Parsing timed out after 5 seconds")
}

Example - User cancellation:

ctx, cancel := context.WithCancel(context.Background())
defer cancel()

go func() {
    ast, err := gosqlx.ParseWithContext(ctx, complexSQL)
    if err == context.Canceled {
        log.Println("User cancelled parsing")
    }
}()

// User clicks cancel button
cancel()

Example - HTTP request timeout:

func handleParse(w http.ResponseWriter, r *http.Request) {
    ast, err := gosqlx.ParseWithContext(r.Context(), sql)
    if err == context.Canceled {
        http.Error(w, "Request cancelled", http.StatusRequestTimeout)
        return
    }
}

See also: ParseWithTimeout for a simpler timeout-only API.

Example

ExampleParseWithContext demonstrates parsing with context for cancellation.

package main

import (
	"context"
	"fmt"
	"log"
	"time"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
	defer cancel()

	sql := "SELECT * FROM users"
	ast, err := gosqlx.ParseWithContext(ctx, sql)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Parsed with context: %d statement(s)\n", len(ast.Statements))
}
Output:
Parsed with context: 1 statement(s)

func ParseWithDialect added in v1.9.3

func ParseWithDialect(sql string, dialect keywords.SQLDialect) (*ast.AST, error)

ParseWithDialect tokenizes and parses SQL using a specific SQL dialect for keyword recognition and dialect-aware parsing rules.

This is a top-level convenience wrapper around pkg/sql/parser.ParseWithDialect. It is equivalent to calling Parse but instructs the tokenizer and parser to apply dialect-specific rules (e.g., MySQL-specific syntax, PostgreSQL extensions).

Supported dialects:

  • keywords.DialectGeneric - generic SQL (default fallback)
  • keywords.DialectMySQL - MySQL / MariaDB
  • keywords.DialectPostgreSQL - PostgreSQL
  • keywords.DialectSQLite - SQLite
  • keywords.DialectSQLServer - Microsoft SQL Server (T-SQL)
  • keywords.DialectOracle - Oracle Database (PL/SQL)
  • keywords.DialectSnowflake - Snowflake SQL

Example - parse MySQL-specific syntax:

sql := "INSERT INTO t (id, name) VALUES (1, 'Alice') ON DUPLICATE KEY UPDATE name=VALUES(name)"
ast, err := gosqlx.ParseWithDialect(sql, keywords.DialectMySQL)
if err != nil {
    log.Fatal(err)
}

Example - parse PostgreSQL JSON operators:

sql := "SELECT data->>'name' FROM users"
ast, err := gosqlx.ParseWithDialect(sql, keywords.DialectPostgreSQL)

Returns an error if the dialect is unknown or if SQL is syntactically invalid. Errors are wrapped with gosqlx sentinel errors (ErrUnsupportedDialect, ErrSyntax, ErrTokenize) so callers can match via errors.Is.

func ParseWithRecovery added in v1.9.3

func ParseWithRecovery(sql string) ([]ast.Statement, []error)

ParseWithRecovery tokenizes and parses SQL with error recovery, returning partial AST statements and all collected errors. Unlike Parse, it does not stop at the first error - it synchronises and continues, collecting every error it can find. This is ideal for IDE / LSP use-cases where the user wants to see all diagnostics at once.

Thread Safety: safe for concurrent use; each call uses pooled objects.

func ParseWithTimeout

func ParseWithTimeout(sql string, timeout time.Duration) (*ast.AST, error)

ParseWithTimeout is a convenience function that parses SQL with a timeout.

This is a wrapper around ParseWithContext that creates a timeout context automatically. It's useful for quick timeout-based parsing without manual context management.

Example:

astNode, err := gosqlx.ParseWithTimeout(sql, 5*time.Second)
if err == context.DeadlineExceeded {
    log.Println("Parsing timed out after 5 seconds")
}
Example

ExampleParseWithTimeout demonstrates parsing with a timeout.

package main

import (
	"fmt"
	"log"
	"time"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := "SELECT * FROM users"
	ast, err := gosqlx.ParseWithTimeout(sql, 5*time.Second)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Parsed with timeout: %d statement(s)\n", len(ast.Statements))
}
Output:
Parsed with timeout: 1 statement(s)

func SplitStatements added in v1.15.0

func SplitStatements(sql, dialect string) []string

SplitStatements splits a SQL source string into top-level statements on unquoted semicolons. It is dialect-aware: depending on the supplied dialect it understands features that the conservative ANSI-compatible path cannot, notably PostgreSQL dollar-quoting, MySQL/ClickHouse backtick identifiers, SQL Server bracketed identifiers, PostgreSQL E-string backslash escapes, and PostgreSQL nested block comments.

Recognised dialect strings (case-insensitive) — unknown values fall back to the conservative ANSI profile:

"postgresql", "postgres", "pg" — dollar-quotes, E-strings, nested /*…*/
"mysql", "mariadb"            — backtick identifiers
"clickhouse"                  — backtick identifiers
"sqlserver", "mssql", "tsql"  — [bracketed identifiers]
""                            — ANSI only (single/double quotes, line/block comments)

The returned slice preserves the original byte ranges between semicolons (the terminating ';' itself is stripped). Empty or whitespace-only segments are NOT filtered out here; callers such as ParseReaderMultiple trim and skip them. This mirrors the long-standing behaviour of the splitter that this function replaces.

The splitter is intentionally a hand-written state machine rather than a full tokenizer: it only needs to know what it is inside of, not what the tokens mean. That keeps it fast, allocation-free on the hot path, and robust against partial or syntactically invalid SQL (the parser still gets a chance to report those problems downstream).

func Transpile added in v1.14.0

func Transpile(sql string, from, to keywords.SQLDialect) (string, error)

Transpile converts SQL from one dialect to another.

Supported dialect pairs:

  • MySQL → PostgreSQL (AUTO_INCREMENT→SERIAL, TINYINT(1)→BOOLEAN)
  • PostgreSQL → MySQL (SERIAL→AUTO_INCREMENT, ILIKE→LOWER() LIKE LOWER())
  • PostgreSQL → SQLite (SERIAL→INTEGER, array types→TEXT)

For unsupported dialect pairs the SQL is parsed and reformatted without any dialect-specific rewrites (passthrough with normalisation).

Example:

result, err := gosqlx.Transpile(
    "CREATE TABLE t (id INT AUTO_INCREMENT PRIMARY KEY)",
    keywords.DialectMySQL,
    keywords.DialectPostgreSQL,
)
// result: "CREATE TABLE t (id SERIAL PRIMARY KEY)"

func Validate

func Validate(sql string) error

Validate checks if the given SQL is syntactically valid.

This is a convenience function that only validates syntax without building the full AST, making it slightly faster than Parse().

Example:

if err := gosqlx.Validate("SELECT * FROM users"); err != nil {
    fmt.Printf("Invalid SQL: %v\n", err)
}

Returns nil if SQL is valid, or an error describing the problem.

Example

Example demonstrating SQL validation

if err := Validate("SELECT * FROM users"); err != nil {
	panic(err)
}
Example

ExampleValidate demonstrates SQL validation.

package main

import (
	"fmt"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := "SELECT * FROM users"
	if err := gosqlx.Validate(sql); err != nil {
		fmt.Printf("Invalid SQL: %v\n", err)
		return
	}

	fmt.Println("Valid SQL")
}
Output:
Valid SQL

func ValidateMultiple

func ValidateMultiple(queries []string) error

ValidateMultiple validates multiple SQL statements.

Returns nil if all statements are valid, or an error for the first invalid statement encountered.

Example:

queries := []string{
    "SELECT * FROM users",
    "INVALID SQL HERE",
}
if err := gosqlx.ValidateMultiple(queries); err != nil {
    fmt.Printf("Validation failed: %v\n", err)
}
Example

ExampleValidateMultiple demonstrates validating multiple SQL statements.

package main

import (
	"fmt"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	queries := []string{
		"SELECT * FROM users",
		"INSERT INTO users (name) VALUES ('test')",
		"UPDATE users SET active = true WHERE id = 1",
	}

	if err := gosqlx.ValidateMultiple(queries); err != nil {
		fmt.Printf("Validation failed: %v\n", err)
		return
	}

	fmt.Println("All queries valid")
}
Output:
All queries valid

func WalkBy added in v1.15.0

func WalkBy[T ast.Node](t *Tree, fn func(T) bool)

WalkBy traverses the tree in depth-first, pre-order fashion and invokes fn for every node whose concrete type is T. Children of the matched node are descended unless fn returns false.

WalkBy is a generic helper that removes the type-assertion boilerplate required when using Tree.Walk. It is implemented on top of ast.Inspect so it follows the same Node.Children() contract and descends into every reachable subtree (subqueries, CTEs, UNION arms, etc.).

Nodes whose concrete type does not match T are skipped but still descended into — returning false from fn only prunes children of a matched node.

Example — collect every table reference inside nested subqueries:

var names []string
gosqlx.WalkBy(tree, func(t *ast.TableReference) bool {
    names = append(names, t.Name)
    return true
})

Example — short-circuit on the first SELECT with a Where clause:

var found *ast.SelectStatement
gosqlx.WalkBy(tree, func(s *ast.SelectStatement) bool {
    if s.Where != nil {
        found = s
        return false // prune this subtree (siblings still visited)
    }
    return true
})

Types

type AnalysisResult added in v1.15.0

type AnalysisResult struct {
	// Suggestions contains optimization recommendations.
	Suggestions []AnalysisSuggestion

	// QueryComplexity is one of "simple", "moderate", or "complex".
	QueryComplexity string

	// Score is 0-100 where 100 means no issues found.
	Score int
}

AnalysisResult contains the output of a SQL optimization analysis.

func Analyze added in v1.15.0

func Analyze(sql string) (*AnalysisResult, error)

Analyze runs basic optimization analysis on the given SQL, checking for common anti-patterns such as SELECT *, missing WHERE clauses, and cartesian products.

For full optimization analysis with all 20 built-in rules (OPT-001 through OPT-020), use pkg/advisor.New().AnalyzeSQL() directly. This function provides a quick check for the most common issues without requiring an additional import.

Thread Safety: safe for concurrent use.

Example:

result, err := gosqlx.Analyze("SELECT * FROM users")
if err != nil {
    log.Fatal(err)
}
fmt.Printf("Complexity: %s\n", result.QueryComplexity)
for _, s := range result.Suggestions {
    fmt.Printf("[%s] %s\n", s.RuleID, s.Message)
}

type AnalysisSuggestion added in v1.15.0

type AnalysisSuggestion struct {
	RuleID   string // e.g., "OPT-001"
	Severity string // "info", "warning", or "error"
	Message  string // Short description
	Detail   string // Detailed explanation
}

AnalysisSuggestion represents a single optimization recommendation.

type FormatOption added in v1.15.0

type FormatOption func(*formatOptions)

FormatOption configures the AST-based formatter exposed by FormatTree and FormatAST. Options are applied in order; later options override earlier ones when they touch the same field.

See WithIndent, WithUppercaseKeywords.

func WithIndent added in v1.15.0

func WithIndent(size int) FormatOption

WithIndent sets the number of spaces per indent level. Pass 0 for compact output (no indentation, single line). Negative values are clamped to 0.

func WithUppercaseKeywords added in v1.15.0

func WithUppercaseKeywords(on bool) FormatOption

WithUppercaseKeywords controls whether SQL keywords are uppercased in the formatted output. When false, keyword case is preserved as emitted by the AST formatter's default (which itself preserves the original parsed case).

type FormatOptions

type FormatOptions struct {
	// IndentSize is the number of spaces to use for each indentation level.
	// Common values are 2 (compact) or 4 (readable).
	//
	// Default: 2 spaces
	// Recommended range: 2-4 spaces
	//
	// Example with IndentSize=2:
	//   SELECT
	//     column1,
	//     column2
	//   FROM table
	IndentSize int

	// UppercaseKeywords determines whether SQL keywords should be converted to uppercase.
	// When true, keywords like SELECT, FROM, WHERE become uppercase.
	// When false, keywords remain in their original case or lowercase.
	//
	// Default: false (preserve original case)
	//
	// Note: PostgreSQL convention typically uses lowercase keywords, while
	// Oracle and SQL Server often use uppercase. Choose based on your dialect.
	UppercaseKeywords bool

	// AddSemicolon ensures a trailing semicolon is added to SQL statements if missing.
	// This is useful for ensuring SQL statements are properly terminated.
	//
	// Default: false (preserve original)
	//
	// When true:  "SELECT * FROM users"  -> "SELECT * FROM users;"
	// When false: "SELECT * FROM users"  -> "SELECT * FROM users"
	AddSemicolon bool

	// SingleLineLimit is the maximum line length in characters.
	//
	// Deprecated: This field currently has no effect on formatting output.
	// Line-breaking support is planned for a future release. The value is
	// still accepted to avoid breaking existing callers that set it.
	SingleLineLimit int
}

FormatOptions controls SQL formatting behavior for the Format function.

This type provides configuration for SQL code formatting, including indentation, keyword casing, and line length limits. The formatting engine aims to produce readable, consistent SQL code following industry best practices.

Default values are optimized for readability and compatibility with most SQL style guides. Use DefaultFormatOptions() to get a pre-configured instance with sensible defaults.

Thread Safety: FormatOptions instances are safe to use concurrently as long as they are not modified after creation. The recommended pattern is to create FormatOptions once and reuse them for all formatting operations.

Example - Custom formatting options:

opts := gosqlx.FormatOptions{
    IndentSize:        4,              // 4 spaces per indent level
    UppercaseKeywords: true,           // SQL keywords in UPPERCASE
    AddSemicolon:      true,           // Ensure trailing semicolon
    SingleLineLimit:   100,            // Break lines at 100 characters
}
formatted, err := gosqlx.Format(sql, opts)

Example - PostgreSQL style:

opts := gosqlx.DefaultFormatOptions()
opts.IndentSize = 2
opts.UppercaseKeywords = false  // PostgreSQL convention: lowercase

Example - Enterprise style (UPPERCASE):

opts := gosqlx.DefaultFormatOptions()
opts.UppercaseKeywords = true
opts.AddSemicolon = true

func DefaultFormatOptions

func DefaultFormatOptions() FormatOptions

DefaultFormatOptions returns a FormatOptions value with sensible defaults.

The defaults are:

  • IndentSize: 2 spaces per indent level
  • UppercaseKeywords: false (preserve original case)
  • AddSemicolon: false (preserve original termination)
  • SingleLineLimit: 80 characters

Use the returned value as a starting point and override individual fields to match your project's SQL style guide:

opts := gosqlx.DefaultFormatOptions()
opts.UppercaseKeywords = true  // enforce UPPERCASE keywords
opts.AddSemicolon = true       // always terminate with ;
formatted, err := gosqlx.Format(sql, opts)

type LintResult added in v1.15.0

type LintResult struct {
	Violations []linter.Violation
}

LintResult represents the result of linting a SQL string.

func Lint added in v1.15.0

func Lint(sql string) (*LintResult, error)

Lint checks SQL for style, safety, and performance issues using all 30 built-in lint rules (L001-L030). Returns violations found and any error encountered.

This is a convenience wrapper around the pkg/linter package that creates a linter with all rules enabled using sensible defaults. For fine-grained control over which rules to enable or their configuration, use pkg/linter directly.

Thread Safety: safe for concurrent use.

Example:

result, err := gosqlx.Lint("SELECT * FROM users")
if err != nil {
    log.Fatal(err)
}
for _, v := range result.Violations {
    fmt.Printf("[%s] %s: %s\n", v.Rule, v.RuleName, v.Message)
}

type Metadata added in v1.6.0

type Metadata struct {
	// Tables contains simple (unqualified) table names extracted from the query.
	// Example: ["users", "orders", "products"]
	//
	// This includes tables from:
	//   - FROM clauses
	//   - JOIN clauses
	//   - INSERT/UPDATE/DELETE statements
	//   - Subqueries and CTEs
	Tables []string

	// TablesQualified contains fully qualified table names with schema information.
	// Example: [QualifiedName{Schema: "public", Name: "users"}]
	//
	// Use this when you need to preserve schema qualifiers from the original query.
	// For queries without schema qualifiers, Schema field will be empty.
	TablesQualified []QualifiedName

	// Columns contains simple (unqualified) column names extracted from the query.
	// Example: ["name", "email", "created_at"]
	//
	// This includes columns from:
	//   - SELECT lists
	//   - WHERE conditions
	//   - GROUP BY clauses
	//   - ORDER BY clauses
	//   - JOIN conditions
	//   - HAVING clauses
	Columns []string

	// ColumnsQualified contains qualified column names with table/alias information.
	// Example: [QualifiedName{Table: "u", Name: "name"}]
	//
	// Use this when you need to preserve table qualifiers (e.g., "u.name" vs "name").
	// For unqualified columns, Table field will be empty.
	ColumnsQualified []QualifiedName

	// Functions contains all function names used in the query.
	// Example: ["COUNT", "SUM", "UPPER", "NOW"]
	//
	// This includes:
	//   - Aggregate functions (COUNT, SUM, AVG, MIN, MAX)
	//   - Window functions (ROW_NUMBER, RANK, DENSE_RANK, LAG, LEAD)
	//   - Scalar functions (UPPER, LOWER, SUBSTRING, COALESCE)
	//   - Date/time functions (NOW, CURRENT_TIMESTAMP, DATE_TRUNC)
	//   - JSON functions (JSON_EXTRACT, JSONB_BUILD_OBJECT)
	//   - PostgreSQL aggregate functions with FILTER clause (v1.6.0)
	Functions []string
}

Metadata contains comprehensive metadata extracted from a SQL query's AST.

This type aggregates all extractable metadata from a SQL query, including tables, columns, and function calls. It provides both simple (unqualified) and qualified versions of identifiers for maximum flexibility in query analysis.

All slices in Metadata are deduplicated - each identifier appears only once regardless of how many times it appears in the original query.

Thread Safety: Metadata instances are safe to read concurrently but should not be modified after creation.

Example - Analyzing query complexity:

metadata := gosqlx.ExtractMetadata(ast)
complexity := len(metadata.Tables) * len(metadata.Columns) * len(metadata.Functions)
fmt.Printf("Query complexity score: %d\n", complexity)

Example - Validating query against schema:

metadata := gosqlx.ExtractMetadata(ast)
for _, table := range metadata.Tables {
    if !schema.TableExists(table) {
        return fmt.Errorf("table %s does not exist", table)
    }
}

Example - Query impact analysis:

metadata := gosqlx.ExtractMetadata(ast)
fmt.Printf("Query Impact Analysis:\n")
fmt.Printf("  Tables accessed: %d (%v)\n", len(metadata.Tables), metadata.Tables)
fmt.Printf("  Columns referenced: %d (%v)\n", len(metadata.Columns), metadata.Columns)
fmt.Printf("  Functions used: %d (%v)\n", len(metadata.Functions), metadata.Functions)

func ExtractMetadata added in v1.6.0

func ExtractMetadata(astNode *ast.AST) *Metadata

ExtractMetadata extracts comprehensive metadata from an AST in a single call.

This is a convenience function that calls all extraction functions (ExtractTables, ExtractTablesQualified, ExtractColumns, ExtractColumnsQualified, ExtractFunctions) and returns the results in a structured Metadata object.

Performance: This function performs multiple AST traversals (one per extraction type). For better performance when you only need specific metadata, call the individual extraction functions directly instead of using ExtractMetadata.

Thread Safety: This function is thread-safe and can be called concurrently on different AST instances.

Use Cases:

  • Query analysis: Understanding what resources a query uses
  • Security scanning: Identifying accessed tables and columns
  • Query optimization: Analyzing function usage and access patterns
  • Documentation: Generating query metadata for documentation
  • Testing: Validating query structure in tests

Example - Basic metadata extraction:

sql := "SELECT COUNT(*), u.name FROM users u WHERE u.active = true"
ast, _ := gosqlx.Parse(sql)
metadata := gosqlx.ExtractMetadata(ast)
fmt.Printf("Tables: %v, Columns: %v, Functions: %v\n",
    metadata.Tables, metadata.Columns, metadata.Functions)
// Output: Tables: [users], Columns: [name active], Functions: [COUNT]

Example - Query dependency analysis:

sql := `SELECT u.name, COUNT(o.id) as order_count
    FROM users u
    LEFT JOIN orders o ON u.id = o.user_id
    GROUP BY u.name`
ast, _ := gosqlx.Parse(sql)
metadata := gosqlx.ExtractMetadata(ast)
fmt.Printf("Query depends on tables: %v\n", metadata.Tables)
// Output: Query depends on tables: [users orders]

Example - Security analysis:

sql := "SELECT password, ssn FROM users WHERE admin = true"
ast, _ := gosqlx.Parse(sql)
metadata := gosqlx.ExtractMetadata(ast)

sensitiveColumns := []string{"password", "ssn", "credit_card"}
for _, col := range metadata.Columns {
    for _, sensitive := range sensitiveColumns {
        if col == sensitive {
            fmt.Printf("WARNING: Query accesses sensitive column: %s\n", col)
        }
    }
}

Example - PostgreSQL v1.6.0 features:

sql := `SELECT data->>'name' as name,
    COUNT(*) FILTER (WHERE status = 'active')
    FROM users u
    LATERAL JOIN orders o ON o.user_id = u.id`
ast, _ := gosqlx.Parse(sql)
metadata := gosqlx.ExtractMetadata(ast)
// Captures JSON operators, FILTER clause, LATERAL joins

See also: Individual extraction functions for targeted metadata retrieval.

Example

ExampleExtractMetadata demonstrates extracting comprehensive metadata from a query.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := `SELECT u.name, COUNT(o.id) as order_count
		FROM users u
		LEFT JOIN orders o ON u.id = o.user_id
		WHERE u.active = true
		GROUP BY u.name
		HAVING COUNT(o.id) > 5`

	ast, err := gosqlx.Parse(sql)
	if err != nil {
		log.Fatal(err)
	}

	metadata := gosqlx.ExtractMetadata(ast)
	fmt.Printf("Tables: %d, Columns: %d, Functions: %d\n",
		len(metadata.Tables), len(metadata.Columns), len(metadata.Functions))
}
Output:
Tables: 2, Columns: 4, Functions: 1

func (*Metadata) String added in v1.6.0

func (m *Metadata) String() string

String returns a human-readable representation of the metadata.

type Option added in v1.15.0

type Option func(*parseOptions)

Option configures parse behavior for ParseTree, ParseReader, and future parsing entry points that follow the functional-options pattern.

Options are applied in the order they are passed. Later options override earlier ones when they touch the same field. Passing zero options selects sensible defaults (generic dialect, no timeout, non-strict parsing).

Options are the recommended configuration surface for new code. The older ParseWithContext / ParseWithDialect / ParseWithTimeout / ParseWithRecovery entry points remain fully supported for backward compatibility but are combinatorial; Option avoids that combinatorial explosion.

Example:

tree, err := gosqlx.ParseTree(
    ctx, sql,
    gosqlx.WithDialect("postgresql"),
    gosqlx.WithTimeout(3*time.Second),
)

func WithDialect added in v1.15.0

func WithDialect(dialect string) Option

WithDialect selects the SQL dialect for keyword recognition and dialect-specific grammar rules.

Supported values include (see pkg/sql/keywords for the canonical list):

  • "generic" — generic SQL (default when empty)
  • "mysql" — MySQL
  • "mariadb" — MariaDB
  • "postgresql" — PostgreSQL
  • "sqlite" — SQLite
  • "sqlserver" — Microsoft SQL Server (T-SQL)
  • "oracle" — Oracle (PL/SQL)
  • "snowflake" — Snowflake
  • "clickhouse" — ClickHouse

Unknown dialect strings are passed through to the parser, which returns an ErrUnsupportedDialect-wrapped error if it cannot resolve the name.

Example:

tree, err := gosqlx.ParseTree(ctx, sql, gosqlx.WithDialect("mysql"))

func WithMaxBytes added in v1.15.0

func WithMaxBytes(n int64) Option

WithMaxBytes caps the number of bytes ParseReader and ParseReaderMultiple will read from an io.Reader. A zero or negative value disables the cap and preserves the original unbounded behaviour (the default).

When the cap is exceeded the reader entry points abort before any parsing work is attempted and return an error that satisfies errors.Is(err, ErrTooLarge). The reader is drained only up to maxBytes+1 bytes, so hostile 100 MB inputs will not cause a ~2x allocation spike.

Example — reject inputs larger than 1 MiB:

tree, err := gosqlx.ParseReader(ctx, r, gosqlx.WithMaxBytes(1<<20))
if errors.Is(err, gosqlx.ErrTooLarge) {
    // surface a 413-style error to the caller
}

func WithRecovery added in v1.15.0

func WithRecovery() Option

WithRecovery enables error-recovery parsing. When set, the parser synchronizes after errors and continues, returning partial statements and the full list of diagnostics rather than failing at the first error.

Consumers that need diagnostics (IDEs, LSP servers, linters) should prefer this mode. When WithRecovery is set, ParseTree returns a *Tree whose Statements may include nil entries for unparseable segments; inspect the returned error via errors.Is(err, ErrSyntax) / ErrTokenize to identify the kinds of problems collected.

func WithStrict added in v1.15.0

func WithStrict() Option

WithStrict enables strict parsing mode. In strict mode the parser rejects constructs it would otherwise silently tolerate (for example, lone semicolons producing empty statements).

Default is lenient (non-strict) for backward compatibility.

func WithTimeout added in v1.15.0

func WithTimeout(d time.Duration) Option

WithTimeout applies a parse-time deadline. If the caller already passes a context with an earlier deadline, the caller's deadline wins; this option only tightens contexts that have no deadline of their own.

A non-positive duration disables the timeout.

Example:

tree, err := gosqlx.ParseTree(ctx, sql, gosqlx.WithTimeout(2*time.Second))

type QualifiedName added in v1.6.0

type QualifiedName struct {
	// Schema is the optional schema or database name (first qualifier).
	// Examples: "public", "mydb", "information_schema"
	Schema string

	// Table is the table name or middle qualifier.
	// For tables: may be the schema when Schema and Name are both set
	// For columns: typically the table name or alias
	Table string

	// Name is the primary identifier (final qualifier).
	// For tables: the table name
	// For columns: the column name
	Name string
}

QualifiedName represents a fully qualified table or column name with optional schema.

This type supports various levels of qualification commonly found in SQL queries:

  • Single-part: "users" (just Name)
  • Two-part: "public.users" (Schema.Name) or "u.name" (Table.Name)
  • Three-part: "db.public.users" (Schema.Table.Name)

The interpretation of fields depends on context:

  • For tables: Schema typically represents database/schema, Name represents table
  • For columns: Table represents table/alias, Name represents column
  • For three-part names: Schema.Table.Name covers all levels

Thread Safety: QualifiedName is a simple struct and safe to use concurrently. The String() and FullName() methods are read-only and safe for concurrent calls.

Example - Table qualification:

// Simple table
QualifiedName{Name: "users"}  // "users"

// Schema-qualified table
QualifiedName{Schema: "public", Name: "users"}  // "public.users"

// Database-schema-table (PostgreSQL)
QualifiedName{Schema: "mydb", Table: "public", Name: "users"}  // "mydb.public.users"

Example - Column qualification:

// Simple column
QualifiedName{Name: "email"}  // "email"

// Table-qualified column
QualifiedName{Table: "u", Name: "email"}  // "u.email"

// Fully qualified column
QualifiedName{Schema: "public", Table: "users", Name: "email"}  // "public.users.email"

Use String() to get the full qualified name, or FullName() to get the name without the schema component (useful for working with qualified identifiers in a single database context).

func ExtractColumnsQualified added in v1.6.0

func ExtractColumnsQualified(astNode *ast.AST) []QualifiedName

ExtractColumnsQualified extracts all column references with their table qualifiers.

This function is similar to ExtractColumns but preserves table qualifier information when present in the original query. It collects column references from:

  • SELECT lists
  • WHERE conditions
  • GROUP BY clauses
  • ORDER BY clauses
  • JOIN conditions
  • HAVING clauses

Returns a deduplicated slice of QualifiedName objects representing columns.

Example:

sql := "SELECT u.name, u.email FROM users u WHERE u.active = true"
ast, _ := gosqlx.Parse(sql)
columns := gosqlx.ExtractColumnsQualified(ast)
// columns contains QualifiedName{Table: "u", Name: "name"},
// QualifiedName{Table: "u", Name: "email"}, QualifiedName{Table: "u", Name: "active"}
Example

ExampleExtractColumnsQualified demonstrates extracting qualified column names.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := "SELECT u.name, u.email FROM users u WHERE u.active = true"

	ast, err := gosqlx.Parse(sql)
	if err != nil {
		log.Fatal(err)
	}

	columns := gosqlx.ExtractColumnsQualified(ast)
	fmt.Printf("Found %d qualified columns\n", len(columns))
}
Output:
Found 3 qualified columns

func ExtractTablesQualified added in v1.6.0

func ExtractTablesQualified(astNode *ast.AST) []QualifiedName

ExtractTablesQualified extracts all table names with their qualifiers (schema.table).

This function is similar to ExtractTables but preserves schema information when present in the original query.

Returns a deduplicated slice of QualifiedName objects.

Example:

sql := "SELECT * FROM public.users JOIN app.orders ON users.id = orders.user_id"
ast, _ := gosqlx.Parse(sql)
tables := gosqlx.ExtractTablesQualified(ast)
// tables contains QualifiedName{Schema: "public", Name: "users"} and
// QualifiedName{Schema: "app", Name: "orders"}
Example

ExampleExtractTablesQualified demonstrates extracting qualified table names.

package main

import (
	"fmt"
	"log"

	"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
)

func main() {
	sql := "SELECT * FROM users JOIN orders ON users.id = orders.user_id"

	ast, err := gosqlx.Parse(sql)
	if err != nil {
		log.Fatal(err)
	}

	tables := gosqlx.ExtractTablesQualified(ast)
	fmt.Printf("Found %d tables\n", len(tables))
}
Output:
Found 2 tables

func (QualifiedName) FullName added in v1.6.0

func (q QualifiedName) FullName() string

FullName returns the full name without schema qualifier. This method strips the schema component and returns the meaningful identifier.

Behavior:

  • For 3-part names (schema.table.column): Returns table.column (drops schema)
  • For 2-part names (table.column OR schema.table): Returns table.column
  • For single-part names: Returns the name

Examples:

  • QualifiedName{Schema: "db", Table: "public", Name: "users"} → "public.users"
  • QualifiedName{Table: "users", Name: "id"} → "users.id"
  • QualifiedName{Name: "id"} → "id"
  • QualifiedName{Schema: "public", Name: "users"} → "users"
  • QualifiedName{Table: "users"} → "users"

func (QualifiedName) String added in v1.6.0

func (q QualifiedName) String() string

String returns the qualified name as a string.

type Tree added in v1.15.0

type Tree struct {
	// contains filtered or unexported fields
}

Tree is an opaque handle to a parsed SQL AST. It is the recommended entry point for new code: by going through Tree instead of the raw *ast.AST, callers gain access to high-level helpers (Walk, Tables, Columns, Functions, Format) without having to import the internal AST package, and the library retains the freedom to evolve AST internals without breaking consumers.

A *ast.AST escape hatch is still available via Raw() for power users who need node-level access.

Tree values are safe to use from a single goroutine. Share across goroutines only for read-only inspection (Walk / Tables / Format). Concurrent mutation through Raw() is not supported.

func ParseReader added in v1.15.0

func ParseReader(ctx context.Context, r io.Reader, opts ...Option) (*Tree, error)

ParseReader reads SQL from r and parses it, returning an opaque Tree.

This is a convenience wrapper for callers who already have an io.Reader (HTTP request body, file handle, strings.Reader, etc.) and don't want to manage the buffering themselves. Input is consumed in full before parsing begins; by default the read is unbounded, but callers that handle hostile or user-supplied input should pass WithMaxBytes to cap allocation.

If ctx is nil, context.Background is used. Options are forwarded to ParseTree unchanged; see ParseTree for the context/dialect/timeout semantics.

Read errors are surfaced verbatim (not wrapped in one of the gosqlx sentinels) because they originate outside the SQL layer. Parse errors follow the normal ParseTree wrapping (ErrSyntax / ErrTokenize / ErrTimeout / ErrUnsupportedDialect). Inputs that exceed WithMaxBytes return an error that satisfies errors.Is(err, ErrTooLarge).

Example:

f, _ := os.Open("query.sql")
defer f.Close()
tree, err := gosqlx.ParseReader(ctx, f,
    gosqlx.WithDialect("postgresql"),
    gosqlx.WithMaxBytes(1<<20), // reject >1 MiB
)
if err != nil {
    return err
}

Cancellation: the reader is wrapped so that each Read call short-circuits with the context error if ctx is cancelled. This does not interrupt a Read that has already entered a syscall — callers dealing with pathological network readers should still enforce deadlines at the transport layer.

func ParseReaderMultiple added in v1.15.0

func ParseReaderMultiple(ctx context.Context, r io.Reader, opts ...Option) ([]*Tree, error)

ParseReaderMultiple reads SQL from r, splits it into individual statements on unquoted top-level semicolons, and parses each, returning one Tree per statement.

The splitter is dialect-aware: pass WithDialect("postgresql") to opt into dollar-quoting and E-string handling, WithDialect("mysql") for backtick identifiers, WithDialect("sqlserver") for [bracketed identifiers], and so on. When no dialect is set the conservative ANSI rules apply (single and double quotes, line comments, and non-nested block comments).

ParseReaderMultiple honours WithMaxBytes the same way ParseReader does: inputs larger than the cap are rejected with ErrTooLarge before any splitting or parsing work begins.

Empty segments (whitespace between consecutive semicolons, trailing whitespace after the last ';', etc.) are skipped. The first segment that fails to parse short-circuits the call and returns its error wrapped in the usual ParseTree sentinels, prefixed with the 1-based statement index.

Example:

tree, err := gosqlx.ParseReaderMultiple(ctx,
    strings.NewReader(script),
    gosqlx.WithDialect("postgresql"),
    gosqlx.WithMaxBytes(4<<20),
)

func ParseTree added in v1.15.0

func ParseTree(ctx context.Context, sql string, opts ...Option) (*Tree, error)

ParseTree parses SQL and returns an opaque Tree, the recommended entry point for new code. Configuration is supplied via functional options (WithDialect, WithStrict, WithTimeout, WithRecovery) rather than through the combinatorial ParseWithX helpers on the top-level API surface.

Context handling:

  • If ctx is nil, context.Background is used.
  • WithTimeout(d) installs a deadline when ctx has none; an existing earlier deadline on ctx wins.

Error handling: returned errors are wrapped with sentinel values — ErrTokenize, ErrSyntax, ErrTimeout, ErrUnsupportedDialect — so callers can match with errors.Is. The underlying structured *errors.Error remains reachable via errors.As or the ErrorCode / ErrorLocation / ErrorHint helpers in this package.

Example:

ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()
tree, err := gosqlx.ParseTree(ctx, sql,
    gosqlx.WithDialect("postgresql"),
)
if err != nil {
    return err
}
for _, tbl := range tree.Tables() {
    ...
}

func (*Tree) Clone added in v1.15.0

func (t *Tree) Clone() *Tree

Clone returns an independent deep copy of the tree. Mutations to the original (via Raw(), WalkSelects field writes, Rewrite, ...) do not affect the clone, and vice versa.

IMPLEMENTATION — Clone is implemented by re-parsing t.SQL() with default options. This is simple and provably correct: the clone has exactly the structure the parser would produce today for the original source, which is the strongest possible guarantee of independence. The tradeoff is cost — Clone is O(parse) rather than O(nodes), and a clone of a tree that was produced with a non-default dialect / recovery mode will not preserve those parse options. For cases where either matters, hold onto the SQL string and call ParseTree yourself with the desired options.

Clone returns nil if the receiver is nil, the original SQL is empty, or the re-parse fails (which would indicate a parser regression since the original SQL was known to parse successfully when t was constructed).

func (*Tree) Columns added in v1.15.0

func (t *Tree) Columns() []string

Columns returns the deduplicated list of column names referenced anywhere in the tree. Delegates to ExtractColumns.

func (*Tree) Format added in v1.15.0

func (t *Tree) Format(opts ...FormatOption) string

Format renders the Tree back to SQL text using the AST-based formatter. Unlike the top-level Format(sql, opts) function, this method does not re-tokenize or re-parse — it walks the already-parsed AST, so it is both faster and guaranteed to match the parsed structure.

Pass zero or more FormatOption values to customize indent width and keyword casing; see WithIndent and WithUppercaseKeywords.

func (*Tree) Functions added in v1.15.0

func (t *Tree) Functions() []string

Functions returns the deduplicated list of function names called anywhere in the tree. Delegates to ExtractFunctions.

func (*Tree) Raw added in v1.15.0

func (t *Tree) Raw() *ast.AST

Raw returns the underlying *ast.AST. It is an escape hatch for callers that need direct node access; prefer the Tree methods (Walk, Tables, Columns, Functions, Format) for forward compatibility.

The returned AST is owned by the Tree — do not call ast.ReleaseAST on it directly. Use Tree.Release instead.

func (*Tree) Release added in v1.15.0

func (t *Tree) Release()

Release returns any pooled resources associated with the tree. It is currently a best-effort no-op: Tree does not own pooled resources because the underlying *ast.AST lifetime is caller-driven (power users can still retrieve Raw()). Calling Release makes your code forward-compatible with future versions that may adopt explicit Tree-level pooling.

func (*Tree) Rewrite added in v1.15.0

func (t *Tree) Rewrite(pre, post func(ast.Statement) ast.Statement)

Rewrite applies pre and post transformation passes to each top-level Statement in the tree. pre runs before any children are considered; post runs after. Either may be nil to skip that pass. The return value of each function replaces the statement in the tree; returning the same node is the no-op case.

SCOPE — Rewrite operates at Statement granularity only. Deeper rewrites (e.g., replacing an expression inside a WHERE clause) require walking the AST via Raw() and mutating the concrete struct fields directly. This is a deliberate design choice: the AST contains ~100 concrete node types with heterogeneous child-field layouts; a generic deep-rewrite API would require either reflection (slow, easy to misuse) or an exhaustive per-type switch (maintenance burden). Until there is a clear user need we prefer the honest narrow API over a permissive one that silently misses cases.

Example — drop every DeleteStatement from a batch:

tree.Rewrite(nil, func(s ast.Statement) ast.Statement {
    if _, ok := s.(*ast.DeleteStatement); ok {
        return nil // filtered out
    }
    return s
})

A nil return from pre or post drops the statement from the tree. Rewrite mutates t in place; call Clone() first to preserve the original.

For intra-statement rewrites, combine Tree.WalkSelects / WalkBinaryExpressions / etc. with direct field assignment on the visited node. Because the walkers return pointers, any field you assign to is visible to subsequent reads through the same Tree.

func (*Tree) SQL added in v1.15.0

func (t *Tree) SQL() string

SQL returns the original SQL source that produced this Tree. This is the unmodified caller input, useful for error context and debugging.

func (*Tree) Statements added in v1.15.0

func (t *Tree) Statements() []ast.Statement

Statements returns the top-level AST statements. This is a power-user escape hatch used when callers want to switch on concrete statement types. For generic traversal prefer Walk.

The returned slice aliases the underlying AST storage — do not mutate it.

func (*Tree) Tables added in v1.15.0

func (t *Tree) Tables() []string

Tables returns the deduplicated list of table names referenced anywhere in the tree, including inside subqueries and CTEs. Delegates to ExtractTables for consistency with the existing top-level helper.

func (*Tree) Walk added in v1.15.0

func (t *Tree) Walk(fn func(ast.Node) bool)

Walk traverses every node in the tree in depth-first, pre-order fashion. The visitor function is invoked for each node. Return true to descend into children, false to skip the current subtree.

Walk correctly descends into nested SELECTs, CTEs, subqueries, UNION arms, and every other position where an ast.Node appears as a child, because it delegates to ast.Inspect which follows the Children() contract.

Example — collect every identifier, including those inside subqueries:

var idents []string
tree.Walk(func(n ast.Node) bool {
    if id, ok := n.(*ast.Identifier); ok {
        idents = append(idents, id.Name)
    }
    return true
})

func (*Tree) WalkBinaryExpressions added in v1.15.0

func (t *Tree) WalkBinaryExpressions(fn func(*ast.BinaryExpression) bool)

WalkBinaryExpressions invokes fn for every *ast.BinaryExpression in the tree. Useful for linting operators (=, <, LIKE, ->>, etc.) or rewriting comparison predicates.

func (*Tree) WalkCTEs added in v1.15.0

func (t *Tree) WalkCTEs(fn func(*ast.CommonTableExpr) bool)

WalkCTEs invokes fn for every *ast.CommonTableExpr in the tree, descending into nested WITH clauses inside subqueries.

func (*Tree) WalkCreateTables added in v1.15.0

func (t *Tree) WalkCreateTables(fn func(*ast.CreateTableStatement) bool)

WalkCreateTables invokes fn for every *ast.CreateTableStatement in the tree.

func (*Tree) WalkDeletes added in v1.15.0

func (t *Tree) WalkDeletes(fn func(*ast.DeleteStatement) bool)

WalkDeletes invokes fn for every *ast.DeleteStatement in the tree.

func (*Tree) WalkFunctionCalls added in v1.15.0

func (t *Tree) WalkFunctionCalls(fn func(*ast.FunctionCall) bool)

WalkFunctionCalls invokes fn for every *ast.FunctionCall in the tree, including window functions, aggregate functions, and scalar functions.

func (*Tree) WalkIdentifiers added in v1.15.0

func (t *Tree) WalkIdentifiers(fn func(*ast.Identifier) bool)

WalkIdentifiers invokes fn for every *ast.Identifier in the tree. Useful for collecting column references, table aliases, or any bare name that the parser lowered to an Identifier node.

func (*Tree) WalkInserts added in v1.15.0

func (t *Tree) WalkInserts(fn func(*ast.InsertStatement) bool)

WalkInserts invokes fn for every *ast.InsertStatement in the tree.

func (*Tree) WalkJoins added in v1.15.0

func (t *Tree) WalkJoins(fn func(*ast.JoinClause) bool)

WalkJoins invokes fn for every *ast.JoinClause in the tree. Because JoinClause is stored by value on SelectStatement.Joins, its Children() method exposes pointer access so WalkBy can locate each join node during traversal. If your parser version stores joins as values and they are not reachable via Children(), use Tree.WalkSelects and iterate s.Joins directly.

func (*Tree) WalkSelects added in v1.15.0

func (t *Tree) WalkSelects(fn func(*ast.SelectStatement) bool)

WalkSelects invokes fn for every *ast.SelectStatement in the tree, including those nested inside subqueries, CTEs, and set-operation arms. Return false from fn to skip descent into the matched SELECT's children (siblings are still visited).

func (*Tree) WalkUpdates added in v1.15.0

func (t *Tree) WalkUpdates(fn func(*ast.UpdateStatement) bool)

WalkUpdates invokes fn for every *ast.UpdateStatement in the tree.

Directories

Path Synopsis
Package testing provides comprehensive SQL parsing test helpers for use in Go test suites.
Package testing provides comprehensive SQL parsing test helpers for use in Go test suites.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL