gosqlx

GoSQLX

⚡ High-Performance SQL Parser for Go ⚡

Production-ready, high-performance SQL parsing SDK for Go
Zero-copy tokenization • Object pooling • Multi-dialect support • Unicode-first design

🚀 Installation • ⚡ Quick Start • 📚 Documentation • 💡 Examples • 📊 Benchmarks

🎯 Overview

GoSQLX is a high-performance SQL parsing library designed for production use. It provides zero-copy tokenization, intelligent object pooling, and comprehensive SQL dialect support while maintaining a simple, idiomatic Go API.

✨ Key Features

• 🚀 Blazing Fast: 2.2M ops/sec, 8M tokens/sec processing speed
• 💾 Memory Efficient: 60-80% reduction through intelligent object pooling
• 🔒 Thread-Safe: Race-free, linear scaling to 128+ cores
• 🌍 Unicode Support: Complete UTF-8 support for international SQL
• 🔧 Multi-Dialect: PostgreSQL, MySQL, SQL Server, Oracle, SQLite
• 📊 Zero-Copy: Direct byte slice operations, < 200ns latency
• 🏗️ Production Ready: Battle-tested with 0 race conditions detected

🎯 Performance Highlights (v1.0.0)

📈 Project Stats

📦 Installation

go get github.com/ajitpratap0/GoSQLX

Requirements:

• Go 1.19 or higher
• No external dependencies

🚀 Quick Start

Basic Usage

package main

import (
    "fmt"
    "log"
    
    "github.com/ajitpratap0/GoSQLX/pkg/sql/tokenizer"
)

func main() {
    // Get tokenizer from pool (always return it!)
    tkz := tokenizer.GetTokenizer()
    defer tokenizer.PutTokenizer(tkz)
    
    // Tokenize SQL
    sql := "SELECT id, name FROM users WHERE age > 18"
    tokens, err := tkz.Tokenize([]byte(sql))
    if err != nil {
        log.Fatal(err)
    }
    
    // Process tokens
    fmt.Printf("Generated %d tokens\n", len(tokens))
    for _, token := range tokens {
        fmt.Printf("  %s (line %d, col %d)\n", 
            token.Token.Value,
            token.Start.Line,
            token.Start.Column)
    }
}

Advanced Example with AST

package main

import (
    "fmt"
    
    "github.com/ajitpratap0/GoSQLX/pkg/sql/tokenizer"
    "github.com/ajitpratap0/GoSQLX/pkg/sql/parser"
)

func AnalyzeSQL(sql string) error {
    // Tokenize
    tkz := tokenizer.GetTokenizer()
    defer tokenizer.PutTokenizer(tkz)
    
    tokens, err := tkz.Tokenize([]byte(sql))
    if err != nil {
        return fmt.Errorf("tokenization failed: %w", err)
    }
    
    // Parse to AST
    p := parser.NewParser()
    defer p.Release()
    
    ast, err := p.Parse(convertTokens(tokens))
    if err != nil {
        return fmt.Errorf("parsing failed: %w", err)
    }
    
    // Analyze AST
    fmt.Printf("Statement type: %T\n", ast)
    return nil
}

📚 Documentation

📖 Comprehensive Guides

🚀 Getting Started

📋 Quick Links

• Installation & Setup
• Basic Usage
• Advanced Patterns
• Performance Tuning
• Error Handling
• FAQ

💻 Examples

Multi-Dialect Support

// PostgreSQL with array operators
sql := `SELECT * FROM users WHERE tags @> ARRAY['admin']`

// MySQL with backticks
sql := "SELECT `user_id`, `name` FROM `users`"

// SQL Server with brackets
sql := "SELECT [user_id], [name] FROM [users]"

Unicode and International SQL

// Japanese
sql := `SELECT "名前", "年齢" FROM "ユーザー"`

// Russian
sql := `SELECT "имя", "возраст" FROM "пользователи"`

// Arabic
sql := `SELECT "الاسم", "العمر" FROM "المستخدمون"`

// Emoji support
sql := `SELECT * FROM users WHERE status = '🚀'`

Concurrent Processing

func ProcessConcurrently(queries []string) {
    var wg sync.WaitGroup
    
    for _, sql := range queries {
        wg.Add(1)
        go func(query string) {
            defer wg.Done()
            
            // Each goroutine gets its own tokenizer
            tkz := tokenizer.GetTokenizer()
            defer tokenizer.PutTokenizer(tkz)
            
            tokens, _ := tkz.Tokenize([]byte(query))
            // Process tokens...
        }(sql)
    }
    
    wg.Wait()
}

📊 Performance

🎯 v1.0.0 Performance Improvements

Latest Benchmark Results

BenchmarkTokenizer/SimpleSQL-16             965,466      1,238 ns/op     1,585 B/op      20 allocs/op
BenchmarkTokenizer/ComplexSQL-16             92,636     13,078 ns/op    13,868 B/op     159 allocs/op
BenchmarkTokenizer/Concurrent-128-16        639,093      1,788 ns/op    10,735 B/op      88 allocs/op

BenchmarkParser/SimpleSelect-16           6,330,259        185 ns/op       536 B/op       9 allocs/op
BenchmarkParser/ParallelSelect-16         8,175,652        154 ns/op       536 B/op       9 allocs/op

BenchmarkThroughput/200_goroutines-16     3,144,678        381 ns/op   2,189,740 ops/sec
BenchmarkTokensPerSecond-16                 733,141      1,619 ns/op   8,032,114 tokens/sec

Performance Characteristics

See PERFORMANCE_REPORT.md for detailed analysis.

🧪 Testing

# Run all tests with race detection
go test -race ./...

# Run benchmarks
go test -bench=. -benchmem ./...

# Generate coverage report
go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out

# Run specific test suites
go test -v ./pkg/sql/tokenizer/
go test -v ./pkg/sql/parser/

🏗️ Project Structure

GoSQLX/
├── pkg/
│   ├── models/              # Core data structures
│   │   ├── token.go        # Token definitions
│   │   └── location.go     # Position tracking
│   └── sql/
│       ├── tokenizer/       # Lexical analysis
│       │   ├── tokenizer.go
│       │   └── pool.go
│       ├── parser/          # Syntax analysis
│       │   ├── parser.go
│       │   └── expressions.go
│       ├── ast/            # Abstract syntax tree
│       │   ├── nodes.go
│       │   └── statements.go
│       └── keywords/        # SQL keywords
├── examples/               # Usage examples
│   └── cmd/
│       ├── example.go
│       └── example_test.go
├── docs/                   # Documentation
│   ├── API_REFERENCE.md
│   ├── USAGE_GUIDE.md
│   ├── ARCHITECTURE.md
│   └── TROUBLESHOOTING.md
└── tools/                  # Development tools

🛠️ Development

Prerequisites

• Go 1.19+
• Make (optional, for Makefile targets)
• golint, staticcheck (for code quality)

Building

# Build the project
make build

# Run quality checks
make quality

# Run all tests
make test

# Clean build artifacts
make clean

Code Quality

# Format code
go fmt ./...

# Vet code
go vet ./...

# Run linter
golint ./...

# Static analysis
staticcheck ./...

🤝 Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines.

How to Contribute

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

Development Guidelines

• Write tests for new features
• Ensure all tests pass with race detection
• Follow Go idioms and best practices
• Update documentation for API changes
• Add benchmarks for performance-critical code

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🚀 Roadmap

Phase 1: Foundation (Q3 2024) - v1.1.0

• ✅ Common Table Expressions (CTEs) with RECURSIVE
• ✅ Complete JOIN support (LEFT/RIGHT/FULL OUTER)
• ✅ Set operations (UNION/EXCEPT/INTERSECT)
• ✅ Comprehensive subquery support
• ✅ Standardized error handling

Phase 2: Advanced Features (Q4 2024) - v1.2.0

• 📋 Window functions (OVER, PARTITION BY)
• 📋 Transaction control (BEGIN/COMMIT/ROLLBACK)
• 📋 Views and materialized views
• 📋 Streaming parser API
• 📋 AST transformation framework

Phase 3: Dialect Specialization (Q1 2025) - v2.0.0

• 📋 PostgreSQL arrays, JSONB, custom types
• 📋 MySQL-specific syntax and functions
• 📋 SQL Server T-SQL extensions
• 📋 Multi-dialect parser with auto-detection

Phase 4: Intelligence Layer (Q2 2025) - v2.1.0

• 📋 Query optimization suggestions
• 📋 Security vulnerability detection
• 📋 Performance analysis and hints
• 📋 Schema validation

📄 Full Architectural Review & Roadmap

🤝 Community & Support

👥 Contributors

Core Team

How to Contribute

We love your input! We want to make contributing as easy and transparent as possible.

Quick Contribution Guide

1. 🍴 Fork the repo
2. 🔨 Make your changes
3. ✅ Ensure tests pass (go test -race ./...)
4. 📝 Update documentation
5. 🚀 Submit a PR

🎯 Use Cases

📊 Who's Using GoSQLX

📈 Project Metrics

graph LR
    A[SQL Input] -->|2.2M ops/sec| B[Tokenizer]
    B -->|8M tokens/sec| C[Parser]
    C -->|Zero-copy| D[AST]
    D -->|60-80% less memory| E[Output]

🗺️ Roadmap

💖 Support This Project

📜 License

Built with ❤️ by the GoSQLX Team

_{Copyright © 2024 GoSQLX. All rights reserved.}