g0database

package module
v0.0.0-...-61987bc Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 17, 2026 License: MIT Imports: 15 Imported by: 0

README

g0database

Github Actions codecov

A MySQL-like database, written from scratch in Go

Inspired by

Documentation

See more in docs

Learning Path: Building a SQL Database

If you want to understand how SQL databases work internally, here are the key topics to study:

1. Compiler Theory & SQL Parsing
  • Lexical Analysis (Lexer/Tokenizer) - Breaking SQL strings into tokens (keywords, identifiers, literals, operators)
  • Syntax Analysis (Parser) - Building an Abstract Syntax Tree (AST) from tokens
  • Grammar & BNF Notation - Understanding SQL grammar rules
  • Recursive Descent Parsing - A common technique for parsing SQL
  • Pratt Parsing - For handling operator precedence in expressions

Resources:

2. Query Processing
  • Query Planning - Converting AST to execution plan
  • Query Optimization - Cost-based optimization, join ordering, index selection
  • Predicate Pushdown - Moving filters closer to data source
  • Expression Evaluation - Evaluating WHERE clauses, functions, operators

Resources:

3. Storage Engines
  • Page-Based Storage - How databases organize data in fixed-size pages
  • B-Tree / B+Tree - The most common index structure for databases
  • LSM Tree (Log-Structured Merge Tree) - Used by LevelDB, RocksDB, Cassandra
  • SSTable (Sorted String Table) - Immutable sorted files used with LSM
  • Write-Ahead Log (WAL) - Durability through logging before writes
  • MVCC (Multi-Version Concurrency Control) - How databases handle concurrent access
  • Buffer Pool Management - Caching pages in memory

Resources:

4. Transaction Management
  • ACID Properties - Atomicity, Consistency, Isolation, Durability
  • Isolation Levels - Read Uncommitted, Read Committed, Repeatable Read, Serializable
  • Locking - Row locks, table locks, deadlock detection
  • 2PC (Two-Phase Commit) - Distributed transaction protocol
5. Network Protocol
  • MySQL Wire Protocol - Packet format, handshake, query/response
  • Connection Management - Connection pooling, session state
  • Result Set Encoding - How query results are serialized

Resources:

6. Data Structures & Algorithms
  • Hash Tables - For hash indexes and hash joins
  • Binary Search - For B-tree traversal
  • Sorting Algorithms - For ORDER BY, merge joins
  • Bloom Filters - For efficient negative lookups
  1. Start here: Crafting Interpreters - Learn lexer/parser basics
  2. Then: How Query Engines Work - Understand query execution
  3. Deep dive: Database Internals - Storage engine details
  4. Big picture: Designing Data-Intensive Applications - Distributed systems context
8. Open Source Databases to Study
Database Language Notable For
SQLite C Simple, well-documented, single-file
CockroachDB Go Distributed SQL, great docs
TiDB Go MySQL compatible, distributed
DuckDB C++ Analytical (OLAP), columnar
go-mysql-server Go MySQL protocol implementation
9. Key Concepts Summary
SQL Query Lifecycle:
┌──────────┐    ┌────────┐    ┌──────────┐    ┌───────────┐    ┌─────────┐
│  SQL     │───▶│ Lexer  │───▶│  Parser  │───▶│ Optimizer │───▶│Executor │
│  String  │    │(Tokens)│    │  (AST)   │    │  (Plan)   │    │(Results)│
└──────────┘    └────────┘    └──────────┘    └───────────┘    └─────────┘
                                                                    │
                                                                    ▼
                                                            ┌─────────────┐
                                                            │   Storage   │
                                                            │   Engine    │
                                                            │ (B-Tree/LSM)│
                                                            └─────────────┘

Documentation

Index

Constants

View Source 
const (
	// Default server version string
	DefaultServerVersion = "8.0.0-g0database"

	// Auth plugin name
	AuthPluginMysqlNativePassword = "mysql_native_password"
)
View Source 
const (
	PREC_LOWEST      = 1
	PREC_OR          = 2  // OR
	PREC_AND         = 3  // AND
	PREC_NOT         = 4  // NOT
	PREC_EQUALS      = 5  // =, <>, !=, IS, IN, LIKE, BETWEEN
	PREC_LESSGREATER = 6  // <, >, <=, >=
	PREC_SUM         = 7  // +, -
	PREC_PRODUCT     = 8  // *, /, %
	PREC_PREFIX      = 9  // -X, NOT X
	PREC_CALL        = 10 // function()
)

Precedence levels for Pratt parsing

View Source 
const (
	COM_QUIT    byte = 0x01
	COM_INIT_DB byte = 0x02
	COM_QUERY   byte = 0x03
	COM_PING    byte = 0x0e
)

MySQL command bytes

View Source 
const (
	MYSQL_TYPE_TINY     byte = 0x01 // BOOLEAN
	MYSQL_TYPE_LONG     byte = 0x03 // INT
	MYSQL_TYPE_LONGLONG byte = 0x08 // BIGINT
	MYSQL_TYPE_DATETIME byte = 0x0c
	MYSQL_TYPE_VARCHAR  byte = 0x0f
	MYSQL_TYPE_BLOB     byte = 0xfc // TEXT
)

MySQL column types

View Source 
const (
	CLIENT_LONG_PASSWORD      uint32 = 0x00000001
	CLIENT_FOUND_ROWS         uint32 = 0x00000002
	CLIENT_LONG_FLAG          uint32 = 0x00000004
	CLIENT_CONNECT_WITH_DB    uint32 = 0x00000008
	CLIENT_PROTOCOL_41        uint32 = 0x00000200
	CLIENT_SECURE_CONNECTION  uint32 = 0x00008000
	CLIENT_PLUGIN_AUTH        uint32 = 0x00080000
	CLIENT_PLUGIN_AUTH_LENENC uint32 = 0x00200000
	CLIENT_DEPRECATE_EOF      uint32 = 0x01000000
)

Capability flags

View Source 
const (
	CHARSET_UTF8_GENERAL_CI byte = 0x21 // utf8_general_ci (33)
)

Character set

View Source 
const (
	SERVER_STATUS_AUTOCOMMIT uint16 = 0x0002
)

Server status flags

Variables

View Source 
var (
	ErrDatabaseExists      = errors.New("database already exists")
	ErrDatabaseNotFound    = errors.New("database not found")
	ErrNoDatabaseSelected  = errors.New("no database selected")
	ErrTableExists         = errors.New("table already exists")
	ErrTableNotFound       = errors.New("table not found")
	ErrPrimaryKeyExists    = errors.New("duplicate primary key")
	ErrColumnCountMismatch = errors.New("column count mismatch")
	ErrColumnNotFound      = errors.New("column not found")
	ErrInvalidType         = errors.New("invalid value type for column")
	ErrNullNotAllowed      = errors.New("null value not allowed for column")
	ErrRowNotFound         = errors.New("row not found")
	ErrNoPrimaryKey        = errors.New("table has no primary key")
)
View Source 
var ErrInvalidPacket = ErrInvalidType // Reuse existing error

ErrInvalidPacket indicates an invalid packet format

Functions

func BuildPredicate 

func BuildPredicate(where Expression, table *Table) (func(*Row) bool, error)

BuildPredicate creates a predicate function from an Expression tree

func BuildPredicateFromWhereClause 

func BuildPredicateFromWhereClause(where *WhereClause, table *Table) (func(*Row) bool, error)

BuildPredicateFromWhereClause builds a predicate from legacy WhereClause Kept for backward compatibility

func ConvertValue 

func ConvertValue(val interface{}, colDef *ColumnDef) (interface{}, error)

ConvertValue converts a value to the appropriate type for a column

Types

type BetweenExpr 

type BetweenExpr struct {
	Expr Expression
	Low  Expression
	High Expression
	Not  bool // true for NOT BETWEEN
}

BetweenExpr represents: expr BETWEEN low AND high

func (*BetweenExpr) String 

func (be *BetweenExpr) String() string

type BinaryExpr 

type BinaryExpr struct {
	Left     Expression
	Operator TokenType
	Right    Expression
}

BinaryExpr represents a binary expression: left OP right Used for AND, OR, =, <>, <, >, <=, >=, +, -, *, /

func (*BinaryExpr) String 

func (b *BinaryExpr) String() string

type BooleanLiteral 

type BooleanLiteral struct {
	Value bool
}

BooleanLiteral represents a boolean value (TRUE/FALSE)

func (*BooleanLiteral) String 

func (bl *BooleanLiteral) String() string

type Column 

type Column struct {
	Name     string
	Datatype DataType
	DataSize int
	Value    interface{}
}

Column represents a parsed column definition from SQL (used by parser)

type ColumnDef 

type ColumnDef struct {
	Name       string
	Type       DataType
	Size       int // For varchar(N)
	PrimaryKey bool
	Nullable   bool
	Default    interface{}
}

ColumnDef defines a column's schema

type Command 

type Command interface {
	GetType() CommandType
}

type CommandCreate 

type CommandCreate struct {
	TableName string
	Columns   []*Column
}

CommandCreate represents CREATE TABLE statement

func (*CommandCreate) GetType 

func (c *CommandCreate) GetType() CommandType

type CommandDelete 

type CommandDelete struct {
	TableName string
	Where     Expression // Changed from *WhereClause to Expression
}

CommandDelete represents DELETE statement

func (*CommandDelete) GetType 

func (c *CommandDelete) GetType() CommandType

type CommandDrop 

type CommandDrop struct {
	TableName string
}

CommandDrop represents DROP TABLE statement

func (*CommandDrop) GetType 

func (c *CommandDrop) GetType() CommandType

type CommandInsert 

type CommandInsert struct {
	TableName string
	Columns   []string      // Column names (optional)
	Values    []interface{} // Values to insert
}

CommandInsert represents INSERT statement

func (*CommandInsert) GetType 

func (c *CommandInsert) GetType() CommandType

type CommandResult 

type CommandResult struct {
	Output      string
	IsTerminate bool

	// Fields for MySQL protocol support
	Type         ResultType
	AffectedRows int64
	Columns      []ResultColumn
	Rows         []ResultRow
}

CommandResult holds execution result (used by executor)

type CommandSelect 

type CommandSelect struct {
	SelectFields []string
	FromTables   []string
	Where        Expression // Changed from *WhereClause to Expression
	OrderBy      []OrderByClause
	Limit        int // 0 = no limit
}

CommandSelect represents SELECT statement

func (*CommandSelect) GetType 

func (c *CommandSelect) GetType() CommandType

type CommandType 

type CommandType string
const (
	CommandTypeSelect CommandType = "select"
	CommandTypeUpdate CommandType = "update"
	CommandTypeDelete CommandType = "delete"
	CommandTypeInsert CommandType = "insert"
	CommandTypeCreate CommandType = "create"
	CommandTypeDrop   CommandType = "drop"
)

type CommandUpdate 

type CommandUpdate struct {
	TableName string
	Updates   map[string]interface{} // column -> new value
	Where     Expression             // Changed from *WhereClause to Expression
}

CommandUpdate represents UPDATE statement

func (*CommandUpdate) GetType 

func (c *CommandUpdate) GetType() CommandType

type Condition 

type Condition struct {
	Column   string
	Operator string // =, <>, <, >, <=, >=
	Value    interface{}
}

Condition represents a single WHERE condition (kept for backward compatibility)

type Connection 

type Connection struct {
	// contains filtered or unexported fields
}

Connection represents a client connection

type DataSource 

type DataSource interface {
	LoadIntoTable(table *Table, filePath string) error
}

func NewDataSource 

func NewDataSource(sourceType DataSourceType) DataSource

type DataSourceType 

type DataSourceType string
const (
	DataSourceTypeCsv DataSourceType = "csv"
)

type DataType 

type DataType string
const (
	DataTypeInt      DataType = "int"
	DataTypeBigInt   DataType = "bigint"
	DataTypeVarchar  DataType = "varchar"
	DataTypeText     DataType = "text"
	DataTypeDateTime DataType = "datetime"
	DataTypeBoolean  DataType = "boolean"
)

type Database 

type Database struct {
	Name   string
	Tables map[string]*Table
}

Database holds multiple tables

func (*Database) CreateTable 

func (db *Database) CreateTable(def *TableDef) error

CreateTable creates a new table in the database

func (*Database) DropTable 

func (db *Database) DropTable(name string) error

DropTable removes a table from the database

func (*Database) GetTable 

func (db *Database) GetTable(name string) *Table

GetTable returns a table by name

type Engine 

type Engine struct {
	Databases map[string]*Database
	Current   *Database
}

Engine is the top-level storage manager

func NewEngine 

func NewEngine() *Engine

NewEngine creates a new storage engine

func (*Engine) CreateDatabase 

func (e *Engine) CreateDatabase(name string) error

CreateDatabase creates a new database

func (*Engine) DropDatabase 

func (e *Engine) DropDatabase(name string) error

DropDatabase removes a database

func (*Engine) GetDatabase 

func (e *Engine) GetDatabase(name string) *Database

GetDatabase returns a database by name

func (*Engine) UseDatabase 

func (e *Engine) UseDatabase(name string) error

UseDatabase selects a database as current

type Executor 

type Executor interface {
	Execute(cmd Command) CommandResult
}

func NewExecutor 

func NewExecutor(engine *Engine) Executor

type Expression 

type Expression interface {
	String() string
	// contains filtered or unexported methods
}

Expression represents an AST node for SQL expressions

type FloatLiteral 

type FloatLiteral struct {
	Value float64
}

FloatLiteral represents a floating-point value

func (*FloatLiteral) String 

func (fl *FloatLiteral) String() string

type FunctionCall 

type FunctionCall struct {
	Name string
	Args []Expression
}

FunctionCall represents a function call: name(args...)

func (*FunctionCall) String 

func (f *FunctionCall) String() string

type GroupedExpr 

type GroupedExpr struct {
	Expr Expression
}

GroupedExpr represents a parenthesized expression

func (*GroupedExpr) String 

func (g *GroupedExpr) String() string

type HandshakeResponse 

type HandshakeResponse struct {
	Capabilities  uint32
	MaxPacketSize uint32
	CharacterSet  uint8
	Username      string
	AuthResponse  []byte
	Database      string
	AuthPlugin    string
}

HandshakeResponse holds parsed client handshake response

type Identifier 

type Identifier struct {
	Name string
}

Identifier represents a column name or table.column reference

func (*Identifier) String 

func (i *Identifier) String() string

type InExpr 

type InExpr struct {
	Left   Expression
	Values []Expression
	Not    bool // true for NOT IN
}

InExpr represents: expr IN (values...)

func (*InExpr) String 

func (ie *InExpr) String() string

type IntegerLiteral 

type IntegerLiteral struct {
	Value int64
}

IntegerLiteral represents an integer value

func (*IntegerLiteral) String 

func (il *IntegerLiteral) String() string

type IsNullExpr 

type IsNullExpr struct {
	Expr Expression
	Not  bool // true for IS NOT NULL
}

IsNullExpr represents: expr IS NULL or expr IS NOT NULL

func (*IsNullExpr) String 

func (in *IsNullExpr) String() string

type Lexer 

type Lexer interface {
	NextToken() Token
	Tokenize() ([]Token, error)
	Analyze(sqlCmd string) ([]Token, error) // backward compatibility
}

Lexer defines the interface for SQL tokenization

func NewLexer 

func NewLexer(input string) Lexer

NewLexer creates a new lexer for the given SQL input

func NewNaiveLexer 

func NewNaiveLexer() Lexer

NewNaiveLexer returns a lexer (kept for backward compatibility)

type LikeExpr 

type LikeExpr struct {
	Left    Expression
	Pattern Expression
	Not     bool // true for NOT LIKE
}

LikeExpr represents: expr LIKE pattern

func (*LikeExpr) String 

func (le *LikeExpr) String() string

type NullLiteral 

type NullLiteral struct{}

NullLiteral represents a NULL value

func (*NullLiteral) String 

func (nl *NullLiteral) String() string

type OrderByClause 

type OrderByClause struct {
	Column string
	Desc   bool
}

OrderByClause represents ORDER BY column

type Parser 

type Parser interface {
	Parse(tokens []Token) (Command, error)
}

Parser defines the interface for SQL parsing

func NewParser 

func NewParser() Parser

NewParser creates a new parser

type ResultColumn 

type ResultColumn struct {
	Name string
	Type DataType
	Size int
}

ResultColumn holds metadata for a column in the result set

type ResultRow 

type ResultRow struct {
	Values []interface{}
}

ResultRow holds a row of data in the result set

type ResultType 

type ResultType int

ResultType indicates the type of query result 

const (
	ResultTypeOK     ResultType = iota // INSERT, UPDATE, DELETE, CREATE, DROP
	ResultTypeSelect                   // SELECT with result set
	ResultTypeError                    // Error occurred
)

type Row 

type Row struct {
	Values []interface{}
}

Row represents a single row of data

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server represents the MySQL protocol server

func NewServer 

func NewServer(config ServerConfig, engine *Engine) *Server

NewServer creates a new MySQL protocol server

func (*Server) Addr 

func (s *Server) Addr() string

Addr returns the server's listen address

func (*Server) Close 

func (s *Server) Close() error

Close stops the server

func (*Server) Start 

func (s *Server) Start() error

Start starts the server and listens for connections

type ServerConfig 

type ServerConfig struct {
	Addr          string // Address to listen on (default ":3306")
	ServerVersion string // Server version string (default "8.0.0-g0database")
}

ServerConfig holds server configuration

type StarExpr 

type StarExpr struct{}

StarExpr represents the * in SELECT *

func (*StarExpr) String 

func (s *StarExpr) String() string

type StringLiteral 

type StringLiteral struct {
	Value string
}

StringLiteral represents a string value

func (*StringLiteral) String 

func (sl *StringLiteral) String() string

type Table 

type Table struct {
	Def  *TableDef
	Rows []*Row
	// contains filtered or unexported fields
}

Table holds the schema and data for a table

func (*Table) DeleteByPK 

func (t *Table) DeleteByPK(pk interface{}) error

DeleteByPK deletes a row by primary key

func (*Table) DeleteWhere 

func (t *Table) DeleteWhere(predicate func(*Row) bool) (int, error)

DeleteWhere deletes all rows matching the predicate

func (*Table) GetColumnIndex 

func (t *Table) GetColumnIndex(name string) int

GetColumnIndex returns the index of a column by name, or -1 if not found

func (*Table) Insert 

func (t *Table) Insert(values []interface{}) error

Insert adds a new row to the table

func (*Table) InsertMap 

func (t *Table) InsertMap(data map[string]interface{}) error

InsertMap inserts a row using column name -> value mapping

func (*Table) SelectAll 

func (t *Table) SelectAll() []*Row

SelectAll returns all rows in the table

func (*Table) SelectByPK 

func (t *Table) SelectByPK(pk interface{}) *Row

SelectByPK returns a row by primary key value (scans all rows)

func (*Table) SelectWhere 

func (t *Table) SelectWhere(predicate func(*Row) bool) []*Row

SelectWhere returns rows matching the predicate

func (*Table) UpdateByPK 

func (t *Table) UpdateByPK(pk interface{}, updates map[string]interface{}) error

UpdateByPK updates a row by primary key

func (*Table) UpdateWhere 

func (t *Table) UpdateWhere(predicate func(*Row) bool, updates map[string]interface{}) (int, error)

UpdateWhere updates all rows matching the predicate

type TableDef 

type TableDef struct {
	Name       string
	Columns    []ColumnDef
	PrimaryKey string // Column name that is primary key
}

TableDef defines a table's schema

type Token 

type Token struct {
	Type    TokenType
	Literal string
	Line    int
	Column  int
}

Token represents a lexical token with position information

func (Token) String 

func (t Token) String() string

String returns a string representation of the token for debugging

type TokenType 

type TokenType int

TokenType represents the type of a token 

const (
	TOKEN_ILLEGAL TokenType = iota
	TOKEN_EOF

	// Identifiers & Literals
	TOKEN_IDENT  // column_name, table_name
	TOKEN_INT    // 123
	TOKEN_FLOAT  // 3.14
	TOKEN_STRING // 'hello'

	// Keywords - DDL
	TOKEN_CREATE
	TOKEN_TABLE
	TOKEN_DROP
	TOKEN_ALTER
	TOKEN_ADD
	TOKEN_COLUMN

	// Keywords - DML
	TOKEN_SELECT
	TOKEN_INSERT
	TOKEN_UPDATE
	TOKEN_DELETE
	TOKEN_FROM
	TOKEN_WHERE
	TOKEN_INTO
	TOKEN_VALUES
	TOKEN_SET

	// Keywords - Clauses
	TOKEN_ORDER
	TOKEN_BY
	TOKEN_ASC
	TOKEN_DESC
	TOKEN_LIMIT
	TOKEN_GROUP
	TOKEN_HAVING

	// Keywords - Logical
	TOKEN_AND
	TOKEN_OR
	TOKEN_NOT

	// Keywords - Values
	TOKEN_NULL
	TOKEN_TRUE
	TOKEN_FALSE

	// Keywords - Constraints
	TOKEN_PRIMARY
	TOKEN_KEY
	TOKEN_UNIQUE
	TOKEN_FOREIGN
	TOKEN_REFERENCES

	// Keywords - Data Types
	TOKEN_INTEGER
	TOKEN_INT_TYPE
	TOKEN_VARCHAR
	TOKEN_TEXT
	TOKEN_BOOLEAN
	TOKEN_FLOAT_TYPE
	TOKEN_DOUBLE
	TOKEN_DATE
	TOKEN_DATETIME
	TOKEN_TIMESTAMP

	// Comparison Operators
	TOKEN_EQ      // =
	TOKEN_NE      // <> or !=
	TOKEN_LT      // <
	TOKEN_GT      // >
	TOKEN_LE      // <=
	TOKEN_GE      // >=
	TOKEN_LIKE    // LIKE
	TOKEN_IN      // IN
	TOKEN_BETWEEN // BETWEEN
	TOKEN_IS      // IS

	// Arithmetic Operators
	TOKEN_PLUS     // +
	TOKEN_MINUS    // -
	TOKEN_ASTERISK // *
	TOKEN_SLASH    // /
	TOKEN_PERCENT  // %

	// Punctuation
	TOKEN_COMMA     // ,
	TOKEN_SEMICOLON // ;
	TOKEN_LPAREN    // (
	TOKEN_RPAREN    // )
	TOKEN_DOT       // .
)

func LookupKeyword 

func LookupKeyword(ident string) TokenType

LookupKeyword returns the token type for an identifier. If the identifier is a keyword, returns the keyword token type. Otherwise, returns TOKEN_IDENT.

func (TokenType) IsKeyword 

func (t TokenType) IsKeyword() bool

IsKeyword returns true if the token type is a keyword

func (TokenType) IsLiteral 

func (t TokenType) IsLiteral() bool

IsLiteral returns true if the token type is a literal value

func (TokenType) IsOperator 

func (t TokenType) IsOperator() bool

IsOperator returns true if the token type is an operator

func (TokenType) String 

func (t TokenType) String() string

String returns the string representation of a TokenType

type UnaryExpr 

type UnaryExpr struct {
	Operator TokenType
	Right    Expression
}

UnaryExpr represents a unary expression: OP expr Used for NOT, unary minus

func (*UnaryExpr) String 

func (u *UnaryExpr) String() string

type WhereClause 

type WhereClause struct {
	Conditions []Condition
	Logic      string // "AND" or "OR" (default AND)
}

WhereClause represents WHERE conditions (kept for backward compatibility)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.