storage

package
v0.5.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 14, 2026 License: AGPL-3.0 Imports: 19 Imported by: 0

Documentation

Overview

Package storage provides persistence primitives and metadata structures used by tinySQL. This file implements a lightweight in-memory system catalog used for introspection (tables, columns, views, functions) and simple job scheduling metadata.

Package storage provides the durable data structures for tinySQL.

What: An in-memory multi-tenant catalog of tables with column metadata, rows, and basic typing. It includes snapshot cloning for MVCC-light, GOB-based checkpoints, and an append-only Write-Ahead Log (WAL) for crash recovery and durability. How: Tables store rows as [][]any for compactness; a lower-cased column index accelerates name lookups. Save/Load serialize the catalog to a file, writing JSON for JSON columns. The WAL logs whole-table changes and drops; recovery replays committed records and truncates partial tails. Why: Favor a simple, explicit model over complex page managers: it keeps the code understandable, testable, and sufficient for embedded/edge use cases.

Package storage – StorageBackend interface and StorageMode definitions.

What: Pluggable storage backends that decouple data management from the in-memory catalog. Each backend decides where table data lives (RAM, disk, or a combination) and how it is persisted. How: The DB struct optionally delegates Get/Put/Drop operations to an attached StorageBackend. Backends may lazily load tables, cache hot data, and flush dirty tables on Sync/Close. Why: Supporting multiple storage modes turns tinySQL into a realistic alternative to SQLite – from purely in-memory analytics right through to disk-resident databases that exceed available RAM.

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrTxNotActive          = fmt.Errorf("transaction is not active")
	ErrSerializationFailure = fmt.Errorf("could not serialize access due to concurrent update")
	ErrRowNotFound          = fmt.Errorf("row not found")
)

Errors

Functions

func EstimateColumnSize 

func EstimateColumnSize(typ ColType) int64

EstimateColumnSize estimates the average size of a column type.

func EstimateTableSize 

func EstimateTableSize(t *Table) int64

EstimateTableSize estimates memory usage of a table in bytes.

func EstimateValueSize 

func EstimateValueSize(val any) int64

EstimateValueSize estimates the memory size of a value.

func FanIn 

func FanIn(ctx context.Context, channels ...<-chan interface{}) <-chan interface{}

FanIn combines multiple channels into one.

func FanOut 

func FanOut(ctx context.Context, input <-chan interface{}, workers int) []<-chan interface{}

FanOut distributes work from one channel to multiple channels.

func SaveToBytes 

func SaveToBytes(db *DB) ([]byte, error)

SaveToBytes serializes the database snapshot to a byte slice.

func SaveToFile 

func SaveToFile(db *DB, filename string) error

SaveToFile writes a snapshot of the database to a file. If the filename ends with .gz, the snapshot is gzip-compressed to reduce size.

func SaveToWriter 

func SaveToWriter(db *DB, w io.Writer) error

SaveToWriter writes a snapshot of the database to an arbitrary writer. It does not attach or alter WAL configuration.

Types

type AdvancedWAL 

type AdvancedWAL struct {
	// contains filtered or unexported fields
}

AdvancedWAL manages row-level write-ahead logging with full ACID guarantees.

func OpenAdvancedWAL 

func OpenAdvancedWAL(config AdvancedWALConfig) (*AdvancedWAL, error)

OpenAdvancedWAL creates or opens a WAL with full ACID semantics.

func (*AdvancedWAL) Checkpoint 

func (w *AdvancedWAL) Checkpoint(db *DB) error

Checkpoint creates a consistent snapshot and truncates the WAL.

func (*AdvancedWAL) Close 

func (w *AdvancedWAL) Close() error

Close flushes and closes the WAL.

func (*AdvancedWAL) GetCommittedLSN 

func (w *AdvancedWAL) GetCommittedLSN() LSN

GetCommittedLSN returns the LSN of the last committed transaction.

func (*AdvancedWAL) GetFlushedLSN 

func (w *AdvancedWAL) GetFlushedLSN() LSN

GetFlushedLSN returns the LSN of the last flushed record.

func (*AdvancedWAL) GetNextLSN 

func (w *AdvancedWAL) GetNextLSN() LSN

GetNextLSN returns the next LSN to be assigned.

func (*AdvancedWAL) LogAbort 

func (w *AdvancedWAL) LogAbort(txID TxID) (LSN, error)

LogAbort logs a transaction abort.

func (*AdvancedWAL) LogBegin 

func (w *AdvancedWAL) LogBegin(txID TxID) (LSN, error)

LogBegin logs the start of a transaction.

func (*AdvancedWAL) LogCommit 

func (w *AdvancedWAL) LogCommit(txID TxID) (LSN, error)

LogCommit logs a transaction commit.

func (*AdvancedWAL) LogDelete 

func (w *AdvancedWAL) LogDelete(txID TxID, tenant, table string, rowID int64, before []any, cols []Column) (LSN, error)

LogDelete logs a row deletion.

func (*AdvancedWAL) LogInsert 

func (w *AdvancedWAL) LogInsert(txID TxID, tenant, table string, rowID int64, data []any, cols []Column) (LSN, error)

LogInsert logs a row insertion.

func (*AdvancedWAL) LogUpdate 

func (w *AdvancedWAL) LogUpdate(txID TxID, tenant, table string, rowID int64, before, after []any, cols []Column) (LSN, error)

LogUpdate logs a row update with before/after images.

func (*AdvancedWAL) Recover 

func (w *AdvancedWAL) Recover(db *DB) (int, error)

Recover replays the WAL to restore database state after a crash.

func (*AdvancedWAL) ShouldCheckpoint 

func (w *AdvancedWAL) ShouldCheckpoint() bool

ShouldCheckpoint checks if a checkpoint is needed.

type AdvancedWALConfig 

type AdvancedWALConfig struct {
	Path               string
	CheckpointPath     string
	CheckpointEvery    uint64        // Checkpoint after N records
	CheckpointInterval time.Duration // Checkpoint after duration
	Compress           bool
	BufferSize         int // Buffer size for writing
}

AdvancedWALConfig configures the advanced WAL.

type BackendStats added in v0.5.4 

type BackendStats struct {
	Mode             StorageMode
	TablesInMemory   int
	TablesOnDisk     int
	MemoryUsedBytes  int64
	MemoryLimitBytes int64
	DiskUsedBytes    int64
	CacheHitRate     float64
	SyncCount        int64
	LoadCount        int64
	EvictionCount    int64
}

BackendStats provides observability into storage backend behaviour.

type BatchHandler 

type BatchHandler func(items []interface{}) error

BatchHandler processes a batch of items.

type BatchProcessor 

type BatchProcessor struct {
	// contains filtered or unexported fields
}

BatchProcessor batches operations for efficiency.

func NewBatchProcessor 

func NewBatchProcessor(maxSize int, interval time.Duration, handler BatchHandler) *BatchProcessor

NewBatchProcessor creates a new batch processor.

func (*BatchProcessor) Add 

func (bp *BatchProcessor) Add(item interface{}) error

Add adds an item to the batch queue.

func (*BatchProcessor) Run 

func (bp *BatchProcessor) Run(ctx context.Context, wg *sync.WaitGroup)

Run starts the batch processor.

type BufferPool 

type BufferPool struct {
	// contains filtered or unexported fields
}

BufferPool manages in-memory tables with configurable eviction.

func NewBufferPool 

func NewBufferPool(policy *MemoryPolicy) *BufferPool

NewBufferPool creates a buffer pool with the given policy.

func (*BufferPool) Get 

func (bp *BufferPool) Get(tenant, name string) (*Table, bool)

Get retrieves a table from the buffer pool.

func (*BufferPool) GetMemoryLimit 

func (bp *BufferPool) GetMemoryLimit() int64

GetMemoryLimit returns the configured memory limit.

func (*BufferPool) GetMemoryUsage 

func (bp *BufferPool) GetMemoryUsage() int64

GetMemoryUsage returns current memory usage in bytes.

func (*BufferPool) GetStats 

func (bp *BufferPool) GetStats() CacheStats

GetStats returns a copy of current statistics.

func (*BufferPool) Put 

func (bp *BufferPool) Put(tenant, name string, table *Table) error

Put adds or updates a table in the buffer pool.

func (*BufferPool) Remove 

func (bp *BufferPool) Remove(tenant, name string)

Remove removes a table from the buffer pool.

type CacheStats 

type CacheStats struct {
	MemoryUsed        int64
	MemoryLimit       int64
	MemoryUtilization float64

	CacheHits   int64
	CacheMisses int64
	HitRate     float64

	EvictionCount int64
	EvictionSize  int64

	TablesInMemory int
	TablesOnDisk   int
	// contains filtered or unexported fields
}

CacheStats tracks buffer pool performance metrics.

type CacheStrategy 

type CacheStrategy int

CacheStrategy defines the eviction policy for the buffer pool. 

const (
	StrategyNone CacheStrategy = iota // No eviction (full in-memory)
	StrategyLRU                       // Least Recently Used
	StrategyLFU                       // Least Frequently Used (future)
	StrategyARC                       // Adaptive Replacement Cache (future)
)

func (CacheStrategy) String 

func (s CacheStrategy) String() string

type CachedTable 

type CachedTable struct {
	Table       *Table
	Size        int64
	LoadedAt    time.Time
	LastAccess  time.Time
	AccessCount int64
	Pinned      bool

	// Disk location (for lazy loading - future)
	OnDisk     bool
	DiskOffset int64
	// contains filtered or unexported fields
}

CachedTable wraps a table with caching metadata.

type CatalogColumn added in v0.5.1 

type CatalogColumn struct {
	Schema       string
	TableName    string
	Name         string
	Position     int
	DataType     string
	IsNullable   bool
	DefaultValue *string
}

CatalogColumn represents metadata for a table column CatalogColumn represents a column in a catalog table including its position and declared data type. `DefaultValue` may be nil if none is defined.

type CatalogFunction added in v0.5.1 

type CatalogFunction struct {
	Schema          string
	Name            string
	FunctionType    string // 'SCALAR', 'TABLE', 'AGGREGATE', 'WINDOW'
	ArgTypes        []string
	ReturnType      string
	Language        string // 'BUILTIN', 'SQL', 'GO'
	IsDeterministic bool
	Description     string
}

CatalogFunction represents metadata for scalar and table-valued functions CatalogFunction describes registered functions (builtin or user defined). `FunctionType` categorizes the function, and `Language` indicates the implementation origin.

type CatalogJob added in v0.5.1 

type CatalogJob struct {
	Name         string
	SQLText      string
	ScheduleType string     // 'CRON', 'INTERVAL', 'ONCE'
	CronExpr     string     // For CRON schedule
	IntervalMs   int64      // For INTERVAL schedule (milliseconds)
	RunAt        *time.Time // For ONCE schedule
	Timezone     string
	Enabled      bool
	CatchUp      bool // Run missed executions
	NoOverlap    bool // Prevent concurrent runs
	MaxRuntimeMs int64
	LastRunAt    *time.Time
	NextRunAt    *time.Time
	CreatedAt    time.Time
	UpdatedAt    time.Time
}

CatalogJob represents a scheduled job CatalogJob describes a scheduled job. The fields provide flexible scheduling options (CRON, interval, or single-run) and execution metadata for bookkeeping and scheduling decisions.

type CatalogManager added in v0.5.1 

type CatalogManager struct {
	// contains filtered or unexported fields
}

CatalogManager manages system catalog tables (`catalog.tables`, `catalog.columns`, etc.) and provides thread-safe registration and lookup helpers used by the rest of the system for introspection and scheduling. CatalogManager is safe for concurrent use.

func NewCatalogManager added in v0.5.1 

func NewCatalogManager() *CatalogManager

NewCatalogManager allocates and returns an initialized CatalogManager.

func (*CatalogManager) DeleteJob added in v0.5.1 

func (c *CatalogManager) DeleteJob(name string) error

DeleteJob removes a job from the catalog, returning an error when the job does not exist.

func (*CatalogManager) GetAllColumns added in v0.5.1 

func (c *CatalogManager) GetAllColumns() []CatalogColumn

GetAllColumns aggregates and returns columns registered for every table in the catalog. The returned slice is a concatenation of the internal per-table column lists.

func (*CatalogManager) GetColumns added in v0.5.1 

func (c *CatalogManager) GetColumns(schema, tableName string) []CatalogColumn

GetColumns returns the column metadata for `schema.tableName`. If the table is unknown an empty slice is returned.

func (*CatalogManager) GetFunctions added in v0.5.1 

func (c *CatalogManager) GetFunctions() []*CatalogFunction

GetFunctions returns metadata for all registered functions.

func (*CatalogManager) GetJob added in v0.5.1 

func (c *CatalogManager) GetJob(name string) (*CatalogJob, error)

GetJob retrieves a job by name, returning an error if not found.

func (*CatalogManager) GetTables added in v0.5.1 

func (c *CatalogManager) GetTables() []*CatalogTable

GetTables returns a slice with metadata for all registered tables and views.

func (*CatalogManager) GetViews added in v0.5.1 

func (c *CatalogManager) GetViews() []*CatalogView

GetViews returns metadata for all registered views.

func (*CatalogManager) ListEnabledJobs added in v0.5.1 

func (c *CatalogManager) ListEnabledJobs() []*CatalogJob

ListEnabledJobs returns only jobs whose `Enabled` flag is true.

func (*CatalogManager) ListJobs added in v0.5.1 

func (c *CatalogManager) ListJobs() []*CatalogJob

ListJobs returns a slice containing all registered jobs.

func (*CatalogManager) RegisterFunction added in v0.5.1 

func (c *CatalogManager) RegisterFunction(fn *CatalogFunction) error

RegisterFunction registers or updates a function's metadata.

func (*CatalogManager) RegisterJob added in v0.5.1 

func (c *CatalogManager) RegisterJob(job *CatalogJob) error

RegisterJob adds a new scheduled job or updates an existing entry. Job names must be non-empty.

func (*CatalogManager) RegisterTable added in v0.5.1 

func (c *CatalogManager) RegisterTable(schema, name string, cols []Column) error

RegisterTable registers a table and its columns in the system catalog. The provided `cols` slice is converted to `CatalogColumn` entries and stored under the key `schema.name`.

func (*CatalogManager) RegisterView added in v0.5.1 

func (c *CatalogManager) RegisterView(schema, name, sqlText string) error

RegisterView registers a view definition under `schema.name`.

func (*CatalogManager) UpdateJobRuntime added in v0.5.1 

func (c *CatalogManager) UpdateJobRuntime(name string, lastRun, nextRun time.Time) error

UpdateJobRuntime updates runtime bookkeeping fields for a named job. It sets `LastRunAt`, `NextRunAt` and marks the job as recently updated.

type CatalogTable added in v0.5.1 

type CatalogTable struct {
	Schema    string
	Name      string
	Type      string // 'TABLE', 'VIEW', 'SYSTEM'
	RowCount  int64
	CreatedAt time.Time
	UpdatedAt time.Time
}

CatalogTable represents metadata for a single table CatalogTable holds basic metadata for a registered table or view.

type CatalogView added in v0.5.1 

type CatalogView struct {
	Schema    string
	Name      string
	SQLText   string
	CreatedAt time.Time
}

CatalogView represents a saved view definition CatalogView stores the definition of a saved view.

type ColType 

type ColType int

ColType enumerates supported column data types. 

const (
	// IntType is a generic integer column type.
	IntType ColType = iota
	// Int8Type is an 8-bit signed integer column type.
	Int8Type
	// Int16Type is a 16-bit signed integer column type.
	Int16Type
	// Int32Type is a 32-bit signed integer column type.
	Int32Type
	// Int64Type is a 64-bit signed integer column type.
	Int64Type
	// UintType is an unsigned integer column type.
	UintType
	// Uint8Type is an 8-bit unsigned integer column type.
	Uint8Type
	// Uint16Type is a 16-bit unsigned integer column type.
	Uint16Type
	// Uint32Type is a 32-bit unsigned integer column type.
	Uint32Type
	// Uint64Type is a 64-bit unsigned integer column type.
	Uint64Type

	// Float32Type is a 32-bit floating point column type.
	Float32Type
	// Float64Type is a 64-bit floating point column type.
	Float64Type
	// FloatType is an alias for Float64Type.
	FloatType // alias for Float64Type

	// StringType represents a variable-length UTF-8 string column.
	StringType
	// TextType is an alias for StringType intended for long text.
	TextType // alias for StringType
	// RuneType stores single Unicode code points.
	RuneType
	// ByteType stores raw byte data.
	ByteType

	// BoolType represents a boolean column (true/false).
	BoolType

	// TimeType stores time-of-day values.
	TimeType
	// DateType stores date-only values.
	DateType
	// DateTimeType stores combined date and time values.
	DateTimeType
	// TimestampType stores an absolute point in time.
	TimestampType
	// DurationType stores a time duration.
	DurationType

	// JsonType stores JSON text.
	JsonType
	// JsonbType stores binary JSON representations.
	JsonbType
	// MapType stores map-like complex values.
	MapType
	// SliceType stores slice-like complex values.
	SliceType
	// ArrayType stores array-like complex values.
	ArrayType

	// Complex64Type stores complex64 numeric values.
	Complex64Type
	// Complex128Type stores complex128 numeric values.
	Complex128Type
	// ComplexType is an alias for Complex128Type.
	ComplexType // alias for Complex128Type
	// PointerType represents a pointer/reference to another object.
	PointerType
	// InterfaceType represents an arbitrary Go interface value.
	InterfaceType

	// VectorType represents a vector/embedding column used by RAG features.
	VectorType
)

func (ColType) String 

func (t ColType) String() string

type Column 

type Column struct {
	Name         string
	Type         ColType
	Constraint   ConstraintType
	ForeignKey   *ForeignKeyRef // Only used if Constraint == ForeignKey
	PointerTable string         // Target table for POINTER type
}

Column holds column schema information in a table.

type ConcurrencyConfig 

type ConcurrencyConfig struct {
	// Worker pool sizes
	ReadWorkers  int
	WriteWorkers int

	// Channel buffer sizes
	ReadQueueSize  int
	WriteQueueSize int

	// Timeouts
	WorkerTimeout time.Duration
	QueueTimeout  time.Duration

	// Batch settings
	BatchSize     int
	BatchInterval time.Duration
}

ConcurrencyConfig configures the concurrency system.

func DefaultConcurrencyConfig 

func DefaultConcurrencyConfig() ConcurrencyConfig

DefaultConcurrencyConfig returns sensible defaults based on CPU count.

type ConcurrencyManager 

type ConcurrencyManager struct {
	// contains filtered or unexported fields
}

ConcurrencyManager orchestrates concurrent operations.

func NewConcurrencyManager 

func NewConcurrencyManager(config ConcurrencyConfig) *ConcurrencyManager

NewConcurrencyManager creates a new concurrency manager.

func (*ConcurrencyManager) Shutdown 

func (cm *ConcurrencyManager) Shutdown(timeout time.Duration) error

Shutdown gracefully shuts down the concurrency manager.

func (*ConcurrencyManager) Stats 

func (cm *ConcurrencyManager) Stats() *ConcurrencyStats

Stats returns current concurrency statistics.

func (*ConcurrencyManager) SubmitRead 

func (cm *ConcurrencyManager) SubmitRead(ctx context.Context, data interface{}) <-chan WorkResult

SubmitRead submits a read request (non-blocking).

func (*ConcurrencyManager) SubmitWrite 

func (cm *ConcurrencyManager) SubmitWrite(ctx context.Context, data interface{}) <-chan WorkResult

SubmitWrite submits a write request (non-blocking).

type ConcurrencyStats 

type ConcurrencyStats struct {
	TotalRequests   atomic.Uint64
	CompletedReads  atomic.Uint64
	CompletedWrites atomic.Uint64
	FailedRequests  atomic.Uint64
	QueuedReads     atomic.Int64
	QueuedWrites    atomic.Int64
	ActiveWorkers   atomic.Int64
}

ConcurrencyStats tracks concurrency metrics.

type ConstraintType 

type ConstraintType int

ConstraintType enumerates supported column constraints. 

const (
	NoConstraint ConstraintType = iota
	PrimaryKey
	ForeignKey
	Unique
)

func (ConstraintType) String 

func (c ConstraintType) String() string

type DB 

type DB struct {
	// contains filtered or unexported fields
}

DB is an in-memory, multi-tenant database catalog with full MVCC support. It optionally delegates storage to a StorageBackend for disk-based or hybrid persistence strategies.

func LoadFromBytes 

func LoadFromBytes(b []byte) (*DB, error)

LoadFromBytes loads a database from a byte slice.

func LoadFromFile 

func LoadFromFile(filename string) (*DB, error)

LoadFromFile loads a database snapshot from a file. It auto-detects gzip compression based on the .gz suffix and attaches a WAL if a path is given.

func LoadFromReader 

func LoadFromReader(r io.Reader) (*DB, error)

LoadFromReader loads a database snapshot from an arbitrary reader. The returned DB has no WAL attached.

func NewDB 

func NewDB() *DB

NewDB creates a new empty database catalog with MVCC support. The database operates in ModeMemory (pure in-memory) by default. Use OpenDB for disk-backed or hybrid storage modes.

func OpenDB added in v0.5.4 

func OpenDB(cfg StorageConfig) (*DB, error)

OpenDB creates or opens a database with the specified storage configuration. For ModeMemory this is equivalent to NewDB (with optional save-on-close). For ModeDisk/ModeHybrid/ModeIndex, tables are stored as individual files in the configured directory. For ModeWAL, the existing WAL mechanism is configured automatically.

func (*DB) AdvancedWAL 

func (db *DB) AdvancedWAL() *AdvancedWAL

AdvancedWAL returns the configured advanced WAL manager (may be nil).

func (*DB) AttachAdvancedWAL 

func (db *DB) AttachAdvancedWAL(wal *AdvancedWAL)

AttachAdvancedWAL attaches an advanced WAL to the database.

func (*DB) Backend added in v0.5.4 

func (db *DB) Backend() StorageBackend

Backend returns the attached StorageBackend (may be nil for pure in-memory databases created with NewDB).

func (*DB) BackendStats added in v0.5.4 

func (db *DB) BackendStats() BackendStats

BackendStats returns statistics from the storage backend. Returns a zero-value BackendStats if no backend is attached.

func (*DB) Catalog added in v0.5.1 

func (db *DB) Catalog() *CatalogManager

Catalog returns the CatalogManager attached to the DB, creating one lazily if necessary.

func (*DB) Close added in v0.5.4 

func (db *DB) Close() error

Close persists all data and releases resources. For ModeMemory with a configured path, this saves a final GOB snapshot. For ModeDisk/ModeHybrid, dirty tables are flushed. WAL and Advanced WAL resources are closed.

func (*DB) Config added in v0.5.4 

func (db *DB) Config() *StorageConfig

Config returns the StorageConfig used to open this database. Returns nil for databases created with NewDB().

func (*DB) DeepClone 

func (db *DB) DeepClone() *DB

DeepClone creates a full copy of the database (MVCC-light snapshot). Note: This is not copy-on-write; it creates a full copy (simple but O(n)).

func (*DB) Drop 

func (db *DB) Drop(tn, name string) error

Drop removes a table from the tenant (and from the backend if attached).

func (*DB) Evict added in v0.5.4 

func (db *DB) Evict(tenant, name string) error

Evict removes a table from the in-memory cache without deleting it from the backend. This is only meaningful for disk-backed modes; in ModeMemory the data would be lost. Returns an error if no backend is attached.

func (*DB) Get 

func (db *DB) Get(tn, name string) (*Table, error)

Get returns a table by name for the given tenant. When a StorageBackend is attached, tables not found in memory are loaded from the backend on demand (lazy loading).

func (*DB) ListTables 

func (db *DB) ListTables(tn string) []*Table

ListTables returns the tables in a tenant sorted by name. When a StorageBackend is attached, tables that exist on disk but are not currently loaded into memory are loaded on demand.

func (*DB) ListTenants added in v0.5.4 

func (db *DB) ListTenants() []string

ListTenants returns the names of all tenants that have at least one table.

func (*DB) MVCC 

func (db *DB) MVCC() *MVCCManager

MVCC returns the MVCC manager.

func (*DB) MigrateToBackend added in v0.5.4 

func (db *DB) MigrateToBackend(b StorageBackend) error

MigrateToBackend copies all in-memory tables to the given backend and attaches it. This enables migrating a ModeMemory database to ModeDisk or ModeHybrid at runtime.

func (*DB) Put 

func (db *DB) Put(tn string, t *Table) error

Put adds a new table to the tenant; returns error if it already exists. When a StorageBackend is attached, the table is also checked against the backend to prevent duplicates, and optionally persisted immediately when SyncOnMutate is configured.

func (*DB) SetBackend added in v0.5.4 

func (db *DB) SetBackend(b StorageBackend)

SetBackend attaches a StorageBackend and sets the storage mode. This is primarily used internally by OpenDB; calling it on a running database should be done with care.

func (*DB) ShallowCloneForTable added in v0.5.4 

func (db *DB) ShallowCloneForTable(tenant, tableName string) *DB

ShallowCloneForTable creates a lightweight copy of the database that deep-copies only the specified table and shares all others by reference. This is safe when the caller knows only the target table will be mutated (single-statement DML). For a database with many tables, this is dramatically cheaper than DeepClone — O(rows in target table) instead of O(rows in all tables).

func (*DB) StorageMode added in v0.5.4 

func (db *DB) StorageMode() StorageMode

StorageMode returns the active storage mode.

func (*DB) Sync added in v0.5.4 

func (db *DB) Sync() error

Sync flushes all dirty in-memory tables to the storage backend. For ModeMemory and ModeWAL this is a no-op (those modes use SaveToFile / WAL checkpoints respectively). For ModeDisk, ModeHybrid, and ModeIndex, tables whose version has changed since the last save are written to disk.

func (*DB) SyncTable added in v0.5.4 

func (db *DB) SyncTable(tenant string, t *Table) error

SyncTable flushes a single table to the backend. This is called by the engine after mutations when SyncOnMutate is enabled.

func (*DB) TableExists added in v0.5.4 

func (db *DB) TableExists(tenant, name string) bool

TableExists reports whether the named table exists, checking both in-memory tables and the storage backend.

func (*DB) WAL 

func (db *DB) WAL() *WALManager

WAL returns the configured WAL manager (may be nil).

type DiskBackend added in v0.5.4 

type DiskBackend struct {
	// contains filtered or unexported fields
}

DiskBackend stores each table as an individual file under a directory tree: 

<dir>/<tenant>/<tablename>.tbl      (GOB-encoded diskTable)
<dir>/manifest.json                 (lightweight metadata index)

Tables are loaded into memory on demand (via LoadTable) and written back on SaveTable / Sync. This minimises RAM usage while keeping the file format compatible with the existing GOB serialisation.

func NewDiskBackend added in v0.5.4 

func NewDiskBackend(dir string, compress bool) (*DiskBackend, error)

NewDiskBackend opens (or creates) a disk-backed storage directory. It reads the manifest to learn which tables exist without loading them.

func (*DiskBackend) Close added in v0.5.4 

func (b *DiskBackend) Close() error

Close is like Sync but also cleans up any resources.

func (*DiskBackend) DeleteTable added in v0.5.4 

func (b *DiskBackend) DeleteTable(tenant, name string) error

DeleteTable removes a table file from disk and updates the manifest.

func (*DiskBackend) GetMeta added in v0.5.4 

func (b *DiskBackend) GetMeta(tenant, name string) *TableMeta

GetMeta returns table metadata without loading the full table. Returns nil if the table is unknown.

func (*DiskBackend) ImportFromDB added in v0.5.4 

func (b *DiskBackend) ImportFromDB(db *DB) error

ImportFromDB takes all tables from an existing in-memory DB and writes them as individual table files. This is the migration path from ModeMemory / GOB checkpoints to ModeDisk.

func (*DiskBackend) IsDirty added in v0.5.4 

func (b *DiskBackend) IsDirty(tenant, name string, currentVersion int) bool

IsDirty reports whether the named table has been modified since it was last saved. Uses the version field for comparison.

func (*DiskBackend) ListTableNames added in v0.5.4 

func (b *DiskBackend) ListTableNames(tenant string) ([]string, error)

ListTableNames returns all known table names for a tenant (on disk or in memory).

func (*DiskBackend) LoadTable added in v0.5.4 

func (b *DiskBackend) LoadTable(tenant, name string) (*Table, error)

LoadTable reads a table from its GOB file on disk. Returns (nil, nil) if the table does not exist.

func (*DiskBackend) Mode added in v0.5.4 

func (b *DiskBackend) Mode() StorageMode

func (*DiskBackend) SaveTable added in v0.5.4 

func (b *DiskBackend) SaveTable(tenant string, t *Table) error

SaveTable writes a table to its GOB file on disk and updates the manifest.

func (*DiskBackend) Stats added in v0.5.4 

func (b *DiskBackend) Stats() BackendStats

func (*DiskBackend) Sync added in v0.5.4 

func (b *DiskBackend) Sync() error

Sync writes the manifest (individual table files are written immediately by SaveTable). The caller should call SaveTable for each dirty table before calling Sync, or use DB.Sync which does this automatically.

func (*DiskBackend) TableExists added in v0.5.4 

func (b *DiskBackend) TableExists(tenant, name string) bool

TableExists reports whether the table is known to the backend.

type ForeignKeyRef 

type ForeignKeyRef struct {
	Table  string
	Column string
}

ForeignKeyRef describes a foreign key reference target.

type HybridBackend added in v0.5.4 

type HybridBackend struct {
	// contains filtered or unexported fields
}

HybridBackend keeps frequently-accessed tables in an LRU memory cache backed by individual table files on disk. When the cache exceeds the configured memory limit, cold tables are evicted. This gives near in- memory performance for the working set while allowing the full database to exceed available RAM.

For ModeIndex, the same structure is used with a much smaller memory limit and aggressive eviction – essentially only table schemas (loaded as metadata from the manifest) stay resident.

func NewHybridBackend added in v0.5.4 

func NewHybridBackend(dir string, maxMemoryBytes int64, compress bool, mode StorageMode) (*HybridBackend, error)

NewHybridBackend creates a HybridBackend.

  • dir: directory for table files
  • maxMemoryBytes: memory limit (0 = 256 MB default)
  • compress: gzip table files
  • mode: ModeHybrid or ModeIndex

func (*HybridBackend) Close added in v0.5.4 

func (h *HybridBackend) Close() error

Close flushes dirty tables and closes the disk backend.

func (*HybridBackend) DeleteTable added in v0.5.4 

func (h *HybridBackend) DeleteTable(tenant, name string) error

DeleteTable removes from both disk and cache.

func (*HybridBackend) Disk added in v0.5.4 

func (h *HybridBackend) Disk() *DiskBackend

Disk returns the underlying DiskBackend for advanced operations (e.g. migration, metadata access).

func (*HybridBackend) ListTableNames added in v0.5.4 

func (h *HybridBackend) ListTableNames(tenant string) ([]string, error)

ListTableNames returns names from the disk manifest (authoritative source).

func (*HybridBackend) LoadTable added in v0.5.4 

func (h *HybridBackend) LoadTable(tenant, name string) (*Table, error)

LoadTable tries the in-memory cache first, then loads from disk.

func (*HybridBackend) MarkDirty added in v0.5.4 

func (h *HybridBackend) MarkDirty(tenant, name string)

MarkDirty records that a table has been modified in memory and needs to be flushed on the next Sync. Called by DB after mutations.

func (*HybridBackend) Mode added in v0.5.4 

func (h *HybridBackend) Mode() StorageMode

func (*HybridBackend) SaveTable added in v0.5.4 

func (h *HybridBackend) SaveTable(tenant string, t *Table) error

SaveTable writes to disk and updates the cache.

func (*HybridBackend) Stats added in v0.5.4 

func (h *HybridBackend) Stats() BackendStats

func (*HybridBackend) Sync added in v0.5.4 

func (h *HybridBackend) Sync() error

Sync writes all dirty in-memory tables to disk.

func (*HybridBackend) TableExists added in v0.5.4 

func (h *HybridBackend) TableExists(tenant, name string) bool

TableExists checks the disk manifest.

type IsolationLevel 

type IsolationLevel uint8

IsolationLevel defines transaction isolation semantics. 

const (
	ReadCommitted IsolationLevel = iota
	RepeatableRead
	SnapshotIsolation
	Serializable
)

type JobExecutor added in v0.5.1 

type JobExecutor interface {
	ExecuteSQL(ctx context.Context, sql string) (interface{}, error)
}

JobExecutor interface allows the scheduler to execute SQL without circular dependencies

type LRUNode 

type LRUNode struct {
	// contains filtered or unexported fields
}

LRUNode is a node in the LRU doubly-linked list.

type LRUQueue 

type LRUQueue struct {
	// contains filtered or unexported fields
}

LRUQueue implements least-recently-used eviction.

func NewLRUQueue 

func NewLRUQueue() *LRUQueue

NewLRUQueue creates an empty LRU queue.

func (*LRUQueue) Access 

func (lru *LRUQueue) Access(key string, table *CachedTable)

Access moves a node to the front (most recent).

func (*LRUQueue) Add 

func (lru *LRUQueue) Add(key string, table *CachedTable)

Add adds a node to the LRU queue (most recent).

func (*LRUQueue) Remove 

func (lru *LRUQueue) Remove(key string)

Remove removes a node from the queue.

func (*LRUQueue) RemoveLRU 

func (lru *LRUQueue) RemoveLRU() (string, *CachedTable)

RemoveLRU removes and returns the least recently used node.

type LSN 

type LSN uint64

LSN (Log Sequence Number) provides total ordering of log records.

type MVCCManager 

type MVCCManager struct {
	// contains filtered or unexported fields
}

MVCCManager coordinates transaction IDs, timestamps, and visibility.

func NewMVCCManager 

func NewMVCCManager() *MVCCManager

NewMVCCManager creates a new MVCC coordinator.

func (*MVCCManager) AbortTx 

func (m *MVCCManager) AbortTx(tx *TxContext)

AbortTx marks a transaction as aborted.

func (*MVCCManager) BeginTx 

func (m *MVCCManager) BeginTx(level IsolationLevel) *TxContext

BeginTx starts a new transaction and returns its context.

func (*MVCCManager) CommitTx 

func (m *MVCCManager) CommitTx(tx *TxContext) (Timestamp, error)

CommitTx marks a transaction as committed and records its commit timestamp.

func (*MVCCManager) GCWatermark 

func (m *MVCCManager) GCWatermark() Timestamp

GCWatermark returns the timestamp before which row versions can be garbage collected.

func (*MVCCManager) IsVisible 

func (m *MVCCManager) IsVisible(tx *TxContext, rv *RowVersion) bool

IsVisible determines if a row version is visible to a transaction.

type MVCCTable 

type MVCCTable struct {
	*Table
	// contains filtered or unexported fields
}

MVCCTable extends Table with version chains.

func NewMVCCTable 

func NewMVCCTable(name string, cols []Column, isTemp bool) *MVCCTable

NewMVCCTable creates a table with MVCC support.

func (*MVCCTable) DeleteVersion 

func (t *MVCCTable) DeleteVersion(tx *TxContext, rowID int64) error

DeleteVersion marks a row version as deleted.

func (*MVCCTable) GarbageCollect 

func (t *MVCCTable) GarbageCollect(watermark Timestamp) int

GarbageCollect removes old row versions that are no longer visible.

func (*MVCCTable) GetVisibleVersion 

func (t *MVCCTable) GetVisibleVersion(mvcc *MVCCManager, tx *TxContext, rowID int64) *RowVersion

GetVisibleVersion returns the visible version of a row for the given transaction.

func (*MVCCTable) InsertVersion 

func (t *MVCCTable) InsertVersion(tx *TxContext, data []any) int64

InsertVersion adds a new row version.

func (*MVCCTable) UpdateVersion 

func (t *MVCCTable) UpdateVersion(tx *TxContext, rowID int64, newData []any) error

UpdateVersion creates a new version for an update.

type MemoryBackend added in v0.5.4 

type MemoryBackend struct {
	// contains filtered or unexported fields
}

MemoryBackend is a no-op backend used by ModeMemory. All data lives exclusively in the DB's in-memory tenants map; the backend simply reports that tables don't exist on any backing store (so the DB never tries to load from disk). Save/Sync are intentional no-ops – persistence is the caller's responsibility (via SaveToFile or the WAL).

func NewMemoryBackend added in v0.5.4 

func NewMemoryBackend(savePath string) *MemoryBackend

NewMemoryBackend creates a MemoryBackend. If savePath is non-empty, Close performs a final SaveToFile to that path.

func (*MemoryBackend) Close added in v0.5.4 

func (m *MemoryBackend) Close() error

func (*MemoryBackend) DeleteTable added in v0.5.4 

func (m *MemoryBackend) DeleteTable(_, _ string) error

func (*MemoryBackend) ListTableNames added in v0.5.4 

func (m *MemoryBackend) ListTableNames(_ string) ([]string, error)

func (*MemoryBackend) LoadTable added in v0.5.4 

func (m *MemoryBackend) LoadTable(_, _ string) (*Table, error)

func (*MemoryBackend) Mode added in v0.5.4 

func (m *MemoryBackend) Mode() StorageMode

func (*MemoryBackend) SaveTable added in v0.5.4 

func (m *MemoryBackend) SaveTable(_ string, _ *Table) error

func (*MemoryBackend) Stats added in v0.5.4 

func (m *MemoryBackend) Stats() BackendStats

func (*MemoryBackend) Sync added in v0.5.4 

func (m *MemoryBackend) Sync() error

func (*MemoryBackend) TableExists added in v0.5.4 

func (m *MemoryBackend) TableExists(_, _ string) bool

type MemoryPolicy 

type MemoryPolicy struct {
	// Maximum memory usage in bytes (0 = unlimited)
	MaxMemoryBytes int64

	// Cache eviction strategy
	Strategy CacheStrategy

	// Start evicting when memory usage exceeds this ratio (0.0-1.0)
	EvictionThreshold float64

	// Tables that should always stay in memory
	PinnedTables []string

	// Tables that should never be cached
	IgnoreTables []string

	// Enable eviction (if false, OOM when limit reached)
	EnableEviction bool

	// Number of tables to evict in one batch
	EvictionBatchSize int

	// Track access patterns for better eviction decisions
	TrackAccessPatterns bool

	// Time window for access tracking
	AccessWindow time.Duration
}

MemoryPolicy defines memory management configuration.

func DefaultMemoryPolicy 

func DefaultMemoryPolicy() *MemoryPolicy

DefaultMemoryPolicy returns a sensible default configuration.

func LimitedMemoryPolicy 

func LimitedMemoryPolicy(maxMB int64) *MemoryPolicy

LimitedMemoryPolicy returns a policy with memory limits.

type ParallelIterator 

type ParallelIterator struct {
	// contains filtered or unexported fields
}

ParallelIterator provides concurrent iteration over data.

func NewParallelIterator 

func NewParallelIterator(items []interface{}, workers int) *ParallelIterator

NewParallelIterator creates a parallel iterator.

func (*ParallelIterator) ForEach 

func (pi *ParallelIterator) ForEach(fn func(item interface{}) error) error

ForEach processes items in parallel.

func (*ParallelIterator) Map 

func (pi *ParallelIterator) Map(fn func(item interface{}) (interface{}, error)) ([]interface{}, error)

Map applies a function to all items in parallel and returns results.

func (*ParallelIterator) WithContext 

func (pi *ParallelIterator) WithContext(ctx context.Context) *ParallelIterator

WithContext sets the context for the iterator.

type Pipeline 

type Pipeline struct {
	// contains filtered or unexported fields
}

Pipeline implements a concurrent pipeline pattern.

func NewPipeline 

func NewPipeline(ctx context.Context, stages ...PipelineStage) *Pipeline

NewPipeline creates a new pipeline.

func (*Pipeline) Execute 

func (p *Pipeline) Execute(input []interface{}) <-chan interface{}

Execute runs the pipeline.

type PipelineStage 

type PipelineStage func(ctx context.Context, input <-chan interface{}) <-chan interface{}

PipelineStage represents a stage in the pipeline.

type RateLimiter 

type RateLimiter struct {
	// contains filtered or unexported fields
}

RateLimiter limits the rate of operations.

func NewRateLimiter 

func NewRateLimiter(opsPerSecond int) *RateLimiter

NewRateLimiter creates a new rate limiter.

func (*RateLimiter) Stop 

func (rl *RateLimiter) Stop()

Stop stops the rate limiter.

func (*RateLimiter) Wait 

func (rl *RateLimiter) Wait(ctx context.Context) error

Wait blocks until a token is available.

type RowVersion 

type RowVersion struct {
	// Transaction that created this version
	XMin TxID

	// Transaction that deleted/updated this version (0 if still valid)
	XMax TxID

	// Creation timestamp
	CreatedAt Timestamp

	// Deletion/update timestamp (0 if still valid)
	DeletedAt Timestamp

	// Actual row data
	Data []any

	// Pointer to next version (for version chain)
	NextVersion *RowVersion
}

RowVersion contains MVCC metadata for a single row version.

type Scheduler added in v0.5.1 

type Scheduler struct {
	// contains filtered or unexported fields
}

Scheduler manages scheduled job execution

func NewScheduler added in v0.5.1 

func NewScheduler(db *DB, executor JobExecutor) *Scheduler

NewScheduler creates a new job scheduler

func (*Scheduler) AddJob added in v0.5.1 

func (s *Scheduler) AddJob(job *CatalogJob) error

AddJob registers a new job and schedules it immediately if enabled

func (*Scheduler) RemoveJob added in v0.5.1 

func (s *Scheduler) RemoveJob(name string) error

RemoveJob unregisters a job and stops its execution

func (*Scheduler) Start added in v0.5.1 

func (s *Scheduler) Start() error

Start begins the scheduler loop

func (*Scheduler) Stop added in v0.5.1 

func (s *Scheduler) Stop()

Stop halts the scheduler and cancels all running jobs

type StorageBackend added in v0.5.4 

type StorageBackend interface {
	// LoadTable retrieves a table from the backing store. It returns nil, nil
	// when the table does not exist (not an error – the table may simply not
	// have been created yet).
	LoadTable(tenant, name string) (*Table, error)

	// SaveTable persists a single table to the backing store.
	SaveTable(tenant string, t *Table) error

	// DeleteTable removes a table from the backing store.
	DeleteTable(tenant, name string) error

	// ListTableNames returns all table names for a tenant that exist in the
	// backing store (including tables not currently loaded in memory).
	ListTableNames(tenant string) ([]string, error)

	// TableExists reports whether the named table exists in the backing store
	// without loading it into memory.
	TableExists(tenant, name string) bool

	// Sync flushes any pending / dirty data to durable storage.
	Sync() error

	// Close releases all resources. Implementations should call Sync
	// internally if there is unsaved data.
	Close() error

	// Mode returns the StorageMode this backend implements.
	Mode() StorageMode

	// Stats returns operational statistics about the backend.
	Stats() BackendStats
}

StorageBackend abstracts the underlying table storage mechanism.

Implementations are responsible for loading tables into *Table structs that the engine can mutate directly. After mutations, Sync writes dirty tables back to durable storage. Close persists pending data and releases resources.

type StorageConfig added in v0.5.4 

type StorageConfig struct {
	// Mode selects the storage strategy. Defaults to ModeMemory.
	Mode StorageMode

	// Path is the root directory (or file path) for persistent storage.
	// Required for all modes except ModeMemory.
	Path string

	// MaxMemoryBytes limits in-memory table data. Used by ModeHybrid and
	// ModeIndex. Zero means use a sensible default (256 MB).
	MaxMemoryBytes int64

	// SyncOnMutate forces a disk write after every INSERT / UPDATE / DELETE.
	// Slower but provides immediate durability for ModeDisk / ModeHybrid.
	SyncOnMutate bool

	// CompressFiles enables gzip compression for table files on disk.
	CompressFiles bool

	// CheckpointEvery controls how many committed WAL transactions trigger
	// an automatic checkpoint (ModeWAL only). Zero means default (32).
	CheckpointEvery uint64

	// CheckpointInterval controls the maximum time between checkpoints
	// (ModeWAL only). Zero means default (30 s).
	CheckpointInterval time.Duration
}

StorageConfig configures database storage behaviour.

func DefaultStorageConfig added in v0.5.4 

func DefaultStorageConfig(mode StorageMode) StorageConfig

DefaultStorageConfig returns a StorageConfig with sensible defaults for the given mode. Path must be supplied by the caller afterwards.

type StorageMode added in v0.5.4 

type StorageMode int

StorageMode defines how the database manages data between memory and disk. 

const (
	// ModeMemory keeps all data in RAM. Persistence only occurs via explicit
	// SaveToFile calls or when DB.Close is invoked. Fastest mode.
	ModeMemory StorageMode = iota

	// ModeWAL keeps all data in RAM and writes a Write-Ahead Log for crash
	// recovery. Periodic checkpoints create full GOB snapshots. Good
	// balance of speed and durability.
	ModeWAL

	// ModeDisk stores each table as a separate GOB file on disk. Tables
	// are loaded into memory on demand and flushed back on Sync/Close.
	// Minimises RAM usage at the cost of disk I/O.
	ModeDisk

	// ModeIndex keeps table schemas (columns, row-counts) permanently in
	// RAM while row data resides on disk. Rows are loaded on demand and
	// evicted aggressively. Memory usage is proportional to schema size,
	// not data size.
	ModeIndex

	// ModeHybrid uses an LRU buffer pool with a configurable memory limit.
	// Hot tables stay in RAM; cold tables spill to disk. Best for mixed
	// workloads where the working set fits in a bounded amount of memory.
	ModeHybrid
)

func ParseStorageMode added in v0.5.4 

func ParseStorageMode(s string) (StorageMode, error)

ParseStorageMode converts a string representation back to a StorageMode. It is case-insensitive and returns an error for unknown values.

func (StorageMode) String added in v0.5.4 

func (m StorageMode) String() string

String returns a human-readable label for the StorageMode.

type Table 

type Table struct {
	Name   string
	Cols   []Column
	Rows   [][]any
	IsTemp bool

	Version int
	// contains filtered or unexported fields
}

Table stores rows along with column metadata and indexes.

func NewTable 

func NewTable(name string, cols []Column, isTemp bool) *Table

NewTable creates a new Table with case-insensitive column lookup indices.

func (*Table) ColIndex 

func (t *Table) ColIndex(name string) (int, error)

ColIndex returns the zero-based index of the named column.

func (*Table) DirtyFrom added in v0.5.4 

func (t *Table) DirtyFrom() int

DirtyFrom returns the first dirty row index, or -1 if non-append-only.

func (*Table) MarkDirtyFrom added in v0.5.4 

func (t *Table) MarkDirtyFrom(idx int)

MarkDirtyFrom records the first row index that was modified. If an earlier index is already set, it is kept. Use -1 for non-append mutations (UPDATE, DELETE) to force a full-table WAL entry.

func (*Table) ResetDirty added in v0.5.4 

func (t *Table) ResetDirty()

ResetDirty marks the table as clean (called after WAL checkpoint).

type TableMeta added in v0.5.4 

type TableMeta struct {
	Tenant   string   `json:"tenant"`
	Name     string   `json:"name"`
	Cols     []Column `json:"cols"`
	RowCount int      `json:"row_count"`
	Version  int      `json:"version"`
	// DiskSize is the file size in bytes on the backing store.
	DiskSize int64 `json:"disk_size"`
	// FilePath is the relative path inside the database directory.
	FilePath string `json:"file_path"`
}

TableMeta stores lightweight metadata for a table that is (potentially) on disk. The disk backend uses this to answer ListTableNames, TableExists, and schema-level queries without loading row data.

type Timestamp 

type Timestamp uint64

Timestamp represents a logical timestamp for MVCC visibility.

type TxContext 

type TxContext struct {
	ID             TxID
	StartTime      Timestamp
	Status         TxStatus
	ReadSnapshot   Timestamp                      // Snapshot timestamp for reads
	WriteSet       map[string]map[int64]bool      // table -> row IDs modified
	ReadSet        map[string]map[int64]Timestamp // table -> row IDs read with version
	IsolationLevel IsolationLevel
	// contains filtered or unexported fields
}

TxContext holds the state of an active transaction.

func (*TxContext) RecordRead 

func (tx *TxContext) RecordRead(table string, rowID int64, version Timestamp)

RecordRead tracks a read operation for conflict detection.

func (*TxContext) RecordWrite 

func (tx *TxContext) RecordWrite(table string, rowID int64)

RecordWrite tracks a write operation.

type TxID 

type TxID uint64

TxID represents a unique transaction identifier.

type TxStatus 

type TxStatus uint8

TxStatus represents the current state of a transaction. 

const (
	TxStatusInProgress TxStatus = iota
	TxStatusCommitted
	TxStatusAborted
)

type WALChange 

type WALChange struct {
	Tenant string
	Name   string
	Table  *Table
	Drop   bool
}

WALChange describes a persistent change that will be written to the WAL.

func CollectWALChanges 

func CollectWALChanges(prev, next *DB) []WALChange

CollectWALChanges computes the delta between two MVCC snapshots.

type WALConfig 

type WALConfig struct {
	Path               string
	CheckpointEvery    uint64
	CheckpointInterval time.Duration
}

WALConfig configures WAL and checkpoint behavior.

type WALManager 

type WALManager struct {
	// contains filtered or unexported fields
}

WALManager encapsulates WAL append, recovery, and checkpoints.

func OpenWAL 

func OpenWAL(db *DB, cfg WALConfig) (*WALManager, error)

ready-to-use manager. It attaches no WAL when Path is empty.

func (*WALManager) Checkpoint 

func (w *WALManager) Checkpoint(db *DB) error

Checkpoint writes a DB snapshot and resets the WAL file.

func (*WALManager) Close 

func (w *WALManager) Close() error

Close flushes, syncs, and closes the WAL resources.

func (*WALManager) LogTransaction 

func (w *WALManager) LogTransaction(changes []WALChange) (bool, error)

LogTransaction appends all changes atomically to the WAL. It returns true when a checkpoint is recommended.

type WALOperationType 

type WALOperationType uint8

WALOperationType defines the type of WAL operation. 

const (
	WALOpBegin WALOperationType = iota + 1
	WALOpInsert
	WALOpUpdate
	WALOpDelete
	WALOpCommit
	WALOpAbort
	WALOpCheckpoint
)

func (WALOperationType) String 

func (t WALOperationType) String() string

type WALRecord 

type WALRecord struct {
	// Log Sequence Number - globally unique, monotonically increasing
	LSN LSN

	// Transaction ID
	TxID TxID

	// Operation type
	OpType WALOperationType

	// Tenant and table
	Tenant string
	Table  string

	// Row ID (for row-level operations)
	RowID int64

	// UNDO image (before state) - for rollback
	BeforeImage []any

	// REDO image (after state) - for recovery
	AfterImage []any

	// Column information (for schema tracking)
	Columns []Column

	// Timestamp
	Timestamp time.Time

	// Checksum for corruption detection
	Checksum uint32
}

WALRecord represents a single log entry with before/after images.

type WALTxState 

type WALTxState struct {
	TxID       TxID
	StartLSN   LSN
	Operations []LSN
	Status     TxStatus
}

WALTxState tracks the state of a transaction in the WAL.

type WorkHandler 

type WorkHandler func(ctx context.Context, req WorkRequest) WorkResult

WorkHandler processes work requests.

type WorkRequest 

type WorkRequest struct {
	ID      uint64
	Context context.Context
	Type    WorkType
	Data    interface{}
	Result  chan WorkResult
}

WorkRequest represents a unit of work to be processed.

type WorkResult 

type WorkResult struct {
	ID    uint64
	Data  interface{}
	Error error
}

WorkResult contains the result of a work request.

type WorkType 

type WorkType uint8

WorkType defines the type of operation. 

const (
	WorkTypeRead WorkType = iota
	WorkTypeWrite
	WorkTypeDelete
	WorkTypeScan
	WorkTypeBatch
)

type WorkerPool 

type WorkerPool struct {
	// contains filtered or unexported fields
}

WorkerPool manages a pool of worker goroutines.

func NewWorkerPool 

func NewWorkerPool(name string, size int, workQueue chan WorkRequest, handler WorkHandler, ctx context.Context, wg *sync.WaitGroup) *WorkerPool

NewWorkerPool creates a new worker pool.

func (*WorkerPool) Start 

func (wp *WorkerPool) Start()

Start launches all worker goroutines.

Source Files

View all Source files

Directories

Path Synopsis
Package pager — PageBackend integrates the page-based storage engine with the tinySQL StorageBackend interface.
 Package pager — PageBackend integrates the page-based storage engine with the tinySQL StorageBackend interface.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.