storage

package
v0.17.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 9, 2026 License: AGPL-3.0 Imports: 34 Imported by: 0

Documentation

Overview

Package storage: tamper-evident audit log.

What: an append-only log of every statement Execute runs (see AttachAuditLog/DB.AuditLog and internal/engine/audit.go for the hook), where each entry's hash covers its own fields plus the previous entry's hash — a hash chain, the same integrity primitive used by blockchains and git commit history. Altering, reordering, or deleting any entry breaks the chain from that point forward in a way Verify detects.

Why this matters for the intended use cases (industry compliance, scientific reproducibility, security-sensitive deployments): a plain log file can be edited by anyone with filesystem access, silently. A hash chain doesn't prevent that access, but it makes silent editing detectable — Verify recomputes every hash and reports exactly where the chain first diverges from what's recorded, which is what audit/compliance tooling actually needs (proof of tampering, not prevention of it — true tamper-*prevention* needs signing with a key the log's own host doesn't hold, which is out of scope for an embedded database).

Package storage provides persistence primitives and metadata structures used by tinySQL. This file implements a lightweight in-memory system catalog used for introspection (tables, columns, views, functions) and simple job scheduling metadata.

Package storage provides the durable data structures for tinySQL.

What: An in-memory multi-tenant catalog of tables with column metadata, rows, and basic typing. It includes snapshot cloning for MVCC-light, GOB-based checkpoints, and an append-only Write-Ahead Log (WAL) for crash recovery and durability. How: Tables store rows as [][]any for compactness; a lower-cased column index accelerates name lookups. Save/Load serialize the catalog to a file, writing JSON for JSON columns. The WAL logs whole-table changes and drops; recovery replays committed records and truncates partial tails. Why: Favor a simple, explicit model over complex page managers: it keeps the code understandable, testable, and sufficient for embedded/edge use cases.

Package storage: encryption at rest for disk-based backends.

What: AES-256-GCM authenticated encryption of table files (ModeDisk, ModeJSON) and the backend catalog file. GCM was chosen over a plain block cipher mode specifically because it's authenticated — decrypting with the wrong key, or decrypting data that was altered after being written, fails closed with an error instead of silently returning garbage or (worse) subtly-wrong-but-plausible data. That gives encryption at rest a side benefit of the same tamper-detection property the audit log provides for statement history, applied to the data files themselves.

Why Argon2id for passphrase-based keys: PBKDF2 (also available in golang.org/x/crypto) is checkable purely with CPU, which makes brute-force guessing cheap on GPUs/ASICs at scale. Argon2id is memory-hard — an attacker also needs the configured amount of RAM per guess, which is the current best-practice recommendation (OWASP, and the Password Hashing Competition it won) for deriving a key from a human-chosen passphrase.

Package storage: role-based access control (RBAC).

What: Users authenticate with a bcrypt-hashed password and are members of one or more roles; roles carry grants (a permission on a schema/table pair, with "*" as a wildcard for "all schemas"/"all tables"). Execute() in internal/engine checks the acting user's resolved permissions before running a statement.

Why opt-in: tinySQL has always been usable with zero setup (NewDB(), start running SQL). Requiring every caller to create a user before their first query would break that and every existing embedder. So RBAC enforcement only activates once at least one user has been created via CreateUser — see HasUsers/IsRBACEnabled. Before that, every permission check is a no-op allow, matching tinySQL's historical behavior exactly. This mirrors the same opt-in pattern already used for read-only mode (SetReadOnly) and cmd/server's optional auth token.

Package storage – StorageBackend interface and StorageMode definitions.

What: Pluggable storage backends that decouple data management from the in-memory catalog. Each backend decides where table data lives (RAM, disk, or a combination) and how it is persisted. How: The DB struct optionally delegates Get/Put/Drop operations to an attached StorageBackend. Backends may lazily load tables, cache hot data, and flush dirty tables on Sync/Close. Why: Supporting multiple storage modes turns tinySQL into a realistic alternative to SQLite – from purely in-memory analytics right through to disk-resident databases that exceed available RAM.

Index

Constants

View Source 
const EncryptionKeySize = 32

EncryptionKeySize is the required AES-256 key size in bytes.

View Source 
const EncryptionSaltSize = 16

EncryptionSaltSize is the recommended salt size for DeriveKeyFromPassphrase.

Variables

View Source 
var (
	ErrTxNotActive          = fmt.Errorf("transaction is not active")
	ErrSerializationFailure = fmt.Errorf("could not serialize access due to concurrent update")
	ErrRowNotFound          = fmt.Errorf("row not found")
)

Errors

Functions

func AsBigRat added in v0.6.0 

func AsBigRat(v any) (*big.Rat, bool)

AsBigRat returns the value as *big.Rat if it is already a rational type (either *big.Rat or big.Rat). Unlike DecimalFromAny this does not coerce ints/floats/strings and is suitable for preserving existing numeric semantics in the engine.

func DecimalAdd added in v0.6.0 

func DecimalAdd(a, b any) (*big.Rat, error)

DecimalAdd returns the sum of two decimal-like values as *big.Rat.

func DecimalFromAny added in v0.6.0 

func DecimalFromAny(v any) (*big.Rat, bool)

DecimalFromAny attempts to convert a value to *big.Rat.

func DecimalToString added in v0.6.0 

func DecimalToString(r *big.Rat) string

DecimalToString returns a plain decimal string representation.

func DeriveKeyFromPassphrase added in v0.13.0 

func DeriveKeyFromPassphrase(passphrase string, salt []byte) []byte

DeriveKeyFromPassphrase derives a 32-byte AES-256 key from a passphrase using Argon2id. salt must be a fixed, persisted value of at least EncryptionSaltSize bytes (see NewEncryptionSalt) — reusing a random salt on every call would derive a different key each time and permanently lock the caller out of data encrypted under the previous key.

The cost parameters (time=1, memory=64 MiB, threads=4) follow OWASP's Argon2id baseline recommendation: expensive enough to resist large-scale offline guessing, cheap enough to run once at database-open time rather than per-operation.

func EstimateColumnSize 

func EstimateColumnSize(typ ColType) int64

EstimateColumnSize estimates the average size of a column type.

func EstimateTableSize 

func EstimateTableSize(t *Table) int64

EstimateTableSize estimates memory usage of a table in bytes.

func EstimateValueSize 

func EstimateValueSize(val any) int64

EstimateValueSize estimates the memory size of a value.

func FanIn 

func FanIn(ctx context.Context, channels ...<-chan interface{}) <-chan interface{}

FanIn combines multiple channels into one.

func FanOut 

func FanOut(ctx context.Context, input <-chan interface{}, workers int) []<-chan interface{}

FanOut distributes work from one channel to multiple channels.

func GenerateRandomPassword added in v0.12.0 

func GenerateRandomPassword(n int) (string, error)

GenerateRandomPassword returns a cryptographically random hex-encoded password of the given byte length (before hex encoding, so the resulting string is 2*n characters) — a convenience for callers provisioning service accounts that don't need a human-memorable password.

func JSONMarshal added in v0.6.0 

func JSONMarshal(v any) ([]byte, error)

JSONMarshal marshals v after converting internal types to JSON-friendly representations (e.g. big.Rat -> string).

func NewEncryptionSalt added in v0.13.0 

func NewEncryptionSalt() ([]byte, error)

NewEncryptionSalt returns a fresh, cryptographically random salt sized for DeriveKeyFromPassphrase. Persist it (e.g. in the backend's manifest) — the same salt must be supplied on every subsequent open for the same passphrase to derive the same key.

func ParseUUID added in v0.6.0 

func ParseUUID(s string) (uuid.UUID, error)

ParseUUID parses a UUID string into uuid.UUID.

func SaveToBytes 

func SaveToBytes(db *DB) ([]byte, error)

SaveToBytes serializes the database snapshot to a byte slice.

func SaveToFile 

func SaveToFile(db *DB, filename string) error

SaveToFile writes a snapshot of the database to a file. If the filename ends with .gz, the snapshot is gzip-compressed to reduce size.

The snapshot is written atomically: data goes to a temporary file in the same directory, is fsynced, and is then renamed over the target. A crash mid-checkpoint therefore never corrupts or truncates the previous snapshot.

func SaveToWriter 

func SaveToWriter(db *DB, w io.Writer) error

SaveToWriter writes a snapshot of the database to an arbitrary writer. It does not attach or alter WAL configuration.

func UUIDToBytes added in v0.6.0 

func UUIDToBytes(u uuid.UUID) []byte

UUIDToBytes returns the 16-byte representation of a uuid.UUID.

Types

type AdvancedWAL 

type AdvancedWAL struct {
	// contains filtered or unexported fields
}

AdvancedWAL manages row-level write-ahead logging with full ACID guarantees.

func OpenAdvancedWAL 

func OpenAdvancedWAL(config AdvancedWALConfig) (*AdvancedWAL, error)

OpenAdvancedWAL creates or opens a WAL with full ACID semantics.

func (*AdvancedWAL) Checkpoint 

func (w *AdvancedWAL) Checkpoint(db *DB) error

Checkpoint creates a consistent snapshot and truncates the WAL.

func (*AdvancedWAL) Close 

func (w *AdvancedWAL) Close() error

Close flushes and closes the WAL.

func (*AdvancedWAL) GetCommittedLSN 

func (w *AdvancedWAL) GetCommittedLSN() LSN

GetCommittedLSN returns the LSN of the last committed transaction.

func (*AdvancedWAL) GetFlushedLSN 

func (w *AdvancedWAL) GetFlushedLSN() LSN

GetFlushedLSN returns the LSN of the last flushed record.

func (*AdvancedWAL) GetNextLSN 

func (w *AdvancedWAL) GetNextLSN() LSN

GetNextLSN returns the next LSN to be assigned.

func (*AdvancedWAL) LogAbort 

func (w *AdvancedWAL) LogAbort(txID TxID) (LSN, error)

LogAbort logs a transaction abort.

func (*AdvancedWAL) LogBegin 

func (w *AdvancedWAL) LogBegin(txID TxID) (LSN, error)

LogBegin logs the start of a transaction.

func (*AdvancedWAL) LogCommit 

func (w *AdvancedWAL) LogCommit(txID TxID) (LSN, error)

LogCommit logs a transaction commit.

func (*AdvancedWAL) LogDelete 

func (w *AdvancedWAL) LogDelete(txID TxID, tenant, table string, rowID int64, before []any, cols []Column) (LSN, error)

LogDelete logs a row deletion.

func (*AdvancedWAL) LogInsert 

func (w *AdvancedWAL) LogInsert(txID TxID, tenant, table string, rowID int64, data []any, cols []Column) (LSN, error)

LogInsert logs a row insertion.

func (*AdvancedWAL) LogUpdate 

func (w *AdvancedWAL) LogUpdate(txID TxID, tenant, table string, rowID int64, before, after []any, cols []Column) (LSN, error)

LogUpdate logs a row update with before/after images.

func (*AdvancedWAL) NewAutoTxID added in v0.12.0 

func (w *AdvancedWAL) NewAutoTxID() TxID

NewAutoTxID returns a fresh transaction ID for the engine to use as an implicit single-statement transaction (one INSERT/UPDATE/DELETE statement = one WAL transaction, autocommitted). Safe for concurrent use.

func (*AdvancedWAL) Recover 

func (w *AdvancedWAL) Recover(db *DB) (int, error)

Recover replays the WAL to restore database state after a crash.

func (*AdvancedWAL) ShouldCheckpoint 

func (w *AdvancedWAL) ShouldCheckpoint() bool

ShouldCheckpoint checks if a checkpoint is needed.

type AdvancedWALConfig 

type AdvancedWALConfig struct {
	Path               string
	CheckpointPath     string
	CheckpointEvery    uint64        // Checkpoint after N records
	CheckpointInterval time.Duration // Checkpoint after duration
	CheckpointMaxBytes int64         // Checkpoint once WAL exceeds this size (0 = 64 MB default, <0 disables)
	Compress           bool
	BufferSize         int // Buffer size for writing
}

AdvancedWALConfig configures the advanced WAL.

type AuditEntry added in v0.13.0 

type AuditEntry struct {
	Seq       uint64
	Timestamp time.Time
	Tenant    string
	User      string // empty if RBAC isn't enabled / no user was set
	Statement string // original SQL text if available, else a best-effort fallback
	Success   bool
	Error     string // populated only when !Success
	PrevHash  string // hex-encoded SHA-256 of the previous entry, "" for the first entry
	Hash      string // hex-encoded SHA-256 of this entry's own fields + PrevHash
}

AuditEntry is one recorded statement execution.

type AuditLog added in v0.13.0 

type AuditLog struct {
	// contains filtered or unexported fields
}

AuditLog is an in-memory, optionally file-backed hash-chained log. Safe for concurrent use.

func NewAuditLog added in v0.13.0 

func NewAuditLog() *AuditLog

NewAuditLog creates an in-memory-only audit log (not persisted to disk — useful for tests, or callers that only care about the current process's lifetime).

func OpenAuditLog added in v0.13.0 

func OpenAuditLog(path string) (*AuditLog, error)

OpenAuditLog opens (creating if necessary) an append-only JSONL file — one JSON-encoded AuditEntry per line, chosen over a binary format for the same reason ModeJSON exists: human-readable, diffable, greppable, and inspectable with ordinary tools during an incident, at the cost of some size on disk. If the file already has entries, they're replayed and the chain is verified as part of opening — a corrupted or tampered existing log is caught immediately at startup rather than silently trusted.

func (*AuditLog) Append added in v0.13.0 

func (a *AuditLog) Append(tenant, user, statement string, success bool, errMsg string) AuditEntry

Append records one statement execution and returns the entry, hash chain already computed. statement should be the original SQL text when available; user may be empty (RBAC not enabled, or no user in context).

func (*AuditLog) Close added in v0.13.0 

func (a *AuditLog) Close() error

Close flushes and closes the backing file, if any. A no-op for an in-memory-only log (NewAuditLog).

func (*AuditLog) Entries added in v0.13.0 

func (a *AuditLog) Entries() []AuditEntry

Entries returns a copy of every logged entry, in order.

func (*AuditLog) Verify added in v0.13.0 

func (a *AuditLog) Verify() error

Verify walks the whole chain and recomputes every hash, returning an error identifying the first entry (by Seq) where the recorded hash doesn't match — either the entry's own fields were altered, or PrevHash no longer matches its predecessor (an entry was inserted, removed, or reordered).

type BackendStats added in v0.5.4 

type BackendStats struct {
	Mode             StorageMode
	TablesInMemory   int
	TablesOnDisk     int
	MemoryUsedBytes  int64
	MemoryLimitBytes int64
	DiskUsedBytes    int64
	CacheHitRate     float64
	SyncCount        int64
	LoadCount        int64
	EvictionCount    int64
}

BackendStats provides observability into storage backend behaviour.

type BatchHandler 

type BatchHandler func(items []interface{}) error

BatchHandler processes a batch of items.

type BatchProcessor 

type BatchProcessor struct {
	// contains filtered or unexported fields
}

BatchProcessor batches operations for efficiency.

func NewBatchProcessor 

func NewBatchProcessor(maxSize int, interval time.Duration, handler BatchHandler) *BatchProcessor

NewBatchProcessor creates a new batch processor.

func (*BatchProcessor) Add 

func (bp *BatchProcessor) Add(item interface{}) error

Add adds an item to the batch queue.

func (*BatchProcessor) Run 

func (bp *BatchProcessor) Run(ctx context.Context, wg *sync.WaitGroup)

Run starts the batch processor.

type BufferPool 

type BufferPool struct {
	// contains filtered or unexported fields
}

BufferPool manages in-memory tables with configurable eviction.

func NewBufferPool 

func NewBufferPool(policy *MemoryPolicy) *BufferPool

NewBufferPool creates a buffer pool with the given policy.

func (*BufferPool) Get 

func (bp *BufferPool) Get(tenant, name string) (*Table, bool)

Get retrieves a table from the buffer pool.

func (*BufferPool) GetMemoryLimit 

func (bp *BufferPool) GetMemoryLimit() int64

GetMemoryLimit returns the configured memory limit.

func (*BufferPool) GetMemoryUsage 

func (bp *BufferPool) GetMemoryUsage() int64

GetMemoryUsage returns current memory usage in bytes.

func (*BufferPool) GetStats 

func (bp *BufferPool) GetStats() CacheStats

GetStats returns a copy of current statistics.

func (*BufferPool) Put 

func (bp *BufferPool) Put(tenant, name string, table *Table) error

Put adds or updates a table in the buffer pool.

func (*BufferPool) Remove 

func (bp *BufferPool) Remove(tenant, name string)

Remove removes a table from the buffer pool.

type CacheStats 

type CacheStats struct {
	MemoryUsed        int64
	MemoryLimit       int64
	MemoryUtilization float64

	CacheHits   int64
	CacheMisses int64
	HitRate     float64

	EvictionCount int64
	EvictionSize  int64

	TablesInMemory int
	TablesOnDisk   int
	// contains filtered or unexported fields
}

CacheStats tracks buffer pool performance metrics.

type CacheStrategy 

type CacheStrategy int

CacheStrategy defines the eviction policy for the buffer pool. 

const (
	StrategyNone CacheStrategy = iota // No eviction (full in-memory)
	StrategyLRU                       // Least Recently Used
	StrategyLFU                       // Least Frequently Used (future)
	StrategyARC                       // Adaptive Replacement Cache (future)
)

func (CacheStrategy) String 

func (s CacheStrategy) String() string

type CachedTable 

type CachedTable struct {
	Table       *Table
	Size        int64
	LoadedAt    time.Time
	LastAccess  time.Time
	AccessCount int64
	Pinned      bool

	// Disk location (for lazy loading - future)
	OnDisk     bool
	DiskOffset int64
	// contains filtered or unexported fields
}

CachedTable wraps a table with caching metadata.

type CatalogColumn added in v0.5.1 

type CatalogColumn struct {
	Schema       string
	TableName    string
	Name         string
	Position     int
	DataType     string
	IsNullable   bool
	DefaultValue *string
}

CatalogColumn represents metadata for a table column CatalogColumn represents a column in a catalog table including its position and declared data type. `DefaultValue` may be nil if none is defined.

type CatalogDependency added in v0.11.0 

type CatalogDependency struct {
	Schema          string
	ObjectName      string
	ObjectType      string
	DependsOnSchema string
	DependsOnName   string
	DependsOnType   string
	DependencyType  string
	CreatedAt       time.Time
}

CatalogDependency records that one catalog object depends on another object.

type CatalogFunction added in v0.5.1 

type CatalogFunction struct {
	Schema          string
	Name            string
	FunctionType    string // 'SCALAR', 'TABLE', 'AGGREGATE', 'WINDOW'
	ArgTypes        []string
	ReturnType      string
	Language        string // 'BUILTIN', 'SQL', 'GO'
	IsDeterministic bool
	Description     string
}

CatalogFunction represents metadata for scalar and table-valued functions CatalogFunction describes registered functions (builtin or user defined). `FunctionType` categorizes the function, and `Language` indicates the implementation origin.

type CatalogJob added in v0.5.1 

type CatalogJob struct {
	Name         string
	SQLText      string
	ScheduleType string     // 'CRON', 'INTERVAL', 'ONCE'
	CronExpr     string     // For CRON schedule
	IntervalMs   int64      // For INTERVAL schedule (milliseconds)
	RunAt        *time.Time // For ONCE schedule
	Timezone     string
	Enabled      bool
	CatchUp      bool // Run missed executions
	NoOverlap    bool // Prevent concurrent runs
	MaxRuntimeMs int64
	LastRunAt    *time.Time
	NextRunAt    *time.Time
	CreatedAt    time.Time
	UpdatedAt    time.Time
}

CatalogJob represents a scheduled job CatalogJob describes a scheduled job. The fields provide flexible scheduling options (CRON, interval, or single-run) and execution metadata for bookkeeping and scheduling decisions.

type CatalogJobHistory added in v0.11.0 

type CatalogJobHistory struct {
	RunID        int64
	JobName      string
	StartedAt    time.Time
	FinishedAt   time.Time
	DurationMs   int64
	Status       string // 'SUCCEEDED', 'FAILED', 'SKIPPED', 'CANCELED'
	ErrorMessage string
}

CatalogJobHistory records one completed, failed, skipped, or canceled job run.

type CatalogManager added in v0.5.1 

type CatalogManager struct {
	// contains filtered or unexported fields
}

CatalogManager manages system catalog tables (`catalog.tables`, `catalog.columns`, etc.) and provides thread-safe registration and lookup helpers used by the rest of the system for introspection and scheduling. CatalogManager is safe for concurrent use.

func NewCatalogManager added in v0.5.1 

func NewCatalogManager() *CatalogManager

NewCatalogManager allocates and returns an initialized CatalogManager.

func (*CatalogManager) AddJobHistory added in v0.11.0 

func (c *CatalogManager) AddJobHistory(run *CatalogJobHistory) error

AddJobHistory appends a job execution history row and assigns a run id.

func (*CatalogManager) Authenticate added in v0.12.0 

func (c *CatalogManager) Authenticate(name, password string) bool

Authenticate verifies a password against the stored bcrypt hash in constant time (bcrypt.CompareHashAndPassword is itself constant-time with respect to the password content). Returns false for an unknown or disabled user without distinguishing why, so a caller can't use timing or error content to enumerate valid usernames.

func (*CatalogManager) CreateRole added in v0.12.0 

func (c *CatalogManager) CreateRole(name string) error

CreateRole adds a new, initially grant-less role.

func (*CatalogManager) CreateUser added in v0.12.0 

func (c *CatalogManager) CreateUser(name, password string, roles []string) error

CreateUser adds a new user with a bcrypt-hashed password. Returns an error if the name is already taken (case-insensitive) or the password is empty (an empty password would make Authenticate meaningless — bcrypt would happily hash "" too, but rejecting it here catches the mistake of e.g. an unpopulated config field at the source instead of silently creating a passwordless account).

func (*CatalogManager) DeleteJob added in v0.5.1 

func (c *CatalogManager) DeleteJob(name string) error

DeleteJob removes a job from the catalog, returning an error when the job does not exist.

func (*CatalogManager) DeleteMaterializedView added in v0.11.0 

func (c *CatalogManager) DeleteMaterializedView(schema, name string) error

DeleteMaterializedView removes a materialized view definition.

func (*CatalogManager) DeleteView added in v0.11.0 

func (c *CatalogManager) DeleteView(schema, name string) error

DeleteView removes a view definition and its catalog table entry.

func (*CatalogManager) DropRole added in v0.12.0 

func (c *CatalogManager) DropRole(name string) error

DropRole removes a role. Existing users keep a dangling membership reference (harmless: GrantRoleToUser/permission resolution both treat a reference to a since-dropped role as granting nothing), consistent with how DropTrigger etc. don't cascade either.

func (*CatalogManager) DropTrigger added in v0.9.0 

func (c *CatalogManager) DropTrigger(name string) error

DropTrigger removes a named trigger from the catalog. It returns an error when the trigger does not exist.

func (*CatalogManager) DropUser added in v0.12.0 

func (c *CatalogManager) DropUser(name string) error

DropUser removes a user. Returns an error if it doesn't exist.

func (*CatalogManager) FinishMaterializedViewRefresh added in v0.11.0 

func (c *CatalogManager) FinishMaterializedViewRefresh(schema, name string, refreshedAt time.Time, durationMs int64, rowCount int64, errMsg string) error

FinishMaterializedViewRefresh updates refresh bookkeeping.

func (*CatalogManager) GetAllColumns added in v0.5.1 

func (c *CatalogManager) GetAllColumns() []CatalogColumn

GetAllColumns aggregates and returns columns registered for every table in the catalog. The returned slice is a concatenation of the internal per-table column lists.

func (*CatalogManager) GetColumns added in v0.5.1 

func (c *CatalogManager) GetColumns(schema, tableName string) []CatalogColumn

GetColumns returns the column metadata for `schema.tableName`. If the table is unknown an empty slice is returned.

func (*CatalogManager) GetDependencies added in v0.11.0 

func (c *CatalogManager) GetDependencies() []CatalogDependency

GetDependencies returns all recorded catalog dependencies.

func (*CatalogManager) GetFunctions added in v0.5.1 

func (c *CatalogManager) GetFunctions() []*CatalogFunction

GetFunctions returns metadata for all registered functions.

func (*CatalogManager) GetJob added in v0.5.1 

func (c *CatalogManager) GetJob(name string) (*CatalogJob, error)

GetJob retrieves a job by name, returning an error if not found.

func (*CatalogManager) GetMaterializedView added in v0.11.0 

func (c *CatalogManager) GetMaterializedView(schema, name string) (*CatalogMaterializedView, bool)

GetMaterializedView retrieves a materialized view definition by schema/name.

func (*CatalogManager) GetMaterializedViews added in v0.11.0 

func (c *CatalogManager) GetMaterializedViews() []*CatalogMaterializedView

GetMaterializedViews returns all materialized view definitions.

func (*CatalogManager) GetTables added in v0.5.1 

func (c *CatalogManager) GetTables() []*CatalogTable

GetTables returns a slice with metadata for all registered tables and views.

func (*CatalogManager) GetTriggers added in v0.9.0 

func (c *CatalogManager) GetTriggers(table string, timing TriggerTiming, event TriggerEvent) []*CatalogTrigger

GetTriggers returns all triggers for the given table, timing, and event.

func (*CatalogManager) GetUser added in v0.12.0 

func (c *CatalogManager) GetUser(name string) (CatalogUser, bool)

GetUser returns a copy of a user's metadata (never the password hash's caller-mutable backing, since Roles is copied) for introspection, or false if it doesn't exist.

func (*CatalogManager) GetView added in v0.11.0 

func (c *CatalogManager) GetView(schema, name string) (*CatalogView, bool)

GetView retrieves a view definition by schema and name.

func (*CatalogManager) GetViews added in v0.5.1 

func (c *CatalogManager) GetViews() []*CatalogView

GetViews returns metadata for all registered views.

func (*CatalogManager) GrantPermission added in v0.12.0 

func (c *CatalogManager) GrantPermission(roleName string, perm Permission, schema, table string) error

GrantPermission adds a grant to an existing role. schema/table accept "*" as a wildcard.

func (*CatalogManager) GrantRoleToUser added in v0.12.0 

func (c *CatalogManager) GrantRoleToUser(userName, roleName string) error

GrantRoleToUser adds a role membership to an existing user. Idempotent — granting a role the user already has is not an error.

func (*CatalogManager) HasPermission added in v0.12.0 

func (c *CatalogManager) HasPermission(userName string, perm Permission, schema, table string) bool

HasPermission resolves userName's roles to their grants and reports whether any grant authorizes perm on (schema, table). Returns false for an unknown user (fail closed).

func (*CatalogManager) HasUsers added in v0.12.0 

func (c *CatalogManager) HasUsers() bool

HasUsers reports whether any user has been created. Note this ignores SetRBACEnabled's override — use IsRBACEnabled to answer "is enforcement actually active right now", which is what Execute checks.

func (*CatalogManager) IsRBACEnabled added in v0.13.0 

func (c *CatalogManager) IsRBACEnabled() bool

IsRBACEnabled reports whether Execute currently enforces permissions: true if SetRBACEnabled(false) was never called (or was reverted via SetRBACEnabled(true)) and at least one user exists.

func (*CatalogManager) ListEnabledJobs added in v0.5.1 

func (c *CatalogManager) ListEnabledJobs() []*CatalogJob

ListEnabledJobs returns only jobs whose `Enabled` flag is true.

func (*CatalogManager) ListJobHistory added in v0.11.0 

func (c *CatalogManager) ListJobHistory() []*CatalogJobHistory

ListJobHistory returns all job execution history rows.

func (*CatalogManager) ListJobs added in v0.5.1 

func (c *CatalogManager) ListJobs() []*CatalogJob

ListJobs returns a slice containing all registered jobs.

func (*CatalogManager) ListRoles added in v0.12.0 

func (c *CatalogManager) ListRoles() []CatalogRole

ListRoles returns all roles, sorted by name.

func (*CatalogManager) ListTriggers added in v0.9.0 

func (c *CatalogManager) ListTriggers() []*CatalogTrigger

ListTriggers returns all registered trigger definitions.

func (*CatalogManager) ListUsers added in v0.12.0 

func (c *CatalogManager) ListUsers() []CatalogUser

ListUsers returns all users (without password hashes), sorted by name.

func (*CatalogManager) MarkMaterializedViewsStaleByDependency added in v0.11.0 

func (c *CatalogManager) MarkMaterializedViewsStaleByDependency(schema, changedName string) []string

MarkMaterializedViewsStaleByDependency marks opt-in materialized views stale when they depend on the changed object. It returns the affected view names.

func (*CatalogManager) RegisterFunction added in v0.5.1 

func (c *CatalogManager) RegisterFunction(fn *CatalogFunction) error

RegisterFunction registers or updates a function's metadata.

func (*CatalogManager) RegisterJob added in v0.5.1 

func (c *CatalogManager) RegisterJob(job *CatalogJob) error

RegisterJob adds a new scheduled job or updates an existing entry. Job names must be non-empty.

func (*CatalogManager) RegisterMaterializedView added in v0.11.0 

func (c *CatalogManager) RegisterMaterializedView(mv *CatalogMaterializedView) error

RegisterMaterializedView registers or updates a materialized view definition.

func (*CatalogManager) RegisterTable added in v0.5.1 

func (c *CatalogManager) RegisterTable(schema, name string, cols []Column) error

RegisterTable registers a table and its columns in the system catalog. The provided `cols` slice is converted to `CatalogColumn` entries and stored under the key `schema.name`.

func (*CatalogManager) RegisterTrigger added in v0.9.0 

func (c *CatalogManager) RegisterTrigger(t *CatalogTrigger) error

RegisterTrigger stores a trigger definition in the catalog. If a trigger with the same name already exists it is replaced.

func (*CatalogManager) RegisterView added in v0.5.1 

func (c *CatalogManager) RegisterView(schema, name, sqlText string) error

RegisterView registers a view definition under `schema.name`.

func (*CatalogManager) RevokePermission added in v0.12.0 

func (c *CatalogManager) RevokePermission(roleName string, perm Permission, schema, table string) error

RevokePermission removes every grant on a role matching perm/schema/table exactly (same wildcard values as originally granted — this is a simple exact-match revoke, not a partial-overlap subtraction).

func (*CatalogManager) RevokeRoleFromUser added in v0.12.0 

func (c *CatalogManager) RevokeRoleFromUser(userName, roleName string) error

RevokeRoleFromUser removes a role membership. Not an error if the user didn't have that role.

func (*CatalogManager) SetDependencies added in v0.11.0 

func (c *CatalogManager) SetDependencies(schema, objectName, objectType string, deps []CatalogDependency)

SetDependencies replaces the dependency list for a catalog object.

func (*CatalogManager) SetRBACEnabled added in v0.13.0 

func (c *CatalogManager) SetRBACEnabled(enabled bool)

SetRBACEnabled overrides the default opt-in-via-CreateUser behavior. Pass false to force RBAC off — every Execute call is permitted regardless of context, even if users/roles already exist — useful for a setup that provisions accounts ahead of time but isn't ready to enforce yet, or a dev/test environment that wants audit-log attribution (see WithUser) without access checks getting in the way. Pass true to restore the default: enforcement follows HasUsers() exactly as if SetRBACEnabled had never been called.

This is a deliberate escape hatch matching SetReadOnly's toggle pattern elsewhere in this package — RBAC being opt-in-by-default already covers "not needed in every setup" for anyone who simply never calls CreateUser; this method is for the narrower case of a database that *has* users defined but shouldn't enforce against them right now.

func (*CatalogManager) SetUserDisabled added in v0.12.0 

func (c *CatalogManager) SetUserDisabled(name string, disabled bool) error

SetUserDisabled enables/disables a user without deleting their account (preserving audit-log attribution history for anything they did while enabled). A disabled user fails Authenticate.

func (*CatalogManager) TryBeginMaterializedViewRefresh added in v0.11.0 

func (c *CatalogManager) TryBeginMaterializedViewRefresh(schema, name string) bool

TryBeginMaterializedViewRefresh marks a materialized view as refreshing.

func (*CatalogManager) UpdateJobRuntime added in v0.5.1 

func (c *CatalogManager) UpdateJobRuntime(name string, lastRun, nextRun time.Time) error

UpdateJobRuntime updates runtime bookkeeping fields for a named job. It sets `LastRunAt`, `NextRunAt` and marks the job as recently updated.

func (*CatalogManager) UpdateJobRuntimePtr added in v0.11.0 

func (c *CatalogManager) UpdateJobRuntimePtr(name string, lastRun time.Time, nextRun *time.Time) error

UpdateJobRuntimePtr updates runtime bookkeeping fields for a named job. Passing nil for nextRun clears NextRunAt, which is expected for completed one-shot jobs.

type CatalogMaterializedView added in v0.11.0 

type CatalogMaterializedView struct {
	Schema             string
	Name               string
	SQLText            string
	CacheTableName     string
	StaleAfterMs       int64
	RefreshEveryMs     int64
	DailyAt            string
	Timezone           string
	WithData           bool
	InvalidateOnChange bool
	IsStale            bool
	LastRefreshAt      *time.Time
	LastDurationMs     int64
	LastError          string
	IsRefreshing       bool
	CreatedAt          time.Time
	UpdatedAt          time.Time
}

CatalogMaterializedView stores a saved query with physical refresh metadata.

type CatalogRole added in v0.12.0 

type CatalogRole struct {
	Name      string
	Grants    []Grant
	CreatedAt time.Time
}

CatalogRole is a named bundle of grants that one or more users can hold.

type CatalogTable added in v0.5.1 

type CatalogTable struct {
	Schema    string
	Name      string
	Type      string // 'TABLE', 'VIEW', 'SYSTEM'
	RowCount  int64
	CreatedAt time.Time
	UpdatedAt time.Time
}

CatalogTable represents metadata for a single table CatalogTable holds basic metadata for a registered table or view.

type CatalogTrigger added in v0.9.0 

type CatalogTrigger struct {
	Name       string
	Table      string
	Timing     TriggerTiming
	Event      TriggerEvent
	ForEachRow bool
	WhenExpr   string // optional WHEN clause SQL text
	Body       string // SQL text of trigger body (semicolon-separated stmts)
	CreatedAt  time.Time
}

CatalogTrigger holds a stored trigger definition.

type CatalogUser added in v0.12.0 

type CatalogUser struct {
	Name         string
	PasswordHash string
	Roles        []string
	Disabled     bool
	CreatedAt    time.Time
}

CatalogUser is an authenticatable principal. Passwords are never stored in plaintext — only a bcrypt hash, which is itself salted and safe to treat as opaque (bcrypt embeds its own per-hash random salt).

type CatalogView added in v0.5.1 

type CatalogView struct {
	Schema    string
	Name      string
	SQLText   string
	CreatedAt time.Time
}

CatalogView represents a saved view definition CatalogView stores the definition of a saved view.

type ColType 

type ColType int

ColType enumerates supported column data types. 

const (
	// IntType is a generic integer column type.
	IntType ColType = iota
	// Int8Type is an 8-bit signed integer column type.
	Int8Type
	// Int16Type is a 16-bit signed integer column type.
	Int16Type
	// Int32Type is a 32-bit signed integer column type.
	Int32Type
	// Int64Type is a 64-bit signed integer column type.
	Int64Type
	// UintType is an unsigned integer column type.
	UintType
	// Uint8Type is an 8-bit unsigned integer column type.
	Uint8Type
	// Uint16Type is a 16-bit unsigned integer column type.
	Uint16Type
	// Uint32Type is a 32-bit unsigned integer column type.
	Uint32Type
	// Uint64Type is a 64-bit unsigned integer column type.
	Uint64Type

	// Float32Type is a 32-bit floating point column type.
	Float32Type
	// Float64Type is a 64-bit floating point column type.
	Float64Type
	// FloatType is an alias for Float64Type.
	FloatType // alias for Float64Type

	// StringType represents a variable-length UTF-8 string column.
	StringType
	// TextType is an alias for StringType intended for long text.
	TextType // alias for StringType
	// RuneType stores single Unicode code points.
	RuneType
	// ByteType stores raw byte data.
	ByteType

	// BoolType represents a boolean column (true/false).
	BoolType

	// TimeType stores time-of-day values.
	TimeType
	// DateType stores date-only values.
	DateType
	// DateTimeType stores combined date and time values.
	DateTimeType
	// TimestampType stores an absolute point in time.
	TimestampType
	// DurationType stores a time duration.
	DurationType

	// DecimalType stores arbitrary-precision decimal numbers.
	DecimalType
	// MoneyType is a convenience alias for DecimalType used for monetary values.
	MoneyType
	// UUIDType stores RFC-4122 UUID values.
	UUIDType
	// BlobType stores binary large objects.
	BlobType
	// XMLType stores XML text.
	XMLType
	// IntervalType stores SQL-like intervals (parsed to time.Duration when possible).
	IntervalType

	// JsonType stores JSON text.
	JsonType
	// JsonbType stores binary JSON representations.
	JsonbType
	// MapType stores map-like complex values.
	MapType
	// SliceType stores slice-like complex values.
	SliceType
	// ArrayType stores array-like complex values.
	ArrayType

	// Complex64Type stores complex64 numeric values.
	Complex64Type
	// Complex128Type stores complex128 numeric values.
	Complex128Type
	// ComplexType is an alias for Complex128Type.
	ComplexType // alias for Complex128Type
	// PointerType represents a pointer/reference to another object.
	PointerType
	// InterfaceType represents an arbitrary Go interface value.
	InterfaceType

	// VectorType represents a vector/embedding column used by RAG features.
	VectorType
	// GeometryType stores spatial geometry values (GeoJSON/WKB) as JSONB or binary payload.
	GeometryType

	// YAMLType stores YAML-formatted text data.
	YAMLType
	// URLType stores URL/URI values with optional validation.
	URLType
	// HASHType stores cryptographic hash digests (hex-encoded).
	HASHType
	// BitmapType stores roaring-bitmap or bitset values as a byte slice.
	BitmapType
)

func (ColType) String 

func (t ColType) String() string

type Column 

type Column struct {
	Name         string
	Type         ColType
	Constraint   ConstraintType
	ForeignKey   *ForeignKeyRef // Only used if Constraint == ForeignKey
	PointerTable string         // Target table for POINTER type
}

Column holds column schema information in a table.

type ConcurrencyConfig 

type ConcurrencyConfig struct {
	// Worker pool sizes
	ReadWorkers  int
	WriteWorkers int

	// Channel buffer sizes
	ReadQueueSize  int
	WriteQueueSize int

	// Timeouts
	WorkerTimeout time.Duration
	QueueTimeout  time.Duration

	// Batch settings
	BatchSize     int
	BatchInterval time.Duration
}

ConcurrencyConfig configures the concurrency system.

func DefaultConcurrencyConfig 

func DefaultConcurrencyConfig() ConcurrencyConfig

DefaultConcurrencyConfig returns sensible defaults based on CPU count.

type ConcurrencyManager 

type ConcurrencyManager struct {
	// contains filtered or unexported fields
}

ConcurrencyManager orchestrates concurrent operations.

func NewConcurrencyManager 

func NewConcurrencyManager(config ConcurrencyConfig) *ConcurrencyManager

NewConcurrencyManager creates a new concurrency manager.

func (*ConcurrencyManager) Shutdown 

func (cm *ConcurrencyManager) Shutdown(timeout time.Duration) error

Shutdown gracefully shuts down the concurrency manager.

func (*ConcurrencyManager) Stats 

func (cm *ConcurrencyManager) Stats() *ConcurrencyStats

Stats returns current concurrency statistics.

func (*ConcurrencyManager) SubmitRead 

func (cm *ConcurrencyManager) SubmitRead(ctx context.Context, data interface{}) <-chan WorkResult

SubmitRead submits a read request (non-blocking).

func (*ConcurrencyManager) SubmitWrite 

func (cm *ConcurrencyManager) SubmitWrite(ctx context.Context, data interface{}) <-chan WorkResult

SubmitWrite submits a write request (non-blocking).

type ConcurrencyStats 

type ConcurrencyStats struct {
	TotalRequests   atomic.Uint64
	CompletedReads  atomic.Uint64
	CompletedWrites atomic.Uint64
	FailedRequests  atomic.Uint64
	QueuedReads     atomic.Int64
	QueuedWrites    atomic.Int64
	ActiveWorkers   atomic.Int64
}

ConcurrencyStats tracks concurrency metrics.

type ConstraintType 

type ConstraintType int

ConstraintType enumerates supported column constraints. 

const (
	NoConstraint ConstraintType = iota
	PrimaryKey
	ForeignKey
	Unique
)

func (ConstraintType) String 

func (c ConstraintType) String() string

type DB 

type DB struct {
	// contains filtered or unexported fields
}

DB is an in-memory, multi-tenant database catalog with full MVCC support. It optionally delegates storage to a StorageBackend for disk-based or hybrid persistence strategies.

func LoadFromBytes 

func LoadFromBytes(b []byte) (*DB, error)

LoadFromBytes loads a database from a byte slice.

func LoadFromFile 

func LoadFromFile(filename string) (*DB, error)

LoadFromFile loads a database snapshot from a file. It auto-detects gzip compression based on the .gz suffix and attaches a WAL if a path is given.

func LoadFromReader 

func LoadFromReader(r io.Reader) (*DB, error)

LoadFromReader loads a database snapshot from an arbitrary reader. The returned DB has no WAL attached.

func NewDB 

func NewDB() *DB

NewDB creates a new empty database catalog with MVCC support. The database operates in ModeMemory (pure in-memory) by default. Use OpenDB for disk-backed or hybrid storage modes.

func OpenDB added in v0.5.4 

func OpenDB(cfg StorageConfig) (*DB, error)

OpenDB creates or opens a database with the specified storage configuration. For ModeMemory this is equivalent to NewDB (with optional save-on-close). For ModeDisk/ModeJSON/ModeHybrid/ModeIndex, tables are stored as individual files in the configured directory. For ModeWAL, the existing WAL mechanism is configured automatically.

func (*DB) AdvancedWAL 

func (db *DB) AdvancedWAL() *AdvancedWAL

AdvancedWAL returns the configured advanced WAL manager (may be nil).

func (*DB) AttachAdvancedWAL 

func (db *DB) AttachAdvancedWAL(wal *AdvancedWAL)

AttachAdvancedWAL attaches an advanced WAL to the database.

func (*DB) AttachAuditLog added in v0.13.0 

func (db *DB) AttachAuditLog(log *AuditLog)

AttachAuditLog attaches a tamper-evident audit log to the database. Once attached, internal/engine.Execute records every statement to it (see internal/engine/audit.go). Pass nil to detach (stop logging); Execute treats a nil audit log as "logging disabled", matching the opt-in pattern used throughout this session's hardening work (RBAC, read-only mode, the security warning in cmd/server).

func (*DB) AuditLog added in v0.13.0 

func (db *DB) AuditLog() *AuditLog

AuditLog returns the attached audit log, or nil if none is configured.

func (*DB) Backend added in v0.5.4 

func (db *DB) Backend() StorageBackend

Backend returns the attached StorageBackend (may be nil for pure in-memory databases created with NewDB).

func (*DB) BackendStats added in v0.5.4 

func (db *DB) BackendStats() BackendStats

BackendStats returns statistics from the storage backend. Returns a zero-value BackendStats if no backend is attached.

func (*DB) Catalog added in v0.5.1 

func (db *DB) Catalog() *CatalogManager

Catalog returns the CatalogManager attached to the DB, creating one lazily if necessary.

func (*DB) Close added in v0.5.4 

func (db *DB) Close() error

Close persists all data and releases resources. For ModeMemory with a configured path, this saves a final GOB snapshot. For ModeDisk/ModeJSON/ ModeHybrid, dirty tables are flushed. WAL and Advanced WAL resources are closed.

func (*DB) Config added in v0.5.4 

func (db *DB) Config() *StorageConfig

Config returns the StorageConfig used to open this database. Returns nil for databases created with NewDB().

func (*DB) DeepClone 

func (db *DB) DeepClone() *DB

DeepClone creates a full copy of the database (MVCC-light snapshot). Note: This is not copy-on-write; it creates a full copy (simple but O(n)).

func (*DB) DeleteJob added in v0.11.0 

func (db *DB) DeleteJob(name string) error

DeleteJob deletes a job and unschedules/cancels it when present.

func (*DB) Drop 

func (db *DB) Drop(tn, name string) error

Drop removes a table from the tenant (and from the backend if attached).

func (*DB) Evict added in v0.5.4 

func (db *DB) Evict(tenant, name string) error

Evict removes a table from the in-memory cache without deleting it from the backend. This is only meaningful for disk-backed modes; in ModeMemory the data would be lost. Returns an error if no backend is attached.

func (*DB) Get 

func (db *DB) Get(tn, name string) (*Table, error)

Get returns a table by name for the given tenant. When a StorageBackend is attached, tables not found in memory are loaded from the backend on demand (lazy loading).

func (*DB) HealthCheck added in v0.11.0 

func (db *DB) HealthCheck() DBHealth

HealthCheck returns a production-oriented lifecycle and storage snapshot.

func (*DB) IsRBACEnabled added in v0.13.0 

func (db *DB) IsRBACEnabled() bool

IsRBACEnabled reports whether Execute currently enforces RBAC permissions. A convenience delegate to db.Catalog().IsRBACEnabled.

func (*DB) IsReadOnly added in v0.11.0 

func (db *DB) IsReadOnly() bool

IsReadOnly reports whether the database is in read-only mode.

func (*DB) JobScheduler added in v0.11.0 

func (db *DB) JobScheduler() *Scheduler

JobScheduler returns the active database job scheduler, if any.

func (*DB) ListTables 

func (db *DB) ListTables(tn string) []*Table

ListTables returns the tables in a tenant sorted by name. When a StorageBackend is attached, tables that exist on disk but are not currently loaded into memory are loaded on demand.

func (*DB) ListTenants added in v0.5.4 

func (db *DB) ListTenants() []string

ListTenants returns the names of all tenants that have at least one table.

func (*DB) LockContentForRead added in v0.12.0 

func (db *DB) LockContentForRead()

LockContentForRead acquires the database's content lock for a read-only statement (SELECT/EXPLAIN/PRAGMA). Multiple readers may hold this concurrently; it excludes concurrent LockContentForWrite callers. Callers must call UnlockContentForRead exactly once, typically via defer, and must not call it again re-entrantly on the same goroutine (sync.RWMutex is not reentrant) — nested statement execution within the engine must bypass this lock rather than re-acquire it.

func (*DB) LockContentForWrite added in v0.12.0 

func (db *DB) LockContentForWrite()

LockContentForWrite acquires the database's content lock exclusively, for any statement that may mutate table rows/columns (INSERT/UPDATE/DELETE and all DDL). See LockContentForRead for the re-entrancy caveat.

func (*DB) MVCC 

func (db *DB) MVCC() *MVCCManager

MVCC returns the MVCC manager.

func (*DB) MigrateToBackend added in v0.5.4 

func (db *DB) MigrateToBackend(b StorageBackend) error

MigrateToBackend copies all in-memory tables to the given backend and attaches it. This enables migrating a ModeMemory database to ModeDisk or ModeHybrid at runtime.

func (*DB) Put 

func (db *DB) Put(tn string, t *Table) error

Put adds a new table to the tenant; returns error if it already exists. When a StorageBackend is attached, the table is also checked against the backend to prevent duplicates, and optionally persisted immediately when SyncOnMutate is configured.

func (*DB) RegisterJob added in v0.11.0 

func (db *DB) RegisterJob(job *CatalogJob) error

RegisterJob registers a job and refreshes the live scheduler when present.

func (*DB) RestartJobScheduler added in v0.11.0 

func (db *DB) RestartJobScheduler(executor JobExecutor) error

RestartJobScheduler restarts the database job scheduler with the executor.

func (*DB) SetBackend added in v0.5.4 

func (db *DB) SetBackend(b StorageBackend)

SetBackend attaches a StorageBackend and sets the storage mode. This is primarily used internally by OpenDB; calling it on a running database should be done with care.

func (*DB) SetRBACEnabled added in v0.13.0 

func (db *DB) SetRBACEnabled(enabled bool)

SetRBACEnabled overrides RBAC's default opt-in-via-CreateUser behavior; see CatalogManager.SetRBACEnabled for the full explanation. A convenience delegate to db.Catalog().SetRBACEnabled, provided directly on DB to mirror SetReadOnly above (the same "toggle a behavior" shape).

func (*DB) SetReadOnly added in v0.11.0 

func (db *DB) SetReadOnly(ro bool)

SetReadOnly toggles read-only mode. While enabled, the SQL engine rejects INSERT/UPDATE/DELETE/DDL with an error; SELECT, EXPLAIN, and PRAGMA still work. Safe to call concurrently with running queries.

func (*DB) ShallowCloneForTable added in v0.5.4 

func (db *DB) ShallowCloneForTable(tenant, tableName string) *DB

ShallowCloneForTable creates a lightweight copy of the database that deep-copies only the specified table and shares all others by reference. This is safe when the caller knows only the target table will be mutated (single-statement DML). For a database with many tables, this is dramatically cheaper than DeepClone — O(rows in target table) instead of O(rows in all tables).

func (*DB) StartJobScheduler added in v0.11.0 

func (db *DB) StartJobScheduler(executor JobExecutor) error

StartJobScheduler starts the database job scheduler with the given executor.

func (*DB) StopJobScheduler added in v0.11.0 

func (db *DB) StopJobScheduler()

StopJobScheduler stops the database job scheduler if it is running.

func (*DB) StorageMode added in v0.5.4 

func (db *DB) StorageMode() StorageMode

StorageMode returns the active storage mode.

func (*DB) Sync added in v0.5.4 

func (db *DB) Sync() error

Sync flushes all dirty in-memory tables to the storage backend. For ModeMemory and ModeWAL this is a no-op (those modes use SaveToFile / WAL checkpoints respectively). For ModeDisk, ModeJSON, ModeHybrid, and ModeIndex, tables whose version has changed since the last save are written to disk.

func (*DB) SyncTable added in v0.5.4 

func (db *DB) SyncTable(tenant string, t *Table) error

SyncTable flushes a single table to the backend. This is called by the engine after mutations when SyncOnMutate is enabled.

func (*DB) TableExists added in v0.5.4 

func (db *DB) TableExists(tenant, name string) bool

TableExists reports whether the named table exists, checking both in-memory tables and the storage backend.

func (*DB) UnlockContentForRead added in v0.12.0 

func (db *DB) UnlockContentForRead()

UnlockContentForRead releases a lock taken by LockContentForRead.

func (*DB) UnlockContentForWrite added in v0.12.0 

func (db *DB) UnlockContentForWrite()

UnlockContentForWrite releases a lock taken by LockContentForWrite.

func (*DB) WAL 

func (db *DB) WAL() *WALManager

WAL returns the configured WAL manager (may be nil).

type DBHealth added in v0.11.0 

type DBHealth struct {
	OK                bool
	Mode              StorageMode
	ModeName          string
	Path              string
	Closed            bool
	Closing           bool
	ReadOnly          bool
	SchedulerRunning  bool
	WALActive         bool
	AdvancedWALActive bool
	Tenants           int
	Tables            int
	BackendStats      BackendStats
	LastSyncAt        time.Time
	LastCloseAt       time.Time
	Recovery          RecoveryStatus
	Error             string
}

DBHealth is a point-in-time operational snapshot for production probes.

type DiskBackend added in v0.5.4 

type DiskBackend struct {
	// contains filtered or unexported fields
}

DiskBackend stores each table as an individual file under a directory tree: 

<dir>/<tenant>/<tablename>.tbl        (GOB-encoded diskTable, default)
<dir>/<tenant>/<tablename>.json       (JSON-encoded diskTable, ModeJSON)
<dir>/manifest.json                   (lightweight metadata index)

Tables are loaded into memory on demand (via LoadTable) and written back on SaveTable / Sync. This minimises RAM usage while keeping the file format compatible with the existing GOB serialisation (or, for ModeJSON, a JSON encoding of the same diskTable structure).

func NewDiskBackend added in v0.5.4 

func NewDiskBackend(dir string, compress bool) (*DiskBackend, error)

NewDiskBackend opens (or creates) a disk-backed storage directory using GOB-encoded per-table files (ModeDisk).

func NewJSONBackend added in v0.12.0 

func NewJSONBackend(dir string, compress bool) (*DiskBackend, error)

NewJSONBackend opens (or creates) a disk-backed storage directory using JSON-encoded per-table files (ModeJSON). Functionally identical to NewDiskBackend — same manifest, same lazy loading, same dirty tracking — only the per-table file encoding differs. JSON files are larger than GOB and big.Rat/uuid.UUID values round-trip as plain strings (see normalizeForJSON), but the files can be read, diffed, or hand-edited with any text tool, which GOB files cannot.

func (*DiskBackend) Close added in v0.5.4 

func (b *DiskBackend) Close() error

Close is like Sync but also cleans up any resources.

func (*DiskBackend) DeleteTable added in v0.5.4 

func (b *DiskBackend) DeleteTable(tenant, name string) error

DeleteTable removes a table file from disk and updates the manifest.

func (*DiskBackend) GetMeta added in v0.5.4 

func (b *DiskBackend) GetMeta(tenant, name string) *TableMeta

GetMeta returns table metadata without loading the full table. Returns nil if the table is unknown.

func (*DiskBackend) ImportFromDB added in v0.5.4 

func (b *DiskBackend) ImportFromDB(db *DB) error

ImportFromDB takes all tables from an existing in-memory DB and writes them as individual table files. This is the migration path from ModeMemory / GOB checkpoints to ModeDisk.

func (*DiskBackend) IsDirty added in v0.5.4 

func (b *DiskBackend) IsDirty(tenant, name string, currentVersion int) bool

IsDirty reports whether the named table has been modified since it was last saved. Uses the version field for comparison.

func (*DiskBackend) ListTableNames added in v0.5.4 

func (b *DiskBackend) ListTableNames(tenant string) ([]string, error)

ListTableNames returns all known table names for a tenant (on disk or in memory).

func (*DiskBackend) LoadTable added in v0.5.4 

func (b *DiskBackend) LoadTable(tenant, name string) (*Table, error)

LoadTable reads a table from its GOB file on disk. Returns (nil, nil) if the table does not exist.

func (*DiskBackend) Mode added in v0.5.4 

func (b *DiskBackend) Mode() StorageMode

func (*DiskBackend) SaveTable added in v0.5.4 

func (b *DiskBackend) SaveTable(tenant string, t *Table) error

SaveTable writes a table to its GOB file on disk and updates the manifest.

func (*DiskBackend) SetEncryptor added in v0.13.0 

func (b *DiskBackend) SetEncryptor(enc *Encryptor)

SetEncryptor enables (or, passing nil, disables) AES-256-GCM encryption at rest for every table file this backend writes from this point on. Existing files already on disk are re-encrypted (or decrypted) lazily, the next time each is saved — SetEncryptor doesn't rewrite files that aren't touched again, so a backend directory can end up with a mix of encrypted and unencrypted files if the encryptor is changed mid-lifetime. For a clean transition, set the encryptor once at open time (see StorageConfig.EncryptionKey) rather than toggling it on a live backend.

func (*DiskBackend) Stats added in v0.5.4 

func (b *DiskBackend) Stats() BackendStats

func (*DiskBackend) Sync added in v0.5.4 

func (b *DiskBackend) Sync() error

Sync writes the manifest (individual table files are written immediately by SaveTable). The caller should call SaveTable for each dirty table before calling Sync, or use DB.Sync which does this automatically.

func (*DiskBackend) TableExists added in v0.5.4 

func (b *DiskBackend) TableExists(tenant, name string) bool

TableExists reports whether the table is known to the backend.

type Encryptor added in v0.13.0 

type Encryptor struct {
	// contains filtered or unexported fields
}

Encryptor performs authenticated AES-256-GCM encryption/decryption of byte blobs, used to encrypt table files at rest.

func NewEncryptor added in v0.13.0 

func NewEncryptor(key []byte) (*Encryptor, error)

NewEncryptor creates an Encryptor from a 32-byte AES-256 key (typically from DeriveKeyFromPassphrase, or any caller-supplied 32 cryptographically random bytes for key-file-based setups).

func (*Encryptor) Decrypt added in v0.13.0 

func (e *Encryptor) Decrypt(blob []byte) ([]byte, error)

Decrypt reverses Encrypt. It returns an error — rather than corrupted or wrong plaintext — if blob is truncated, was encrypted under a different key, or was altered after encryption: GCM's authentication tag check fails closed on any of those.

func (*Encryptor) Encrypt added in v0.13.0 

func (e *Encryptor) Encrypt(plaintext []byte) ([]byte, error)

Encrypt returns nonce||ciphertext||tag as a single self-contained blob safe to write directly to disk (Decrypt expects exactly this layout). A fresh random nonce is generated per call, which is required for GCM's security guarantees — reusing a nonce under the same key breaks confidentiality — and cheap enough to pay per file write.

type ForeignKeyRef 

type ForeignKeyRef struct {
	Table    string
	Column   string
	OnDelete ReferentialAction
	OnUpdate ReferentialAction
}

ForeignKeyRef describes a foreign key reference target.

type Grant added in v0.12.0 

type Grant struct {
	Permission Permission
	Schema     string
	Table      string
}

Grant authorizes Permission on every table in Schema (or every schema, if Schema is "*") named Table (or every table, if Table is "*").

type HybridBackend added in v0.5.4 

type HybridBackend struct {
	// contains filtered or unexported fields
}

HybridBackend keeps frequently-accessed tables in an LRU memory cache backed by individual table files on disk. When the cache exceeds the configured memory limit, cold tables are evicted. This gives near in- memory performance for the working set while allowing the full database to exceed available RAM.

For ModeIndex, the same structure is used with a much smaller memory limit and aggressive eviction – essentially only table schemas (loaded as metadata from the manifest) stay resident.

func NewHybridBackend added in v0.5.4 

func NewHybridBackend(dir string, maxMemoryBytes int64, compress bool, mode StorageMode) (*HybridBackend, error)

NewHybridBackend creates a HybridBackend.

  • dir: directory for table files
  • maxMemoryBytes: memory limit (0 = 256 MB default)
  • compress: gzip table files
  • mode: ModeHybrid or ModeIndex

func (*HybridBackend) Close added in v0.5.4 

func (h *HybridBackend) Close() error

Close flushes dirty tables and closes the disk backend.

func (*HybridBackend) DeleteTable added in v0.5.4 

func (h *HybridBackend) DeleteTable(tenant, name string) error

DeleteTable removes from both disk and cache.

func (*HybridBackend) Disk added in v0.5.4 

func (h *HybridBackend) Disk() *DiskBackend

Disk returns the underlying DiskBackend for advanced operations (e.g. migration, metadata access).

func (*HybridBackend) ListTableNames added in v0.5.4 

func (h *HybridBackend) ListTableNames(tenant string) ([]string, error)

ListTableNames returns names from the disk manifest (authoritative source).

func (*HybridBackend) LoadTable added in v0.5.4 

func (h *HybridBackend) LoadTable(tenant, name string) (*Table, error)

LoadTable tries the in-memory cache first, then loads from disk.

func (*HybridBackend) MarkDirty added in v0.5.4 

func (h *HybridBackend) MarkDirty(tenant, name string)

MarkDirty records that a table has been modified in memory and needs to be flushed on the next Sync. Called by DB after mutations.

func (*HybridBackend) Mode added in v0.5.4 

func (h *HybridBackend) Mode() StorageMode

func (*HybridBackend) SaveTable added in v0.5.4 

func (h *HybridBackend) SaveTable(tenant string, t *Table) error

SaveTable writes to disk and updates the cache.

func (*HybridBackend) Stats added in v0.5.4 

func (h *HybridBackend) Stats() BackendStats

func (*HybridBackend) Sync added in v0.5.4 

func (h *HybridBackend) Sync() error

Sync writes all dirty in-memory tables to disk.

func (*HybridBackend) TableExists added in v0.5.4 

func (h *HybridBackend) TableExists(tenant, name string) bool

TableExists checks the disk manifest.

type IsolationLevel 

type IsolationLevel uint8

IsolationLevel defines transaction isolation semantics. 

const (
	ReadCommitted IsolationLevel = iota
	RepeatableRead
	SnapshotIsolation
	Serializable
)

type JobExecutor added in v0.5.1 

type JobExecutor interface {
	ExecuteSQL(ctx context.Context, sql string) (interface{}, error)
}

JobExecutor interface allows the scheduler to execute SQL without circular dependencies

type LRUNode 

type LRUNode struct {
	// contains filtered or unexported fields
}

LRUNode is a node in the LRU doubly-linked list.

type LRUQueue 

type LRUQueue struct {
	// contains filtered or unexported fields
}

LRUQueue implements least-recently-used eviction.

func NewLRUQueue 

func NewLRUQueue() *LRUQueue

NewLRUQueue creates an empty LRU queue.

func (*LRUQueue) Access 

func (lru *LRUQueue) Access(key string, table *CachedTable)

Access moves a node to the front (most recent).

func (*LRUQueue) Add 

func (lru *LRUQueue) Add(key string, table *CachedTable)

Add adds a node to the LRU queue (most recent).

func (*LRUQueue) Remove 

func (lru *LRUQueue) Remove(key string)

Remove removes a node from the queue.

func (*LRUQueue) RemoveLRU 

func (lru *LRUQueue) RemoveLRU() (string, *CachedTable)

RemoveLRU removes and returns the least recently used node.

type LSN 

type LSN uint64

LSN (Log Sequence Number) provides total ordering of log records.

type MVCCManager 

type MVCCManager struct {
	// contains filtered or unexported fields
}

MVCCManager coordinates transaction IDs, timestamps, and visibility.

func NewMVCCManager 

func NewMVCCManager() *MVCCManager

NewMVCCManager creates a new MVCC coordinator.

func (*MVCCManager) AbortTx 

func (m *MVCCManager) AbortTx(tx *TxContext)

AbortTx marks a transaction as aborted.

func (*MVCCManager) BeginTx 

func (m *MVCCManager) BeginTx(level IsolationLevel) *TxContext

BeginTx starts a new transaction and returns its context.

func (*MVCCManager) CommitTx 

func (m *MVCCManager) CommitTx(tx *TxContext) (Timestamp, error)

CommitTx marks a transaction as committed and records its commit timestamp.

func (*MVCCManager) GCWatermark 

func (m *MVCCManager) GCWatermark() Timestamp

GCWatermark returns the timestamp before which row versions can be garbage collected.

func (*MVCCManager) IsVisible 

func (m *MVCCManager) IsVisible(tx *TxContext, rv *RowVersion) bool

IsVisible determines if a row version is visible to a transaction.

type MVCCTable 

type MVCCTable struct {
	*Table
	// contains filtered or unexported fields
}

MVCCTable extends Table with version chains.

func NewMVCCTable 

func NewMVCCTable(name string, cols []Column, isTemp bool) *MVCCTable

NewMVCCTable creates a table with MVCC support.

func (*MVCCTable) DeleteVersion 

func (t *MVCCTable) DeleteVersion(tx *TxContext, rowID int64) error

DeleteVersion marks a row version as deleted.

func (*MVCCTable) GarbageCollect 

func (t *MVCCTable) GarbageCollect(watermark Timestamp) int

GarbageCollect removes old row versions that are no longer visible.

func (*MVCCTable) GetVisibleVersion 

func (t *MVCCTable) GetVisibleVersion(mvcc *MVCCManager, tx *TxContext, rowID int64) *RowVersion

GetVisibleVersion returns the visible version of a row for the given transaction.

func (*MVCCTable) InsertVersion 

func (t *MVCCTable) InsertVersion(tx *TxContext, data []any) int64

InsertVersion adds a new row version.

func (*MVCCTable) UpdateVersion 

func (t *MVCCTable) UpdateVersion(tx *TxContext, rowID int64, newData []any) error

UpdateVersion creates a new version for an update.

type MemoryBackend added in v0.5.4 

type MemoryBackend struct {
	// contains filtered or unexported fields
}

MemoryBackend is a no-op backend used by ModeMemory. All data lives exclusively in the DB's in-memory tenants map; the backend simply reports that tables don't exist on any backing store (so the DB never tries to load from disk). Save/Sync are intentional no-ops – persistence is the caller's responsibility (via SaveToFile or the WAL).

func NewMemoryBackend added in v0.5.4 

func NewMemoryBackend(savePath string) *MemoryBackend

NewMemoryBackend creates a MemoryBackend. If savePath is non-empty, Close performs a final SaveToFile to that path.

func (*MemoryBackend) Close added in v0.5.4 

func (m *MemoryBackend) Close() error

func (*MemoryBackend) DeleteTable added in v0.5.4 

func (m *MemoryBackend) DeleteTable(_, _ string) error

func (*MemoryBackend) ListTableNames added in v0.5.4 

func (m *MemoryBackend) ListTableNames(_ string) ([]string, error)

func (*MemoryBackend) LoadTable added in v0.5.4 

func (m *MemoryBackend) LoadTable(_, _ string) (*Table, error)

func (*MemoryBackend) Mode added in v0.5.4 

func (m *MemoryBackend) Mode() StorageMode

func (*MemoryBackend) SaveTable added in v0.5.4 

func (m *MemoryBackend) SaveTable(_ string, _ *Table) error

func (*MemoryBackend) Stats added in v0.5.4 

func (m *MemoryBackend) Stats() BackendStats

func (*MemoryBackend) Sync added in v0.5.4 

func (m *MemoryBackend) Sync() error

func (*MemoryBackend) TableExists added in v0.5.4 

func (m *MemoryBackend) TableExists(_, _ string) bool

type MemoryPolicy 

type MemoryPolicy struct {
	// Maximum memory usage in bytes (0 = unlimited)
	MaxMemoryBytes int64

	// Cache eviction strategy
	Strategy CacheStrategy

	// Start evicting when memory usage exceeds this ratio (0.0-1.0)
	EvictionThreshold float64

	// Tables that should always stay in memory
	PinnedTables []string

	// Tables that should never be cached
	IgnoreTables []string

	// Enable eviction (if false, OOM when limit reached)
	EnableEviction bool

	// Number of tables to evict in one batch
	EvictionBatchSize int

	// Track access patterns for better eviction decisions
	TrackAccessPatterns bool

	// Time window for access tracking
	AccessWindow time.Duration
}

MemoryPolicy defines memory management configuration.

func DefaultMemoryPolicy 

func DefaultMemoryPolicy() *MemoryPolicy

DefaultMemoryPolicy returns a sensible default configuration.

func LimitedMemoryPolicy 

func LimitedMemoryPolicy(maxMB int64) *MemoryPolicy

LimitedMemoryPolicy returns a policy with memory limits.

type ParallelIterator 

type ParallelIterator struct {
	// contains filtered or unexported fields
}

ParallelIterator provides concurrent iteration over data.

func NewParallelIterator 

func NewParallelIterator(items []interface{}, workers int) *ParallelIterator

NewParallelIterator creates a parallel iterator.

func (*ParallelIterator) ForEach 

func (pi *ParallelIterator) ForEach(fn func(item interface{}) error) error

ForEach processes items in parallel.

func (*ParallelIterator) Map 

func (pi *ParallelIterator) Map(fn func(item interface{}) (interface{}, error)) ([]interface{}, error)

Map applies a function to all items in parallel and returns results.

func (*ParallelIterator) WithContext 

func (pi *ParallelIterator) WithContext(ctx context.Context) *ParallelIterator

WithContext sets the context for the iterator.

type Permission added in v0.12.0 

type Permission string

Permission enumerates the operations a grant can authorize. 

const (
	PermSelect Permission = "SELECT"
	PermInsert Permission = "INSERT"
	PermUpdate Permission = "UPDATE"
	PermDelete Permission = "DELETE"
	// PermDDL covers CREATE/DROP/ALTER TABLE/INDEX/VIEW/TRIGGER/JOB and
	// similar schema-changing statements.
	PermDDL Permission = "DDL"
	// PermAll grants every permission above at once.
	PermAll Permission = "ALL"
)

func ParsePermission added in v0.12.0 

func ParsePermission(s string) (Permission, error)

ParsePermission validates and normalizes a permission keyword from SQL (case-insensitive).

type Pipeline 

type Pipeline struct {
	// contains filtered or unexported fields
}

Pipeline implements a concurrent pipeline pattern.

func NewPipeline 

func NewPipeline(ctx context.Context, stages ...PipelineStage) *Pipeline

NewPipeline creates a new pipeline.

func (*Pipeline) Execute 

func (p *Pipeline) Execute(input []interface{}) <-chan interface{}

Execute runs the pipeline.

type PipelineStage 

type PipelineStage func(ctx context.Context, input <-chan interface{}) <-chan interface{}

PipelineStage represents a stage in the pipeline.

type RateLimiter 

type RateLimiter struct {
	// contains filtered or unexported fields
}

RateLimiter limits the rate of operations.

func NewRateLimiter 

func NewRateLimiter(opsPerSecond int) *RateLimiter

NewRateLimiter creates a new rate limiter.

func (*RateLimiter) Stop 

func (rl *RateLimiter) Stop()

Stop stops the rate limiter.

func (*RateLimiter) Wait 

func (rl *RateLimiter) Wait(ctx context.Context) error

Wait blocks until a token is available.

type RecoveryStatus added in v0.11.0 

type RecoveryStatus struct {
	Mode                  StorageMode
	Path                  string
	CheckpointLoaded      bool
	RecoveredTransactions uint64
	RecoveredOperations   int
	Truncated             bool
	RecoveredAt           time.Time
}

RecoveryStatus describes the last recovery pass performed while opening a DB.

type ReferentialAction added in v0.12.0 

type ReferentialAction int

ReferentialAction enumerates what happens to a child row when the parent row it references is deleted (ON DELETE) or its referenced column value changes (ON UPDATE). 

const (
	// NoAction means no ON DELETE/ON UPDATE clause was given. tinySQL checks
	// foreign key constraints immediately (it has no deferred-constraint
	// mode), so NoAction and Restrict behave identically here — the SQL
	// standard only distinguishes them for deferred checking.
	NoAction ReferentialAction = iota
	// Restrict blocks the parent-side mutation while a referencing child row exists.
	Restrict
	// Cascade propagates the parent-side delete/update to matching child rows.
	Cascade
	// SetNull nulls the child row's foreign key column instead of deleting/blocking.
	SetNull
)

func (ReferentialAction) String added in v0.12.0 

func (a ReferentialAction) String() string

type RowVersion 

type RowVersion struct {
	// Transaction that created this version
	XMin TxID

	// Transaction that deleted/updated this version (0 if still valid)
	XMax TxID

	// Creation timestamp
	CreatedAt Timestamp

	// Deletion/update timestamp (0 if still valid)
	DeletedAt Timestamp

	// Actual row data
	Data []any

	// Pointer to next version (for version chain)
	NextVersion *RowVersion
}

RowVersion contains MVCC metadata for a single row version.

type Scheduler added in v0.5.1 

type Scheduler struct {
	// contains filtered or unexported fields
}

Scheduler manages scheduled job execution

func NewScheduler added in v0.5.1 

func NewScheduler(db *DB, executor JobExecutor) *Scheduler

NewScheduler creates a new job scheduler

func (*Scheduler) AddJob added in v0.5.1 

func (s *Scheduler) AddJob(job *CatalogJob) error

AddJob registers a new job and schedules it immediately if enabled

func (*Scheduler) RemoveJob added in v0.5.1 

func (s *Scheduler) RemoveJob(name string) error

RemoveJob unregisters a job and stops its execution

func (*Scheduler) Start added in v0.5.1 

func (s *Scheduler) Start() error

Start begins the scheduler loop

func (*Scheduler) Stop added in v0.5.1 

func (s *Scheduler) Stop()

Stop halts the scheduler and cancels all running jobs

func (*Scheduler) UpsertJob added in v0.11.0 

func (s *Scheduler) UpsertJob(job *CatalogJob) error

UpsertJob registers or updates a job and refreshes its live schedule.

type StorageBackend added in v0.5.4 

type StorageBackend interface {
	// LoadTable retrieves a table from the backing store. It returns nil, nil
	// when the table does not exist (not an error – the table may simply not
	// have been created yet).
	LoadTable(tenant, name string) (*Table, error)

	// SaveTable persists a single table to the backing store.
	SaveTable(tenant string, t *Table) error

	// DeleteTable removes a table from the backing store.
	DeleteTable(tenant, name string) error

	// ListTableNames returns all table names for a tenant that exist in the
	// backing store (including tables not currently loaded in memory).
	ListTableNames(tenant string) ([]string, error)

	// TableExists reports whether the named table exists in the backing store
	// without loading it into memory.
	TableExists(tenant, name string) bool

	// Sync flushes any pending / dirty data to durable storage.
	Sync() error

	// Close releases all resources. Implementations should call Sync
	// internally if there is unsaved data.
	Close() error

	// Mode returns the StorageMode this backend implements.
	Mode() StorageMode

	// Stats returns operational statistics about the backend.
	Stats() BackendStats
}

StorageBackend abstracts the underlying table storage mechanism.

Implementations are responsible for loading tables into *Table structs that the engine can mutate directly. After mutations, Sync writes dirty tables back to durable storage. Close persists pending data and releases resources.

type StorageConfig added in v0.5.4 

type StorageConfig struct {
	// Mode selects the storage strategy. Defaults to ModeMemory.
	Mode StorageMode

	// Path is the root directory (or file path) for persistent storage.
	// Required for all modes except ModeMemory.
	Path string

	// MaxMemoryBytes limits in-memory table data. Used by ModeHybrid and
	// ModeIndex. Zero means use a sensible default (256 MB).
	MaxMemoryBytes int64

	// SyncOnMutate forces a disk write after every INSERT / UPDATE / DELETE.
	// Slower but provides immediate durability for ModeDisk / ModeHybrid.
	SyncOnMutate bool

	// CompressFiles enables gzip compression for table files on disk.
	CompressFiles bool

	// CheckpointEvery controls how many committed WAL transactions trigger
	// an automatic checkpoint (ModeWAL only). Zero means default (32).
	CheckpointEvery uint64

	// CheckpointInterval controls the maximum time between checkpoints
	// (ModeWAL only). Zero means default (30 s).
	CheckpointInterval time.Duration

	// CheckpointMaxBytes forces a checkpoint once the WAL file exceeds this
	// size, bounding WAL growth independently of transaction count and time
	// (ModeWAL / ModeAdvancedWAL). Zero means default (64 MB); negative
	// disables the size trigger.
	CheckpointMaxBytes int64

	// ReadOnly opens the database in read-only mode: the SQL engine rejects
	// all mutating statements. Ideal for serve-only phases (e.g. load at
	// night, query during the day) — vector index and column caches can never
	// be invalidated and the WAL is never appended to.
	ReadOnly bool

	// EncryptionKey, if set, must be exactly EncryptionKeySize (32) bytes
	// and enables AES-256-GCM encryption at rest for table files (ModeDisk,
	// ModeJSON). Derive it from a passphrase with DeriveKeyFromPassphrase
	// (persist the salt yourself — this package doesn't manage key
	// storage) or supply 32 cryptographically random bytes directly for a
	// key-file-based setup. Leave nil for the default, unencrypted
	// behavior. Not yet supported for ModeWAL/ModeAdvancedWAL/ModeHybrid/
	// ModeIndex — see README's Limitations section.
	EncryptionKey []byte
}

StorageConfig configures database storage behaviour.

func DefaultStorageConfig added in v0.5.4 

func DefaultStorageConfig(mode StorageMode) StorageConfig

DefaultStorageConfig returns a StorageConfig with sensible defaults for the given mode. Path must be supplied by the caller afterwards.

type StorageMode added in v0.5.4 

type StorageMode int

StorageMode defines how the database manages data between memory and disk. 

const (
	// ModeMemory keeps all data in RAM. Persistence only occurs via explicit
	// SaveToFile calls or when DB.Close is invoked. Fastest mode.
	ModeMemory StorageMode = iota

	// ModeWAL keeps all data in RAM and writes a Write-Ahead Log for crash
	// recovery. Periodic checkpoints create full GOB snapshots. Good
	// balance of speed and durability.
	ModeWAL

	// ModeDisk stores each table as a separate GOB file on disk. Tables
	// are loaded into memory on demand and flushed back on Sync/Close.
	// Minimises RAM usage at the cost of disk I/O.
	ModeDisk

	// ModeIndex keeps table schemas (columns, row-counts) permanently in
	// RAM while row data resides on disk. Rows are loaded on demand and
	// evicted aggressively. Memory usage is proportional to schema size,
	// not data size.
	ModeIndex

	// ModeHybrid uses an LRU buffer pool with a configurable memory limit.
	// Hot tables stay in RAM; cold tables spill to disk. Best for mixed
	// workloads where the working set fits in a bounded amount of memory.
	ModeHybrid

	// ModeAdvancedWAL keeps all data in RAM and uses a row-level Write-Ahead Log
	// for full ACID transaction durability, crash recovery, and point-in-time recovery.
	ModeAdvancedWAL

	// ModeJSON stores each table as a separate human-readable JSON file on
	// disk (same lazy-load/dirty-tracking behaviour as ModeDisk, which uses
	// GOB instead). Larger on disk than GOB and big.Rat/uuid.UUID values
	// round-trip as plain strings, but files can be read, diffed, or
	// hand-edited with any text tool.
	ModeJSON
)

func ParseStorageMode added in v0.5.4 

func ParseStorageMode(s string) (StorageMode, error)

ParseStorageMode converts a string representation back to a StorageMode. It is case-insensitive and returns an error for unknown values.

func (StorageMode) String added in v0.5.4 

func (m StorageMode) String() string

String returns a human-readable label for the StorageMode.

type Table 

type Table struct {
	Name   string
	Cols   []Column
	Rows   [][]any
	IsTemp bool

	Version int
	// contains filtered or unexported fields
}

Table stores rows along with column metadata and indexes.

func NewTable 

func NewTable(name string, cols []Column, isTemp bool) *Table

NewTable creates a new Table with case-insensitive column lookup indices.

func (*Table) ColIndex 

func (t *Table) ColIndex(name string) (int, error)

ColIndex returns the zero-based index of the named column.

func (*Table) DirtyFrom added in v0.5.4 

func (t *Table) DirtyFrom() int

DirtyFrom returns the first dirty row index, or -1 if non-append-only.

func (*Table) MarkDirtyFrom added in v0.5.4 

func (t *Table) MarkDirtyFrom(idx int)

MarkDirtyFrom records the first row index that was modified. If an earlier index is already set, it is kept. Use -1 for non-append mutations (UPDATE, DELETE) to force a full-table WAL entry.

func (*Table) ResetDirty added in v0.5.4 

func (t *Table) ResetDirty()

ResetDirty marks the table as clean (called after WAL checkpoint).

type TableMeta added in v0.5.4 

type TableMeta struct {
	Tenant   string   `json:"tenant"`
	Name     string   `json:"name"`
	Cols     []Column `json:"cols"`
	RowCount int      `json:"row_count"`
	Version  int      `json:"version"`
	// DiskSize is the file size in bytes on the backing store.
	DiskSize int64 `json:"disk_size"`
	// FilePath is the relative path inside the database directory.
	FilePath string `json:"file_path"`
}

TableMeta stores lightweight metadata for a table that is (potentially) on disk. The disk backend uses this to answer ListTableNames, TableExists, and schema-level queries without loading row data.

type Timestamp 

type Timestamp uint64

Timestamp represents a logical timestamp for MVCC visibility.

type TriggerEvent added in v0.9.0 

type TriggerEvent string

TriggerEvent indicates what event fires the trigger. 

const (
	// TriggerInsert fires on INSERT.
	TriggerInsert TriggerEvent = "INSERT"
	// TriggerUpdate fires on UPDATE.
	TriggerUpdate TriggerEvent = "UPDATE"
	// TriggerDelete fires on DELETE.
	TriggerDelete TriggerEvent = "DELETE"
)

type TriggerTiming added in v0.9.0 

type TriggerTiming string

TriggerTiming indicates when a trigger fires. 

const (
	// TriggerBefore fires before the DML operation.
	TriggerBefore TriggerTiming = "BEFORE"
	// TriggerAfter fires after the DML operation.
	TriggerAfter TriggerTiming = "AFTER"
	// TriggerInsteadOf replaces the DML operation (views).
	TriggerInsteadOf TriggerTiming = "INSTEAD OF"
)

type TxContext 

type TxContext struct {
	ID             TxID
	StartTime      Timestamp
	Status         TxStatus
	ReadSnapshot   Timestamp                      // Snapshot timestamp for reads
	WriteSet       map[string]map[int64]bool      // table -> row IDs modified
	ReadSet        map[string]map[int64]Timestamp // table -> row IDs read with version
	IsolationLevel IsolationLevel
	// contains filtered or unexported fields
}

TxContext holds the state of an active transaction.

func (*TxContext) RecordRead 

func (tx *TxContext) RecordRead(table string, rowID int64, version Timestamp)

RecordRead tracks a read operation for conflict detection.

func (*TxContext) RecordWrite 

func (tx *TxContext) RecordWrite(table string, rowID int64)

RecordWrite tracks a write operation.

type TxID 

type TxID uint64

TxID represents a unique transaction identifier.

type TxStatus 

type TxStatus uint8

TxStatus represents the current state of a transaction. 

const (
	TxStatusInProgress TxStatus = iota
	TxStatusCommitted
	TxStatusAborted
)

type WALChange 

type WALChange struct {
	Tenant string
	Name   string
	Table  *Table
	Drop   bool
}

WALChange describes a persistent change that will be written to the WAL.

func CollectWALChanges 

func CollectWALChanges(prev, next *DB) []WALChange

CollectWALChanges computes the delta between two MVCC snapshots.

type WALConfig 

type WALConfig struct {
	Path               string
	CheckpointEvery    uint64
	CheckpointInterval time.Duration
	// CheckpointMaxBytes forces a checkpoint once the WAL file exceeds this
	// size, bounding WAL growth independently of transaction count and time.
	// Zero means default (64 MB); negative disables the size trigger.
	CheckpointMaxBytes int64
}

WALConfig configures WAL and checkpoint behavior.

type WALManager 

type WALManager struct {
	// contains filtered or unexported fields
}

WALManager encapsulates WAL append, recovery, and checkpoints.

func OpenWAL 

func OpenWAL(db *DB, cfg WALConfig) (*WALManager, error)

ready-to-use manager. It attaches no WAL when Path is empty.

func (*WALManager) Checkpoint 

func (w *WALManager) Checkpoint(db *DB) error

Checkpoint writes a DB snapshot and resets the WAL file.

func (*WALManager) Close 

func (w *WALManager) Close() error

Close flushes, syncs, and closes the WAL resources.

func (*WALManager) LogTransaction 

func (w *WALManager) LogTransaction(changes []WALChange) (bool, error)

LogTransaction appends all changes atomically to the WAL. It returns true when a checkpoint is recommended.

type WALOperationType 

type WALOperationType uint8

WALOperationType defines the type of WAL operation. 

const (
	WALOpBegin WALOperationType = iota + 1
	WALOpInsert
	WALOpUpdate
	WALOpDelete
	WALOpCommit
	WALOpAbort
	WALOpCheckpoint
)

func (WALOperationType) String 

func (t WALOperationType) String() string

type WALRecord 

type WALRecord struct {
	// Log Sequence Number - globally unique, monotonically increasing
	LSN LSN

	// Transaction ID
	TxID TxID

	// Operation type
	OpType WALOperationType

	// Tenant and table
	Tenant string
	Table  string

	// Row ID (for row-level operations)
	RowID int64

	// UNDO image (before state) - for rollback
	BeforeImage []any

	// REDO image (after state) - for recovery
	AfterImage []any

	// Column information (for schema tracking)
	Columns []Column

	// Timestamp
	Timestamp time.Time

	// Checksum for corruption detection
	Checksum uint32
}

WALRecord represents a single log entry with before/after images.

type WALTxState 

type WALTxState struct {
	TxID       TxID
	StartLSN   LSN
	Operations []LSN
	Status     TxStatus
}

WALTxState tracks the state of a transaction in the WAL.

type WorkHandler 

type WorkHandler func(ctx context.Context, req WorkRequest) WorkResult

WorkHandler processes work requests.

type WorkRequest 

type WorkRequest struct {
	ID      uint64
	Context context.Context
	Type    WorkType
	Data    interface{}
	Result  chan WorkResult
}

WorkRequest represents a unit of work to be processed.

type WorkResult 

type WorkResult struct {
	ID    uint64
	Data  interface{}
	Error error
}

WorkResult contains the result of a work request.

type WorkType 

type WorkType uint8

WorkType defines the type of operation. 

const (
	WorkTypeRead WorkType = iota
	WorkTypeWrite
	WorkTypeDelete
	WorkTypeScan
	WorkTypeBatch
)

type WorkerPool 

type WorkerPool struct {
	// contains filtered or unexported fields
}

WorkerPool manages a pool of worker goroutines.

func NewWorkerPool 

func NewWorkerPool(name string, size int, workQueue chan WorkRequest, handler WorkHandler, ctx context.Context, wg *sync.WaitGroup) *WorkerPool

NewWorkerPool creates a new worker pool.

func (*WorkerPool) Start 

func (wp *WorkerPool) Start()

Start launches all worker goroutines.

Source Files

View all Source files

Directories

Path Synopsis
Package pager — PageBackend integrates the page-based storage engine with the tinySQL StorageBackend interface.
 Package pager — PageBackend integrates the page-based storage engine with the tinySQL StorageBackend interface.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.