storage

package
v0.24.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 26, 2026 License: AGPL-3.0 Imports: 35 Imported by: 0

Documentation

Overview

Package storage: tamper-evident audit log.

What: an append-only log of every statement Execute runs (see AttachAuditLog/DB.AuditLog and internal/engine/audit.go for the hook), where each entry's hash covers its own fields plus the previous entry's hash — a hash chain, the same integrity primitive used by blockchains and git commit history. Altering, reordering, or deleting any entry breaks the chain from that point forward in a way Verify detects.

Why this matters for the intended use cases (industry compliance, scientific reproducibility, security-sensitive deployments): a plain log file can be edited by anyone with filesystem access, silently. A hash chain doesn't prevent that access, but it makes silent editing detectable — Verify recomputes every hash and reports exactly where the chain first diverges from what's recorded, which is what audit/compliance tooling actually needs (proof of tampering, not prevention of it — true tamper-*prevention* needs signing with a key the log's own host doesn't hold, which is out of scope for an embedded database).

Package storage provides persistence primitives and metadata structures used by tinySQL. This file implements a lightweight in-memory system catalog used for introspection (tables, columns, views, functions) and simple job scheduling metadata.

Reading and writing a whole-database snapshot: to a file, a writer, or a byte slice. The file variant writes to a temporary file and renames it, so a crash during a save leaves the previous snapshot intact rather than a half-written one.

The column type system: storage types, SQLite type affinities, column constraints and referential actions. These are the vocabulary the schema is written in, so they are kept apart from the database that stores it.

Package storage provides the durable data structures for tinySQL.

What: An in-memory multi-tenant catalog of tables with column metadata, rows, and basic typing. It includes snapshot cloning for MVCC-light, GOB-based checkpoints, and an append-only Write-Ahead Log (WAL) for crash recovery and durability. How: Tables store rows as [][]any for compactness; a lower-cased column index accelerates name lookups. Save/Load serialize the catalog to a file, writing JSON for JSON columns. The WAL logs whole-table changes and drops; recovery replays committed records and truncates partial tails. Why: Favor a simple, explicit model over complex page managers: it keeps the code understandable, testable, and sufficient for embedded/edge use cases.

The on-disk representation of a database: the serializable forms of columns, tables and the system catalog, and the conversions to and from them. This is the format compatibility surface, so a change here changes what older files mean.

Converting one table to and from its on-disk form, including the row ranges and row sets the write-ahead log writes as deltas.

Package storage: encryption at rest for disk-based backends.

What: AES-256-GCM authenticated encryption of table files (ModeDisk, ModeJSON) and the backend catalog file. GCM was chosen over a plain block cipher mode specifically because it's authenticated — decrypting with the wrong key, or decrypting data that was altered after being written, fails closed with an error instead of silently returning garbage or (worse) subtly-wrong-but-plausible data. That gives encryption at rest a side benefit of the same tamper-detection property the audit log provides for statement history, applied to the data files themselves.

Why Argon2id for passphrase-based keys: PBKDF2 (also available in golang.org/x/crypto) is checkable purely with CPU, which makes brute-force guessing cheap on GPUs/ASICs at scale. Argon2id is memory-hard — an attacker also needs the configured amount of RAM per guess, which is the current best-practice recommendation (OWASP, and the Password Hashing Competition it won) for deriving a key from a human-chosen passphrase.

The operational surface: health, recovery status, syncing to the backend, closing, evicting a table from memory, and migrating an in-memory database onto a backend.

Opening a database: allocating an empty one, and OpenDB's per-mode construction of the backend, write-ahead log and checkpoint that a storage mode implies. Also loading a GOB checkpoint and reading the watermark that says how far it reflects the log.

Package storage: role-based access control (RBAC).

What: Users authenticate with a bcrypt-hashed password and are members of one or more roles; roles carry grants (a permission on a schema/table pair, with "*" as a wildcard for "all schemas"/"all tables"). Execute() in internal/engine checks the acting user's resolved permissions before running a statement.

Why opt-in: tinySQL has always been usable with zero setup (NewDB(), start running SQL). Requiring every caller to create a user before their first query would break that and every existing embedder. So RBAC enforcement only activates once at least one user has been created via CreateUser — see HasUsers/IsRBACEnabled. Before that, every permission check is a no-op allow, matching tinySQL's historical behavior exactly. This mirrors the same opt-in pattern already used for read-only mode (SetReadOnly) and cmd/server's optional auth token.

Copies of a database: the deep clone and the transaction snapshot pair the SQL driver runs a transaction against, and the row-less metadata snapshot the engine diffs for write-ahead logging.

Every clone must carry the runtime state that is not tenant data — see DB.copyRuntimeState in db.go. Hand-copying individual fields here is what once left a promoted clone without its write-ahead log.

Package storage – StorageBackend interface and StorageMode definitions.

What: Pluggable storage backends that decouple data management from the in-memory catalog. Each backend decides where table data lives (RAM, disk, or a combination) and how it is persisted. How: The DB struct optionally delegates Get/Put/Drop operations to an attached StorageBackend. Backends may lazily load tables, cache hot data, and flush dirty tables on Sync/Close. Why: Supporting multiple storage modes turns tinySQL into a realistic alternative to SQLite – from purely in-memory analytics right through to disk-resident databases that exceed available RAM.

Table: the rows of one table plus the schema and statistics describing them, and the dirty tracking that lets the write-ahead log write a delta instead of a whole table.

Finding, adding and removing tables within a tenant, including the did-you-mean suggestion an unknown table name produces and the lazy load a disk-backed mode performs on a cache miss.

The unit of work the write-ahead log records: which tables a change touched, how to compute that from two database states, and how to apply it. Shared by the log itself and by the SQL driver's commit path.

Index

Constants

View Source 
const EncryptionKeySize = 32

EncryptionKeySize is the required AES-256 key size in bytes.

View Source 
const EncryptionSaltSize = 16

EncryptionSaltSize is the recommended salt size for DeriveKeyFromPassphrase.

Variables

View Source 
var (
	ErrTxNotActive          = fmt.Errorf("transaction is not active")
	ErrSerializationFailure = fmt.Errorf("could not serialize access due to concurrent update")
	ErrRowNotFound          = fmt.Errorf("row not found")
)

Errors

View Source 
var ErrReadOnlyStorage = errors.New("storage is read-only")

ErrReadOnlyStorage is returned by a persistence backend when a caller tries to modify an artifact opened for serving. It is deliberately distinct from an OS permission error so callers can surface an actionable configuration error before any file is touched.

Functions

func AsBigRat added in v0.6.0 

func AsBigRat(v any) (*big.Rat, bool)

AsBigRat returns the value as *big.Rat if it is already a rational type (either *big.Rat or big.Rat). Unlike DecimalFromAny this does not coerce ints/floats/strings and is suitable for preserving existing numeric semantics in the engine.

func CanonicalIndexKey added in v0.20.0 

func CanonicalIndexKey(values []any) []byte

CanonicalIndexKey returns the durable, type-tagged composite encoding used by materialized secondary indexes. Paged storage uses exactly the same encoding so a SQL-bound key and an on-disk B+Tree key compare identically.

func CanonicalIndexValueEqual added in v0.22.0 

func CanonicalIndexValueEqual(left, right any) bool

CanonicalIndexValueEqual reports whether two individual values have the same durable index encoding. The hot primitive forms avoid constructing temporary []any and []byte values, while the fallback deliberately uses the canonical encoder so newly supported value types retain identical behavior.

func DecimalAdd added in v0.6.0 

func DecimalAdd(a, b any) (*big.Rat, error)

DecimalAdd returns the sum of two decimal-like values as *big.Rat.

func DecimalFromAny added in v0.6.0 

func DecimalFromAny(v any) (*big.Rat, bool)

DecimalFromAny attempts to convert a value to *big.Rat.

func DecimalToString added in v0.6.0 

func DecimalToString(r *big.Rat) string

DecimalToString returns a plain decimal string representation.

func DeriveKeyFromPassphrase added in v0.13.0 

func DeriveKeyFromPassphrase(passphrase string, salt []byte) []byte

DeriveKeyFromPassphrase derives a 32-byte AES-256 key from a passphrase using Argon2id. salt must be a fixed, persisted value of at least EncryptionSaltSize bytes (see NewEncryptionSalt) — reusing a random salt on every call would derive a different key each time and permanently lock the caller out of data encrypted under the previous key.

The cost parameters (time=1, memory=64 MiB, threads=4) follow OWASP's Argon2id baseline recommendation: expensive enough to resist large-scale offline guessing, cheap enough to run once at database-open time rather than per-operation.

func EstimateColumnSize 

func EstimateColumnSize(typ ColType) int64

EstimateColumnSize estimates the average size of a column type.

func EstimateTableSize 

func EstimateTableSize(t *Table) int64

EstimateTableSize estimates memory usage of a table in bytes.

func EstimateValueSize 

func EstimateValueSize(val any) int64

EstimateValueSize estimates the memory size of a value.

func GenerateRandomPassword added in v0.12.0 

func GenerateRandomPassword(n int) (string, error)

GenerateRandomPassword returns a cryptographically random hex-encoded password of the given byte length (before hex encoding, so the resulting string is 2*n characters) — a convenience for callers provisioning service accounts that don't need a human-memorable password.

func JSONMarshal added in v0.6.0 

func JSONMarshal(v any) ([]byte, error)

JSONMarshal marshals v after converting internal types to JSON-friendly representations (e.g. big.Rat -> string).

func NewEncryptionSalt added in v0.13.0 

func NewEncryptionSalt() ([]byte, error)

NewEncryptionSalt returns a fresh, cryptographically random salt sized for DeriveKeyFromPassphrase. Persist it (e.g. in the backend's manifest) — the same salt must be supplied on every subsequent open for the same passphrase to derive the same key.

func ParseUUID added in v0.6.0 

func ParseUUID(s string) (uuid.UUID, error)

ParseUUID parses a UUID string into uuid.UUID.

func ReadCheckpointWatermark added in v0.22.0 

func ReadCheckpointWatermark(filename string) (uint64, error)

ReadCheckpointWatermark reads the trailing uint64 a checkpoint file may carry after its table dump and catalog (see SaveToFile's extra parameter) — the LSN/Seq up to which that checkpoint already reflects every operation, used by AdvancedWAL/WALManager to skip re-applying already-checkpointed WAL records on recovery (see each Checkpoint/Recover pair). Returns 0 with no error if the file doesn't exist, is empty, or simply predates this watermark (an older checkpoint format) — all of which mean "nothing to skip," the safe default.

func SaveToBytes 

func SaveToBytes(db *DB) ([]byte, error)

SaveToBytes serializes the database snapshot to a byte slice.

func SaveToFile 

func SaveToFile(db *DB, filename string, extra ...any) error

SaveToFile writes a snapshot of the database to a file. If the filename ends with .gz, the snapshot is gzip-compressed to reduce size.

The snapshot is written atomically: data goes to a temporary file in the same directory, is fsynced, and is then renamed over the target. A crash mid-checkpoint therefore never corrupts or truncates the previous snapshot. SaveToFile writes an atomic snapshot of db (every table plus the catalog) to filename, then gob-encodes each value in extra, in order, immediately after — letting a caller persist small auxiliary state (e.g. a WAL checkpoint's last-applied LSN/Seq watermark, see AdvancedWAL.Checkpoint/WALManager.Checkpoint) atomically with the snapshot itself, via the same temp-file-then-rename step, rather than a separate file whose write could complete independently of this one and leave the two inconsistent after a crash. Existing callers that pass no extra values are unaffected; the file format is unchanged for them.

func SaveToWriter 

func SaveToWriter(db *DB, w io.Writer) error

SaveToWriter writes a snapshot of the database to an arbitrary writer. It does not attach or alter WAL configuration.

func UUIDToBytes added in v0.6.0 

func UUIDToBytes(u uuid.UUID) []byte

UUIDToBytes returns the 16-byte representation of a uuid.UUID.

Types

type AdvancedWAL 

type AdvancedWAL struct {
	// contains filtered or unexported fields
}

AdvancedWAL manages row-level write-ahead logging with full ACID guarantees.

func OpenAdvancedWAL 

func OpenAdvancedWAL(config AdvancedWALConfig) (*AdvancedWAL, error)

OpenAdvancedWAL creates or opens a WAL with full ACID semantics.

func (*AdvancedWAL) Checkpoint 

func (w *AdvancedWAL) Checkpoint(db *DB) error

Checkpoint creates a consistent snapshot and truncates the WAL.

func (*AdvancedWAL) Close 

func (w *AdvancedWAL) Close() error

Close flushes and closes the WAL.

func (*AdvancedWAL) GetCommittedLSN 

func (w *AdvancedWAL) GetCommittedLSN() LSN

GetCommittedLSN returns the LSN of the last committed transaction.

func (*AdvancedWAL) GetFlushedLSN 

func (w *AdvancedWAL) GetFlushedLSN() LSN

GetFlushedLSN returns the LSN of the last flushed record.

func (*AdvancedWAL) GetNextLSN 

func (w *AdvancedWAL) GetNextLSN() LSN

GetNextLSN returns the next LSN to be assigned.

func (*AdvancedWAL) LogAbort 

func (w *AdvancedWAL) LogAbort(txID TxID) (LSN, error)

LogAbort logs a transaction abort.

func (*AdvancedWAL) LogBegin 

func (w *AdvancedWAL) LogBegin(txID TxID) (LSN, error)

LogBegin logs the start of a transaction.

func (*AdvancedWAL) LogCommit 

func (w *AdvancedWAL) LogCommit(txID TxID) (LSN, error)

LogCommit logs a transaction commit.

func (*AdvancedWAL) LogDelete 

func (w *AdvancedWAL) LogDelete(txID TxID, tenant, table string, rowID int64, before []any, cols []Column) (LSN, error)

LogDelete logs a row deletion.

func (*AdvancedWAL) LogInsert 

func (w *AdvancedWAL) LogInsert(txID TxID, tenant, table string, rowID int64, data []any, cols []Column) (LSN, error)

LogInsert logs a row insertion.

func (*AdvancedWAL) LogUpdate 

func (w *AdvancedWAL) LogUpdate(txID TxID, tenant, table string, rowID int64, before, after []any, cols []Column) (LSN, error)

LogUpdate logs a row update with before/after images.

func (*AdvancedWAL) NewAutoTxID added in v0.12.0 

func (w *AdvancedWAL) NewAutoTxID() TxID

NewAutoTxID returns a fresh transaction ID for the engine to use as an implicit single-statement transaction (one INSERT/UPDATE/DELETE statement = one WAL transaction, autocommitted). Safe for concurrent use.

func (*AdvancedWAL) Recover 

func (w *AdvancedWAL) Recover(db *DB) (int, error)

Recover replays the WAL to restore database state after a crash.

func (*AdvancedWAL) ShouldCheckpoint 

func (w *AdvancedWAL) ShouldCheckpoint() bool

ShouldCheckpoint checks if a checkpoint is needed.

type AdvancedWALConfig 

type AdvancedWALConfig struct {
	Path               string
	CheckpointPath     string
	CheckpointEvery    uint64        // Checkpoint after N records
	CheckpointInterval time.Duration // Checkpoint after duration
	CheckpointMaxBytes int64         // Checkpoint once WAL exceeds this size (0 = 64 MB default, <0 disables)
	Compress           bool
	BufferSize         int // Buffer size for writing
}

AdvancedWALConfig configures the advanced WAL.

type AuditEntry added in v0.13.0 

type AuditEntry struct {
	Seq       uint64
	Timestamp time.Time
	Tenant    string
	User      string // empty if RBAC isn't enabled / no user was set
	Statement string // original SQL text if available, else a best-effort fallback
	Success   bool
	Error     string // populated only when !Success
	PrevHash  string // hex-encoded SHA-256 of the previous entry, "" for the first entry
	Hash      string // hex-encoded SHA-256 of this entry's own fields + PrevHash
}

AuditEntry is one recorded statement execution.

type AuditLog added in v0.13.0 

type AuditLog struct {
	// contains filtered or unexported fields
}

AuditLog is an in-memory, optionally file-backed hash-chained log. Safe for concurrent use.

func NewAuditLog added in v0.13.0 

func NewAuditLog() *AuditLog

NewAuditLog creates an in-memory-only audit log (not persisted to disk — useful for tests, or callers that only care about the current process's lifetime).

func OpenAuditLog added in v0.13.0 

func OpenAuditLog(path string) (*AuditLog, error)

OpenAuditLog opens (creating if necessary) an append-only JSONL file — one JSON-encoded AuditEntry per line, chosen over a binary format for the same reason ModeJSON exists: human-readable, diffable, greppable, and inspectable with ordinary tools during an incident, at the cost of some size on disk. If the file already has entries, they're replayed and the chain is verified as part of opening — a corrupted or tampered existing log is caught immediately at startup rather than silently trusted.

func (*AuditLog) Append added in v0.13.0 

func (a *AuditLog) Append(tenant, user, statement string, success bool, errMsg string) AuditEntry

Append records one statement execution and returns the entry, hash chain already computed. statement should be the original SQL text when available; user may be empty (RBAC not enabled, or no user in context).

func (*AuditLog) Close added in v0.13.0 

func (a *AuditLog) Close() error

Close flushes and closes the backing file, if any. A no-op for an in-memory-only log (NewAuditLog).

func (*AuditLog) Entries added in v0.13.0 

func (a *AuditLog) Entries() []AuditEntry

Entries returns a copy of every logged entry, in order.

func (*AuditLog) Verify added in v0.13.0 

func (a *AuditLog) Verify() error

Verify walks the whole chain and recomputes every hash, returning an error identifying the first entry (by Seq) where the recorded hash doesn't match — either the entry's own fields were altered, or PrevHash no longer matches its predecessor (an entry was inserted, removed, or reordered).

type BackendStats added in v0.5.4 

type BackendStats struct {
	Mode             StorageMode
	TablesInMemory   int
	TablesOnDisk     int
	MemoryUsedBytes  int64
	MemoryLimitBytes int64
	DiskUsedBytes    int64
	CacheHitRate     float64
	SyncCount        int64
	LoadCount        int64
	EvictionCount    int64
	// PageReads and cache counters are populated by page-oriented backends.
	// They are zero for legacy table-file backends which do not expose pages.
	PageReads   int64
	CacheHits   int64
	CacheMisses int64
	CachedPages int
	PinnedPages int
	// Transient pages are read-only query scratch that could not be admitted
	// while all cache frames were pinned. They are never retained by the LRU.
	TransientPages  int
	TransientFrames int
	MaxCachePages   int
}

BackendStats provides observability into storage backend behaviour.

type BufferPool 

type BufferPool struct {
	// contains filtered or unexported fields
}

BufferPool manages in-memory tables with configurable eviction.

func NewBufferPool 

func NewBufferPool(policy *MemoryPolicy) *BufferPool

NewBufferPool creates a buffer pool with the given policy.

func (*BufferPool) Get 

func (bp *BufferPool) Get(tenant, name string) (*Table, bool)

Get retrieves a table from the buffer pool.

func (*BufferPool) GetMemoryLimit 

func (bp *BufferPool) GetMemoryLimit() int64

GetMemoryLimit returns the configured memory limit.

func (*BufferPool) GetMemoryUsage 

func (bp *BufferPool) GetMemoryUsage() int64

GetMemoryUsage returns current memory usage in bytes.

func (*BufferPool) GetStats 

func (bp *BufferPool) GetStats() CacheStats

GetStats returns a copy of current statistics.

func (*BufferPool) Put 

func (bp *BufferPool) Put(tenant, name string, table *Table) error

Put adds or updates a table in the buffer pool.

func (*BufferPool) Remove 

func (bp *BufferPool) Remove(tenant, name string)

Remove removes a table from the buffer pool.

func (*BufferPool) SetEvictionSaver added in v0.23.1 

func (bp *BufferPool) SetEvictionSaver(f func(tenant, name string, table *Table))

SetEvictionSaver installs the callback evictLRU invokes on each table it is about to drop from the cache. See the evictionSaver field for the no-reentrancy constraint.

func (*BufferPool) Snapshot added in v0.23.1 

func (bp *BufferPool) Snapshot() []PooledEntry

Snapshot returns every table currently resident in the pool. The returned Table pointers are shared with the cache (and with whatever caller is still holding the lease DB.Get returned) — callers must not assume exclusive access to them.

type CacheStats 

type CacheStats struct {
	MemoryUsed        int64
	MemoryLimit       int64
	MemoryUtilization float64

	CacheHits   int64
	CacheMisses int64
	HitRate     float64

	EvictionCount int64
	EvictionSize  int64

	TablesInMemory int
	TablesOnDisk   int
	// contains filtered or unexported fields
}

CacheStats tracks buffer pool performance metrics.

type CacheStrategy 

type CacheStrategy int

CacheStrategy defines the eviction policy for the buffer pool. 

const (
	StrategyNone CacheStrategy = iota // No eviction (full in-memory)
	StrategyLRU                       // Least Recently Used
	StrategyLFU                       // Least Frequently Used (future)
	StrategyARC                       // Adaptive Replacement Cache (future)
)

func (CacheStrategy) String 

func (s CacheStrategy) String() string

type CachedTable 

type CachedTable struct {
	Table       *Table
	Size        int64
	LoadedAt    time.Time
	LastAccess  time.Time
	AccessCount int64
	Pinned      bool

	// Disk location (for lazy loading - future)
	OnDisk     bool
	DiskOffset int64
	// contains filtered or unexported fields
}

CachedTable wraps a table with caching metadata.

type CatalogColumn added in v0.5.1 

type CatalogColumn struct {
	Schema       string
	TableName    string
	Name         string
	Position     int
	DataType     string
	IsNullable   bool
	DefaultValue *string
}

CatalogColumn represents metadata for a table column CatalogColumn represents a column in a catalog table including its position and declared data type. `DefaultValue` may be nil if none is defined.

type CatalogDependency added in v0.11.0 

type CatalogDependency struct {
	Schema          string
	ObjectName      string
	ObjectType      string
	DependsOnSchema string
	DependsOnName   string
	DependsOnType   string
	DependencyType  string
	CreatedAt       time.Time
}

CatalogDependency records that one catalog object depends on another object.

type CatalogFunction added in v0.5.1 

type CatalogFunction struct {
	Schema          string
	Name            string
	FunctionType    string // 'SCALAR', 'TABLE', 'AGGREGATE', 'WINDOW'
	ArgTypes        []string
	ReturnType      string
	Language        string // 'BUILTIN', 'SQL', 'GO'
	IsDeterministic bool
	Description     string
}

CatalogFunction represents metadata for scalar and table-valued functions CatalogFunction describes registered functions (builtin or user defined). `FunctionType` categorizes the function, and `Language` indicates the implementation origin.

type CatalogIndex added in v0.18.0 

type CatalogIndex struct {
	// Tenant scopes the index definition to one logical database tenant.
	// Unlike views and functions, materialized secondary indexes belong to a
	// physical tenant table, so catalog metadata must never be shared across
	// tenants with the same schema/name.
	Tenant    string
	Schema    string
	Name      string
	Table     string
	Columns   []string
	Unique    bool
	CreatedAt time.Time
}

CatalogIndex stores metadata for a CREATE INDEX statement. tinySQL records index definitions for introspection, but the query planner does not consume them yet.

type CatalogJob added in v0.5.1 

type CatalogJob struct {
	Name         string
	SQLText      string
	ScheduleType string     // 'CRON', 'INTERVAL', 'ONCE'
	CronExpr     string     // For CRON schedule
	IntervalMs   int64      // For INTERVAL schedule (milliseconds)
	RunAt        *time.Time // For ONCE schedule
	Timezone     string
	Enabled      bool
	CatchUp      bool // Run missed executions
	NoOverlap    bool // Prevent concurrent runs
	MaxRuntimeMs int64
	LastRunAt    *time.Time
	NextRunAt    *time.Time
	CreatedAt    time.Time
	UpdatedAt    time.Time
}

CatalogJob represents a scheduled job CatalogJob describes a scheduled job. The fields provide flexible scheduling options (CRON, interval, or single-run) and execution metadata for bookkeeping and scheduling decisions.

type CatalogJobHistory added in v0.11.0 

type CatalogJobHistory struct {
	RunID        int64
	JobName      string
	StartedAt    time.Time
	FinishedAt   time.Time
	DurationMs   int64
	Status       string // 'SUCCEEDED', 'FAILED', 'SKIPPED', 'CANCELED'
	ErrorMessage string
}

CatalogJobHistory records one completed, failed, skipped, or canceled job run.

type CatalogManager added in v0.5.1 

type CatalogManager struct {
	// contains filtered or unexported fields
}

CatalogManager manages system catalog tables (`catalog.tables`, `catalog.columns`, etc.) and provides thread-safe registration and lookup helpers used by the rest of the system for introspection and scheduling. CatalogManager is safe for concurrent use.

func NewCatalogManager added in v0.5.1 

func NewCatalogManager() *CatalogManager

NewCatalogManager allocates and returns an initialized CatalogManager.

func (*CatalogManager) AddJobHistory added in v0.11.0 

func (c *CatalogManager) AddJobHistory(run *CatalogJobHistory) error

AddJobHistory appends a job execution history row and assigns a run id.

func (*CatalogManager) Authenticate added in v0.12.0 

func (c *CatalogManager) Authenticate(name, password string) bool

Authenticate verifies a password against the stored bcrypt hash in constant time (bcrypt.CompareHashAndPassword is itself constant-time with respect to the password content). Returns false for an unknown or disabled user without distinguishing why, so a caller can't use timing or error content to enumerate valid usernames.

func (*CatalogManager) CreateRole added in v0.12.0 

func (c *CatalogManager) CreateRole(name string) error

CreateRole adds a new, initially grant-less role.

func (*CatalogManager) CreateUser added in v0.12.0 

func (c *CatalogManager) CreateUser(name, password string, roles []string) error

CreateUser adds a new user with a bcrypt-hashed password. Returns an error if the name is already taken (case-insensitive) or the password is empty (an empty password would make Authenticate meaningless — bcrypt would happily hash "" too, but rejecting it here catches the mistake of e.g. an unpopulated config field at the source instead of silently creating a passwordless account).

func (*CatalogManager) DeleteIndex added in v0.18.0 

func (c *CatalogManager) DeleteIndex(schema, name string) error

DeleteIndex removes an index definition from the default tenant. New engine code should use DeleteIndexForTenant so tenant ownership is explicit.

func (*CatalogManager) DeleteIndexForTenant added in v0.22.0 

func (c *CatalogManager) DeleteIndexForTenant(tenant, schema, name string) error

DeleteIndexForTenant removes a stored index definition from one tenant.

func (*CatalogManager) DeleteIndexesForTable added in v0.18.0 

func (c *CatalogManager) DeleteIndexesForTable(table string)

DeleteIndexesForTable removes default-tenant indexes registered for a table. New engine code should use DeleteIndexesForTenantTable.

func (*CatalogManager) DeleteIndexesForTenantTable added in v0.22.0 

func (c *CatalogManager) DeleteIndexesForTenantTable(tenant, table string)

DeleteIndexesForTenantTable removes all indexes registered for a table in one tenant only.

func (*CatalogManager) DeleteJob added in v0.5.1 

func (c *CatalogManager) DeleteJob(name string) error

DeleteJob removes a job from the catalog, returning an error when the job does not exist.

func (*CatalogManager) DeleteMaterializedView added in v0.11.0 

func (c *CatalogManager) DeleteMaterializedView(schema, name string) error

DeleteMaterializedView removes a materialized view definition.

func (*CatalogManager) DeleteView added in v0.11.0 

func (c *CatalogManager) DeleteView(schema, name string) error

DeleteView removes a view definition and its catalog table entry.

func (*CatalogManager) DropRole added in v0.12.0 

func (c *CatalogManager) DropRole(name string) error

DropRole removes a role. Existing users keep a dangling membership reference (harmless: GrantRoleToUser/permission resolution both treat a reference to a since-dropped role as granting nothing), consistent with how DropTrigger etc. don't cascade either.

func (*CatalogManager) DropTrigger added in v0.9.0 

func (c *CatalogManager) DropTrigger(name string) error

DropTrigger removes a named trigger from the catalog. It returns an error when the trigger does not exist.

func (*CatalogManager) DropUser added in v0.12.0 

func (c *CatalogManager) DropUser(name string) error

DropUser removes a user. Returns an error if it doesn't exist.

func (*CatalogManager) FinishMaterializedViewRefresh added in v0.11.0 

func (c *CatalogManager) FinishMaterializedViewRefresh(schema, name string, refreshedAt time.Time, durationMs int64, rowCount int64, errMsg string) error

FinishMaterializedViewRefresh updates refresh bookkeeping.

func (*CatalogManager) GetAllColumns added in v0.5.1 

func (c *CatalogManager) GetAllColumns() []CatalogColumn

GetAllColumns aggregates and returns columns registered for every table in the catalog. The returned slice is a concatenation of the internal per-table column lists.

func (*CatalogManager) GetColumns added in v0.5.1 

func (c *CatalogManager) GetColumns(schema, tableName string) []CatalogColumn

GetColumns returns the column metadata for `schema.tableName`. If the table is unknown an empty slice is returned.

func (*CatalogManager) GetDependencies added in v0.11.0 

func (c *CatalogManager) GetDependencies() []CatalogDependency

GetDependencies returns all recorded catalog dependencies.

func (*CatalogManager) GetFunctions added in v0.5.1 

func (c *CatalogManager) GetFunctions() []*CatalogFunction

GetFunctions returns metadata for all registered functions.

func (*CatalogManager) GetIndex added in v0.18.0 

func (c *CatalogManager) GetIndex(schema, name string) (*CatalogIndex, bool)

GetIndex retrieves a default-tenant index definition by schema and name. New engine code should use GetIndexForTenant.

func (*CatalogManager) GetIndexForTenant added in v0.22.0 

func (c *CatalogManager) GetIndexForTenant(tenant, schema, name string) (*CatalogIndex, bool)

GetIndexForTenant retrieves one tenant's index definition by schema and name. Legacy unscoped catalog records intentionally do not match: guessing their owner would let one tenant read or delete another tenant's metadata.

func (*CatalogManager) GetIndexes added in v0.18.0 

func (c *CatalogManager) GetIndexes() []*CatalogIndex

GetIndexes returns all registered index definitions, including legacy unscoped records. It exists for administrative callers; query execution and virtual tables must use GetIndexesForTenant.

func (*CatalogManager) GetIndexesForTenant added in v0.22.0 

func (c *CatalogManager) GetIndexesForTenant(tenant string) []*CatalogIndex

GetIndexesForTenant returns only index definitions owned by tenant. Legacy records with an empty tenant are deliberately excluded rather than being guessed as a particular tenant.

func (*CatalogManager) GetJob added in v0.5.1 

func (c *CatalogManager) GetJob(name string) (*CatalogJob, error)

GetJob retrieves a job by name, returning an error if not found.

func (*CatalogManager) GetMaterializedView added in v0.11.0 

func (c *CatalogManager) GetMaterializedView(schema, name string) (*CatalogMaterializedView, bool)

GetMaterializedView retrieves a materialized view definition by schema/name.

func (*CatalogManager) GetMaterializedViews added in v0.11.0 

func (c *CatalogManager) GetMaterializedViews() []*CatalogMaterializedView

GetMaterializedViews returns all materialized view definitions.

func (*CatalogManager) GetTables added in v0.5.1 

func (c *CatalogManager) GetTables() []*CatalogTable

GetTables returns a slice with metadata for all registered tables and views.

func (*CatalogManager) GetTriggers added in v0.9.0 

func (c *CatalogManager) GetTriggers(table string, timing TriggerTiming, event TriggerEvent) []*CatalogTrigger

GetTriggers returns all triggers for the given table, timing, and event.

func (*CatalogManager) GetTriggersForEvent added in v0.23.1 

func (c *CatalogManager) GetTriggersForEvent(table string, event TriggerEvent) (before, after []*CatalogTrigger)

GetTriggersForEvent returns the BEFORE and AFTER row-trigger lists for one table/event pair while holding the catalog lock only once. DML executors resolve both timings together before entering their row loop, avoiding two full catalog scans per statement and any lookup work per affected row.

func (*CatalogManager) GetUser added in v0.12.0 

func (c *CatalogManager) GetUser(name string) (CatalogUser, bool)

GetUser returns a copy of a user's metadata (never the password hash's caller-mutable backing, since Roles is copied) for introspection, or false if it doesn't exist.

func (*CatalogManager) GetView added in v0.11.0 

func (c *CatalogManager) GetView(schema, name string) (*CatalogView, bool)

GetView retrieves a view definition by schema and name.

func (*CatalogManager) GetViews added in v0.5.1 

func (c *CatalogManager) GetViews() []*CatalogView

GetViews returns metadata for all registered views.

func (*CatalogManager) GrantPermission added in v0.12.0 

func (c *CatalogManager) GrantPermission(roleName string, perm Permission, schema, table string) error

GrantPermission adds a grant to an existing role. schema/table accept "*" as a wildcard.

func (*CatalogManager) GrantRoleToUser added in v0.12.0 

func (c *CatalogManager) GrantRoleToUser(userName, roleName string) error

GrantRoleToUser adds a role membership to an existing user. Idempotent — granting a role the user already has is not an error.

func (*CatalogManager) HasPermission added in v0.12.0 

func (c *CatalogManager) HasPermission(userName string, perm Permission, schema, table string) bool

HasPermission resolves userName's roles to their grants and reports whether any grant authorizes perm on (schema, table). Returns false for an unknown user (fail closed).

func (*CatalogManager) HasUsers added in v0.12.0 

func (c *CatalogManager) HasUsers() bool

HasUsers reports whether any user has been created. Note this ignores SetRBACEnabled's override — use IsRBACEnabled to answer "is enforcement actually active right now", which is what Execute checks.

func (*CatalogManager) IsRBACEnabled added in v0.13.0 

func (c *CatalogManager) IsRBACEnabled() bool

IsRBACEnabled reports whether Execute currently enforces permissions: true if SetRBACEnabled(false) was never called (or was reverted via SetRBACEnabled(true)) and at least one user exists.

func (*CatalogManager) ListEnabledJobs added in v0.5.1 

func (c *CatalogManager) ListEnabledJobs() []*CatalogJob

ListEnabledJobs returns only jobs whose `Enabled` flag is true.

func (*CatalogManager) ListJobHistory added in v0.11.0 

func (c *CatalogManager) ListJobHistory() []*CatalogJobHistory

ListJobHistory returns all job execution history rows.

func (*CatalogManager) ListJobs added in v0.5.1 

func (c *CatalogManager) ListJobs() []*CatalogJob

ListJobs returns a slice containing all registered jobs.

func (*CatalogManager) ListRoles added in v0.12.0 

func (c *CatalogManager) ListRoles() []CatalogRole

ListRoles returns all roles, sorted by name.

func (*CatalogManager) ListTriggers added in v0.9.0 

func (c *CatalogManager) ListTriggers() []*CatalogTrigger

ListTriggers returns all registered trigger definitions.

func (*CatalogManager) ListUsers added in v0.12.0 

func (c *CatalogManager) ListUsers() []CatalogUser

ListUsers returns all users (without password hashes), sorted by name.

func (*CatalogManager) MarkMaterializedViewsStaleByDependency added in v0.11.0 

func (c *CatalogManager) MarkMaterializedViewsStaleByDependency(schema, changedName string) []string

MarkMaterializedViewsStaleByDependency marks opt-in materialized views stale when they depend on the changed object. It returns the affected view names.

func (*CatalogManager) RegisterFunction added in v0.5.1 

func (c *CatalogManager) RegisterFunction(fn *CatalogFunction) error

RegisterFunction registers or updates a function's metadata.

func (*CatalogManager) RegisterIndex added in v0.18.0 

func (c *CatalogManager) RegisterIndex(idx *CatalogIndex) error

RegisterIndex stores index metadata in the default tenant. New engine code should use RegisterIndexForTenant so tenant ownership is explicit.

func (*CatalogManager) RegisterIndexForTenant added in v0.22.0 

func (c *CatalogManager) RegisterIndexForTenant(tenant string, idx *CatalogIndex) error

RegisterIndexForTenant stores index metadata for one tenant. If an index with the same tenant/schema/name exists, it is replaced.

func (*CatalogManager) RegisterJob added in v0.5.1 

func (c *CatalogManager) RegisterJob(job *CatalogJob) error

RegisterJob adds a new scheduled job or updates an existing entry. Job names must be non-empty.

func (*CatalogManager) RegisterMaterializedView added in v0.11.0 

func (c *CatalogManager) RegisterMaterializedView(mv *CatalogMaterializedView) error

RegisterMaterializedView registers or updates a materialized view definition.

func (*CatalogManager) RegisterTable added in v0.5.1 

func (c *CatalogManager) RegisterTable(schema, name string, cols []Column) error

RegisterTable registers a table and its columns in the system catalog. The provided `cols` slice is converted to `CatalogColumn` entries and stored under the key `schema.name`.

func (*CatalogManager) RegisterTrigger added in v0.9.0 

func (c *CatalogManager) RegisterTrigger(t *CatalogTrigger) error

RegisterTrigger stores a trigger definition in the catalog. If a trigger with the same name already exists it is replaced.

func (*CatalogManager) RegisterView added in v0.5.1 

func (c *CatalogManager) RegisterView(schema, name, sqlText string) error

RegisterView registers a view definition under `schema.name`.

func (*CatalogManager) Revision added in v0.24.0 

func (c *CatalogManager) Revision() uint64

Revision returns a counter that increases on every catalog mutation. It is only meaningful for comparison against an earlier reading of the same catalog instance (or of a snapshot taken from it), never across instances.

func (*CatalogManager) RevokePermission added in v0.12.0 

func (c *CatalogManager) RevokePermission(roleName string, perm Permission, schema, table string) error

RevokePermission removes every grant on a role matching perm/schema/table exactly (same wildcard values as originally granted — this is a simple exact-match revoke, not a partial-overlap subtraction).

func (*CatalogManager) RevokeRoleFromUser added in v0.12.0 

func (c *CatalogManager) RevokeRoleFromUser(userName, roleName string) error

RevokeRoleFromUser removes a role membership. Not an error if the user didn't have that role.

func (*CatalogManager) SetDependencies added in v0.11.0 

func (c *CatalogManager) SetDependencies(schema, objectName, objectType string, deps []CatalogDependency)

SetDependencies replaces the dependency list for a catalog object.

func (*CatalogManager) SetRBACEnabled added in v0.13.0 

func (c *CatalogManager) SetRBACEnabled(enabled bool)

SetRBACEnabled overrides the default opt-in-via-CreateUser behavior. Pass false to force RBAC off — every Execute call is permitted regardless of context, even if users/roles already exist — useful for a setup that provisions accounts ahead of time but isn't ready to enforce yet, or a dev/test environment that wants audit-log attribution (see WithUser) without access checks getting in the way. Pass true to restore the default: enforcement follows HasUsers() exactly as if SetRBACEnabled had never been called.

This is a deliberate escape hatch matching SetReadOnly's toggle pattern elsewhere in this package — RBAC being opt-in-by-default already covers "not needed in every setup" for anyone who simply never calls CreateUser; this method is for the narrower case of a database that *has* users defined but shouldn't enforce against them right now.

func (*CatalogManager) SetUserDisabled added in v0.12.0 

func (c *CatalogManager) SetUserDisabled(name string, disabled bool) error

SetUserDisabled enables/disables a user without deleting their account (preserving audit-log attribution history for anything they did while enabled). A disabled user fails Authenticate.

func (*CatalogManager) TryBeginMaterializedViewRefresh added in v0.11.0 

func (c *CatalogManager) TryBeginMaterializedViewRefresh(schema, name string) bool

TryBeginMaterializedViewRefresh marks a materialized view as refreshing.

func (*CatalogManager) UpdateJobRuntime added in v0.5.1 

func (c *CatalogManager) UpdateJobRuntime(name string, lastRun, nextRun time.Time) error

UpdateJobRuntime updates runtime bookkeeping fields for a named job. It sets `LastRunAt`, `NextRunAt` and marks the job as recently updated.

func (*CatalogManager) UpdateJobRuntimePtr added in v0.11.0 

func (c *CatalogManager) UpdateJobRuntimePtr(name string, lastRun time.Time, nextRun *time.Time) error

UpdateJobRuntimePtr updates runtime bookkeeping fields for a named job. Passing nil for nextRun clears NextRunAt, which is expected for completed one-shot jobs.

type CatalogMaterializedView added in v0.11.0 

type CatalogMaterializedView struct {
	Schema             string
	Name               string
	SQLText            string
	CacheTableName     string
	StaleAfterMs       int64
	RefreshEveryMs     int64
	DailyAt            string
	Timezone           string
	WithData           bool
	InvalidateOnChange bool
	IsStale            bool
	LastRefreshAt      *time.Time
	LastDurationMs     int64
	LastError          string
	IsRefreshing       bool
	CreatedAt          time.Time
	UpdatedAt          time.Time
}

CatalogMaterializedView stores a saved query with physical refresh metadata.

type CatalogRole added in v0.12.0 

type CatalogRole struct {
	Name      string
	Grants    []Grant
	CreatedAt time.Time
}

CatalogRole is a named bundle of grants that one or more users can hold.

type CatalogSnapshot added in v0.24.0 

type CatalogSnapshot struct {
	// contains filtered or unexported fields
}

CatalogSnapshot is an opaque, comparable copy of a catalog's contents. It exists so the SQL driver can tell "this transaction created a view" from "this transaction ran an INSERT that happened to touch catalog bookkeeping", which a revision counter alone cannot distinguish.

func (CatalogSnapshot) Equal added in v0.24.0 

func (s CatalogSnapshot) Equal(other CatalogSnapshot) bool

Equal reports whether two snapshots hold the same catalog contents.

type CatalogTable added in v0.5.1 

type CatalogTable struct {
	Schema    string
	Name      string
	Type      string // 'TABLE', 'VIEW', 'SYSTEM'
	RowCount  int64
	CreatedAt time.Time
	UpdatedAt time.Time
}

CatalogTable represents metadata for a single table CatalogTable holds basic metadata for a registered table or view.

type CatalogTrigger added in v0.9.0 

type CatalogTrigger struct {
	Name       string
	Table      string
	Timing     TriggerTiming
	Event      TriggerEvent
	ForEachRow bool
	WhenExpr   string // optional WHEN clause SQL text
	Body       string // SQL text of trigger body (semicolon-separated stmts)
	CreatedAt  time.Time
}

CatalogTrigger holds a stored trigger definition.

type CatalogUser added in v0.12.0 

type CatalogUser struct {
	Name         string
	PasswordHash string
	Roles        []string
	Disabled     bool
	CreatedAt    time.Time
}

CatalogUser is an authenticatable principal. Passwords are never stored in plaintext — only a bcrypt hash, which is itself salted and safe to treat as opaque (bcrypt embeds its own per-hash random salt).

type CatalogView added in v0.5.1 

type CatalogView struct {
	Schema    string
	Name      string
	SQLText   string
	CreatedAt time.Time
}

CatalogView represents a saved view definition CatalogView stores the definition of a saved view.

type ColType 

type ColType int

ColType enumerates supported column data types. 

const (
	// IntType is a generic integer column type.
	IntType ColType = iota
	// Int8Type is an 8-bit signed integer column type.
	Int8Type
	// Int16Type is a 16-bit signed integer column type.
	Int16Type
	// Int32Type is a 32-bit signed integer column type.
	Int32Type
	// Int64Type is a 64-bit signed integer column type.
	Int64Type
	// UintType is an unsigned integer column type.
	UintType
	// Uint8Type is an 8-bit unsigned integer column type.
	Uint8Type
	// Uint16Type is a 16-bit unsigned integer column type.
	Uint16Type
	// Uint32Type is a 32-bit unsigned integer column type.
	Uint32Type
	// Uint64Type is a 64-bit unsigned integer column type.
	Uint64Type

	// Float32Type is a 32-bit floating point column type.
	Float32Type
	// Float64Type is a 64-bit floating point column type.
	Float64Type
	// FloatType is an alias for Float64Type.
	FloatType // alias for Float64Type

	// StringType represents a variable-length UTF-8 string column.
	StringType
	// TextType is an alias for StringType intended for long text.
	TextType // alias for StringType
	// RuneType stores single Unicode code points.
	RuneType
	// ByteType stores raw byte data.
	ByteType

	// BoolType represents a boolean column (true/false).
	BoolType

	// TimeType stores time-of-day values.
	TimeType
	// DateType stores date-only values.
	DateType
	// DateTimeType stores combined date and time values.
	DateTimeType
	// TimestampType stores an absolute point in time.
	TimestampType
	// DurationType stores a time duration.
	DurationType

	// DecimalType stores arbitrary-precision decimal numbers.
	DecimalType
	// MoneyType is a convenience alias for DecimalType used for monetary values.
	MoneyType
	// UUIDType stores RFC-4122 UUID values.
	UUIDType
	// BlobType stores binary large objects.
	BlobType
	// XMLType stores XML text.
	XMLType
	// IntervalType stores SQL-like intervals (parsed to time.Duration when possible).
	IntervalType

	// JsonType stores JSON text.
	JsonType
	// JsonbType stores binary JSON representations.
	JsonbType
	// MapType stores map-like complex values.
	MapType
	// SliceType stores slice-like complex values.
	SliceType
	// ArrayType stores array-like complex values.
	ArrayType

	// Complex64Type stores complex64 numeric values.
	Complex64Type
	// Complex128Type stores complex128 numeric values.
	Complex128Type
	// ComplexType is an alias for Complex128Type.
	ComplexType // alias for Complex128Type
	// PointerType represents a pointer/reference to another object.
	PointerType
	// InterfaceType represents an arbitrary Go interface value.
	InterfaceType

	// VectorType represents a vector/embedding column used by RAG features.
	VectorType
	// GeometryType stores spatial geometry values (GeoJSON/WKB) as JSONB or binary payload.
	GeometryType

	// YAMLType stores YAML-formatted text data.
	YAMLType
	// URLType stores URL/URI values with optional validation.
	URLType
	// HASHType stores cryptographic hash digests (hex-encoded).
	HASHType
	// BitmapType stores roaring-bitmap or bitset values as a byte slice.
	BitmapType
)

func (ColType) String 

func (t ColType) String() string

type Column 

type Column struct {
	Name string
	Type ColType
	// DeclaredType retains the source SQL spelling (for example VARCHAR(80)
	// or DOUBLE PRECISION). It is intentionally metadata; the physical value
	// representation remains ColType plus SQLiteAffinity.
	DeclaredType string
	// Affinity is populated for SQLite-style declarations. AffinityDefault
	// means this column was declared using a native tinySQL type.
	Affinity SQLiteAffinity
	// NotNull and default metadata are independent of Constraint because SQL
	// permits combinations such as "PRIMARY KEY NOT NULL DEFAULT 0".
	NotNull      bool
	HasDefault   bool
	DefaultValue any
	Constraint   ConstraintType
	ForeignKey   *ForeignKeyRef // Only used if Constraint == ForeignKey
	PointerTable string         // Target table for POINTER type
}

Column holds column schema information in a table.

type ColumnStats added in v0.21.0 

type ColumnStats struct {
	NullCount     int
	DistinctCount int
	Min           string
	Max           string
	HasMinMax     bool
}

ColumnStats summarizes one column as of TableStats.AnalyzedAt. Min and Max are display values for introspection; the planner currently uses row and distinct counts, which remain meaningful across all supported column types.

type ConstraintType 

type ConstraintType int

ConstraintType enumerates supported column constraints. 

const (
	NoConstraint ConstraintType = iota
	PrimaryKey
	ForeignKey
	Unique
)

func (ConstraintType) String 

func (c ConstraintType) String() string

type DB 

type DB struct {
	// contains filtered or unexported fields
}

DB is an in-memory, multi-tenant database catalog with full MVCC support. It optionally delegates storage to a StorageBackend for disk-based or hybrid persistence strategies.

func LoadFromBytes 

func LoadFromBytes(b []byte) (*DB, error)

LoadFromBytes loads a database from a byte slice.

func LoadFromFile 

func LoadFromFile(filename string) (*DB, error)

LoadFromFile loads a database snapshot from a file. It auto-detects gzip compression based on the .gz suffix and attaches a WAL if a path is given.

func LoadFromReader 

func LoadFromReader(r io.Reader) (*DB, error)

LoadFromReader loads a database snapshot from an arbitrary reader. The returned DB has no WAL attached.

func NewDB 

func NewDB() *DB

NewDB creates a new empty database catalog with MVCC support. The database operates in ModeMemory (pure in-memory) by default. Use OpenDB for disk-backed or hybrid storage modes.

func OpenDB added in v0.5.4 

func OpenDB(cfg StorageConfig) (*DB, error)

OpenDB creates or opens a database with the specified storage configuration. For ModeMemory this is equivalent to NewDB (with optional save-on-close). For ModeDisk/ModeJSON/ModeHybrid/ModeIndex, tables are stored as individual files in the configured directory. For ModeWAL, the existing WAL mechanism is configured automatically.

func (*DB) AbortAmbientWALTx added in v0.24.0 

func (db *DB) AbortAmbientWALTx() error

AbortAmbientWALTx writes the abort record for the ambient transaction and clears it, so recovery discards whatever the rolled-back block logged.

func (*DB) AdoptCatalog added in v0.24.0 

func (db *DB) AdoptCatalog(catalog *CatalogManager)

AdoptCatalog installs a catalog produced elsewhere as this database's own. The SQL driver calls it at COMMIT to publish the catalog work a transaction did on its private copy — creating a view or trigger, granting a permission, marking a materialized view stale. Those changes are not table data and so are invisible to ApplyWALChanges; before this existed they were silently discarded by a successful commit.

The caller is responsible for having established that no concurrent change is being overwritten (see conn.commitTx's revision check). The revision counter is carried forward so it stays monotonic for the next comparison.

func (*DB) AdvancedWAL 

func (db *DB) AdvancedWAL() *AdvancedWAL

AdvancedWAL returns the configured advanced WAL manager (may be nil).

func (*DB) AmbientWALTx added in v0.24.0 

func (db *DB) AmbientWALTx() (TxID, bool)

AmbientWALTx returns the ambient AdvancedWAL transaction, if one is open.

func (*DB) ApplyWALChanges added in v0.18.0 

func (db *DB) ApplyWALChanges(changes []WALChange) error

ApplyWALChanges applies a set of table-level changes to the database. It is used by the SQL driver to commit a transaction by merging the transaction's delta into the latest shared database instead of replacing the whole DB with an older snapshot.

func (*DB) AttachAdvancedWAL 

func (db *DB) AttachAdvancedWAL(wal *AdvancedWAL)

AttachAdvancedWAL attaches an advanced WAL to the database.

func (*DB) AttachAuditLog added in v0.13.0 

func (db *DB) AttachAuditLog(log *AuditLog)

AttachAuditLog attaches a tamper-evident audit log to the database. Once attached, internal/engine.Execute records every statement to it (see internal/engine/audit.go). Pass nil to detach (stop logging); Execute treats a nil audit log as "logging disabled", matching the opt-in pattern used throughout this session's hardening work (RBAC, read-only mode, the security warning in cmd/server).

func (*DB) AuditLog added in v0.13.0 

func (db *DB) AuditLog() *AuditLog

AuditLog returns the attached audit log, or nil if none is configured.

func (*DB) Backend added in v0.5.4 

func (db *DB) Backend() StorageBackend

Backend returns the attached StorageBackend (may be nil for pure in-memory databases created with NewDB).

func (*DB) BackendStats added in v0.5.4 

func (db *DB) BackendStats() BackendStats

BackendStats returns statistics from the storage backend. Returns a zero-value BackendStats if no backend is attached.

func (*DB) BeginAmbientWALTx added in v0.24.0 

func (db *DB) BeginAmbientWALTx() (bool, error)

BeginAmbientWALTx opens one AdvancedWAL transaction that every subsequent statement executed against db joins, instead of each statement logging its own implicitly-committed transaction. The SQL driver calls it on a transaction shadow at BEGIN so that recovery replays the block only if CommitAmbientWALTx ran; AbortAmbientWALTx marks it rolled back.

It is idempotent, so the driver can call it lazily before the first write in a block instead of writing a begin record for every read-only BEGIN. It is a no-op returning false when no AdvancedWAL is attached.

func (*DB) Catalog added in v0.5.1 

func (db *DB) Catalog() *CatalogManager

Catalog returns the CatalogManager attached to the DB, creating one lazily if necessary. The catalog is reached by every SQL statement for RBAC checks, so its one-time construction must be synchronized when a database/sql connector opens several physical reader connections at once.

func (*DB) Close added in v0.5.4 

func (db *DB) Close() error

Close persists all data and releases resources. For ModeMemory with a configured path, this saves a final GOB snapshot. For ModeDisk/ModeJSON/ ModeHybrid, dirty tables are flushed. WAL and Advanced WAL resources are closed.

func (*DB) CommitAmbientWALTx added in v0.24.0 

func (db *DB) CommitAmbientWALTx() error

CommitAmbientWALTx writes the commit record for the ambient transaction and clears it. Recovery replays the transaction's operations only after seeing this record.

func (*DB) Config added in v0.5.4 

func (db *DB) Config() *StorageConfig

Config returns the StorageConfig used to open this database. Returns nil for databases created with NewDB().

func (*DB) DeepClone 

func (db *DB) DeepClone() *DB

DeepClone creates a full copy of the database (MVCC-light snapshot). Note: This is not copy-on-write; it creates a full copy (simple but O(n)).

The result is a shadow: it carries the full runtime state and schema of the original, but statements executed against it do not write to the WAL. Call PromoteShadow if it is going to replace the live database.

func (*DB) DeleteJob added in v0.11.0 

func (db *DB) DeleteJob(name string) error

DeleteJob deletes a job and unschedules/cancels it when present.

func (*DB) Drop 

func (db *DB) Drop(tn, name string) error

Drop removes a table from the tenant (and from the backend if attached).

func (*DB) Evict added in v0.5.4 

func (db *DB) Evict(tenant, name string) error

Evict removes a table from the in-memory cache without deleting it from the backend. This is only meaningful for disk-backed modes; in ModeMemory the data would be lost. Returns an error if no backend is attached.

func (*DB) Extensions added in v0.21.0 

func (db *DB) Extensions() []ExtensionInfo

Extensions returns a deterministic snapshot of extensions active for this DB. Mutating the returned values cannot change the database registry.

func (*DB) Get 

func (db *DB) Get(tn, name string) (*Table, error)

Get returns a table by name for the given tenant. When a StorageBackend is attached, tables not found in memory are loaded from the backend on demand (lazy loading).

func (*DB) HealthCheck added in v0.11.0 

func (db *DB) HealthCheck() DBHealth

HealthCheck returns a production-oriented lifecycle and storage snapshot.

func (*DB) IsClosed added in v0.24.0 

func (db *DB) IsClosed() bool

IsClosed reports whether Close has completed for this database. The SQL driver uses it to let the connection pool discard connections whose database went away underneath them, rather than returning errors from each one.

func (*DB) IsRBACEnabled added in v0.13.0 

func (db *DB) IsRBACEnabled() bool

IsRBACEnabled reports whether Execute currently enforces RBAC permissions. A convenience delegate to db.Catalog().IsRBACEnabled.

func (*DB) IsReadOnly added in v0.11.0 

func (db *DB) IsReadOnly() bool

IsReadOnly reports whether the database is in read-only mode.

func (*DB) IsShadow added in v0.24.0 

func (db *DB) IsShadow() bool

IsShadow reports whether this database is an uncommitted working copy.

func (*DB) JobScheduler added in v0.11.0 

func (db *DB) JobScheduler() *Scheduler

JobScheduler returns the active database job scheduler, if any.

func (*DB) ListTables 

func (db *DB) ListTables(tn string) []*Table

ListTables returns the tables in a tenant sorted by name. When a StorageBackend is attached, tables that exist on disk but are not currently loaded into memory are loaded on demand.

func (*DB) ListTenants added in v0.5.4 

func (db *DB) ListTenants() []string

ListTenants returns the names of all tenants that have at least one table.

func (*DB) LockContentForRead added in v0.12.0 

func (db *DB) LockContentForRead()

LockContentForRead acquires the database's content lock for a read-only statement (SELECT/EXPLAIN/PRAGMA). Multiple readers may hold this concurrently; it excludes concurrent LockContentForWrite callers. Callers must call UnlockContentForRead exactly once, typically via defer, and must not call it again re-entrantly on the same goroutine (sync.RWMutex is not reentrant) — nested statement execution within the engine must bypass this lock rather than re-acquire it.

func (*DB) LockContentForWrite added in v0.12.0 

func (db *DB) LockContentForWrite()

LockContentForWrite acquires the database's content lock exclusively, for any statement that may mutate table rows/columns (INSERT/UPDATE/DELETE and all DDL). See LockContentForRead for the re-entrancy caveat.

func (*DB) MVCC 

func (db *DB) MVCC() *MVCCManager

MVCC returns the MVCC manager.

func (*DB) MetaSnapshot added in v0.24.0 

func (db *DB) MetaSnapshot() *DB

MetaSnapshot captures every table's identity and Version but none of its rows. It is the "before" image the engine diffs against to decide what one statement changed for WALManager logging.

It replaces the previous approach of reusing the statement's full rollback snapshot for that diff, which forced a deep copy of the entire database on every INSERT into a WAL-backed database and still could not see a CREATE or DROP TABLE, because DDL takes no rollback snapshot at all. A metadata snapshot costs O(number of tables) and detects created, dropped and mutated tables alike.

func (*DB) MigrateToBackend added in v0.5.4 

func (db *DB) MigrateToBackend(b StorageBackend) error

MigrateToBackend copies all in-memory tables to the given backend and attaches it. This enables migrating a ModeMemory database to ModeDisk or ModeHybrid at runtime.

func (*DB) PagedIndexMetadata added in v0.20.0 

func (db *DB) PagedIndexMetadata(tenant, table string) (*Table, bool, error)

PagedIndexMetadata returns a schema-only table for the immutable ModePagedIndex backend. The returned table has column and secondary-index metadata but no rows; it lets a planner select an on-disk index before a full-table compatibility load is considered.

func (*DB) PagedIndexRows added in v0.20.0 

func (db *DB) PagedIndexRows(tenant, table, indexName string, values []any) ([][]any, bool, error)

PagedIndexRows performs an exact composite seek in a ModePagedIndex artifact. The boolean is false when no physical index with that name exists; a true result with an empty row slice is a valid negative lookup.

func (*DB) PromoteShadow added in v0.24.0 

func (db *DB) PromoteShadow()

PromoteShadow clears the shadow flag, declaring that this database's contents are committed and that statements executed against it from now on are responsible for their own write-ahead logging. Only the SQL driver calls it, after it has durably logged the transaction that produced the shadow.

func (*DB) Put 

func (db *DB) Put(tn string, t *Table) error

Put adds a new table to the tenant; returns error if it already exists. When a StorageBackend is attached, the table is also checked against the backend to prevent duplicates, and optionally persisted immediately when SyncOnMutate is configured.

func (*DB) RegisterJob added in v0.11.0 

func (db *DB) RegisterJob(job *CatalogJob) error

RegisterJob registers a job and refreshes the live scheduler when present.

func (*DB) RestartJobScheduler added in v0.11.0 

func (db *DB) RestartJobScheduler(executor JobExecutor) error

RestartJobScheduler restarts the database job scheduler with the executor.

func (*DB) RestoreStatementSnapshot added in v0.21.0 

func (db *DB) RestoreStatementSnapshot(snapshot *StatementSnapshot)

RestoreStatementSnapshot rolls a database back to snapshot. It restores pre-existing tables in place and removes tables created by the failed statement. The caller must hold DB's content write lock.

func (*DB) SetBackend added in v0.5.4 

func (db *DB) SetBackend(b StorageBackend)

SetBackend attaches a StorageBackend and sets the storage mode. This is primarily used internally by OpenDB; calling it on a running database should be done with care.

func (*DB) SetRBACEnabled added in v0.13.0 

func (db *DB) SetRBACEnabled(enabled bool)

SetRBACEnabled overrides RBAC's default opt-in-via-CreateUser behavior; see CatalogManager.SetRBACEnabled for the full explanation. A convenience delegate to db.Catalog().SetRBACEnabled, provided directly on DB to mirror SetReadOnly above (the same "toggle a behavior" shape).

func (*DB) SetReadOnly added in v0.11.0 

func (db *DB) SetReadOnly(ro bool)

SetReadOnly toggles read-only mode. While enabled, the SQL engine rejects INSERT/UPDATE/DELETE/DDL with an error; SELECT, EXPLAIN, and PRAGMA still work. Safe to call concurrently with running queries.

func (*DB) SnapshotCatalog added in v0.24.0 

func (db *DB) SnapshotCatalog() CatalogSnapshot

SnapshotCatalog copies this database's catalog contents for later comparison.

func (*DB) SnapshotForAppendOnlyTableStatement added in v0.22.0 

func (db *DB) SnapshotForAppendOnlyTableStatement(tenant, name string) (*StatementSnapshot, error)

SnapshotForAppendOnlyTableStatement captures the lightweight rollback state for a statement that can only append rows to one table. It is intended for the executor's triggerless, index-free INSERT fast path; other callers must use SnapshotForTableStatement or SnapshotForStatement.

func (*DB) SnapshotForStatement added in v0.21.0 

func (db *DB) SnapshotForStatement() *StatementSnapshot

SnapshotForStatement captures all table and catalog state needed to undo a failed mutating statement, including mutations issued by its triggers. The caller must already hold DB's content write lock.

func (*DB) SnapshotForTableStatement added in v0.22.0 

func (db *DB) SnapshotForTableStatement(tenant, name string) (*StatementSnapshot, error)

SnapshotForTableStatement captures one table and the catalog state needed to roll back a statement known to mutate only that table. Callers must use SnapshotForStatement instead when triggers or foreign-key actions could affect other tables. The caller must already hold DB's content write lock.

func (*DB) SnapshotForTx added in v0.21.0 

func (db *DB) SnapshotForTx() (base *DB, shadow *DB)

SnapshotForTx creates the pair of snapshots a SQL transaction needs while copying row data only once instead of twice.

shadow is a full deep clone that receives the transaction's writes. base is a lightweight snapshot that records each table's identity and Version but no rows: the only consumers of the base — CollectWALChanges and the driver's conflict detection — read Table.Version and table existence exclusively and never inspect rows. Copying rows into the base (as DeepClonePair does) would therefore waste memory proportional to the entire database on every Begin.

func (*DB) StartJobScheduler added in v0.11.0 

func (db *DB) StartJobScheduler(executor JobExecutor) error

StartJobScheduler starts the database job scheduler with the given executor.

func (*DB) StatementAdvancedWAL added in v0.24.0 

func (db *DB) StatementAdvancedWAL() *AdvancedWAL

StatementAdvancedWAL is the AdvancedWAL counterpart of StatementWAL. Unlike the basic WALManager, AdvancedWAL records row-level operations grouped into explicit transactions, so a shadow does not have to stop logging: it logs into the ambient transaction the driver opened at BEGIN (see BeginAmbientWALTx), which recovery only replays once a matching commit record exists.

func (*DB) StatementWAL added in v0.24.0 

func (db *DB) StatementWAL() *WALManager

StatementWAL returns the WALManager that a statement executing against db must append to, or nil when statement-level logging is not this database's responsibility.

It differs from WAL in exactly one case: a transaction shadow, where it returns nil. A statement inside BEGIN…COMMIT is not durable until the transaction commits, so logging it as it runs would leave a committed-looking record on disk that recovery replays even when the transaction was rolled back. The driver logs the transaction as a whole against the live database instead.

func (*DB) StopJobScheduler added in v0.11.0 

func (db *DB) StopJobScheduler()

StopJobScheduler stops the database job scheduler if it is running.

func (*DB) StorageMode added in v0.5.4 

func (db *DB) StorageMode() StorageMode

StorageMode returns the active storage mode.

func (*DB) Sync added in v0.5.4 

func (db *DB) Sync() error

Sync flushes all dirty in-memory tables to the storage backend. For ModeMemory and ModeWAL this is a no-op (those modes use SaveToFile / WAL checkpoints respectively). For ModeDisk, ModeJSON, ModeHybrid, and ModeIndex, tables whose version has changed since the last save are written to disk.

func (*DB) SyncTable added in v0.5.4 

func (db *DB) SyncTable(tenant string, t *Table) error

SyncTable flushes a single table to the backend. This is called by the engine after mutations when SyncOnMutate is enabled.

func (*DB) TableExists added in v0.5.4 

func (db *DB) TableExists(tenant, name string) bool

TableExists reports whether the named table exists, checking both in-memory tables and the storage backend.

func (*DB) UnlockContentForRead added in v0.12.0 

func (db *DB) UnlockContentForRead()

UnlockContentForRead releases a lock taken by LockContentForRead.

func (*DB) UnlockContentForWrite added in v0.12.0 

func (db *DB) UnlockContentForWrite()

UnlockContentForWrite releases a lock taken by LockContentForWrite.

func (*DB) Use added in v0.21.0 

func (db *DB) Use(extension Extension) error

Use activates one statically linked Go extension for this database. An extension name can be activated only once per DB instance. Extensions are not unloadable because their registration may affect executing queries or other database state.

func (*DB) WAL 

func (db *DB) WAL() *WALManager

WAL returns the configured WAL manager (may be nil).

type DBHealth added in v0.11.0 

type DBHealth struct {
	OK                bool
	Mode              StorageMode
	ModeName          string
	Path              string
	Closed            bool
	Closing           bool
	ReadOnly          bool
	SchedulerRunning  bool
	WALActive         bool
	AdvancedWALActive bool
	Tenants           int
	Tables            int
	BackendStats      BackendStats
	LastSyncAt        time.Time
	LastCloseAt       time.Time
	Recovery          RecoveryStatus
	Error             string
}

DBHealth is a point-in-time operational snapshot for production probes.

type DiskBackend added in v0.5.4 

type DiskBackend struct {
	// contains filtered or unexported fields
}

DiskBackend stores each table as an individual file under a directory tree: 

<dir>/<tenant>/<tablename>.tbl        (GOB-encoded diskTable, default)
<dir>/<tenant>/<tablename>.json       (JSON-encoded diskTable, ModeJSON)
<dir>/manifest.json                   (lightweight metadata index)

Tables are loaded into memory on demand (via LoadTable) and written back on SaveTable / Sync. This minimises RAM usage while keeping the file format compatible with the existing GOB serialisation (or, for ModeJSON, a JSON encoding of the same diskTable structure).

func NewDiskBackend added in v0.5.4 

func NewDiskBackend(dir string, compress bool) (*DiskBackend, error)

NewDiskBackend opens (or creates) a disk-backed storage directory using GOB-encoded per-table files (ModeDisk).

func NewJSONBackend added in v0.12.0 

func NewJSONBackend(dir string, compress bool) (*DiskBackend, error)

NewJSONBackend opens (or creates) a disk-backed storage directory using JSON-encoded per-table files (ModeJSON). Functionally identical to NewDiskBackend — same manifest, same lazy loading, same dirty tracking — only the per-table file encoding differs. JSON files are larger than GOB and big.Rat/uuid.UUID values round-trip as plain strings (see normalizeForJSON), but the files can be read, diffed, or hand-edited with any text tool, which GOB files cannot.

func (*DiskBackend) Close added in v0.5.4 

func (b *DiskBackend) Close() error

Close is like Sync but also cleans up any resources.

func (*DiskBackend) DeleteTable added in v0.5.4 

func (b *DiskBackend) DeleteTable(tenant, name string) error

DeleteTable removes a table file from disk and updates the manifest.

func (*DiskBackend) GetMeta added in v0.5.4 

func (b *DiskBackend) GetMeta(tenant, name string) *TableMeta

GetMeta returns table metadata without loading the full table. Returns nil if the table is unknown.

func (*DiskBackend) ImportFromDB added in v0.5.4 

func (b *DiskBackend) ImportFromDB(db *DB) error

ImportFromDB takes all tables from an existing in-memory DB and writes them as individual table files. This is the migration path from ModeMemory / GOB checkpoints to ModeDisk.

func (*DiskBackend) IsDirty added in v0.5.4 

func (b *DiskBackend) IsDirty(tenant, name string, currentVersion int) bool

IsDirty reports whether the named table has been modified since it was last saved. Uses the version field for comparison.

func (*DiskBackend) IsReadOnly added in v0.20.0 

func (b *DiskBackend) IsReadOnly() bool

IsReadOnly reports whether persistence has been disabled for this backend.

func (*DiskBackend) ListTableNames added in v0.5.4 

func (b *DiskBackend) ListTableNames(tenant string) ([]string, error)

ListTableNames returns all known table names for a tenant (on disk or in memory).

func (*DiskBackend) LoadTable added in v0.5.4 

func (b *DiskBackend) LoadTable(tenant, name string) (*Table, error)

LoadTable reads a table from its GOB file on disk. Returns (nil, nil) if the table does not exist.

func (*DiskBackend) Mode added in v0.5.4 

func (b *DiskBackend) Mode() StorageMode

func (*DiskBackend) SaveTable added in v0.5.4 

func (b *DiskBackend) SaveTable(tenant string, t *Table) error

SaveTable writes a table to its GOB file on disk and updates the manifest.

func (*DiskBackend) SetEncryptor added in v0.13.0 

func (b *DiskBackend) SetEncryptor(enc *Encryptor)

SetEncryptor enables (or, passing nil, disables) AES-256-GCM encryption at rest for every table file this backend writes from this point on. Existing files already on disk are re-encrypted (or decrypted) lazily, the next time each is saved — SetEncryptor doesn't rewrite files that aren't touched again, so a backend directory can end up with a mix of encrypted and unencrypted files if the encryptor is changed mid-lifetime. For a clean transition, set the encryptor once at open time (see StorageConfig.EncryptionKey) rather than toggling it on a live backend.

func (*DiskBackend) SetReadOnly added in v0.20.0 

func (b *DiskBackend) SetReadOnly(readOnly bool)

SetReadOnly makes the backend observational. It is normally set by OpenDB before the backend is exposed to callers.

func (*DiskBackend) Stats added in v0.5.4 

func (b *DiskBackend) Stats() BackendStats

func (*DiskBackend) Sync added in v0.5.4 

func (b *DiskBackend) Sync() error

Sync writes the manifest (individual table files are written immediately by SaveTable). The caller should call SaveTable for each dirty table before calling Sync, or use DB.Sync which does this automatically.

func (*DiskBackend) TableExists added in v0.5.4 

func (b *DiskBackend) TableExists(tenant, name string) bool

TableExists reports whether the table is known to the backend.

type Encryptor added in v0.13.0 

type Encryptor struct {
	// contains filtered or unexported fields
}

Encryptor performs authenticated AES-256-GCM encryption/decryption of byte blobs, used to encrypt table files at rest.

func NewEncryptor added in v0.13.0 

func NewEncryptor(key []byte) (*Encryptor, error)

NewEncryptor creates an Encryptor from a 32-byte AES-256 key (typically from DeriveKeyFromPassphrase, or any caller-supplied 32 cryptographically random bytes for key-file-based setups).

func (*Encryptor) Decrypt added in v0.13.0 

func (e *Encryptor) Decrypt(blob []byte) ([]byte, error)

Decrypt reverses Encrypt. It returns an error — rather than corrupted or wrong plaintext — if blob is truncated, was encrypted under a different key, or was altered after encryption: GCM's authentication tag check fails closed on any of those.

func (*Encryptor) Encrypt added in v0.13.0 

func (e *Encryptor) Encrypt(plaintext []byte) ([]byte, error)

Encrypt returns nonce||ciphertext||tag as a single self-contained blob safe to write directly to disk (Decrypt expects exactly this layout). A fresh random nonce is generated per call, which is required for GCM's security guarantees — reusing a nonce under the same key breaks confidentiality — and cheap enough to pay per file write.

type Extension added in v0.21.0 

type Extension interface {
	ExtensionInfo() ExtensionInfo
	Register(db *DB) error
}

Extension is the stable entry point for statically linked Go extensions. An extension package is imported by the application and explicitly enabled with db.Use. tinySQL intentionally does not use Go's plugin package: its shared-object ABI is platform/toolchain-dependent and unavailable to many tinySQL targets, including WebAssembly and TinyGo.

Register may register metadata, functions, table functions, storage helpers, or other supported public extension points on db. Returning an error leaves the extension absent from DB.Extensions and sys.extensions.

type ExtensionCapability added in v0.21.0 

type ExtensionCapability string

ExtensionCapability is a declared resource category an extension may need. Declarations are metadata only for now; callers can use them to review an extension before activation. Enforcement is intentionally left to a future capability policy rather than pretending that in-process Go code is sandboxed. 

const (
	CapabilityFilesystem ExtensionCapability = "filesystem"
	CapabilityNetwork    ExtensionCapability = "network"
	CapabilityWrite      ExtensionCapability = "write"
	CapabilitySecrets    ExtensionCapability = "secrets"
)

type ExtensionInfo added in v0.21.0 

type ExtensionInfo struct {
	Name         string
	Version      string
	Description  string
	Capabilities []ExtensionCapability
	LoadedAt     time.Time
}

ExtensionInfo describes one statically linked Go extension activated for a database instance. LoadedAt is set by DB.Use and is not supplied by the extension implementation.

type ForeignKeyRef 

type ForeignKeyRef struct {
	Table    string
	Column   string
	OnDelete ReferentialAction
	OnUpdate ReferentialAction
}

ForeignKeyRef describes a foreign key reference target.

type Grant added in v0.12.0 

type Grant struct {
	Permission Permission
	Schema     string
	Table      string
}

Grant authorizes Permission on every table in Schema (or every schema, if Schema is "*") named Table (or every table, if Table is "*").

type HybridBackend added in v0.5.4 

type HybridBackend struct {
	// contains filtered or unexported fields
}

HybridBackend keeps frequently-accessed tables in an LRU memory cache backed by individual table files on disk. When the cache exceeds the configured memory limit, cold tables are evicted. This gives near in- memory performance for the working set while allowing the full database to exceed available RAM.

For ModeIndex, the same structure is used with a much smaller memory limit and aggressive eviction – essentially only table schemas (loaded as metadata from the manifest) stay resident.

func NewHybridBackend added in v0.5.4 

func NewHybridBackend(dir string, maxMemoryBytes int64, compress bool, mode StorageMode) (*HybridBackend, error)

NewHybridBackend creates a HybridBackend.

  • dir: directory for table files
  • maxMemoryBytes: memory limit (0 = 256 MB default)
  • compress: gzip table files
  • mode: ModeHybrid or ModeIndex

func (*HybridBackend) Close added in v0.5.4 

func (h *HybridBackend) Close() error

Close flushes dirty tables and closes the disk backend.

func (*HybridBackend) DeleteTable added in v0.5.4 

func (h *HybridBackend) DeleteTable(tenant, name string) error

DeleteTable removes from both disk and cache.

func (*HybridBackend) Disk added in v0.5.4 

func (h *HybridBackend) Disk() *DiskBackend

Disk returns the underlying DiskBackend for advanced operations (e.g. migration, metadata access).

func (*HybridBackend) IsDirty added in v0.23.1 

func (h *HybridBackend) IsDirty(tenant, name string, currentVersion int) bool

IsDirty delegates to the underlying DiskBackend's version-based dirty check, satisfying the dirtyTracker interface DB.Sync uses for both db.tenants entries and PooledTables leases.

func (*HybridBackend) ListTableNames added in v0.5.4 

func (h *HybridBackend) ListTableNames(tenant string) ([]string, error)

ListTableNames returns names from the disk manifest (authoritative source).

func (*HybridBackend) LoadTable added in v0.5.4 

func (h *HybridBackend) LoadTable(tenant, name string) (*Table, error)

LoadTable tries the in-memory cache first, then loads from disk.

func (*HybridBackend) MarkDirty added in v0.5.4 

func (h *HybridBackend) MarkDirty(tenant, name string)

MarkDirty records that a table has been modified in memory and needs to be flushed on the next Sync. Called by DB after mutations.

func (*HybridBackend) Mode added in v0.5.4 

func (h *HybridBackend) Mode() StorageMode

Mode reports the configured hybrid storage mode.

func (*HybridBackend) PooledTables added in v0.23.1 

func (h *HybridBackend) PooledTables() []PooledTableRef

PooledTables returns every table currently resident in the buffer pool, satisfying the pooledTableSource interface. DB.Sync/DB.Close use this to find and flush a table that DB.Get loaded lazily for this evictable storage mode (ModeHybrid/ModeIndex) without registering it in DB.tenants.

func (*HybridBackend) SaveTable added in v0.5.4 

func (h *HybridBackend) SaveTable(tenant string, t *Table) error

SaveTable writes to disk and updates the cache.

func (*HybridBackend) SetEncryptor added in v0.18.0 

func (h *HybridBackend) SetEncryptor(enc *Encryptor)

SetEncryptor forwards table-file encryption to the underlying DiskBackend.

func (*HybridBackend) SetReadOnly added in v0.20.0 

func (h *HybridBackend) SetReadOnly(readOnly bool)

SetReadOnly forwards the serving-only contract to the durable backend.

func (*HybridBackend) Stats added in v0.5.4 

func (h *HybridBackend) Stats() BackendStats

Stats returns combined disk and in-memory cache statistics.

func (*HybridBackend) Sync added in v0.5.4 

func (h *HybridBackend) Sync() error

Sync writes all dirty in-memory tables to disk.

func (*HybridBackend) TableExists added in v0.5.4 

func (h *HybridBackend) TableExists(tenant, name string) bool

TableExists checks the disk manifest.

type IndexEntry added in v0.19.0 

type IndexEntry struct {
	Key    []byte
	RowIDs []int
}

IndexEntry is one canonical composite key and the table row positions that match it. Entries are sorted by Key, making equality and prefix seeks a binary-search operation rather than a table scan.

type IsolationLevel 

type IsolationLevel uint8

IsolationLevel defines transaction isolation semantics. 

const (
	ReadCommitted IsolationLevel = iota
	RepeatableRead
	SnapshotIsolation
	Serializable
)

type JobExecutor added in v0.5.1 

type JobExecutor interface {
	ExecuteSQL(ctx context.Context, sql string) (interface{}, error)
}

JobExecutor interface allows the scheduler to execute SQL without circular dependencies

type LRUNode 

type LRUNode struct {
	// contains filtered or unexported fields
}

LRUNode is a node in the LRU doubly-linked list.

type LRUQueue 

type LRUQueue struct {
	// contains filtered or unexported fields
}

LRUQueue implements least-recently-used eviction.

func NewLRUQueue 

func NewLRUQueue() *LRUQueue

NewLRUQueue creates an empty LRU queue.

func (*LRUQueue) Access 

func (lru *LRUQueue) Access(key string, table *CachedTable)

Access moves a node to the front (most recent).

func (*LRUQueue) Add 

func (lru *LRUQueue) Add(key string, table *CachedTable)

Add adds a node to the LRU queue (most recent).

func (*LRUQueue) Remove 

func (lru *LRUQueue) Remove(key string)

Remove removes a node from the queue.

func (*LRUQueue) RemoveLRU 

func (lru *LRUQueue) RemoveLRU() (string, *CachedTable)

RemoveLRU removes and returns the least recently used node.

type LSN 

type LSN uint64

LSN (Log Sequence Number) provides total ordering of log records.

type MVCCManager 

type MVCCManager struct {
	// contains filtered or unexported fields
}

MVCCManager coordinates transaction IDs, timestamps, and visibility.

func NewMVCCManager 

func NewMVCCManager() *MVCCManager

NewMVCCManager creates a new MVCC coordinator.

func (*MVCCManager) AbortTx 

func (m *MVCCManager) AbortTx(tx *TxContext)

AbortTx marks a transaction as aborted.

func (*MVCCManager) BeginTx 

func (m *MVCCManager) BeginTx(level IsolationLevel) *TxContext

BeginTx starts a new transaction and returns its context.

func (*MVCCManager) CommitTx 

func (m *MVCCManager) CommitTx(tx *TxContext) (Timestamp, error)

CommitTx marks a transaction as committed and records its commit timestamp.

func (*MVCCManager) GCWatermark 

func (m *MVCCManager) GCWatermark() Timestamp

GCWatermark returns the timestamp before which row versions can be garbage collected.

func (*MVCCManager) IsVisible 

func (m *MVCCManager) IsVisible(tx *TxContext, rv *RowVersion) bool

IsVisible determines if a row version is visible to a transaction.

type MVCCTable 

type MVCCTable struct {
	*Table
	// contains filtered or unexported fields
}

MVCCTable extends Table with version chains.

func NewMVCCTable 

func NewMVCCTable(name string, cols []Column, isTemp bool) *MVCCTable

NewMVCCTable creates a table with MVCC support.

func (*MVCCTable) DeleteVersion 

func (t *MVCCTable) DeleteVersion(tx *TxContext, rowID int64) error

DeleteVersion marks a row version as deleted.

func (*MVCCTable) GarbageCollect 

func (t *MVCCTable) GarbageCollect(watermark Timestamp) int

GarbageCollect removes old row versions that are no longer visible.

func (*MVCCTable) GetVisibleVersion 

func (t *MVCCTable) GetVisibleVersion(mvcc *MVCCManager, tx *TxContext, rowID int64) *RowVersion

GetVisibleVersion returns the visible version of a row for the given transaction.

func (*MVCCTable) InsertVersion 

func (t *MVCCTable) InsertVersion(tx *TxContext, data []any) int64

InsertVersion adds a new row version.

func (*MVCCTable) UpdateVersion 

func (t *MVCCTable) UpdateVersion(tx *TxContext, rowID int64, newData []any) error

UpdateVersion creates a new version for an update.

type MemoryBackend added in v0.5.4 

type MemoryBackend struct {
	// contains filtered or unexported fields
}

MemoryBackend is a no-op backend used by ModeMemory. All data lives exclusively in the DB's in-memory tenants map; the backend simply reports that tables don't exist on any backing store (so the DB never tries to load from disk). Save/Sync are intentional no-ops – persistence is the caller's responsibility (via SaveToFile or the WAL).

func NewMemoryBackend added in v0.5.4 

func NewMemoryBackend(savePath string) *MemoryBackend

NewMemoryBackend creates a MemoryBackend. If savePath is non-empty, Close performs a final SaveToFile to that path.

func (*MemoryBackend) Close added in v0.5.4 

func (m *MemoryBackend) Close() error

func (*MemoryBackend) DeleteTable added in v0.5.4 

func (m *MemoryBackend) DeleteTable(_, _ string) error

DeleteTable is a no-op because memory mode has no durable table files.

func (*MemoryBackend) ListTableNames added in v0.5.4 

func (m *MemoryBackend) ListTableNames(_ string) ([]string, error)

ListTableNames reports no durable table names in memory mode.

func (*MemoryBackend) LoadTable added in v0.5.4 

func (m *MemoryBackend) LoadTable(_, _ string) (*Table, error)

LoadTable reports no durable table because memory mode has no backing store.

func (*MemoryBackend) Mode added in v0.5.4 

func (m *MemoryBackend) Mode() StorageMode

func (*MemoryBackend) SaveTable added in v0.5.4 

func (m *MemoryBackend) SaveTable(_ string, _ *Table) error

SaveTable is a no-op because memory mode keeps rows in the owning DB.

func (*MemoryBackend) Stats added in v0.5.4 

func (m *MemoryBackend) Stats() BackendStats

func (*MemoryBackend) Sync added in v0.5.4 

func (m *MemoryBackend) Sync() error

Sync is a no-op because memory mode has no durable writes to flush.

func (*MemoryBackend) TableExists added in v0.5.4 

func (m *MemoryBackend) TableExists(_, _ string) bool

TableExists reports false because memory mode has no durable table files.

type MemoryPolicy 

type MemoryPolicy struct {
	// Maximum memory usage in bytes (0 = unlimited)
	MaxMemoryBytes int64

	// Cache eviction strategy
	Strategy CacheStrategy

	// Start evicting when memory usage exceeds this ratio (0.0-1.0)
	EvictionThreshold float64

	// Tables that should always stay in memory
	PinnedTables []string

	// Tables that should never be cached
	IgnoreTables []string

	// Enable eviction (if false, OOM when limit reached)
	EnableEviction bool

	// Number of tables to evict in one batch
	EvictionBatchSize int

	// Track access patterns for better eviction decisions
	TrackAccessPatterns bool

	// Time window for access tracking
	AccessWindow time.Duration
}

MemoryPolicy defines memory management configuration.

func DefaultMemoryPolicy 

func DefaultMemoryPolicy() *MemoryPolicy

DefaultMemoryPolicy returns a sensible default configuration.

func LimitedMemoryPolicy 

func LimitedMemoryPolicy(maxMB int64) *MemoryPolicy

LimitedMemoryPolicy returns a policy with memory limits.

type PagedIndexBackend added in v0.20.0 

type PagedIndexBackend struct {
	// contains filtered or unexported fields
}

PagedIndexBackend adapts the pager B+Tree format to StorageBackend. Unlike ModeIndex's legacy GOB table files, it persists table rows and materialized secondary indexes in distinct B+Trees. Exact seeks can therefore read an index key, its row locator and only the requested rows/BLOBs.

It is deliberately a separate mode: existing ModeIndex directories retain their GOB compatibility, while published `paged_index` artifacts have an explicit format boundary and can be opened safely read-only.

func NewPagedIndexBackend added in v0.20.0 

func NewPagedIndexBackend(dir string, maxMemoryBytes int64, readOnly bool) (*PagedIndexBackend, error)

func (*PagedIndexBackend) Close added in v0.20.0 

func (b *PagedIndexBackend) Close() error

func (*PagedIndexBackend) DeleteTable added in v0.20.0 

func (b *PagedIndexBackend) DeleteTable(tenant, name string) error

func (*PagedIndexBackend) IndexMetadata added in v0.20.0 

func (b *PagedIndexBackend) IndexMetadata(tenant, name string) (*Table, error)

IndexMetadata creates a schema-only Table with physical index metadata. It is a cheap operation (catalog B+Tree only) and lets the SQL planner choose a page index before calling LoadTable.

func (*PagedIndexBackend) IsDirty added in v0.23.1 

func (b *PagedIndexBackend) IsDirty(tenant, name string, currentVersion int) bool

IsDirty reports whether name's in-memory version has diverged from the version last saved to the pager. Satisfies the dirtyTracker interface.

func (*PagedIndexBackend) IsReadOnly added in v0.20.0 

func (b *PagedIndexBackend) IsReadOnly() bool

func (*PagedIndexBackend) ListTableNames added in v0.20.0 

func (b *PagedIndexBackend) ListTableNames(tenant string) ([]string, error)

func (*PagedIndexBackend) LoadTable added in v0.20.0 

func (b *PagedIndexBackend) LoadTable(tenant, name string) (*Table, error)

func (*PagedIndexBackend) LookupIndexRows added in v0.20.0 

func (b *PagedIndexBackend) LookupIndexRows(tenant, table, indexName string, values []any) ([][]any, bool, error)

LookupIndexRows resolves a complete canonical composite key through the on-disk index and returns only matching rows. The caller owns the returned slices and can safely project/copy BLOBs without exposing page-cache memory.

func (*PagedIndexBackend) Mode added in v0.20.0 

func (b *PagedIndexBackend) Mode() StorageMode

func (*PagedIndexBackend) PooledTables added in v0.23.1 

func (b *PagedIndexBackend) PooledTables() []PooledTableRef

PooledTables returns every table currently resident in the table-object pool, satisfying the pooledTableSource interface. DB.Sync/DB.Close use this to find and flush a table that DB.Get loaded lazily for ModePagedIndex without registering it in DB.tenants.

func (*PagedIndexBackend) SaveTable added in v0.20.0 

func (b *PagedIndexBackend) SaveTable(tenant string, t *Table) error

func (*PagedIndexBackend) SetReadOnly added in v0.20.0 

func (b *PagedIndexBackend) SetReadOnly(readOnly bool)

func (*PagedIndexBackend) Stats added in v0.20.0 

func (b *PagedIndexBackend) Stats() BackendStats

func (*PagedIndexBackend) Sync added in v0.20.0 

func (b *PagedIndexBackend) Sync() error

func (*PagedIndexBackend) TableExists added in v0.20.0 

func (b *PagedIndexBackend) TableExists(tenant, name string) bool

type Permission added in v0.12.0 

type Permission string

Permission enumerates the operations a grant can authorize. 

const (
	PermSelect Permission = "SELECT"
	PermInsert Permission = "INSERT"
	PermUpdate Permission = "UPDATE"
	PermDelete Permission = "DELETE"
	// PermDDL covers CREATE/DROP/ALTER TABLE/INDEX/VIEW/TRIGGER/JOB and
	// similar schema-changing statements.
	PermDDL Permission = "DDL"
	// PermAll grants every permission above at once.
	PermAll Permission = "ALL"
)

func ParsePermission added in v0.12.0 

func ParsePermission(s string) (Permission, error)

ParsePermission validates and normalizes a permission keyword from SQL (case-insensitive).

type PooledEntry added in v0.23.1 

type PooledEntry struct {
	Tenant string
	Name   string
	Table  *Table
}

PooledEntry is one resident cache entry returned by Snapshot.

type PooledTableRef added in v0.23.1 

type PooledTableRef struct {
	Tenant string
	Table  *Table
}

PooledTableRef identifies one table object currently resident in a storage backend's own bounded cache — outside DB.tenants — for an evictable storage mode (ModeIndex, ModeHybrid, ModePagedIndex; see DB.backendTablesEvictable). DB.Get returns exactly this *Table pointer as a query-scoped lease without registering it in DB.tenants, so a mutation applied in place (as the SQL engine's INSERT/UPDATE/DELETE do) is only reachable through the backend's own pool. DB.Sync and DB.Close use pooledTableSource.PooledTables to find and flush these leases; without it a table that was never re-registered in DB.tenants could be mutated, "successfully" synced/closed, and then lose the mutation on next open.

type RecoveryStatus added in v0.11.0 

type RecoveryStatus struct {
	Mode                  StorageMode
	Path                  string
	CheckpointLoaded      bool
	RecoveredTransactions uint64
	RecoveredOperations   int
	Truncated             bool
	RecoveredAt           time.Time
}

RecoveryStatus describes the last recovery pass performed while opening a DB.

type ReferentialAction added in v0.12.0 

type ReferentialAction int

ReferentialAction enumerates what happens to a child row when the parent row it references is deleted (ON DELETE) or its referenced column value changes (ON UPDATE). 

const (
	// NoAction means no ON DELETE/ON UPDATE clause was given. tinySQL checks
	// foreign key constraints immediately (it has no deferred-constraint
	// mode), so NoAction and Restrict behave identically here — the SQL
	// standard only distinguishes them for deferred checking.
	NoAction ReferentialAction = iota
	// Restrict blocks the parent-side mutation while a referencing child row exists.
	Restrict
	// Cascade propagates the parent-side delete/update to matching child rows.
	Cascade
	// SetNull nulls the child row's foreign key column instead of deleting/blocking.
	SetNull
)

func (ReferentialAction) String added in v0.12.0 

func (a ReferentialAction) String() string

type RowVersion 

type RowVersion struct {
	// Transaction that created this version
	XMin TxID

	// Transaction that deleted/updated this version (0 if still valid)
	XMax TxID

	// Creation timestamp
	CreatedAt Timestamp

	// Deletion/update timestamp (0 if still valid)
	DeletedAt Timestamp

	// Actual row data
	Data []any

	// Pointer to next version (for version chain)
	NextVersion *RowVersion
}

RowVersion contains MVCC metadata for a single row version.

type SQLiteAffinity added in v0.19.0 

type SQLiteAffinity uint8

SQLiteAffinity is the five-class type system used by SQLite declarations. It is schema metadata, not another runtime value type: tinySQL continues to store NULL, integer, real, text and binary values directly. Keeping the declared affinity separate lets imported SQLite schemas retain their lossless-coercion behaviour without multiplying ColType values. 

const (
	// AffinityDefault retains tinySQL's native, strongly typed coercion rules.
	AffinityDefault SQLiteAffinity = iota
	AffinityInteger
	AffinityText
	AffinityNumeric
	AffinityReal
	AffinityBlob
)

func (SQLiteAffinity) String added in v0.19.0 

func (a SQLiteAffinity) String() string

type Scheduler added in v0.5.1 

type Scheduler struct {
	// contains filtered or unexported fields
}

Scheduler manages scheduled job execution

func NewScheduler added in v0.5.1 

func NewScheduler(db *DB, executor JobExecutor) *Scheduler

NewScheduler creates a new job scheduler

func (*Scheduler) AddJob added in v0.5.1 

func (s *Scheduler) AddJob(job *CatalogJob) error

AddJob registers a new job and schedules it immediately if enabled

func (*Scheduler) RemoveJob added in v0.5.1 

func (s *Scheduler) RemoveJob(name string) error

RemoveJob unregisters a job and stops its execution

func (*Scheduler) Start added in v0.5.1 

func (s *Scheduler) Start() error

Start begins the scheduler loop

func (*Scheduler) Stop added in v0.5.1 

func (s *Scheduler) Stop()

Stop halts the scheduler and cancels all running jobs

func (*Scheduler) UpsertJob added in v0.11.0 

func (s *Scheduler) UpsertJob(job *CatalogJob) error

UpsertJob registers or updates a job and refreshes its live schedule.

type SecondaryIndex added in v0.19.0 

type SecondaryIndex struct {
	Name    string
	Columns []string
	Unique  bool
	Entries []IndexEntry
}

SecondaryIndex is a materialized, persistent secondary index. It is kept alongside table rows so GOB snapshots, disk and hybrid backends preserve the index itself, not merely CREATE INDEX catalog metadata.

type StatementSnapshot added in v0.21.0 

type StatementSnapshot struct {
	// contains filtered or unexported fields
}

StatementSnapshot is an in-memory rollback point for one SQL statement. It is deliberately internal storage machinery: the engine holds DB's content lock for the whole statement, so a snapshot does not need to solve concurrent writers or provide an application-level transaction API.

Table pointers that existed when the snapshot was taken are retained and restored in place. This keeps a caller holding a *Table obtained through DB.Get from observing a failed statement's half-applied row changes.

type StorageBackend added in v0.5.4 

type StorageBackend interface {
	// LoadTable retrieves a table from the backing store. It returns nil, nil
	// when the table does not exist (not an error – the table may simply not
	// have been created yet).
	LoadTable(tenant, name string) (*Table, error)

	// SaveTable persists a single table to the backing store.
	SaveTable(tenant string, t *Table) error

	// DeleteTable removes a table from the backing store.
	DeleteTable(tenant, name string) error

	// ListTableNames returns all table names for a tenant that exist in the
	// backing store (including tables not currently loaded in memory).
	ListTableNames(tenant string) ([]string, error)

	// TableExists reports whether the named table exists in the backing store
	// without loading it into memory.
	TableExists(tenant, name string) bool

	// Sync flushes any pending / dirty data to durable storage.
	Sync() error

	// Close releases all resources. Implementations should call Sync
	// internally if there is unsaved data.
	Close() error

	// Mode returns the StorageMode this backend implements.
	Mode() StorageMode

	// Stats returns operational statistics about the backend.
	Stats() BackendStats
}

StorageBackend abstracts the underlying table storage mechanism.

Implementations are responsible for loading tables into *Table structs that the engine can mutate directly. After mutations, Sync writes dirty tables back to durable storage. Close persists pending data and releases resources.

type StorageConfig added in v0.5.4 

type StorageConfig struct {
	// Mode selects the storage strategy. Defaults to ModeMemory.
	Mode StorageMode

	// Path is the root directory (or file path) for persistent storage.
	// Required for all modes except ModeMemory.
	Path string

	// MaxMemoryBytes limits in-memory table data. Used by ModeHybrid and
	// ModeIndex. Zero means use a sensible default (256 MB).
	MaxMemoryBytes int64

	// SyncOnMutate forces a disk write after every INSERT / UPDATE / DELETE.
	// Slower but provides immediate durability for ModeDisk / ModeHybrid.
	SyncOnMutate bool

	// CompressFiles enables gzip compression for table files on disk.
	CompressFiles bool

	// CheckpointEvery controls how many committed WAL transactions trigger
	// an automatic checkpoint (ModeWAL only). Zero means default (32).
	CheckpointEvery uint64

	// CheckpointInterval controls the maximum time between checkpoints
	// (ModeWAL only). Zero means default (30 s).
	CheckpointInterval time.Duration

	// CheckpointMaxBytes forces a checkpoint once the WAL file exceeds this
	// size, bounding WAL growth independently of transaction count and time
	// (ModeWAL / ModeAdvancedWAL). Zero means default (64 MB); negative
	// disables the size trigger.
	CheckpointMaxBytes int64

	// ReadOnly opens the database in read-only mode: the SQL engine rejects
	// all mutating statements. Ideal for serve-only phases (e.g. load at
	// night, query during the day) — vector index and column caches can never
	// be invalidated and the WAL is never appended to.
	ReadOnly bool

	// EncryptionKey, if set, must be exactly EncryptionKeySize (32) bytes
	// and enables AES-256-GCM encryption at rest for table files (ModeDisk,
	// ModeJSON, ModeHybrid, ModeIndex). Derive it from a passphrase with
	// DeriveKeyFromPassphrase (persist the salt yourself — this package
	// doesn't manage key storage) or supply 32 cryptographically random
	// bytes directly for a key-file-based setup. Leave nil for the default,
	// unencrypted behavior. Not yet supported for ModeWAL/ModeAdvancedWAL —
	// see README's Limitations section.
	EncryptionKey []byte
}

StorageConfig configures database storage behaviour.

func DefaultStorageConfig added in v0.5.4 

func DefaultStorageConfig(mode StorageMode) StorageConfig

DefaultStorageConfig returns a StorageConfig with sensible defaults for the given mode. Path must be supplied by the caller afterwards.

type StorageMode added in v0.5.4 

type StorageMode int

StorageMode defines how the database manages data between memory and disk. 

const (
	// ModeMemory keeps all data in RAM. Persistence only occurs via explicit
	// SaveToFile calls or when DB.Close is invoked. Fastest mode.
	ModeMemory StorageMode = iota

	// ModeWAL keeps all data in RAM and writes a Write-Ahead Log for crash
	// recovery. Periodic checkpoints create full GOB snapshots. Good
	// balance of speed and durability.
	ModeWAL

	// ModeDisk stores each table as a separate GOB file on disk. Tables
	// are loaded into memory on demand and flushed back on Sync/Close.
	// Minimises RAM usage at the cost of disk I/O.
	ModeDisk

	// ModeIndex keeps table schemas (columns, row-counts) permanently in
	// RAM while row data resides on disk. Rows are loaded on demand and
	// evicted aggressively. Memory usage is proportional to schema size,
	// not data size.
	ModeIndex

	// ModeHybrid uses an LRU buffer pool with a configurable memory limit.
	// Hot tables stay in RAM; cold tables spill to disk. Best for mixed
	// workloads where the working set fits in a bounded amount of memory.
	ModeHybrid

	// ModeAdvancedWAL keeps all data in RAM and uses a row-level Write-Ahead Log
	// for full ACID transaction durability, crash recovery, and point-in-time recovery.
	ModeAdvancedWAL

	// ModeJSON stores each table as a separate human-readable JSON file on
	// disk (same lazy-load/dirty-tracking behaviour as ModeDisk, which uses
	// GOB instead). Larger on disk than GOB and big.Rat/uuid.UUID values
	// round-trip as plain strings, but files can be read, diffed, or
	// hand-edited with any text tool.
	ModeJSON

	// ModePagedIndex is an immutable-page-oriented artifact for read-mostly
	// lookups. It stores rows and materialized secondary indexes in separate
	// B+Trees, so an exact index seek can load only index and referenced row
	// pages instead of decoding a complete table file.
	ModePagedIndex
)

func ParseStorageMode added in v0.5.4 

func ParseStorageMode(s string) (StorageMode, error)

ParseStorageMode converts a string representation back to a StorageMode. It is case-insensitive and returns an error for unknown values.

func (StorageMode) String added in v0.5.4 

func (m StorageMode) String() string

String returns a human-readable label for the StorageMode.

type Table 

type Table struct {
	Name string
	Cols []Column
	Rows [][]any
	// Indexes contains materialized secondary and composite indexes keyed by
	// lower-case SQL index name. Unlike catalog metadata these entries are
	// used by the executor and persisted with table snapshots.
	Indexes map[string]*SecondaryIndex
	IsTemp  bool

	Version int
	// Stats is populated by ANALYZE and persisted with the table. DML marks it
	// stale rather than trying to estimate distinct values incrementally.
	Stats *TableStats
	// contains filtered or unexported fields
}

Table stores rows along with column metadata and indexes.

func NewTable 

func NewTable(name string, cols []Column, isTemp bool) *Table

NewTable creates a new Table with case-insensitive column lookup indices.

func (*Table) Analyze added in v0.21.0 

func (t *Table) Analyze() *TableStats

Analyze computes exact cardinality, null and simple range summaries for the current table contents. The first statistics implementation scans all rows deliberately: transparent and correct inputs are more useful than a sampled model whose accuracy would need separate policy and tuning.

func (*Table) CheckSecondaryIndexConstraints added in v0.19.0 

func (t *Table) CheckSecondaryIndexConstraints(row []any, skipRow int) error

CheckSecondaryIndexConstraints rejects a duplicate before a new row is appended. skipRow is used by UPDATE to ignore a row's current key.

func (*Table) ClearSecondaryIndexes added in v0.21.0 

func (t *Table) ClearSecondaryIndexes()

ClearSecondaryIndexes removes all RowIDs while retaining CREATE INDEX metadata, as required after DELETE without a WHERE clause.

func (*Table) ColIndex 

func (t *Table) ColIndex(name string) (int, error)

ColIndex returns the zero-based index of the named column.

func (*Table) CreateSecondaryIndex added in v0.19.0 

func (t *Table) CreateSecondaryIndex(name string, columns []string, unique bool) error

CreateSecondaryIndex builds an index over both existing and future table rows. Names are case-insensitive, matching SQL identifiers.

func (*Table) DirtyFrom added in v0.5.4 

func (t *Table) DirtyFrom() int

DirtyFrom returns the first dirty row index, or -1 if non-append-only.

func (*Table) DirtyRows added in v0.24.0 

func (t *Table) DirtyRows() ([]int, bool)

DirtyRows returns the in-place updated row indices and whether that list is known to be complete. A false second result means the caller must treat the whole table as changed.

func (*Table) DropSecondaryIndex added in v0.19.0 

func (t *Table) DropSecondaryIndex(name string) bool

DropSecondaryIndex removes the materialized structure from the table.

func (*Table) FindSecondaryIndex added in v0.19.0 

func (t *Table) FindSecondaryIndex(columns []string) *SecondaryIndex

FindSecondaryIndex returns an index whose leading columns exactly match the requested equality predicates. The caller may provide a prefix shorter than the full composite index, enabling prefix seeks.

func (*Table) InsertSecondaryIndexRow added in v0.21.0 

func (t *Table) InsertSecondaryIndexRow(rowID int, row []any) error

InsertSecondaryIndexRow adds one appended table row to every materialized secondary index. It avoids rebuilding unaffected keys after every INSERT. Call it only after the row has been appended to t.Rows and constraints have been checked.

func (*Table) InvalidateStats added in v0.21.0 

func (t *Table) InvalidateStats()

InvalidateStats marks the previous ANALYZE result stale after a mutation. RowCount remains useful for observability while distinct/range values are excluded from planner decisions until ANALYZE is run again.

func (*Table) LookupSecondaryIndexPoint added in v0.19.0 

func (t *Table) LookupSecondaryIndexPoint(idx *SecondaryIndex, values []any) ([]int, error)

LookupSecondaryIndexPoint returns the immutable RowID run for a complete composite key. Unlike a prefix seek it neither merges nor sorts entries: RebuildSecondaryIndexes appends RowIDs in table order, so the entry is already in the observable order of a table scan. The returned slice aliases the index and is read-only; it remains valid until the table is mutated.

This is intentionally a separate API from LookupSecondaryIndexPrefix. A general caller may need an owned prefix result, while the engine's locked read path can safely avoid an allocation on every point lookup.

func (*Table) LookupSecondaryIndexPrefix added in v0.19.0 

func (t *Table) LookupSecondaryIndexPrefix(idx *SecondaryIndex, values []any) ([]int, error)

LookupSecondaryIndexPrefix performs a binary search followed by a compact prefix walk. Returned row IDs are sorted in table order to preserve the observable order of a scan when a query has no ORDER BY clause.

func (*Table) MarkDirtyFrom added in v0.5.4 

func (t *Table) MarkDirtyFrom(idx int)

MarkDirtyFrom records the first row index that was modified. If an earlier index is already set, it is kept. Use -1 for non-append mutations (UPDATE, DELETE) to force a full-table WAL entry.

The -1 (full-table) sentinel is sticky: once a mutation within a transaction forces a full-table entry, a later append-only INSERT must not downgrade it to a delta, or the earlier UPDATE/DELETE would be lost on WAL recovery. Any call also gives up on the in-place row list: an append shifts nothing but adds rows the list cannot describe, and an explicit -1 from a caller that is not reporting individual rows means the shape of the change is unknown.

func (*Table) MarkRowUpdated added in v0.24.0 

func (t *Table) MarkRowUpdated(idx int)

MarkRowUpdated records that row idx was replaced in place, so the WAL can log that row instead of the entire table. UPDATE is the only shape this fits: it changes row contents without changing how many rows there are or where they sit.

Callers must report every row they change. Reporting none is safe — the WAL writes the whole table, as it always did. Reporting some but not all is not, so a mutation path that cannot enumerate its rows must call MarkDirtyFrom(-1) instead, which permanently gives up the list for this dirty window.

func (*Table) RebuildSecondaryIndexes added in v0.19.0 

func (t *Table) RebuildSecondaryIndexes() error

RebuildSecondaryIndexes rebuilds every materialized index from table rows. It is called after DML, during recovery and before persistence boundaries so index/table versions cannot diverge across snapshots or WAL replay.

func (*Table) ReindexSecondaryIndexRows added in v0.21.0 

func (t *Table) ReindexSecondaryIndexRows(oldToNew map[int]int)

ReindexSecondaryIndexRows applies the old-to-new row-position mapping made by DELETE. Keys stay sorted and are not recomputed; only RowIDs belonging to removed rows disappear. This is deliberately named "reindex" rather than "rebuild": it preserves the materialized key structures.

func (*Table) ResetDirty added in v0.5.4 

func (t *Table) ResetDirty()

ResetDirty marks the table as clean (called after WAL checkpoint).

func (*Table) Statistics added in v0.21.0 

func (t *Table) Statistics() *TableStats

Statistics returns a defensive copy of the latest ANALYZE result.

func (*Table) UpdateSecondaryIndexRow added in v0.21.0 

func (t *Table) UpdateSecondaryIndexRow(rowID int, before, after []any) error

UpdateSecondaryIndexRow moves one stable row position between composite keys. Row positions do not change during UPDATE, so this is O(indexes · log(keys)) instead of rescanning the table.

type TableMeta added in v0.5.4 

type TableMeta struct {
	Tenant   string   `json:"tenant"`
	Name     string   `json:"name"`
	Cols     []Column `json:"cols"`
	RowCount int      `json:"row_count"`
	Version  int      `json:"version"`
	// DiskSize is the file size in bytes on the backing store.
	DiskSize int64 `json:"disk_size"`
	// FilePath is the relative path inside the database directory.
	FilePath string `json:"file_path"`
}

TableMeta stores lightweight metadata for a table that is (potentially) on disk. The disk backend uses this to answer ListTableNames, TableExists, and schema-level queries without loading row data.

type TableStats added in v0.21.0 

type TableStats struct {
	RowCount   int
	Columns    map[string]ColumnStats // lower-cased column name → statistics
	AnalyzedAt time.Time
	Stale      bool
}

TableStats is the persisted result of ANALYZE for one table.

type Timestamp 

type Timestamp uint64

Timestamp represents a logical timestamp for MVCC visibility.

type TriggerEvent added in v0.9.0 

type TriggerEvent string

TriggerEvent indicates what event fires the trigger. 

const (
	// TriggerInsert fires on INSERT.
	TriggerInsert TriggerEvent = "INSERT"
	// TriggerUpdate fires on UPDATE.
	TriggerUpdate TriggerEvent = "UPDATE"
	// TriggerDelete fires on DELETE.
	TriggerDelete TriggerEvent = "DELETE"
)

type TriggerTiming added in v0.9.0 

type TriggerTiming string

TriggerTiming indicates when a trigger fires. 

const (
	// TriggerBefore fires before the DML operation.
	TriggerBefore TriggerTiming = "BEFORE"
	// TriggerAfter fires after the DML operation.
	TriggerAfter TriggerTiming = "AFTER"
	// TriggerInsteadOf replaces the DML operation (views).
	TriggerInsteadOf TriggerTiming = "INSTEAD OF"
)

type TxContext 

type TxContext struct {
	ID             TxID
	StartTime      Timestamp
	Status         TxStatus
	ReadSnapshot   Timestamp                      // Snapshot timestamp for reads
	WriteSet       map[string]map[int64]bool      // table -> row IDs modified
	ReadSet        map[string]map[int64]Timestamp // table -> row IDs read with version
	IsolationLevel IsolationLevel
	// contains filtered or unexported fields
}

TxContext holds the state of an active transaction.

func (*TxContext) RecordRead 

func (tx *TxContext) RecordRead(table string, rowID int64, version Timestamp)

RecordRead tracks a read operation for conflict detection.

func (*TxContext) RecordWrite 

func (tx *TxContext) RecordWrite(table string, rowID int64)

RecordWrite tracks a write operation.

type TxID 

type TxID uint64

TxID represents a unique transaction identifier.

type TxStatus 

type TxStatus uint8

TxStatus represents the current state of a transaction. 

const (
	TxStatusInProgress TxStatus = iota
	TxStatusCommitted
	TxStatusAborted
)

type WALChange 

type WALChange struct {
	Tenant string
	Name   string
	Table  *Table
	Drop   bool
}

WALChange describes a persistent change that will be written to the WAL.

func CollectWALChanges 

func CollectWALChanges(prev, next *DB) []WALChange

CollectWALChanges computes the delta between two MVCC snapshots.

type WALConfig 

type WALConfig struct {
	Path               string
	CheckpointEvery    uint64
	CheckpointInterval time.Duration
	// CheckpointMaxBytes forces a checkpoint once the WAL file exceeds this
	// size, bounding WAL growth independently of transaction count and time.
	// Zero means default (64 MB); negative disables the size trigger.
	CheckpointMaxBytes int64
}

WALConfig configures WAL and checkpoint behavior.

type WALManager 

type WALManager struct {
	// contains filtered or unexported fields
}

WALManager encapsulates WAL append, recovery, and checkpoints.

func OpenWAL 

func OpenWAL(db *DB, cfg WALConfig) (*WALManager, error)

ready-to-use manager. It attaches no WAL when Path is empty.

func (*WALManager) Checkpoint 

func (w *WALManager) Checkpoint(db *DB) error

Checkpoint writes a DB snapshot and resets the WAL file.

func (*WALManager) Close 

func (w *WALManager) Close() error

Close flushes, syncs, and closes the WAL resources.

func (*WALManager) LogTransaction 

func (w *WALManager) LogTransaction(changes []WALChange) (bool, error)

LogTransaction appends all changes atomically to the WAL. It returns true when a checkpoint is recommended.

type WALOperationType 

type WALOperationType uint8

WALOperationType defines the type of WAL operation. 

const (
	WALOpBegin WALOperationType = iota + 1
	WALOpInsert
	WALOpUpdate
	WALOpDelete
	WALOpCommit
	WALOpAbort
	WALOpCheckpoint
)

func (WALOperationType) String 

func (t WALOperationType) String() string

type WALRecord 

type WALRecord struct {
	// Log Sequence Number - globally unique, monotonically increasing
	LSN LSN

	// Transaction ID
	TxID TxID

	// Operation type
	OpType WALOperationType

	// Tenant and table
	Tenant string
	Table  string

	// Row ID (for row-level operations)
	RowID int64

	// UNDO image (before state) - for rollback
	BeforeImage []any

	// REDO image (after state) - for recovery
	AfterImage []any

	// Column information (for schema tracking)
	Columns []Column

	// Timestamp
	Timestamp time.Time

	// Checksum for corruption detection
	Checksum uint32
}

WALRecord represents a single log entry with before/after images.

type WALTxState 

type WALTxState struct {
	TxID       TxID
	StartLSN   LSN
	Operations []LSN
	Status     TxStatus
}

WALTxState tracks the state of a transaction in the WAL.

Source Files

View all Source files

Directories

Path Synopsis
Package pager — PageBackend integrates the page-based storage engine with the tinySQL StorageBackend interface.
 Package pager — PageBackend integrates the page-based storage engine with the tinySQL StorageBackend interface.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.