Documentation
¶
Overview ¶
Package engine provides ExecDB's in-memory SQL engine as a Go library.
It wraps modernc.org/sqlite to offer an in-memory database that can be persisted either as a new executable (Snapshot) or by overwriting the host process's own executable in place (Overwrite). DB.Session hands out dedicated connections so independent clients (e.g. one per pgwire connection) can each hold their own transaction. The package does not itself import net or net/http; all network I/O (REPL, PostgreSQL wire protocol) is the responsibility of the caller (cmd/execdb).
See execdb_spec.md §6-7 for the full design.
Index ¶
- Constants
- Variables
- func Complete(sql string) (bool, error)
- func RegisterScalarFunction(name string, nArgs int32, ...) error
- type DB
- func (db *DB) Close() error
- func (db *DB) Exec(query string, args ...any) (sql.Result, error)
- func (db *DB) ExecContext(ctx context.Context, query string, args ...any) (sql.Result, error)
- func (db *DB) Info() Info
- func (db *DB) Load(path string) error
- func (db *DB) LoadFrom(r io.Reader) error
- func (db *DB) Overwrite() error
- func (db *DB) Query(query string, args ...any) (*sql.Rows, error)
- func (db *DB) QueryContext(ctx context.Context, query string, args ...any) (*sql.Rows, error)
- func (db *DB) QueryRow(query string, args ...any) *sql.Row
- func (db *DB) QueryRowContext(ctx context.Context, query string, args ...any) *sql.Row
- func (db *DB) Session(ctx context.Context) (*Session, error)
- func (db *DB) Snapshot(path string) error
- func (db *DB) SnapshotTo(w io.Writer) error
- type FunctionContext
- type Info
- type Session
- func (s *Session) Close() error
- func (s *Session) Exec(query string, args ...any) (sql.Result, error)
- func (s *Session) ExecContext(ctx context.Context, query string, args ...any) (sql.Result, error)
- func (s *Session) Prepare(query string) (*sql.Stmt, error)
- func (s *Session) PrepareContext(ctx context.Context, query string) (*sql.Stmt, error)
- func (s *Session) Query(query string, args ...any) (*sql.Rows, error)
- func (s *Session) QueryContext(ctx context.Context, query string, args ...any) (*sql.Rows, error)
- func (s *Session) QueryRow(query string, args ...any) *sql.Row
- func (s *Session) QueryRowContext(ctx context.Context, query string, args ...any) *sql.Row
Constants ¶
const ( // Magic identifies an ExecDB footer at the end of a file (spec §7). Magic = "EXECDB01" // Magic(8) + Version(4) + DataOffset(8) + DataLength(8) + Reserved(4). // All integer fields are big-endian. FooterSize = 32 // FormatVersion is the current data-blob format version written into // a footer's Version field. FormatVersion = 1 // MaxDataSize is the largest data blob engine can hold. It matches // modernc.org/sqlite's memdb VFS default ceiling // (SQLITE_MEMDB_DEFAULT_MAXSIZE = 1GiB); Step 1's spike found // SQLITE_FULL in practice a bit below that, around 960MiB (PLAN.md // "フェーズ②Step 1で確定した事実", .claude/rules/sqlite-quirks.md). A // footer claiming a DataLength larger than this is necessarily // corrupt. MaxDataSize = 1 << 30 )
Variables ¶
var ErrBusy = errors.New("engine: database is busy (another session is writing)")
ErrBusy is returned when a Snapshot/Overwrite/Load could not acquire the serialization barrier because another session held a conflicting write transaction open past the live database's busy_timeout.
var ErrClosed = errors.New("engine: database is closed")
ErrClosed is returned by DB methods called after Close. It is not returned by QueryRow: *sql.Row has no way to carry a caller-supplied error before Scan is called, so a post-Close QueryRow instead surfaces database/sql's own "sql: database is closed" through Scan.
var ErrNoData = errors.New("engine: no ExecDB data found")
ErrNoData is returned by Load when path holds no ExecDB data (spec §4).
var ErrNotOverwritable = errors.New("engine: running executable is not overwritable (looks like a `go run` temporary binary)")
ErrNotOverwritable is returned by Overwrite when the running process's own executable path looks like a `go run` temporary binary: `go run` deletes it as soon as the process exits, which would make Overwrite a no-op that looks like it succeeded.
var ErrTooLarge = errors.New("engine: database is too large to serialize")
ErrTooLarge is returned when the in-memory database cannot be serialized as a single contiguous byte slice -- either because it exceeds what Serialize can represent, or because modernc.org/sqlite returned an empty buffer for a large database rather than an error (see .claude/rules/sqlite-quirks.md).
Functions ¶
func Complete ¶
Complete reports whether sql is a syntactically complete SQL statement (or a semicolon-separated sequence of them), by calling SQLite's own sqlite3_complete() C routine -- the same one the real sqlite3 CLI shell uses to decide, while reading interactive input, whether to run what has been typed so far or keep waiting for more lines. A caller that reads SQL interactively (cmd/execdb's REPL, spec §3) needs this exact question answered correctly for a CREATE TRIGGER's BEGIN...END body: a hand-rolled scanner that just tracks "BEGIN"/"END" token nesting would be fooled by a CASE...END expression inside that body (CASE's END does not close a trigger body, but such a scanner has no way to tell the two apart), where sqlite3_complete's real tokenizer does not make that mistake.
Complete does not touch any database -- sqlite3_complete is a pure text-scanning routine that takes no DB handle -- so this works without an open DB (New/Open/OpenSelf) by design.
Reaching sqlite3_complete requires calling into modernc.org/sqlite's generated internals (modernc.org/sqlite/lib, not the stable driver package) rather than a documented public API; see .claude/rules/sqlite-quirks.md for what to recheck if modernc.org/sqlite is ever upgraded. modernc.org/libc and modernc.org/sqlite/lib are already transitive dependencies of modernc.org/sqlite itself (.claude/rules/binary-size.md, check-deps' "net" note in the Makefile), so this adds no new third-party dependency.
func RegisterScalarFunction ¶ added in v0.1.1
func RegisterScalarFunction(name string, nArgs int32, fn func(ctx *FunctionContext, args []driver.Value) (driver.Value, error)) error
RegisterScalarFunction registers a custom scalar SQL function named name, usable from any SQL text executed against any engine.DB opened afterward. This wraps modernc.org/sqlite.RegisterDeterministicScalarFunction (the top-level, stable driver package -- unlike Complete's modernc.org/sqlite/lib, this needs no .claude/rules/sqlite-quirks.md upgrade caveat). Registration is process-wide and one-shot per name (modernc.org/sqlite errors if the same name is registered twice), not per-DB or per-Session, matching the underlying library's own semantics.
This exists for cmd/execdb's pgwire layer to expose a handful of PostgreSQL built-in functions (current_schema(), pg_get_expr(), ...) that its pg_catalog-compatibility views (phase 4 follow-up: ODBC/ psqlODBC support) need callable from plain SQL -- engine itself has no notion of PostgreSQL; it only provides the mechanism.
Types ¶
type DB ¶
type DB struct {
// contains filtered or unexported fields
}
DB is an in-memory SQL database backed by modernc.org/sqlite, whose state can be persisted as a new executable (Snapshot) or by overwriting the host process's own executable in place (Overwrite). See execdb_spec.md §6 for the full design. The zero value is not usable; construct a DB with New, Open, or OpenSelf.
func Open ¶
Open loads the data embedded in path, which may be either a plain ExecDB data file or an ExecDB-formatted executable (spec §6). A path that does not exist, or one with no ExecDB footer, yields an empty database; the bytes preceding any footer are remembered so a later Snapshot can carry them forward as the new file's engine prefix (spec §7).
func OpenSelf ¶
OpenSelf loads the data embedded in the running process's own executable (os.Executable()). It also removes a leftover ".execdb_old" sidecar left behind by a previous Overwrite, on a best-effort basis (spec §7).
func (*DB) Close ¶
Close releases the database's resources. It does not persist anything; callers that want to keep the data must call Snapshot or Overwrite first (spec §4: persistence is explicit only). Close is idempotent: a second call is a no-op that returns nil.
func (*DB) Exec ¶
Exec runs a DDL/DML/TCL statement. There is no restriction here: a Go caller using engine as a library is trusted (spec §6); access control by caller (REPL vs. external I/F) is cmd/execdb's responsibility.
Exec, Query and QueryRow (and their *Context variants) each run on a connection borrowed from the pool (db.sdb), used once and returned -- they are one-shot operations, not a session. A statement like BEGIN executed this way appears to succeed, but the transaction it opens is invisible to every later call: database/sql's ResetSession does not roll back an open transaction before returning a connection to the pool, so the next unrelated caller could silently inherit it (.claude/rules/sqlite-quirks.md). Callers that need BEGIN/COMMIT/ROLLBACK to mean anything must hold a single dedicated connection across those statements: use Session instead.
func (*DB) ExecContext ¶
func (*DB) Load ¶
Load replaces the in-memory state with the data embedded in path, discarding whatever this DB previously held; it does not merge (spec §4, REPL ".load"). Load writes no file, and it never references path's engine portion -- only its data -- even when path is itself an ExecDB executable (spec §4: "常に今動いているプロセス自身のエンジンで、 データのみを読み込む"). Load does not warn about a footer format-version mismatch; call Inspect first if the caller wants that (spec §4).
Load replaces the live database's content in place (via SQLite's online Backup API, see loadBlobInto) rather than swapping in a new keeper connection. This means any connection obtained before Load ran -- including a Session from a still-open pgwire/REPL client -- keeps working afterward and sees the newly loaded data, instead of failing mid-response because its underlying connection was closed out from under it.
func (*DB) LoadFrom ¶
LoadFrom behaves like Load, but reads the image (engine bytes + data + footer) from r instead of a named file (spec §6: "汎用的な読み書き（任意の ファイル、io.Writer）" -- the read-side counterpart to SnapshotTo). Because r has no name, the resulting Info's Path is "".
func (*DB) Overwrite ¶
Overwrite replaces the running process's own executable (os.Executable()) in place with its own engine bytes followed by the current DB state (spec §6, §7; REPL ".overwrite"). It always targets the current process's executable, regardless of which file this DB was opened from (spec §6: "ホストアプリ全体がそのままエンジン部分として扱われる"). The process itself is not terminated.
func (*DB) QueryContext ¶
func (*DB) QueryRowContext ¶
QueryRowContext behaves like ExecContext/QueryContext, with one difference after Close: it cannot return ErrClosed directly, because *sql.Row has no way to carry a caller-supplied error before Scan is called. A QueryRowContext call made after Close instead surfaces database/sql's own "sql: database is closed" once Scan is called on the result.
func (*DB) Session ¶
Session opens a dedicated connection to db's live database: one independent client (spec §2/§8), such as a single pgwire connection or the REPL's own connection. Unlike Exec/Query/QueryRow, which borrow a connection from the pool for a single statement, a Session holds the same underlying connection across calls, so BEGIN/COMMIT/ROLLBACK executed through it behave like a real SQL transaction. The caller must Close it when done.
func (*DB) Snapshot ¶
Snapshot writes the current state to path: the engine bytes this DB was opened with (if any) followed by a fresh data blob and footer (spec §4, REPL ".snapshot", CLI "--snapshot-as"). The write is atomic (a temporary file in the same directory, then renamed into place). The output is executable (0755) if this DB was opened from an executable, or a plain data file (0644) otherwise (spec §6).
Snapshot does not generate a file name (timestamps, ".exe" completion): that is cmd/execdb's responsibility (naming.md); path is used as-is.
type FunctionContext ¶ added in v0.1.1
type FunctionContext = sqlite.FunctionContext
FunctionContext is passed to a function registered via RegisterScalarFunction. It carries no methods callers currently need; it exists so RegisterScalarFunction's signature does not depend on modernc.org/sqlite's own type outside this file.
type Session ¶
type Session struct {
// contains filtered or unexported fields
}
Session is a dedicated connection to a DB's live database, obtained via DB.Session -- see that method's doc comment for what it is for. A Session deliberately has no Begin/BeginTx: BEGIN/COMMIT/ROLLBACK are plain SQL statements to it, executed like any other, which is enough to give them real transactional meaning on a connection that is not shared with anyone else. Wrapping them in *sql.Tx would create a second, conflicting notion of "the current transaction" that database/sql itself does not know about.
func (*Session) Close ¶
Close releases the Session's connection, rolling back any transaction left open on it (closing a *sql.Conn does this automatically). Close is idempotent: a second call is a no-op that returns nil.
func (*Session) ExecContext ¶
func (*Session) Prepare ¶
Prepare and PrepareContext return a *sql.Stmt bound to this Session's own connection, for callers that run the same statement many times (e.g. bulk-loading a CSV file's rows, cmd/execdb's ".import") and want to avoid re-preparing it on every call.
Source Files