Documentation
¶
Overview ¶
Package data is Loom's SQL-first persistence layer: a database handle, transactions with AfterCommit hooks, the Querier interface that read helpers and jobs accept, typed reads (Select, Get), mapped errors, migrations and change tracking. The root package re-exports it as loom.DB, loom.Tx, loom.Select and so on; use those names. See docs/data.md.
db, err := loom.OpenDB(ctx, os.Getenv("DATABASE_URL"))
err = db.Tx(ctx, func(tx loom.Tx) error {
if _, err := tx.Exec(ctx, `update todos set done = not done where id = $1`, id); err != nil {
return err
}
jobs.Enqueue(tx, NotifyToggle{ID: id})
tx.AfterCommit(func() { app.PubSub().Publish(loom.Message{Topic: "todos", Event: "changed"}) })
return nil
})
Postgres comes first, through the pgx driver; nothing in these interfaces is Postgres-specific, so a SQLite backend can follow.
Index ¶
- Constants
- Variables
- func Get[T any](ctx context.Context, q Querier, query string, args ...any) (T, error)
- func IsCode(err error, code string) bool
- func MapError(err error) error
- func Select[T any](ctx context.Context, q Querier, query string, args ...any) ([]T, error)
- func SnakeCase(name string) string
- func Tracked(ctx context.Context, q Querier) ([]string, error)
- type Applied
- type Column
- type DB
- func (db *DB) Close() error
- func (db *DB) Exec(ctx context.Context, query string, args ...any) (res Result, err error)
- func (db *DB) IsInTx() bool
- func (db *DB) Observe(stream *trace.Stream, slow time.Duration)
- func (db *DB) Query(ctx context.Context, query string, args ...any) (rows Rows, err error)
- func (db *DB) QueryRow(ctx context.Context, query string, args ...any) Row
- func (db *DB) QueryStats() []QueryStat
- func (db *DB) Record(ctx context.Context, query string, start time.Time, rows int64, err error)
- func (db *DB) SQL() *sql.DB
- func (db *DB) SetValue(key, value any)
- func (db *DB) Stats(ctx context.Context) (ServerStats, error)
- func (db *DB) Tx(ctx context.Context, f func(tx Tx) error) (err error)
- func (db *DB) Value(key any) any
- type Error
- type Migration
- type MigrationState
- type Querier
- type QueryResult
- type QueryStat
- type Result
- type Row
- type Rows
- type SchemaInfo
- type ServerStats
- type StatementStat
- type Status
- type Table
- type TableStat
- type Tx
Constants ¶
const ( DB601 = "DB601" // can't connect to the database DB602 = "DB602" // SQL on a DB with no database DB603 = "DB603" // a Tx used after its transaction ended DB604 = "DB604" // Get found no row (ErrNotFound) DB605 = "DB605" // Get found more than one row DB606 = "DB606" // unique violation DB607 = "DB607" // foreign key violation DB608 = "DB608" // not-null violation DB609 = "DB609" // the table doesn't exist DB610 = "DB610" // a column has no matching struct field DB611 = "DB611" // a column's value doesn't fit its field DB612 = "DB612" // the query's columns don't fit the type DB613 = "DB613" // a badly named or duplicate migration file DB614 = "DB614" // an applied migration was edited DB615 = "DB615" // a migration failed DB616 = "DB616" // a bad loom: annotation in a migration DB618 = "DB618" // the statement names a column that doesn't exist DB619 = "DB619" // the statement has a syntax error DB620 = "DB620" // a value's type doesn't fit its column DB621 = "DB621" // no function or operator takes these argument types DB622 = "DB622" // loom.Watch names a table that isn't tracked, or the app has no database DB623 = "DB623" // the connection doesn't keep LISTEN: a transaction-mode pooler such as pgbouncer DB624 = "DB624" // an AfterCommit or AfterRollback hook panicked )
Data error codes.
const ChangeChannel = "loom_sync"
ChangeChannel is the channel of the NOTIFY sent when a transaction changes a tracked table. The payload is the table's name. The channel keeps the name it had when it served the sync engine, so databases tracked then keep notifying it without a migration.
Variables ¶
var ( ErrNotFound = &Error{Code: DB604, Msg: "no row"} ErrUniqueViolation = &Error{Code: DB606, Msg: "unique violation"} ErrForeignKeyViolation = &Error{Code: DB607, Msg: "foreign key violation"} ErrNotNullViolation = &Error{Code: DB608, Msg: "not-null violation"} )
Errors to test for with errors.Is. Each matches every error of its code, whatever its message.
var LockTimeout = 10 * time.Second
LockTimeout is how long a migration waits for a lock on a table before failing (DB615). A migration that queues behind a long transaction would otherwise block every query on the table meanwhile. A migration can set its own with SET LOCAL lock_timeout.
Functions ¶
func Get ¶
Get runs a query that should return one row and scans it into a T, as Select does. It returns an error matching ErrNotFound (DB604) if there is no row, and DB605 if there is more than one.
todo, err := loom.Get[Todo](ctx, db, `select * from todos where id = $1`, id)
if errors.Is(err, loom.ErrNotFound) { ... }
func MapError ¶
MapError turns an error the database reported into an *Error with a code and a fix, where Loom knows one: unique, foreign key and not-null violations, missing tables and columns, syntax errors, and type mismatches. Other errors are returned unchanged. The driver's error stays reachable with errors.As. Querier methods already map their errors; call MapError on errors from the *sql.DB that db.SQL() returns.
func Select ¶
Select runs a query and scans every row into a T.
todos, err := loom.Select[Todo](ctx, db, `select * from todos where user_id = $1 order by id`, userID)
T is a struct, whose fields take the columns of the same name: the field's db tag (`db:"user_id"`), or else its name in snake_case (UserID takes user_id). Fields tagged `db:"-"` are skipped, embedded structs contribute their fields, and fields with no column are left alone. A column with no field is an error (DB610). T may also be a single value, such as int64, string or time.Time, for a query of one column. A NULL needs a pointer or sql.Null* field (DB611).
Types ¶
type Applied ¶
Applied is a migration Migrate applied.
func Migrate ¶
Migrate applies the pending migrations in fsys to db, in order, each in a transaction with its tracking triggers and its loom_migrations row (unless it is marked `-- loom:no-transaction`), waiting at most LockTimeout for each lock. It holds a Postgres advisory lock throughout, so processes migrating at once take turns and each migration runs once. Before applying anything it checks that no applied migration was edited (DB614). A failing migration is rolled back and stops the run (DB615); those before it stay applied.
type Column ¶
type Column struct {
Name string `json:"name"`
Type string `json:"type"` // as Postgres writes it, e.g. "timestamp with time zone"
Nullable bool `json:"nullable"`
Default string `json:"default,omitempty"` // SQL expression
}
Column describes a column.
type DB ¶
type DB struct {
// contains filtered or unexported fields
}
DB is a database handle, safe for concurrent use. It is one of three kinds:
- a Postgres database, from Open or FromSQL;
- a local DB with no database (NewLocal), whose transactions still commit, roll back and run hooks, for in-memory apps;
- a DB inside one transaction (InTx), whose Tx calls are savepoints, for tests that roll everything back (loomtest.DB).
func FromSQL ¶
FromSQL wraps a *sql.DB. Use the pgx driver (sql.Open("pgx", url)) for Postgres; the jobs store's LISTEN needs it.
func InTx ¶
InTx returns a DB whose SQL all runs inside stx. Its Tx method makes a savepoint, released on success (running AfterCommit hooks) and rolled back on error or panic; statements run on the DB itself get a savepoint each too, so a failed one doesn't abort stx. Roll stx back afterwards and nothing persists: loomtest.DB gives each test one of these. Use it from one goroutine at a time, like the *sql.Tx it wraps. Its SQL method returns nil, so it can't host a Postgres job store or LISTEN.
func NewLocal ¶
func NewLocal() *DB
NewLocal returns a DB with no database behind it. Its transactions run AfterCommit and AfterRollback hooks, so jobs enqueued in them follow the same rules, but any SQL fails with DB602.
func (*DB) Observe ¶
Observe makes db report to stream: tx.commit and tx.rollback for each transaction, and db.slow for each statement that runs for slow or longer (zero reports none). The trace id of each event is the one in the context the transaction or statement was given, so a page handler's commit belongs to the click that caused it. The app calls it for its DB.
func (*DB) QueryStats ¶
QueryStats returns every statement the DB has run, the busiest first: by the time they took in all, so a query that is quick but frequent ranks with one that is slow.
func (*DB) Record ¶
Record counts a statement that code ran on db.SQL() directly, as Exec and Query count their own: in QueryStats, and as db.slow when it ran for the slow threshold or longer. rows is the rows it returned or changed, if known (else 0). The statement also counts toward the action ctx belongs to (trace.Tally). Loom's Postgres job store uses it, since it needs database/sql's own types for its locks and LISTEN; app code uses Exec and Query, which record themselves.
func (*DB) SetValue ¶
SetValue attaches a value to the DB under key, for code that works with a DB's transactions: a jobs store binds itself this way, so jobs.Enqueue(tx, ...) finds the store of tx's DB.
func (*DB) Stats ¶
func (db *DB) Stats(ctx context.Context) (ServerStats, error)
Stats reads the database's own statistics for its tables in the current schema and, if installed, pg_stat_statements.
type Error ¶
type Error struct {
Code string `json:"code"` // e.g. "DB601"
Msg string `json:"message"`
Fix string `json:"fix"`
Err error `json:"-"`
}
Error is a data-layer error with a stable code, documented in docs/errors.md, and a fix. Errors of the same code match with errors.Is, so errors.Is(err, loom.ErrNotFound) works whatever the message. Err is the driver's error, if any.
type Migration ¶
type Migration struct {
Version int64 `json:"version"`
Name string `json:"name"`
File string `json:"file"`
Checksum string `json:"checksum"` // sha256 of the file, in hex
Track []string `json:"track,omitempty"`
From string `json:"from,omitempty"` // "<package> <file>": installed from a sub-app by `loom db install`
// NoTransaction is set by `-- loom:no-transaction`: the migration runs
// outside a transaction, for statements that can't run in one, such
// as CREATE INDEX CONCURRENTLY.
NoTransaction bool `json:"no_transaction,omitempty"`
SQL string `json:"-"`
}
Migration is one migration file.
type MigrationState ¶
type MigrationState struct {
Migration
// State is "applied", "pending", "changed" (applied, then edited: its
// checksum no longer matches) or "missing" (applied, but its file is
// gone).
State string `json:"state"`
AppliedAt *time.Time `json:"applied_at,omitempty"`
}
MigrationState is a migration with its state in a database.
type Querier ¶
type Querier interface {
Exec(ctx context.Context, query string, args ...any) (Result, error)
Query(ctx context.Context, query string, args ...any) (Rows, error)
QueryRow(ctx context.Context, query string, args ...any) Row
}
Querier runs SQL. Both *DB and Tx implement it, so read helpers accept either. Errors the database reports are mapped to *Error where Loom knows a fix (unique and foreign key violations, a missing table); the driver's error stays available through errors.As.
type QueryResult ¶
type QueryResult struct {
Columns []string `json:"columns"`
Rows [][]any `json:"rows"`
Truncated bool `json:"truncated"` // there were more than limit rows
}
QueryResult is the result of QueryReadOnly: what `loom db query` prints.
func QueryReadOnly ¶
QueryReadOnly runs a query in a read-only transaction that is rolled back, with a 10 second statement timeout, and returns at most limit rows. It is for dev tools (`loom db query`): writes fail.
type QueryStat ¶
type QueryStat struct {
// Query is the statement on one line, cut after 500 bytes; never its
// arguments. "other" counts statements past the first 500 seen.
Query string `json:"query"`
// Rows is the rows queries returned and statements changed, in all.
Rows uint64 `json:"rows"`
Errors uint64 `json:"errors"`
trace.TimingSummary
}
QueryStat is how often one statement ran on a DB since it opened, and how long it took.
type Row ¶
Row is one row of a query; Scan returns sql.ErrNoRows if there was none. *sql.Row implements it.
type Rows ¶
type Rows interface {
Next() bool
Scan(dest ...any) error
Columns() ([]string, error)
Err() error
Close() error
}
Rows are the rows of a query. *sql.Rows implements it.
type SchemaInfo ¶
type SchemaInfo struct {
Tables []Table `json:"tables"`
}
SchemaInfo describes a database's tables: what `loom db schema` prints.
type ServerStats ¶
type ServerStats struct {
// Since is when Postgres last reset the counts, if it says.
Since *time.Time `json:"since"`
Tables []TableStat `json:"tables"` // the most rows read by sequential scans first
// Statements are pg_stat_statements' busiest, by total time, or
// empty without the extension; StatementsNote then says why.
Statements []StatementStat `json:"statements"`
StatementsNote string `json:"statements_note,omitempty"`
}
ServerStats is what Postgres itself counts about a database: how each table is read (sequential against index scans) and, with the pg_stat_statements extension, every statement's totals across every node. QueryStats counts one node's statements as Loom ran them; this is the database's side, and where a missing index shows.
type StatementStat ¶
type StatementStat struct {
Query string `json:"query"`
Calls int64 `json:"calls"`
TotalMS float64 `json:"total_ms"`
MeanMS float64 `json:"mean_ms"`
Rows int64 `json:"rows"`
}
StatementStat is one statement's totals in pg_stat_statements.
type Status ¶
type Status struct {
Migrations []MigrationState `json:"migrations"`
Pending int `json:"pending"`
// Problems are errors that stop Migrate: edited migrations (DB614).
Problems []*Error `json:"problems"`
}
Status is the state of a database's migrations: what `loom migrate status` prints.
type Table ¶
type Table struct {
Name string `json:"name"`
Columns []Column `json:"columns"`
PrimaryKey []string `json:"primary_key"`
Tracked bool `json:"tracked"`
}
Table describes a table.
type TableStat ¶
type TableStat struct {
Name string `json:"name"`
Rows int64 `json:"rows"` // live rows, as Postgres estimates
Bytes int64 `json:"bytes"`
SeqScans int64 `json:"seq_scans"`
SeqRows int64 `json:"seq_rows"` // rows those scans read
IndexScans int64 `json:"index_scans"`
Inserts int64 `json:"inserts"`
Updates int64 `json:"updates"`
Deletes int64 `json:"deletes"`
// Warning says when the table is read mostly by sequential scans of
// many rows: a query on it has no index it can use.
Warning string `json:"warning,omitempty"`
}
TableStat is how one table has been read and written.
type Tx ¶
type Tx interface {
Querier
// Context is the context the transaction was started with.
Context() context.Context
// AfterCommit runs f after the transaction commits, never if it rolls
// back. Use it to notify live pages of committed changes.
AfterCommit(f func())
// AfterRollback runs f if the transaction rolls back, for code that
// keeps state on the transaction's behalf (in-memory job stores).
AfterRollback(f func())
// DB is the database the transaction belongs to.
DB() *DB
}
Tx is a database transaction, given to the function passed to DB.Tx. It is the one transaction type in Loom: app code, jobs.Enqueue and jobs take a Tx.