Help us shape the future of Go! Take the 2026 Go Developer Survey

data

package
v0.0.0-...-765e509 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 23, 2026 License: MIT Imports: 23 Imported by: 0

Documentation

Overview

Package data is Loom's SQL-first persistence layer: a database handle, transactions with AfterCommit hooks, the Querier interface that read helpers and jobs accept, typed reads (Select, Get), mapped errors, migrations and change tracking. The root package re-exports it as loom.DB, loom.Tx, loom.Select and so on; use those names. See docs/data.md.

db, err := loom.OpenDB(ctx, os.Getenv("DATABASE_URL"))
err = db.Tx(ctx, func(tx loom.Tx) error {
	if _, err := tx.Exec(ctx, `update todos set done = not done where id = $1`, id); err != nil {
		return err
	}
	jobs.Enqueue(tx, NotifyToggle{ID: id})
	tx.AfterCommit(func() { app.PubSub().Publish(loom.Message{Topic: "todos", Event: "changed"}) })
	return nil
})

Postgres comes first, through the pgx driver; nothing in these interfaces is Postgres-specific, so a SQLite backend can follow.

Index

Constants

View Source
const (
	DB601 = "DB601" // can't connect to the database
	DB602 = "DB602" // SQL on a DB with no database
	DB603 = "DB603" // a Tx used after its transaction ended
	DB604 = "DB604" // Get found no row (ErrNotFound)
	DB605 = "DB605" // Get found more than one row
	DB606 = "DB606" // unique violation
	DB607 = "DB607" // foreign key violation
	DB608 = "DB608" // not-null violation
	DB609 = "DB609" // the table doesn't exist
	DB610 = "DB610" // a column has no matching struct field
	DB611 = "DB611" // a column's value doesn't fit its field
	DB612 = "DB612" // the query's columns don't fit the type
	DB613 = "DB613" // a badly named or duplicate migration file
	DB614 = "DB614" // an applied migration was edited
	DB615 = "DB615" // a migration failed
	DB616 = "DB616" // a bad loom: annotation in a migration
	DB618 = "DB618" // the statement names a column that doesn't exist
	DB619 = "DB619" // the statement has a syntax error
	DB620 = "DB620" // a value's type doesn't fit its column
	DB621 = "DB621" // no function or operator takes these argument types
	DB622 = "DB622" // loom.Watch names a table that isn't tracked, or the app has no database
	DB623 = "DB623" // the connection doesn't keep LISTEN: a transaction-mode pooler such as pgbouncer
	DB624 = "DB624" // an AfterCommit or AfterRollback hook panicked
)

Data error codes.

View Source
const ChangeChannel = "loom_sync"

ChangeChannel is the channel of the NOTIFY sent when a transaction changes a tracked table. The payload is the table's name. The channel keeps the name it had when it served the sync engine, so databases tracked then keep notifying it without a migration.

Variables

View Source
var (
	ErrNotFound            = &Error{Code: DB604, Msg: "no row"}
	ErrUniqueViolation     = &Error{Code: DB606, Msg: "unique violation"}
	ErrForeignKeyViolation = &Error{Code: DB607, Msg: "foreign key violation"}
	ErrNotNullViolation    = &Error{Code: DB608, Msg: "not-null violation"}
)

Errors to test for with errors.Is. Each matches every error of its code, whatever its message.

View Source
var LockTimeout = 10 * time.Second

LockTimeout is how long a migration waits for a lock on a table before failing (DB615). A migration that queues behind a long transaction would otherwise block every query on the table meanwhile. A migration can set its own with SET LOCAL lock_timeout.

Functions

func Get

func Get[T any](ctx context.Context, q Querier, query string, args ...any) (T, error)

Get runs a query that should return one row and scans it into a T, as Select does. It returns an error matching ErrNotFound (DB604) if there is no row, and DB605 if there is more than one.

todo, err := loom.Get[Todo](ctx, db, `select * from todos where id = $1`, id)
if errors.Is(err, loom.ErrNotFound) { ... }

func IsCode

func IsCode(err error, code string) bool

IsCode reports whether err is a data error with the given code.

func MapError

func MapError(err error) error

MapError turns an error the database reported into an *Error with a code and a fix, where Loom knows one: unique, foreign key and not-null violations, missing tables and columns, syntax errors, and type mismatches. Other errors are returned unchanged. The driver's error stays reachable with errors.As. Querier methods already map their errors; call MapError on errors from the *sql.DB that db.SQL() returns.

func Select

func Select[T any](ctx context.Context, q Querier, query string, args ...any) ([]T, error)

Select runs a query and scans every row into a T.

todos, err := loom.Select[Todo](ctx, db, `select * from todos where user_id = $1 order by id`, userID)

T is a struct, whose fields take the columns of the same name: the field's db tag (`db:"user_id"`), or else its name in snake_case (UserID takes user_id). Fields tagged `db:"-"` are skipped, embedded structs contribute their fields, and fields with no column are left alone. A column with no field is an error (DB610). T may also be a single value, such as int64, string or time.Time, for a query of one column. A NULL needs a pointer or sql.Null* field (DB611).

func SnakeCase

func SnakeCase(name string) string

SnakeCase converts a Go field name to the column name it takes by default: UserID -> user_id, HTMLBody -> html_body, DoneAt -> done_at.

func Tracked

func Tracked(ctx context.Context, q Querier) ([]string, error)

Tracked returns the names of the tracked tables.

Types

type Applied

type Applied struct {
	Migration
	Ms int64 `json:"ms"`
}

Applied is a migration Migrate applied.

func Migrate

func Migrate(ctx context.Context, db *DB, fsys fs.FS) ([]Applied, error)

Migrate applies the pending migrations in fsys to db, in order, each in a transaction with its tracking triggers and its loom_migrations row (unless it is marked `-- loom:no-transaction`), waiting at most LockTimeout for each lock. It holds a Postgres advisory lock throughout, so processes migrating at once take turns and each migration runs once. Before applying anything it checks that no applied migration was edited (DB614). A failing migration is rolled back and stops the run (DB615); those before it stay applied.

type Column

type Column struct {
	Name     string `json:"name"`
	Type     string `json:"type"` // as Postgres writes it, e.g. "timestamp with time zone"
	Nullable bool   `json:"nullable"`
	Default  string `json:"default,omitempty"` // SQL expression
}

Column describes a column.

type DB

type DB struct {
	// contains filtered or unexported fields
}

DB is a database handle, safe for concurrent use. It is one of three kinds:

  • a Postgres database, from Open or FromSQL;
  • a local DB with no database (NewLocal), whose transactions still commit, roll back and run hooks, for in-memory apps;
  • a DB inside one transaction (InTx), whose Tx calls are savepoints, for tests that roll everything back (loomtest.DB).

func FromSQL

func FromSQL(sdb *sql.DB) *DB

FromSQL wraps a *sql.DB. Use the pgx driver (sql.Open("pgx", url)) for Postgres; the jobs store's LISTEN needs it.

func InTx

func InTx(stx *sql.Tx) *DB

InTx returns a DB whose SQL all runs inside stx. Its Tx method makes a savepoint, released on success (running AfterCommit hooks) and rolled back on error or panic; statements run on the DB itself get a savepoint each too, so a failed one doesn't abort stx. Roll stx back afterwards and nothing persists: loomtest.DB gives each test one of these. Use it from one goroutine at a time, like the *sql.Tx it wraps. Its SQL method returns nil, so it can't host a Postgres job store or LISTEN.

func NewLocal

func NewLocal() *DB

NewLocal returns a DB with no database behind it. Its transactions run AfterCommit and AfterRollback hooks, so jobs enqueued in them follow the same rules, but any SQL fails with DB602.

func Open

func Open(ctx context.Context, url string) (*DB, error)

Open connects to a Postgres database URL with the pgx driver and checks the connection.

func (*DB) Close

func (db *DB) Close() error

Close closes the database. It does nothing for a local or InTx DB.

func (*DB) Exec

func (db *DB) Exec(ctx context.Context, query string, args ...any) (res Result, err error)

Exec runs a statement outside a transaction.

func (*DB) IsInTx

func (db *DB) IsInTx() bool

IsInTx reports whether db was made by InTx.

func (*DB) Observe

func (db *DB) Observe(stream *trace.Stream, slow time.Duration)

Observe makes db report to stream: tx.commit and tx.rollback for each transaction, and db.slow for each statement that runs for slow or longer (zero reports none). The trace id of each event is the one in the context the transaction or statement was given, so a page handler's commit belongs to the click that caused it. The app calls it for its DB.

func (*DB) Query

func (db *DB) Query(ctx context.Context, query string, args ...any) (rows Rows, err error)

Query runs a query outside a transaction.

func (*DB) QueryRow

func (db *DB) QueryRow(ctx context.Context, query string, args ...any) Row

QueryRow runs a query that returns at most one row.

func (*DB) QueryStats

func (db *DB) QueryStats() []QueryStat

QueryStats returns every statement the DB has run, the busiest first: by the time they took in all, so a query that is quick but frequent ranks with one that is slow.

func (*DB) Record

func (db *DB) Record(ctx context.Context, query string, start time.Time, rows int64, err error)

Record counts a statement that code ran on db.SQL() directly, as Exec and Query count their own: in QueryStats, and as db.slow when it ran for the slow threshold or longer. rows is the rows it returned or changed, if known (else 0). The statement also counts toward the action ctx belongs to (trace.Tally). Loom's Postgres job store uses it, since it needs database/sql's own types for its locks and LISTEN; app code uses Exec and Query, which record themselves.

func (*DB) SQL

func (db *DB) SQL() *sql.DB

SQL returns the underlying *sql.DB, or nil for a local or InTx DB.

func (*DB) SetValue

func (db *DB) SetValue(key, value any)

SetValue attaches a value to the DB under key, for code that works with a DB's transactions: a jobs store binds itself this way, so jobs.Enqueue(tx, ...) finds the store of tx's DB.

func (*DB) Stats

func (db *DB) Stats(ctx context.Context) (ServerStats, error)

Stats reads the database's own statistics for its tables in the current schema and, if installed, pg_stat_statements.

func (*DB) Tx

func (db *DB) Tx(ctx context.Context, f func(tx Tx) error) (err error)

Tx runs f in a transaction. It commits if f returns nil, and rolls back if f returns an error or panics (the panic continues after the rollback). AfterCommit hooks run after a successful commit, in order; AfterRollback hooks run after a rollback, including a failed commit.

func (*DB) Value

func (db *DB) Value(key any) any

Value returns the value attached under key, or nil.

type Error

type Error struct {
	Code string `json:"code"` // e.g. "DB601"
	Msg  string `json:"message"`
	Fix  string `json:"fix"`
	Err  error  `json:"-"`
}

Error is a data-layer error with a stable code, documented in docs/errors.md, and a fix. Errors of the same code match with errors.Is, so errors.Is(err, loom.ErrNotFound) works whatever the message. Err is the driver's error, if any.

func (*Error) Error

func (e *Error) Error() string

func (*Error) Is

func (e *Error) Is(target error) bool

Is reports whether target is an *Error with the same code.

func (*Error) Unwrap

func (e *Error) Unwrap() error

Unwrap returns the driver's error.

type Migration

type Migration struct {
	Version  int64    `json:"version"`
	Name     string   `json:"name"`
	File     string   `json:"file"`
	Checksum string   `json:"checksum"` // sha256 of the file, in hex
	Track    []string `json:"track,omitempty"`
	From     string   `json:"from,omitempty"` // "<package> <file>": installed from a sub-app by `loom db install`
	// NoTransaction is set by `-- loom:no-transaction`: the migration runs
	// outside a transaction, for statements that can't run in one, such
	// as CREATE INDEX CONCURRENTLY.
	NoTransaction bool   `json:"no_transaction,omitempty"`
	SQL           string `json:"-"`
}

Migration is one migration file.

func LoadMigrations

func LoadMigrations(fsys fs.FS) ([]Migration, error)

LoadMigrations reads the migration files at the root of fsys, in order. Other .sql files, and duplicate numbers, are errors (DB613); files that don't end in .sql are ignored.

type MigrationState

type MigrationState struct {
	Migration
	// State is "applied", "pending", "changed" (applied, then edited: its
	// checksum no longer matches) or "missing" (applied, but its file is
	// gone).
	State     string     `json:"state"`
	AppliedAt *time.Time `json:"applied_at,omitempty"`
}

MigrationState is a migration with its state in a database.

type Querier

type Querier interface {
	Exec(ctx context.Context, query string, args ...any) (Result, error)
	Query(ctx context.Context, query string, args ...any) (Rows, error)
	QueryRow(ctx context.Context, query string, args ...any) Row
}

Querier runs SQL. Both *DB and Tx implement it, so read helpers accept either. Errors the database reports are mapped to *Error where Loom knows a fix (unique and foreign key violations, a missing table); the driver's error stays available through errors.As.

type QueryResult

type QueryResult struct {
	Columns   []string `json:"columns"`
	Rows      [][]any  `json:"rows"`
	Truncated bool     `json:"truncated"` // there were more than limit rows
}

QueryResult is the result of QueryReadOnly: what `loom db query` prints.

func QueryReadOnly

func QueryReadOnly(ctx context.Context, db *DB, query string, limit int) (*QueryResult, error)

QueryReadOnly runs a query in a read-only transaction that is rolled back, with a 10 second statement timeout, and returns at most limit rows. It is for dev tools (`loom db query`): writes fail.

type QueryStat

type QueryStat struct {
	// Query is the statement on one line, cut after 500 bytes; never its
	// arguments. "other" counts statements past the first 500 seen.
	Query string `json:"query"`
	// Rows is the rows queries returned and statements changed, in all.
	Rows   uint64 `json:"rows"`
	Errors uint64 `json:"errors"`
	trace.TimingSummary
}

QueryStat is how often one statement ran on a DB since it opened, and how long it took.

type Result

type Result interface {
	RowsAffected() (int64, error)
}

Result is the outcome of Exec.

type Row

type Row interface {
	Scan(dest ...any) error
}

Row is one row of a query; Scan returns sql.ErrNoRows if there was none. *sql.Row implements it.

type Rows

type Rows interface {
	Next() bool
	Scan(dest ...any) error
	Columns() ([]string, error)
	Err() error
	Close() error
}

Rows are the rows of a query. *sql.Rows implements it.

type SchemaInfo

type SchemaInfo struct {
	Tables []Table `json:"tables"`
}

SchemaInfo describes a database's tables: what `loom db schema` prints.

func Schema

func Schema(ctx context.Context, q Querier) (*SchemaInfo, error)

Schema describes the tables of the current schema (usually public), leaving out Loom's own loom_* tables and tracked tables' loom_xid.

type ServerStats

type ServerStats struct {
	// Since is when Postgres last reset the counts, if it says.
	Since  *time.Time  `json:"since"`
	Tables []TableStat `json:"tables"` // the most rows read by sequential scans first
	// Statements are pg_stat_statements' busiest, by total time, or
	// empty without the extension; StatementsNote then says why.
	Statements     []StatementStat `json:"statements"`
	StatementsNote string          `json:"statements_note,omitempty"`
}

ServerStats is what Postgres itself counts about a database: how each table is read (sequential against index scans) and, with the pg_stat_statements extension, every statement's totals across every node. QueryStats counts one node's statements as Loom ran them; this is the database's side, and where a missing index shows.

type StatementStat

type StatementStat struct {
	Query   string  `json:"query"`
	Calls   int64   `json:"calls"`
	TotalMS float64 `json:"total_ms"`
	MeanMS  float64 `json:"mean_ms"`
	Rows    int64   `json:"rows"`
}

StatementStat is one statement's totals in pg_stat_statements.

type Status

type Status struct {
	Migrations []MigrationState `json:"migrations"`
	Pending    int              `json:"pending"`
	// Problems are errors that stop Migrate: edited migrations (DB614).
	Problems []*Error `json:"problems"`
}

Status is the state of a database's migrations: what `loom migrate status` prints.

func MigrationStatus

func MigrationStatus(ctx context.Context, db *DB, fsys fs.FS) (*Status, error)

MigrationStatus reports which migrations in fsys are applied to db, pending, changed since they were applied, or missing from fsys.

type Table

type Table struct {
	Name       string   `json:"name"`
	Columns    []Column `json:"columns"`
	PrimaryKey []string `json:"primary_key"`
	Tracked    bool     `json:"tracked"`
}

Table describes a table.

type TableStat

type TableStat struct {
	Name       string `json:"name"`
	Rows       int64  `json:"rows"` // live rows, as Postgres estimates
	Bytes      int64  `json:"bytes"`
	SeqScans   int64  `json:"seq_scans"`
	SeqRows    int64  `json:"seq_rows"` // rows those scans read
	IndexScans int64  `json:"index_scans"`
	Inserts    int64  `json:"inserts"`
	Updates    int64  `json:"updates"`
	Deletes    int64  `json:"deletes"`
	// Warning says when the table is read mostly by sequential scans of
	// many rows: a query on it has no index it can use.
	Warning string `json:"warning,omitempty"`
}

TableStat is how one table has been read and written.

type Tx

type Tx interface {
	Querier
	// Context is the context the transaction was started with.
	Context() context.Context
	// AfterCommit runs f after the transaction commits, never if it rolls
	// back. Use it to notify live pages of committed changes.
	AfterCommit(f func())
	// AfterRollback runs f if the transaction rolls back, for code that
	// keeps state on the transaction's behalf (in-memory job stores).
	AfterRollback(f func())
	// DB is the database the transaction belongs to.
	DB() *DB
}

Tx is a database transaction, given to the function passed to DB.Tx. It is the one transaction type in Loom: app code, jobs.Enqueue and jobs take a Tx.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL