Documentation
¶
Overview ¶
Package migrate is a versioned SQL migration runner with optional per-group migration streams.
An app composed of optional features can organize migrations into named GROUPS so each module owns its schema independently. A migration with an empty Group belongs to the DEFAULT group — the single flat, version-ordered list every existing app uses. Two groups may each have a version 1; version uniqueness is enforced per group.
-- +migrate Version 1 -- +migrate Group knowledge -- +migrate Name create_kb_tables -- +migrate Up CREATE TABLE kb_docs (...);
Selection ¶
Up/Down/Status take an optional variadic list of group names. With no arguments every registered group is in scope (the existing behavior). With one or more names, only those groups' migrations are applied, rolled back, or reported — so enabling a feature later runs just its pending set. Force takes at most one group (0 = default, 1 = that group, more = error).
Ordering ¶
Within a group migrations apply strictly by version. When one run applies multiple groups the global order is (version, group_name) ascending. Groups MUST be self-contained: no cross-group schema dependencies. The ordering rule is a deterministic tiebreak, not a dependency mechanism.
Compatibility ¶
An app that never uses groups sees byte-identical behavior: the same tracking table, the same SQL, the same advisory lock. The group_name column and the composite (group_name, version) primary key are created only when a non-default group is registered or selected. A pre-existing default-group table that later needs groups is upgraded in place (ALTER on Postgres, an atomic create/copy/rename rebuild on SQLite) the first time a group-aware operation runs.
See https://github.com/DonaldMurillo/gofastr for documentation.
Index ¶
- Constants
- Variables
- func EnsureDatabase(driver, dsn string) (bool, error)
- func ValidateGroupName(name string) error
- func WithAdvisoryLock(ctx context.Context, db *sql.DB, dialect Dialect, ...) error
- func WithAdvisoryLockKey(ctx context.Context, db *sql.DB, dialect Dialect, key int64, ...) error
- type ChecksumMismatchError
- type Dialect
- type Migration
- type MigrationRecord
- type Migrator
- func (m *Migrator) CreateMigrationsTable(ctx context.Context) error
- func (m *Migrator) Down(ctx context.Context, n int, groups ...string) error
- func (m *Migrator) Force(ctx context.Context, version uint64, applied bool, groups ...string) error
- func (m *Migrator) Register(mig Migration) error
- func (m *Migrator) RegisterFromReader(r io.Reader) error
- func (m *Migrator) Status(ctx context.Context, groups ...string) (*Status, error)
- func (m *Migrator) Up(ctx context.Context, groups ...string) error
- type Option
- type Status
Constants ¶
const AdvisoryLockKey int64 = 6724469554113028193
AdvisoryLockKey is the fixed 64-bit key used for the migration advisory lock on Postgres. It MUST stay stable across releases: during a rolling deploy an old and a new instance both try to migrate, and they only mutually exclude if they agree on this key. Changing it would let two instances run DDL concurrently — the exact race the lock exists to prevent.
The value is arbitrary but fixed (derived once from "gofastr.migrate" and frozen). Hosts that run unrelated migration tooling against the same database can override it via WithAdvisoryLockKey to avoid cross-tool contention.
const SeedAdvisoryLockKey int64 = 7583194026157293042
SeedAdvisoryLockKey is the fixed 64-bit key used for the SEED advisory lock on Postgres — DISTINCT from AdvisoryLockKey so that two replicas booting simultaneously serialize schema migration and seeding independently. Without a separate key, a replica that finished its DDL but still holds the migration lock to run its seeds would block every other replica's DDL phase (and vice versa). Same stability rule as AdvisoryLockKey: derived once from "gofastr.seed" and frozen — changing it would let two replicas run a Seed func concurrently.
Combined with the _gofastr_seeded ledger (framework/migrate/seed.go), the lock turns RunSeeds into run-ONCE-globally: whichever replica wins the lock runs the Seed body and records the ledger row; the others wait for the lock, then short-circuit on the ledger. A crashed lock holder's session-level lock is released automatically by Postgres when the connection closes — no permanent block.
Variables ¶
var ErrDirty = errors.New("migrate: database is in a dirty state from a failed migration — reconcile manually and call Force")
ErrDirty is returned by Up and Down when the tracking table records a migration left in a dirty state by a previously failed no-transaction migration. The database may be partially migrated; an operator must reconcile it by hand and then call Force to clear the state before migrations can proceed again.
Functions ¶
func EnsureDatabase ¶
EnsureDatabase creates the target database if it does not already exist, returning whether it had to create it. This is the "first migration can create the DB" capability — call it before running migrations.
- SQLite (and any file/embedded driver): a no-op. The database file is created automatically when the runner opens it, so there is nothing to do; returns (false, nil).
- Postgres: connects to the maintenance `postgres` database (you cannot CREATE a database while connected to the one being created), checks pg_database, and issues CREATE DATABASE when absent. Requires the driver to be linked into the binary and the role to have CREATEDB.
dsn is the normal target DSN (URL form `postgres://…/dbname` or keyword form `host=… dbname=…`); EnsureDatabase derives the admin DSN from it.
func ValidateGroupName ¶ added in v0.17.0
ValidateGroupName is the exported form of validateGroupName, for callers outside the package (e.g. the CLI generate command validating --group before writing a migration file).
func WithAdvisoryLock ¶
func WithAdvisoryLock(ctx context.Context, db *sql.DB, dialect Dialect, fn func(conn *sql.Conn) error) error
WithAdvisoryLock runs fn while holding a database-level lock that serializes migration across every process pointed at the same database. fn receives the pinned *sql.Conn that holds the lock and MUST do all of its work on that connection — running the migration on the same session as the lock is what keeps the whole thing correct on a single-connection pool (MaxOpenConns(1)), which a separate lock connection would deadlock.
- Postgres: a session-level advisory lock on the pinned connection, acquired via pg_try_advisory_lock in a ctx-aware poll loop. A second instance waits until the first releases, or returns promptly if its ctx is cancelled. (A poll loop is used rather than the blocking pg_advisory_lock because lib/pq does not interrupt a blocked pg_advisory_lock on context cancellation — a stuck holder would otherwise hang boot forever.) This is the guard that makes auto-migrate-on-boot safe across N replicas.
- SQLite: no lock is taken (SQLite serializes writers at the file level), but fn still gets a pinned connection so callers have one uniform code path. The PK upgrade (rebuildTableSQLite) is NOT idempotent against a table that already has group_name values — that is why only Up/Down/ Force call ensureCompositeKey, never Status (which is unlocked).
db == nil runs fn(nil) — callers already treat a nil db as a no-op.
func WithAdvisoryLockKey ¶
func WithAdvisoryLockKey(ctx context.Context, db *sql.DB, dialect Dialect, key int64, fn func(conn *sql.Conn) error) error
WithAdvisoryLockKey is WithAdvisoryLock with an explicit lock key, for hosts that need to namespace the lock away from other migration tooling sharing the database.
Types ¶
type ChecksumMismatchError ¶
ChecksumMismatchError is returned by Up when an already-applied migration's recorded checksum no longer matches the registered migration's Up SQL. That means the migration file was edited after it was applied — a drift the runner refuses to paper over, because the live schema no longer matches what the (edited) file says it should be.
func (*ChecksumMismatchError) Error ¶
func (e *ChecksumMismatchError) Error() string
type Migration ¶
type Migration struct {
Version uint64
Name string
Up string // SQL to apply the migration
Down string // SQL to roll back the migration
// Group is the migration group this migration belongs to. An empty value
// means the DEFAULT group — the single flat, version-ordered list every
// existing app uses. A non-empty group lets an optional feature or module
// own an independent migration stream that can be applied selectively
// (e.g. apply the knowledge module's migrations only when that module is
// enabled). Version uniqueness is scoped PER GROUP: two groups may both
// have a version 1. Set via the `-- +migrate Group <name>` directive.
Group string
// NoTransaction runs Up/Down WITHOUT wrapping them in a transaction. The
// escape hatch for statements that cannot run inside a transaction —
// CREATE INDEX CONCURRENTLY, VACUUM, CREATE DATABASE on Postgres. The cost
// is that a failure mid-statement leaves the database partially migrated:
// the runner records the migration dirty before running it and only clears
// that flag on success, so a later run refuses to proceed (ErrDirty) until
// an operator reconciles and calls Force. Set via the `-- +migrate
// NoTransaction` directive in a SQL migration file.
NoTransaction bool
}
Migration represents a single versioned database migration.
type MigrationRecord ¶
type MigrationRecord struct {
Group string
Version uint64
Name string
AppliedAt time.Time
Checksum string // SHA-256 of the Up SQL recorded at apply time
Dirty bool // true if a no-transaction migration failed mid-apply
}
MigrationRecord is a row in the _migrations tracking table.
type Migrator ¶
type Migrator struct {
// contains filtered or unexported fields
}
Migrator manages database migrations.
func New ¶
New creates a new Migrator with the given database connection and options. By default the tracking table is "_migrations".
func (*Migrator) CreateMigrationsTable ¶
CreateMigrationsTable ensures the migrations tracking table exists. Public entry point (used by Status and tests); runs on the pool.
func (*Migrator) Down ¶
Down rolls back the last n applied migrations in reverse (Version, Group) order among the selected groups. Serialized by the same advisory lock as Up.
With no group arguments every group is in scope (the existing behavior). Passing group names limits the rollback to applied migrations in those groups, preserving the most-recent-first semantics within the selection.
func (*Migrator) Force ¶
Force reconciles the tracking table by hand, the recovery path out of a dirty state and the way to adopt an existing database (baseline).
- applied == true: mark `version` as cleanly applied without running its Up SQL. Inserts the row if missing (baseline), and clears any dirty flag on it. If the version is registered, its name and checksum are recorded so future drift checks line up.
- applied == false: remove `version` from the tracking table entirely, so it is treated as pending again (e.g. a no-tx migration that did not actually take effect).
Either way the dirty state for that version is cleared, unblocking Up/Down.
With no group argument the default group is targeted. Exactly one group argument targets that group; more than one is an error.
func (*Migrator) Register ¶
Register adds a migration, rejecting an invalid group name or a duplicate (Group, Version) pair. Migrations are kept sorted by (Version, Group) so the pending list is already in apply order: within a group versions ascend, and when multiple groups run together the tiebreak is group name — deterministic and byte-identical to the pre-group sort when everything is in the default group. A returned error does not mutate the migrator.
func (*Migrator) RegisterFromReader ¶
RegisterFromReader parses migration SQL from a reader and registers it. The expected format uses comment directives:
-- +migrate Version <n> -- +migrate Name <description> -- +migrate Up <up SQL statements> -- +migrate Down <down SQL statements>
func (*Migrator) Status ¶
Status returns the current migration state: which are applied and which are pending. With no group arguments every group is in scope (the existing behavior); passing group names scopes both lists to those groups.
Status is read-only: it never upgrades the tracking-table PK (that is an unlocked check-then-act left to Up/Down/Force, which hold the advisory lock). It detects the table's actual shape and reads accordingly — legacy SELECT when the group_name column is absent, group-aware SELECT when it is present — so a de-registered group's rows are still shown with their real group.
func (*Migrator) Up ¶
Up runs all pending migrations in (Version, Group) order. Each migration executes in its own transaction. Already-applied migrations are skipped. The whole run is serialized by a database advisory lock so two instances (e.g. rolling-deploy replicas) cannot apply migrations concurrently.
With no group arguments every registered group is applied (the existing behavior). Passing one or more group names limits the run to those groups' pending migrations only.
type Option ¶
type Option func(*Migrator)
Option configures a Migrator.
func WithDialect ¶
WithDialect sets the SQL dialect for the migrator. Defaults to DialectPostgres if not specified.
func WithTableName ¶
WithTableName sets a custom name for the migrations tracking table.
type Status ¶
type Status struct {
Applied []MigrationRecord
Pending []Migration
}
Status holds the result of querying migration state.
Source Files