Documentation
¶
Overview ¶
Package migrate runs versioned *.sql migration files against a database, tracking each file's state (applied/rolled back, batch, run count, checksum) in a tracking table it manages itself. It is dialect-agnostic: see the postgres and mysql subpackages for the two supported databases, and RunCLI for wiring it into an application's own command-line entrypoint.
File layout ¶
Migrations live as pairs of files named "YYYYMMDD_HHMMSS_description.up.sql" and "YYYYMMDD_HHMMSS_description.down.sql" — a UTC timestamp, to the second, then a snake_case description — in a single directory; Load discovers and validates them. The timestamp (not a small sequential counter) is deliberate: two people working on separate branches each pick "the next number" independently and collide or, worse, don't collide but sort in an order neither of them intended once merged. A timestamp taken at the moment the file is created makes that collision astronomically unlikely and keeps the merged order matching the order each file was actually written in. New creates a fresh pair with the current timestamp.
Concurrency ¶
A Migrator is not safe for concurrent use by multiple goroutines within one process — each exported method holds the underlying *sql.DB open for its own duration and returns before the next call should start. Concurrent processes are handled separately: every method acquires the dialect's advisory lock for its full duration, so two OS processes (e.g. two deploys racing) serialize instead of corrupting the tracking table.
Index ¶
- Constants
- Variables
- func RunCLI(args []string, dialect Dialect, dir string) int
- type Dialect
- type Migration
- type Migrator
- func (m *Migrator) Baseline(ctx context.Context, version string, force bool) error
- func (m *Migrator) Down(ctx context.Context, all bool, dryRun bool) (*Result, error)
- func (m *Migrator) Status(ctx context.Context) ([]StatusRow, error)
- func (m *Migrator) Up(ctx context.Context, all bool, dryRun bool) (*Result, error)
- type Record
- type Result
- type StatusRow
Constants ¶
const ( // DefaultTable is the tracking table name every adopter of this package uses. It is not // configurable (see NewMigrator) — one gFly app has one migrations directory and one // tracking table, and a configurable name would only invite drift between the table a // health check reads and the one db:migrate writes. DefaultTable = "migrations" // StatusUp and StatusDown are the two values Record.Status takes. They are exported so a // caller inspecting Status's results doesn't need to guess the tracking table's raw string // values, and so this package's own SQL (store.go) and each Dialect's generated SQL // (postgres, mysql) share one definition instead of three independently hand-typed copies. StatusUp = "up" StatusDown = "down" )
Variables ¶
var ( // ErrChecksumMismatch means a migration recorded as applied ("up") no longer matches the // sha256 checksum stored when it ran — its .up.sql file changed on disk since then. Up, // Down and Baseline all refuse to proceed past this unless force is true. ErrChecksumMismatch = errors.New("checksum mismatch: an applied migration changed on disk since it ran") // ErrNonEmptyTable means Baseline was called with force == false against a tracking table // that has a row this tool applied or rolled back itself. A table holding only an earlier // baseline is not refused: Baseline extends it (see ErrBaselineNotAhead). ErrNonEmptyTable = errors.New("baseline refuses to run on a non-empty tracking table") // ErrBaselineNotAhead means Baseline was called with force == false to extend an existing // baseline, but version is not later than the baseline's last migration. A baseline only // moves forward; shrinking it would need rows removed, which Baseline never does. ErrBaselineNotAhead = errors.New("baseline version must be later than the current baseline") // ErrUnknownVersion means the version passed to Baseline does not match any migration // discovered by Load. ErrUnknownVersion = errors.New("baseline version does not match any known migration") // ErrFilesMissing means the tracking table has a row for a migration whose .sql files are // no longer present in the migrations directory, so Down has nothing to execute for it. ErrFilesMissing = errors.New("migration is recorded as applied but its files are missing from disk") // ErrConflictingFlags means RunCLI was given --baseline together with --down or --all, // which the CLI does not support (baseline never applies or rolls back a range of files). ErrConflictingFlags = errors.New("--baseline cannot be combined with --down or --all") )
Sentinel errors returned by Migrator. Wrap these with errors.New("%w: ...", ErrX) (this package's convention — see Migrator's own error sites — is github.com/gflydev/core/errors, not the standard library's, matching the rest of the gFly framework) rather than building a new, unmatched error string for the same condition. Callers, including this package's own CLI and tests, must be able to distinguish them with errors.Is instead of parsing error text.
These stay distinct from github.com/gflydev/core/errors' own sentinels (ItemNotFound, InvalidParameter, ...): those are shaped for mapping an HTTP request to a status code, and reusing one here would make an unrelated part of a gFly app's API layer match on a condition that has nothing to do with it.
Functions ¶
func RunCLI ¶
RunCLI parses args (everything after "db:migrate" on the command line — see the package README for the full flag list), runs the requested operation against dialect's database, and returns a process exit code: 0 on success (including "nothing to do"), 1 on any error (printed to os.Stderr). A typical caller wires it in as:
case args[0] == "db:migrate":
os.Exit(migrate.RunCLI(args[1:], postgres.Dialect{}, "database/migrations"))
Types ¶
type Dialect ¶
type Dialect interface {
// Name identifies the dialect in log/error output, e.g. "postgres", "mysql".
Name() string
// Open reads DB_HOST/DB_PORT/DB_NAME/DB_USERNAME/DB_PASSWORD/DB_SSL_MODE and returns a
// ready connection pool.
Open() (*sql.DB, error)
// CreateMigrationsTableSQL returns the CREATE TABLE IF NOT EXISTS statement for the
// tracking table named table.
CreateMigrationsTableSQL(table string) string
// UpsertMigrationSQL returns a statement that inserts a new row for table, or updates the
// existing row for the same migration name, setting status='up'. Parameter order is
// (migration, batch, run_count, checksum).
UpsertMigrationSQL(table string) string
// Placeholder returns the positional-parameter placeholder for argument position argPos
// (1-based): "$1", "$2", ... for Postgres, "?" for every position in MySQL.
Placeholder(argPos int) string
// Lock acquires a database-wide advisory lock scoped to conn, blocking (up to the
// dialect's own timeout) until it is available or returning an error if it cannot be
// acquired.
Lock(ctx context.Context, conn *sql.Conn, key string) error
// Unlock releases a lock acquired by Lock, using the same conn.
Unlock(ctx context.Context, conn *sql.Conn, key string) error
// SupportsTransactionalDDL reports whether a failing statement partway through a
// migration file can be rolled back (true for Postgres, false for MySQL).
SupportsTransactionalDDL() bool
}
Dialect is the seam between this package's orchestration and a specific database.
type Migration ¶
type Migration struct {
// Version is the UTC timestamp prefix in timestampLayout's shape, e.g. "20260512_230412".
Version string
// Name is the filename's identity: the timestamp and description, without the
// .up.sql/.down.sql suffix, e.g. "20260512_230412_create_widgets_table".
Name string
// UpPath and DownPath are absolute (or working-directory-relative) paths to the two files.
UpPath, DownPath string
}
Migration is one file pair discovered on disk by Load.
func Load ¶
Load reads every *.sql file directly inside dir (not recursively), validates that each one matches "YYYYMMDD_HHMMSS_description.(up|down).sql" with a real, parseable timestamp, and that every "up" file has a matching "down" file and vice versa, and returns the migrations sorted ascending by version (equivalently, by timestamp, equivalently, by creation order).
Returns an error listing every offending filename if any file fails the naming pattern, has an invalid timestamp (e.g. month 13), or is missing its up/down counterpart — Load fails all-or-nothing rather than returning a partial, silently-incomplete set.
Load lists dir with the standard library's os.ReadDir rather than gflydev/storage: the migrations directory is part of the deployed source tree (like the framework's own view templates), not a caller-configurable storage disk, and IStorage has no directory-listing method to begin with.
func New ¶
New creates a fresh migration pair in dir, named "{current UTC timestamp}_{description}", and returns it as a Migration (with UpPath/DownPath already pointing at the new files). Both files are created from upTemplate/downTemplate: guidance and numbered section headers written only as SQL comments, so they are valid, no-op SQL until filled in.
description must already be snake_case ([a-z0-9_]+); RunCLI's --new flag validates this before calling New, and New itself re-validates so a direct library caller gets the same guarantee. Returns an error without creating anything if description is invalid, or if either target file already exists (only possible if New is called twice for the same description within the same second).
type Migrator ¶
type Migrator struct {
// contains filtered or unexported fields
}
Migrator runs the *.sql files in a directory against a database, tracking each one's state in a table it manages itself. Construct one with NewMigrator; a Migrator is cheap to create and holds no long-lived connection until a method is called.
A Migrator is not safe for concurrent use by multiple goroutines in the same process — call its methods sequentially. Concurrent *processes* (e.g. two deploys racing) are handled by the Dialect's advisory lock, which every method holds for its full duration.
func NewMigrator ¶
NewMigrator constructs a Migrator that runs the migration files in dir against db, using dialect for every dialect-specific SQL fragment and table as the tracking table's name.
table is normally DefaultTable ("migrations"); RunCLI always passes DefaultTable, since the spec fixes the table name rather than making it configurable (a health check and db:migrate must agree on one name — see the migrate package's README).
func (*Migrator) Baseline ¶
Baseline marks every migration up to and including version as already applied — batch 0, run count 1, real checksum — without running any SQL. Use it against a database that already has this schema from before it was managed by this tool.
A baseline can be extended: if the tracking table holds only an earlier baseline (every row in batch 0 and still up) and version is later than that baseline's last migration, Baseline marks just the migrations after it, leaving the existing rows untouched. This covers a baseline that was first set too early.
Returns ErrNonEmptyTable if, with force false, the tracking table has any row this tool applied or rolled back itself (batch > 0, or status down). Returns ErrBaselineNotAhead if, with force false, version is not later than the existing baseline. Returns ErrUnknownVersion if version does not match any migration Load discovers. With force == true, Baseline proceeds over any existing rows, overwriting the affected ones.
func (*Migrator) Down ¶
Down rolls back applied migrations, descending by version. With all == false (the default), it rolls back exactly one — the single most-recently-applied migration, regardless of which batch it belongs to. With all == true, it rolls back every migration in the current latest batch (every StatusUp row sharing the highest Record.Batch value). With dryRun == true, it reports which migration(s) it would roll back without executing any SQL.
Returns ErrChecksumMismatch under the same condition as Up. Returns ErrFilesMissing if a migration recorded as applied has no matching files in the migrations directory. As with Up, a failure partway through a multi-file --all rollback stops the run immediately.
func (*Migrator) Status ¶
Status reports every migration file's state: joined with its tracking-table row if it has one, or reported as pending if it doesn't. Unlike Up and Down, Status never fails on ErrChecksumMismatch — it reports the mismatch per row (StatusRow.ChecksumMatches) instead of refusing to run, since inspecting state should always be safe.
It also reports every tracking-table row that has no matching file on disk anymore (StatusRow.FileMissing) — Load itself simply never sees these names, so without this check they would silently sit in the table forever, invisible to every other command.
func (*Migrator) Up ¶
Up applies pending migrations, ascending by version. With all == false (the default), it applies exactly one — the earliest pending migration. With all == true, it applies every pending migration, all sharing one freshly incremented batch number. With dryRun == true, it reports which migration(s) it would apply without executing any SQL or writing to the tracking table.
Returns ErrChecksumMismatch if a migration currently marked applied no longer matches its on-disk checksum (see prepare). If a file fails partway through a multi-file --all run, Up stops immediately: earlier files in the run stay applied and recorded, the failing file is not recorded as applied, and no later file is attempted.
type Record ¶
type Record struct {
// Migration is the tracked migration's identity — the same value as the matching
// Migration.Name (e.g. "000024_create_widgets_table").
Migration string
// Batch is the batch number assigned the last time this migration was applied. It keeps
// its value after a rollback (Status becomes StatusDown) so it remains a historical record;
// it is only reassigned by a fresh "up".
Batch int
// Status is either StatusUp or StatusDown.
Status string
// RunCount is the number of times this migration has been successfully applied, including
// re-applications after a rollback. It is never decremented.
RunCount int
// Checksum is the sha256 (hex-encoded) of the .up.sql file's contents as of the last time
// this migration was applied. It is compared against the current file on disk before every
// operation; see ErrChecksumMismatch.
Checksum string
// MigratedAt is the time of the last successful "up"; zero-valued (Valid == false) if this
// migration has never been applied.
MigratedAt sql.NullTime
// RolledBackAt is the time of the last "down"; zero-valued (Valid == false) if this
// migration has never been rolled back.
RolledBackAt sql.NullTime
}
Record is one row of the tracking table, as last read by store.list.
type Result ¶
type Result struct {
Applied []string
}
Result reports what a single Up or Down call did, in the order it happened. An empty Applied means there was nothing pending (Up) or nothing applied to roll back (Down) — not an error.
type StatusRow ¶
type StatusRow struct {
Migration Migration
// Record is nil if Migration has never been applied.
Record *Record
// ChecksumMatches is meaningless (true) if Record is nil; see Record.Checksum.
ChecksumMatches bool
// FileMissing is true when Record is non-nil but its .sql files are no longer present in
// the migrations directory — e.g. deleted or renamed after the migration ran. Migration's
// UpPath/DownPath are empty in that case; only Version (parsed from the tracking table's
// migration name) and Name are populated. Such a row can never be rolled back by Down until
// its files are restored (see ErrFilesMissing).
FileMissing bool
}
StatusRow is one line of `db:migrate --status` output: a Migration joined with its Record, if any, plus whether its on-disk checksum still matches what was recorded.