Documentation
¶
Overview ¶
Package orchestrator drives `golang-migrate/migrate/v4` Up against one or more registered Sources behind a single advisory-lock-guarded startup path. Each Source owns its own Postgres schema and may supply a `LegacyRun` callback that runs once between the first migration (`mg.Steps(1)`) and the rest (`mg.Up()`).
The orchestrator is the merge gate's contract: server startup (`internal/registry/database/postgres.go`) and the CLI's `arctl db migrate up` both invoke `RunUp` so the same legacy-bridging logic fires on every up path.
Schema is a resolved value (extension authors) ¶
A source's schema is a database.Schema, not a string: it is resolved (validated + its quoted identifier precomputed) once when the Source is built, and that value is threaded everywhere the schema is needed — nothing re-derives it per operation. An extension registering its own source builds the value once at init:
orchestrator.Source{
Name: "ext",
Schema: database.MustNewSchema("agentregistry_ext"),
Files: extMigrations,
Dir: "migrations",
LegacyRun: ext.RunBridge, // func(ctx, *sql.DB, database.Schema) error
}
Use database.NewSchema(name) (returns an error) when the name is operator/runtime input; database.MustNewSchema(name) (panics) for a const or an init-time registration value. A LegacyRun callback receives the source's Schema so it can address its destination tables via schema.Qualify("table") without re-resolving it.
For a query that spans schemas (e.g. an extension joining its table against an OSS table), build a database.SchemaRegistry at the composition root, register each source's schema, and inject it; then reg.Get("oss").Qualify("agents") yields a safe, schema-qualified reference. App queries that stay within one schema don't need the registry — they go through a v1alpha1store.Store, which already holds its resolved schema and qualifies for them.
LegacyRun ordering for Source authors ¶
Sources that supply a `LegacyRun` callback can rely on the following invariant: `LegacyRun` is invoked after `mg.Steps(1)` has applied the source's first migration and before `mg.Up()` runs the rest. So at LegacyRun time, the schema reflects migration 001's tables and indexes but not 002+. Sources whose `LegacyRun` copies into specific tables must either keep those tables in 001 or accept that the destination shape will not have advanced past 001.
`LegacyRun` is gated on `public.schema_migrations` (the prior custom migrator's bookkeeping table) existing. The data-copy must be idempotent under re-invocation (the OSS source uses `INSERT ... ON CONFLICT DO NOTHING`) because after a successful run the orchestrator renames `public.schema_migrations` aside, which closes the gate naturally on subsequent runs. Gating on `public.schema_migrations` alone (rather than also on the source's own `schema_migrations` row count) is what makes the bridge survive a partial run that committed `Steps(1)` and then aborted before `LegacyRun` fired.
After every source's per-source sequence completes, if at least one source's `LegacyRun` actually fired this run, the orchestrator renames `public.schema_migrations` to `public.schema_migrations_v0_legacy`. The rename is gated on the bridge so an external user of the `public.schema_migrations` table (e.g. an unrelated golang-migrate setup against the same database) is never silently touched.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func RunUp ¶
RunUp serializes the entire migrate sequence behind a single global advisory lock, then for each Source opens a dedicated single-connection handle, acquires a per-source `pg_advisory_lock`, applies `Steps(1)`, invokes the legacy bridge if applicable, then `Up()`. After every Source succeeds, if at least one source's `LegacyRun` fired this run, `public.schema_migrations` (the prior custom migrator's bookkeeping table) is renamed to `public.schema_migrations_v0_legacy`. The bridge-gate keeps RunUp from touching an unrelated owner of that table.
Concurrency ¶
The global lock means only one pod is ever inside the source loop or the failure-restore path; losers block, then re-probe and no-op once they acquire it. This is what makes the cross-source restore safe — otherwise one pod restoring a source could clobber a version another pod legitimately advanced. The per-source locks (runSource / WithSourceLock) are retained to serialize RunUp against the CLI's `down`/`goto`/`force`; RunUp always takes the global lock first and the CLI never takes it, so there is no lock-ordering cycle.
Cross-source atomicity ¶
If any source fails, every source this run touched — the failing one and all earlier-applied ones — is restored to its pre-run version, so the database returns to the prior release's version combo instead of a cross-track state no release ships. This is a compensating rollback, not a transaction (each migrator owns its own connection), so it depends on two invariants: every incremental migration ships a real, idempotent `.down.sql`, and each migration file applies as one implicit transaction (so a failed file's DDL rolls back atomically, leaving only a dirty marker — never half-applied schema). The lint test enforces the latter for the common cases; see pkg/registry/v1alpha1store/migrations/README.md.
Sources first-installed this run are reset to NilVersion if they carry a dirty marker (their idempotent floor re-applies on a re-run) and otherwise left at their freshly-applied version. A source found already dirty at entry is not migrated: RunUp restores the prior sources and surfaces it for operator `force`, since its true state is ambiguous.
func WithSourceLock ¶
WithSourceLock opens a dedicated single-connection database handle, acquires the orchestrator's per-source `pg_advisory_lock`, runs fn, then releases the lock and closes the connection. Exposed so CLI per-source operations (down / goto / force) can serialize against orchestrator-driven `up` and against each other — without it, two CLI invocations would only share go-migrate's internal lock on schema_migrations, leaving the LegacyRun window unguarded.
fn receives the underlying *sql.DB so callers that need to issue auxiliary queries (e.g. probing schema state) can share the locked session.
Types ¶
type Source ¶
type Source struct {
// Name is the operator-visible label (shown in logs, accepted by the
// CLI's `--source` flag) and the input to the advisory-lock key hash.
// It is never interpolated into SQL, so it only needs to be a stable,
// unique-per-source token; keep it short and lowercase for readability.
Name string
// Schema is the Postgres schema this source's tables live in,
// resolved once (its quoted identifier is precomputed). Build it with
// database.NewSchema(name) — or database.MustNewSchema(name) at
// init/registration where the name is a known-valid const. The
// orchestrator threads this value everywhere it needs the schema
// (migrator construction, the schema_migrations probes); nothing
// re-derives it from a string. `golang-migrate`'s pgx/v5 driver is
// configured with Schema.Name(); the source's `schema_migrations`
// table is created in that schema.
Schema database.Schema
// AdditionalSchemas are appended to the migration connection's search_path.
// Use this when an extension migration calls a shared function owned by an
// earlier source. The source's own Schema always remains first.
AdditionalSchemas []database.Schema
// Files is the embedded filesystem holding NNN_name.up.sql /
// NNN_name.down.sql pairs.
Files fs.FS
// Dir is the directory inside Files containing the migration
// pairs.
Dir string
// LegacyRun, when non-nil, is invoked between `mg.Steps(1)` and
// `mg.Up()` whenever `public.schema_migrations` exists. It receives
// the source's resolved Schema so the bridge can address the
// destination tables (e.g. schema.Qualify("agents")) without
// re-resolving it. The callback must be idempotent under
// re-invocation: on a successful run the orchestrator renames
// `public.schema_migrations` aside, which closes the gate naturally
// on subsequent runs, but the gate also fires on the recovery path
// after a partial run that committed `Steps(1)` and aborted before
// this callback ran. Fresh installs (no `public.schema_migrations`
// ever) skip cleanly.
LegacyRun func(ctx context.Context, db *sql.DB, schema database.Schema) error
}
Source describes one set of migrations to be applied as part of the orchestrator's startup sequence.
Source Files