migrate

package
v1.2.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 19, 2026 License: Apache-2.0 Imports: 46 Imported by: 0

README

HCL migrations

This package owns the HCL and script migration engine extracted from commons-db. See standalone database documentation for the unified CLI, read-only bundle preview and reviewed PostgreSQL/SQLite schema artifacts. commons-db retains compatible API shims.

migrate.Apply is the single filesystem-backed migration seam for PostgreSQL, SQLite and SQL Server. The connection string selects the target, while callers pass the same embedded Atlas HCL bundle and options. SQL Server bundles are SQL Server-shaped rather than portable; see SQL Server.

err := migrate.Apply(ctx, dsn, schemaFS,
    migrate.WithDir("migrations"),
    migrate.WithName("runtime"),
)

Target selection

PostgreSQL accepts postgres://, postgresql://, and keyword connection strings such as host=localhost dbname=runtime sslmode=disable.

SQLite accepts sqlite:// URLs and plain file paths ending in .db:

err := migrate.Apply(ctx, "sqlite://state/runtime.sqlite", schemaFS, migrate.WithDir("migrations"))
err := migrate.Apply(ctx, "/var/lib/example/runtime.db", schemaFS, migrate.WithDir("migrations"))

SQL Server accepts sqlserver:// URLs in the go-mssqldb form, e.g. sqlserver://user:password@host:1433?database=app. mssql:// is rejected so that one spelling reaches the migrator. ADO connection strings (;-separated pairs that name the server with server, data source, address, addr or network address, e.g. server=db;user id=app;password=secret;database=acme) and odbc: connection strings are SQL Server too, and are passed to go-mssqldb unchanged.

Existing connections

WithDB runs a SQL Server or SQLite migration on a *sql.DB the caller already holds, with the connection string left empty. The dialect comes from the handle's driver. Apply neither closes the handle nor changes its pool settings:

err := migrate.Apply(ctx, "", bundle,
	migrate.WithDB(writeDB),
	migrate.WithRoutines("routines"),
	migrate.WithName("acme/routines"))

Passing both a connection string and WithDB is an error. A PostgreSQL handle is refused, because PostgreSQL migrations scope search_path and lock_timeout on connections they open themselves.

Relative SQLite paths are resolved from the process working directory. SQLite is always file-backed; :memory: and mode=memory are rejected. The normalized connection enables foreign keys, a 5-second busy timeout, and WAL while preserving caller-supplied query parameters and unmanaged pragmas.

Unknown URI schemes written as scheme://... fail instead of being passed to PostgreSQL accidentally. Scheme-like strings without :// are treated as PostgreSQL DSNs unless they are SQL Server ADO/ODBC strings or their path ends in .db; use one of the explicit forms above rather than relying on that ambiguity. A PostgreSQL keyword DSN remains PostgreSQL even when its dbname ends in .db.

db.NewGorm uses the same DSN resolver, so migration and application connections cannot disagree about the target.

Portable canonical HCL

A cross-database bundle declares PostgreSQL-shaped Atlas HCL. PostgreSQL evaluates the declaration directly. SQLite evaluates the same declaration with the PostgreSQL evaluator, validates the resulting realm against the compatibility contract below, and then projects it to SQLite. Acceptance by the HCL parser alone does not mean an object is SQLite-compatible.

All .hcl files below WithDir are loaded recursively in lexical path order and evaluated as one realm. variable blocks are supported on both targets when callers provide values through WithVariables; the resulting schema must still fit the portable set.

Supported cross-database blocks
HCL block Supported portable form SQLite behavior
variable Atlas variable declarations referenced by schema expressions Evaluated before projection using WithVariables values
schema Exactly one schema "public" {} Projected to SQLite's main schema
table One or more tables assigned to schema.public Created with their declared portable columns and constraints
column type, null, and a literal or sql(...) default Type is mapped as documented below; default SQL is preserved verbatim
primary_key Column references only Preserved when the table is first created or safely rebuilt
unique Named constraint over column references Projected to a named unique index
index Column references, optional unique, and optional where; default or explicit B-tree method only Projected to a SQLite index; where is preserved verbatim
foreign_key Local and referenced columns plus on_update and on_delete actions Preserved when the table is first created or safely rebuilt
check Named check with expr and no dialect attributes Preserved verbatim when the table is first created or safely rebuilt

Index expressions, per-column index attributes such as sort direction or operator classes, foreign-key attributes, and table or check attributes outside the forms above are not in the portable set.

Supported portable column types

The following table lists the PostgreSQL type families accepted after HCL evaluation, not every type accepted by Atlas's PostgreSQL parser. Underscored names such as character_varying and double_precision are direct HCL aliases. Space-containing SQL spellings and the less common aliases noted below are accepted through sql("...") and normalize to the same family.

PostgreSQL HCL type SQLite declaration Compatibility notes
uuid text No SQLite UUID-format constraint is added
text, char, character, varchar, character_varying, bpchar, name; SQL spelling character varying text Length modifiers and PostgreSQL's internal name limit are not preserved
json, jsonb text plus json_valid(column) The generated check is named <table>_<column>_json
date, time, timetz, timestamp, timestamptz; SQL spellings time with time zone, time without time zone, timestamp with time zone, timestamp without time zone datetime Precision and timezone enforcement are not preserved by SQLite
int2, int4, int8, int, integer, smallint, bigint, xid, xid8 integer PostgreSQL width and transaction-ID distinctions collapse to SQLite integer affinity
bool, boolean bool SQLite gives the declared bool type NUMERIC affinity; it has no native boolean type
bytea blob Binary values remain binary
numeric, decimal numeric Precision and scale modifiers are not preserved
real, float, float4, float8, double_precision real Precision modifiers and PostgreSQL width distinctions are not preserved

Use only type modifiers whose loss is acceptable to the application, or enforce the invariant with a portable named check. Evaluated type families outside this table fail projection; they are not coerced to text.

Portable example
schema "public" {}

table "projects" {
  schema = schema.public

  column "id" {
    type = uuid
    null = false
  }
  column "payload" {
    type    = jsonb
    null    = false
    default = sql("'{}'")
  }
  column "created_at" {
    type    = timestamptz
    null    = false
    default = sql("CURRENT_TIMESTAMP")
  }

  primary_key {
    columns = [column.id]
  }
}

table "events" {
  schema = schema.public

  column "id" {
    type = uuid
    null = false
  }
  column "project_id" {
    type = uuid
    null = false
  }
  column "event_key" {
    type = text
    null = false
  }
  column "active" {
    type    = bool
    null    = false
    default = false
  }

  primary_key {
    columns = [column.id]
  }
  foreign_key "events_project_id_fkey" {
    columns     = [column.project_id]
    ref_columns = [table.projects.column.id]
    on_update   = NO_ACTION
    on_delete   = CASCADE
  }
  index "events_active_key" {
    unique  = true
    columns = [column.project_id, column.event_key]
    where   = "active"
  }
  check "events_event_key_nonempty" {
    expr = "length(event_key) > 0"
  }
}

Defaults, checks, and partial-index predicates are SQL strings, not a portable expression language. commons-db does not translate them. Every expression in a cross-database bundle must be accepted with the intended meaning by both PostgreSQL and SQLite. CURRENT_TIMESTAMP and sql("'{}'") are portable examples; PostgreSQL-only casts such as ::jsonb, functions such as now(), operators, and JSON expressions must not be assumed portable.

SQLite rejection list

SQLite rejects the following before schema mutation:

Incompatible declaration or option Reason
No table declarations The SQLite reconciler requires at least one declared table
More than one schema, or a schema other than public A portable bundle maps exactly one canonical schema to SQLite main
Realm or schema attributes and objects, including enums, domains, extensions, sequences, views, functions, and procedures There is no defined lossless projection
Table triggers Trigger SQL is dialect-specific
Column types outside the supported type table The projector never guesses an affinity
Column attributes such as generated or identity behavior PostgreSQL attributes cannot be preserved by the current projection
Table attributes other than plain named checks Storage parameters, partitioning, and other PostgreSQL metadata are not portable
Check attributes Only the check name and expression are portable
Non-B-tree index methods SQLite cannot preserve PostgreSQL access-method behavior
Expression indexes or index-part attributes Only direct column references are portable
Foreign-key attributes beyond columns, references, and actions PostgreSQL-only metadata would be lost
A user check named <table>_<json-column>_json with a different expression That name is reserved for the generated json_valid check
Any .sql migration file SQL phases are PostgreSQL-only; SQLite does not skip them silently
role or permission blocks Managed security is PostgreSQL-only
WithExclude(...) Partial inspection could hide SQLite drift
WithSchema(...) with a value other than public SQLite has no PostgreSQL schema namespace

These declarations are parser-valid PostgreSQL HCL but are not SQLite-compatible: bit strings, network and MAC types, spatial types, intervals, serial and identity types, XML, money, arrays, ranges and multiranges, full-text search types, enums, domains, and other user-defined types.

Accepted declarations with different SQLite semantics

Some declarations are accepted because their storage representation is useful on both targets, but SQLite cannot enforce every PostgreSQL semantic:

  • UUIDs are stored as text without a canonical-format check.
  • String length, numeric precision and scale, floating-point width, and time precision are normalized to SQLite affinities.
  • json and jsonb both become validated JSON text; PostgreSQL JSONB indexing and operators are not provided.
  • Date and time families become datetime; SQLite does not enforce PostgreSQL timezone or temporal-type distinctions.
  • Integer widths collapse to SQLite integer affinity.
  • Boolean declarations have NUMERIC affinity and do not restrict stored values to 0 or 1; add a portable check when that invariant matters.
  • Defaults, checks, foreign-key actions, and partial-index predicates are carried as written. Syntax or semantics that differ between engines fail at apply time or behave according to the target engine.
SQLite migration-time limits

The SQLite apply runs in one transaction. It reconciles supported indexes, constraints, column types, defaults and nullability, rebuilding populated tables when needed. Column and table drops are skipped by default; valid changes in the same migration still apply. Omitted columns remain in the effective desired table definition so a rebuild preserves their values. PostgreSQL also skips omitted columns and tables by default.

WithDrops() opts into dropping columns and tables. On SQLite it inspects all user tables and treats the supplied HCL as the complete desired table schema: user tables omitted from the bundle are removed. Without this option, inspection stays limited to the declared tables. Index and constraint removal do not require WithDrops(). A rename represented by Atlas as a column removal and addition preserves the old column by default; this API does not infer rename mappings.

The transaction disables foreign-key enforcement while applying changes, verifies that no new foreign-key violations appear before commit, and restores enforcement afterward. An incompatible value or constraint rolls the transaction back. A rebuild that would discard an unmanaged trigger is refused. SQLite's native column-addition rules still apply, and the migrator does not backfill values. Views remain outside the table bundle; callers must assess views that depend on changed columns. Rebuilds no longer require the removed WithRebuilds() option; remove it from existing callers.

PostgreSQL schema-scoped bundles

WithSchema applies one bundle inside a normalized PostgreSQL schema and leaves the default behavior on public unchanged:

err := migrate.Apply(ctx, dsn, schemaFS,
    migrate.WithName("runtime"),
    migrate.WithSchema("agent_tenant_context"),
)

The schema name must be a lowercase PostgreSQL identifier no longer than 63 bytes. The migration creates the schema and scopes Atlas inspection, HCL objects, SQL migration metadata, dependent-view handling, and managed security to it. A reusable HCL bundle still declares only schema "public" {}; migration remaps that declaration to the selected schema.

SQL files run with the selected schema as search_path. Keep objects owned by the bundle unqualified so the same bundle can target different schemas. Explicitly qualified SQL remains explicit and is not rewritten.

PostgreSQL SQL phases and security

SQL files default to the post-Atlas phase. Directives are read from the leading comment header:

-- phase: pre
-- dependsOn: 001_extensions.sql
-- runs: always
-- transaction: false
  • phase is pre or post; omitted means post.
  • dependsOn uses paths relative to the migration root. Dependencies are topologically ordered and changes rerun all transitive dependents.
  • Scripts are transactional by default; the SQL and its SHA-256 migration log commit atomically.
  • runs: always re-runs a script on every apply. It is an escape hatch for reconciliation that truly must occur at every boot, not a replacement for dependency tracking.

Views are restored automatically when Atlas reshapes a base table they depend on. Prefer small hash-gated files, such as one file per view or shared dependency, so a table reshape re-creates only affected objects and steady-state startup remains DDL-free.

Atlas-style PostgreSQL role and permission blocks may live in the same HCL files as tables. The OSS migration layer supports roles and memberships plus schema, table, column, and sequence permissions. Managed grants are reconciled exactly within the bundle's metadata scope; unrelated roles and grants remain untouched. Different scopes must not manage the same role or grantee/object pair.

Reusable PostgreSQL test templates

migrate.NewProvisioner adapts an HCL and SQL bundle to dbtest.Options.Provisioner:

handle := dbtest.ForT(t, dbtest.Options{
    Name:    "query_sessions",
    LogName: "query-sessions-test",
    Provisioner: migrate.NewProvisioner(
        querySchema,
        migrate.WithDir("migrations"),
        migrate.WithName("query"),
    ),
})

The provisioner fingerprints the migration name, directory, Atlas options, variables, and every HCL and SQL file. dbtest prepares one sealed PostgreSQL template for that fingerprint, clones an isolated database for each call, and keeps the template for later tests. Normal hash-gated SQL is already present in the clone; runs: always scripts and security reconciliation still run for every instance.

This option requires database creation. dbtest fails before invoking the provisioner when COMMONS_DB_CREATE=false because that mode targets the configured database directly.

Programmatic SQLite declarations

The migrate/sqlite subpackage remains the lower-level seam for callers that already own a *sql.DB or *sql.Tx and construct Atlas *schema.Table declarations in Go. Its ReconcileTables function uses the same additive-only policy but does not load or project HCL.

declared, err := sqlitetable.Table{Name: "events", Columns: columns, StoredAs: stored, PrimaryKey: []string{"id"}}.Declare()
if err != nil {
    return err
}
if err := sqlitemigrate.ReconcileTables(ctx, tx, sqlitemigrate.ReconcileOptions{}, declared); err != nil {
    return err
}

The record store uses this lower-level API to add newly declared columns to tables that may already contain rows. New filesystem-backed migrations should call the target-aware parent migrate.Apply instead.

SQL Server

Reviewed CSV data

Plan against a saved SQL Server connection. Review the counts and selected files in review.json, then pass the artifact hash, source root, and filters to apply. This example uses jq to read the hash and assumes the review lists zero deletes.

sqlglot migrate plan --csv-root ./data --dsn connection://example \
  --country=none,BW --env='!dev' --out ./review.json
reviewed_hash="$(jq -er .hash ./review.json)"
sqlglot migrate apply --csv-root ./data --dsn connection://example \
  --country=none,BW --env='!dev' --artifact ./review.json --hash "$reviewed_hash"

For a plan with deletes, review the count for each table and pass each exact acknowledgement reported by the preview using repeatable --ack flags. The artifact contains source digests, selected files, filters, and operation counts. Source CSVs stay in the import root. The HTTP workflow accepts uploaded files with source.kind: csv and a layerFilters map. Apply takes both the artifact and the uploaded files.

Select files and layers

Place this AS.csv.yaml at the import root to map an example AsCode.csv and its country and environment overlays.

version: 1
defaults:
  schema: dbo
  tablePattern: '^(.*/)?(?P<table>AsCode)(_[A-Z]{2})?\.csv$'
  delimiter: "~"
layers:
  country:
    values: [BW, KE]
    match: ["**/*_${value}.csv"]
    valuePattern: "[A-Z]{2}"
  env:
    values: [dev, qa, prod]
    match: ["overrides/${value}/**/*.csv"]
tables:
  dbo.AsCode:
    key: [CODENAME, CODEVALUE]

Use tables.<schema.table>.sources for exact paths or globs with different names. Use a sidecar such as lookup.csv.yaml with version: 1 and table: dbo.Lookup to route a single file. Settings inherit in order from defaults, the table, and the sidecar. Configure phase: post-rules for datasets loaded after rule deployment.

Layer filters are independent. none selects untagged files, * selects every value, and !value excludes a value. Within a layer, a comma-separated filter includes the union of positive terms and subtracts exclusions. Files pass every layer. Use --layer name=filter for additional layers. An overlay row supersedes a less specific row when its tags include every tag of that row. Incomparable records for the same key require a more specific record or a narrower filter.

Unquoted \N defaults to SQL NULL. Quoting preserves it as text. An unquoted blank follows values.blank (empty-string, sql-null, or error). A quoted empty field always means an empty string. Configure alternatives through values.nullTokens, values.nullTokenCase, and per-column settings under values.columns.

The optional $delete column deletes the exact key on true. Missing, blank, and false mean upsert. Under the default delete: explicit, unmatched database rows remain. A table using delete: snapshot must list every untagged source in snapshotFiles and retain a nonempty effective dataset after filtering.

Bulk staging and replay

Plan and apply use SQL Server bulk copy to stream input into transaction-local staging tables. SQL Server converts the staged values to the destination column types before comparing rows. Apply revalidates the reviewed inputs and target state under its migration lock. If the source or target state changed, generate and review a fresh plan. After a successful apply, a fresh plan shows zero writes when the data matches.

In a three-table SQL Server clone with 17,900 selected CSV rows, median native apply time across three runs fell from 6.07 to 1.97 seconds when rows already matched, and from 12.73 to 2.62 seconds for 487 inserts and 475 updates. The final row hashes matched across both staging implementations. These timings cover apply with an existing reviewed artifact.

Schema bundles

sqlserver:// targets reconcile through the Atlas SQL Server driver in atlas/sqlserver, which commons-db ships because Atlas OSS has none. A bundle declares tables in SQL Server-shaped HCL and stored procedures and functions as plain .sql files:

The driver delegates its differ, SQL Server types and migration planner to github.com/flanksource/sqlglot-go/migrate/atlas/sqlserver. Commons-db owns catalog inspection, HCL loading, connections, locks, transactions and applying the resulting SQL. Consumers that need a database-independent typed plan can use sqlglot's migrate/sqlserver package directly; this does not expand the HCL or inspection support described below.

//go:embed db
var bundle embed.FS

err := migrate.Apply(ctx, "sqlserver://app:secret@db:1433?database=acme", bundle,
    migrate.WithDir("db"),
    migrate.WithRoutines("db/routines"),
    migrate.WithName("acme/orders"),
)
Routines come from .sql files

WithRoutines(dir) declares one routine per .sql file under dir. Each file holds exactly one CREATE [OR ALTER] PROCEDURE|FUNCTION batch, optionally followed by GO, and is named after the routine it creates (acme_count_orders.sql creates acme_count_orders). The file is the routine's only source: there are no procedure or function HCL blocks. It runs as written in SSMS or sqlcmd, and migrate runs the same text, with CREATE rewritten to CREATE OR ALTER and every other byte kept, as its own batch.

A routine is compared with the live sys.sql_modules definition through sqlglot's canonical T-SQL: comments, whitespace, keyword case, [brackets] and OR ALTER do not count as changes, while a literal or any other token does and produces one CREATE OR ALTER. A definition, declared or live, that sqlglot cannot parse is an error naming the routine, never a guessed change. Routine files are never hash-gated SQL scripts.

A routine that is removed from the bundle is left in the database; migrate never drops procedures or functions.

Tables from HCL

.hcl files under WithDir are optional when the bundle has routines. They declare exactly schema "dbo" {} and its tables:

Block Supported form
column type, null, default (a number, a string, or sql("getdate()")), and an identity { seed, increment } block
primary_key columns, optional name, clustered = false for a nonclustered key (clustered by default)
index columns (with desc parts), unique, clustered = true (nonclustered by default), include = [column.x], where = "..."
foreign_key columns, ref_columns, on_delete, on_update
check expr

Types: bit, tinyint, smallint, int, bigint, decimal(p,s), numeric(p,s), money, smallmoney, float(n) (stored as real up to 24 bits), real, char(n), varchar(n), nchar(n), nvarchar(n), text, ntext, binary(n), varbinary(n), image, date, time(p), datetime, datetime2(p), smalldatetime, datetimeoffset(p), uniqueidentifier, xml, sql_variant and rowversion (timestamp). A (max) size is written with sql(): type = sql("nvarchar(max)"). Defaults are named DF_<table>_<column>; changing one drops the live default constraint, whatever its name, and adds the new one. String defaults on nchar/nvarchar/ntext columns are written as N'...'.

Scoped inspection

An OIPA-sized database holds thousands of objects, so migrate inspects only the tables and routines the bundle names. Undeclared objects are never read, and so never changed. Inside declared tables, columns, keys, foreign keys and checks are reconciled to the declaration, but indexes the bundle does not declare are kept, so tuning indexes added by a DBA survive. An apply that finds nothing to change runs no DDL. Every apply holds a session sp_getapplock named after WithName (one-minute wait), so concurrent applies of one bundle serialize.

Scripts are sqlcmd files

Every other .sql file under WithDir (outside the routine directory) is a hash-gated script with the same -- phase:, -- dependsOn:, -- runs: always and -- transaction: headers as on PostgreSQL. It is plain T-SQL that runs unchanged with sqlcmd -i or in SSMS: migrate splits it at GO on its own line (never inside a string or comment) and sends each batch in turn. sqlcmd commands (:setvar, :r, :on error, !!) and $(var) scripting variables, which sqlcmd substitutes even inside strings, are refused, so a file never means something different here than in sqlcmd. pre scripts run before the table and routine reconcile, post scripts after, all under the bundle's sp_getapplock. A transactional script runs every batch and records its SHA-256 in dbo.sqlglot_migrations (created on first use) in one transaction under SET LOCK_TIMEOUT 15000; deadlocks (1205) and lock timeouts (1222) are retried. PendingSQLServer reports what an Apply would do without executing anything: the reconcile statements, and each script it would run with why (new, changed, dependency or always).

Compatibility level 100

Generated DDL and the catalog queries avoid syntax newer than compatibility level 100 (no STRING_AGG, OPENJSON or DROP ... IF EXISTS). CREATE OR ALTER depends on the engine version (2016 SP1+), not the database's compatibility level.

Refused on SQL Server

These fail with an error instead of being ignored:

  • sqlcmd commands in scripts (see above);
  • WithSchema other than dbo and WithDrops;
  • role and permission blocks;
  • view, trigger, sequence, function and procedure HCL blocks, computed columns, user-defined and alias types (including sysname), and any unknown column, index or table attribute;
  • declared tables that the database has partitioned or carries triggers on, and IDENTITY changes on an existing column.

Documentation

Overview

Package migrate applies declarative Atlas HCL schemas to PostgreSQL, SQLite and SQL Server databases, and on SQL Server also the stored procedures and functions declared as .sql files (WithRoutines).

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Apply

func Apply(ctx context.Context, connection string, schemaFS fs.FS, opts ...Option) (returnErr error)

Apply resolves the database target from connection, or from the WithDB handle, and reconciles HCL schemas. PostgreSQL supports the complete HCL, SQL, security, and destructive-change lifecycle. SQLite projects portable PostgreSQL-shaped HCL to tables, columns, and indexes, rebuilding tables when needed and skipping column and table drops unless WithDrops is enabled. SQL Server (sqlserver:// URLs, ADO and odbc: strings) reconciles SQL Server HCL tables and the stored procedures and functions declared with WithRoutines, inspecting only the objects the bundle names; it refuses SQL scripts, security blocks, drops, rebuilds and schemas other than dbo.

func ConnectionForSchema

func ConnectionForSchema(connection, schemaName string) (string, error)

ConnectionForSchema returns a PostgreSQL URL whose connections select schemaName.

func ValidateSchemaName

func ValidateSchemaName(name string) error

ValidateSchemaName validates an application-normalized PostgreSQL schema identifier.

Types

type Option

type Option func(*options)

Option configures an HCL migration.

func WithDB

func WithDB(db *sql.DB) Option

WithDB runs the migration on an already open SQL Server or SQLite handle, with Apply's connection string left empty; the dialect comes from the handle's driver. Apply never closes db and never changes its pool settings. PostgreSQL handles are refused: those migrations scope search_path and lock_timeout on connections they open themselves.

func WithDir

func WithDir(dir string) Option

WithDir selects the root containing colocated HCL and SQL migrations.

func WithDrops

func WithDrops() Option

WithDrops allows HCL files to remove columns, tables and types. Drops are suppressed by default so a partial schema bundle cannot delete objects owned by consumers.

Tables and types share one switch because suppressing only tables is worse than suppressing neither: the surviving table keeps a column typed by the enum the diff still wants to drop, and PostgreSQL rejects the DROP TYPE.

func WithExclude

func WithExclude(patterns ...string) Option

WithExclude excludes database objects from Atlas inspection. Values use Atlas's schema inspection patterns (for example "table.column").

func WithName

func WithName(name string) Option

WithName sets the metadata scope used for SQL hashes and managed security. It should be stable and unique for each migration bundle sharing a database.

func WithRebuilds

func WithRebuilds() Option

WithRebuilds retains source compatibility. Supported SQLite rebuilds are now automatic.

func WithRoutines

func WithRoutines(dir string) Option

WithRoutines declares stored procedures and functions from the .sql files under dir (relative to the schema filesystem root). Each file holds exactly one CREATE [OR ALTER] PROCEDURE|FUNCTION batch, optionally followed by GO, and is named after the routine it creates (mc_ClonePlan.sql creates mc_ClonePlan). The files are the routines' single source: they run as written, so the same file works in SSMS or sqlcmd, and they are never treated as hash-gated SQL scripts. Only SQL Server supports routines.

func WithSchema

func WithSchema(name string) Option

WithSchema selects the PostgreSQL schema used for the complete migration lifecycle. SQLite accepts only the default public schema declared by portable HCL bundles, and SQL Server only dbo.

func WithScriptPolicy added in v1.2.0

func WithScriptPolicy(policy ScriptPolicy) Option

WithScriptPolicy enables SQL Server dependency tracking for named script paths.

func WithVariables

func WithVariables(input map[string]cty.Value) Option

WithVariables supplies values for HCL variable blocks and security expressions.

type PendingScript added in v1.1.2

type PendingScript struct {
	Path    string   `json:"path"`
	Phase   string   `json:"phase"`
	Reason  string   `json:"reason"` // new | changed | dependency | always
	Reasons []string `json:"reasons,omitempty"`
	Blocked []string `json:"blocked,omitempty"`
	Hash    string   `json:"hash,omitempty"`
}

PendingScript is one script Apply would run, and why: new or changed since it last ran, dependency (a script it depends on changed), or always (a `runs: always` script, which runs on every Apply and so is never a change by itself).

type Preview

type Preview struct {
	Artifact          *review.Artifact      `json:"artifact,omitempty"`
	SQLServerArtifact *native.Artifact      `json:"sqlServerArtifact,omitempty"`
	SQLServerPlan     *native.Plan          `json:"sqlServerPlan,omitempty"`
	Scripts           []ScriptAnalysis      `json:"scripts,omitempty"`
	ScriptPlan        *scriptserver.Preview `json:"scriptPlan,omitempty"`
	Warnings          []string              `json:"warnings,omitempty"`
	Error             string                `json:"error,omitempty"`
}

func PreviewBundle

func PreviewBundle(ctx context.Context, db *sql.DB, dialect string, files fs.FS, opts ...Option) (result Preview, returnErr error)

PreviewBundle inspects and plans without executing scripts or creating metadata. Script and security bundles yield advisory schema plans until phased artifacts are supported.

type SQLServerPending added in v1.1.2

type SQLServerPending struct {
	Statements []string         `json:"statements,omitempty"`
	Scripts    []PendingScript  `json:"scripts,omitempty"`
	ScriptPlan *backend.Preview `json:"scriptPlan,omitempty"`
}

SQLServerPending is what Apply would do to a SQL Server database: the statements that reconcile the declared tables and routines, and the scripts it would run. It inspects only the objects the bundle names.

func PendingSQLServer added in v1.1.2

func PendingSQLServer(ctx context.Context, db *sql.DB, schemaFS fs.FS, opts ...Option) (SQLServerPending, error)

PendingSQLServer plans Apply on db without executing or recording anything.

func (SQLServerPending) Changed added in v1.1.2

func (p SQLServerPending) Changed() bool

Changed reports whether Apply would change anything beyond running its `runs: always` scripts.

type SchemaProvisioner

type SchemaProvisioner struct {
	// contains filtered or unexported fields
}

SchemaProvisioner prepares reusable dbtest templates and reconciles each cloned database using the same migration behavior as Apply.

func NewProvisioner

func NewProvisioner(schemaFS fs.FS, opts ...Option) *SchemaProvisioner

NewProvisioner adapts an HCL migration bundle to dbtest.Provisioner without coupling the migrate package to dbtest. Options are frozen at construction.

func (*SchemaProvisioner) Fingerprint

func (p *SchemaProvisioner) Fingerprint(ctx context.Context) (string, error)

Fingerprint returns a deterministic identity for all inputs that affect the database produced by this provisioner.

func (*SchemaProvisioner) PrepareInstance

func (p *SchemaProvisioner) PrepareInstance(ctx context.Context, connection string) error

PrepareInstance reconciles instance-specific and always-run migrations after cloning a prepared template.

func (*SchemaProvisioner) PrepareTemplate

func (p *SchemaProvisioner) PrepareTemplate(ctx context.Context, connection string) error

PrepareTemplate applies the full migration bundle to a new template.

type ScriptAnalysis

type ScriptAnalysis struct {
	Path        string                    `json:"path,omitempty"`
	Phase       string                    `json:"phase,omitempty"`
	Kind        string                    `json:"kind"`
	Complete    bool                      `json:"complete"`
	Diagnostics []sqlglot.ParseDiagnostic `json:"diagnostics,omitempty"`
	Error       string                    `json:"error,omitempty"`
}

func AnalyzeScript

func AnalyzeScript(text, dialect string) ScriptAnalysis

type ScriptPolicy added in v1.2.0

type ScriptPolicy struct {
	Functions                   []backend.Function
	Dependencies                map[string][]scripts.Dependency
	ExplicitDependencies        []string
	AcknowledgeNonTransactional bool
}

Directories

Path Synopsis
Package review plans and applies hash-bound, transactional schema migrations.
Package review plans and applies hash-bound, transactional schema migrations.
Package scripts owns migration source identity, dependencies and consumed fingerprints.
Package scripts owns migration source identity, dependencies and consumed fingerprints.
sqlserver
Package sqlserver observes and executes dependency-aware scripts on SQL Server.
Package sqlserver observes and executes dependency-aware scripts on SQL Server.
Package sqlite reconciles SQLite tables with programmatic Atlas declarations.
Package sqlite reconciles SQLite tables with programmatic Atlas declarations.
Package sqlserver executes immutable sqlglot artifacts against SQL Server.
Package sqlserver executes immutable sqlglot artifacts against SQL Server.
Package viewdeps finds, drops, and restores the views and materialized views that transitively depend on a set of tables, so DDL PostgreSQL refuses while a dependent view exists — DROP TABLE, DROP COLUMN, ALTER COLUMN TYPE — can proceed.
Package viewdeps finds, drops, and restores the views and materialized views that transitively depend on a set of tables, so DDL PostgreSQL refuses while a dependent view exists — DROP TABLE, DROP COLUMN, ALTER COLUMN TYPE — can proceed.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL