Documentation
¶
Overview ¶
Package schemadiff is the foundation of a real schema-migration engine for Appximo — the eventual replacement for the idempotent table "converger" in pkg/migration (which only ever runs CREATE TABLE / ADD COLUMN IF NOT EXISTS and therefore loses data on rename, ignores NOT NULL, no-ops a type change, and emits no foreign keys — see docs/MIGRATION_DIAG.md).
Isolation (important) ¶
This package is built ALONGSIDE the engine and is deliberately NOT imported by it. The running engine and its converger are untouched; integration (swapping the converger for a diff-driven planner) is a much later session, once the diff engine is complete and proven. The only dependency this package takes is pgx (a database library, not engine code), so it stays a self-contained library the engine can adopt later without a circular dependency.
What lives here (this session) ¶
- A canonical, COMPARABLE model of a Postgres schema (model.go): typed structs indexed by name (map[string]) for O(1) lookup — never slices with linear scans. Column types are a canonical struct (Type), never a string, so two textual spellings of the same type ("varchar(255)" and "character varying(255)") compare equal. Foreign keys carry their ON DELETE / ON UPDATE actions (the converger has none today). Rename intent is EXPLICIT (RenamedFrom), never a heuristic.
- The type alias map + ParseType (parsetype.go): normalizes any Postgres type spelling — and the Appximo schema-JSON type vocabulary — into the canonical Type.
- The pg_catalog introspector (introspect.go): reads the REAL state of a Postgres schema into the canonical model in a fixed, small number of queries (NOT information_schema, NOT one-query-per-table).
What comes next (later sessions) ¶
The diff itself — comparing a desired canonical schema against the introspected real one to produce a typed plan of operations — then topological ordering of those operations (FK cycle breaking), safe DDL rendering, and finally the integration that replaces the converger. The foundational property the diff will rely on, prepared here, is determinism: Introspect is repeatable, so diff(Introspect(x), Introspect(x)) is empty.
Index ¶
- func DestructiveKey(op Operation) (key string, destructive bool)
- func IsDestructive(op Operation) bool
- type AddCheck
- type AddColumn
- type AddForeignKey
- type AddIndex
- type AddPrimaryKey
- type AddUnique
- type AlterColumn
- func (o AlterColumn) DefaultChanged() bool
- func (AlterColumn) Kind() OpKind
- func (o AlterColumn) NullabilityAdded() bool
- func (o AlterColumn) NullabilityDropped() bool
- func (o AlterColumn) RequiresBackfill() bool
- func (o AlterColumn) Reversible() bool
- func (o AlterColumn) Risk() RiskClass
- func (o AlterColumn) String() string
- func (o AlterColumn) TypeChanged() bool
- type BaseType
- type Check
- type Column
- type Concern
- type CreateTable
- type DropCheck
- type DropColumn
- type DropForeignKey
- type DropIndex
- type DropPrimaryKey
- type DropTable
- type DropUnique
- type EnumType
- type Executor
- type Expr
- type ForeignKey
- type Identity
- type Index
- type OpKind
- type Operation
- type Plan
- type PrimaryKey
- type Querier
- type RefAction
- type RenameColumn
- type RenameTable
- type RiskClass
- type Schema
- type Statement
- type Table
- type Type
- type UniqueConstraint
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DestructiveKey ¶
DestructiveKey returns the stable approval token for a data-losing operation and true, or ("", false) for any non-destructive operation. The token is exactly what an operator lists to approve the drop, and is unambiguous because resource and field names match ^[a-z][a-z0-9_]*$ (no dots), so a table key never collides with a column key:
DropTable → "<table>" e.g. "proyectos" DropColumn → "<table>.<column>" e.g. "empleados.telefono"
func IsDestructive ¶
IsDestructive reports whether op loses data (DropTable / DropColumn). It is the boolean companion to DestructiveKey for callers that only need the predicate.
Types ¶
type AddForeignKey ¶
type AddForeignKey struct {
Table string
FK *ForeignKey
}
func (AddForeignKey) Kind ¶
func (AddForeignKey) Kind() OpKind
func (AddForeignKey) RequiresBackfill ¶
func (AddForeignKey) RequiresBackfill() bool
func (AddForeignKey) Reversible ¶
func (AddForeignKey) Reversible() bool
func (AddForeignKey) Risk ¶
func (AddForeignKey) Risk() RiskClass
func (AddForeignKey) String ¶
func (o AddForeignKey) String() string
type AddPrimaryKey ¶
type AddPrimaryKey struct {
Table string
PK *PrimaryKey
}
func (AddPrimaryKey) Kind ¶
func (AddPrimaryKey) Kind() OpKind
func (AddPrimaryKey) RequiresBackfill ¶
func (AddPrimaryKey) RequiresBackfill() bool
func (AddPrimaryKey) Reversible ¶
func (AddPrimaryKey) Reversible() bool
func (AddPrimaryKey) Risk ¶
func (AddPrimaryKey) Risk() RiskClass
func (AddPrimaryKey) String ¶
func (o AddPrimaryKey) String() string
type AddUnique ¶
type AddUnique struct {
Table string
Unique *UniqueConstraint
}
func (AddUnique) RequiresBackfill ¶
func (AddUnique) Reversible ¶
type AlterColumn ¶
AlterColumn changes a column in place. From is the current column, To the desired one; the specific sub-changes (type / nullability / default) are derivable.
func (AlterColumn) DefaultChanged ¶
func (o AlterColumn) DefaultChanged() bool
func (AlterColumn) Kind ¶
func (AlterColumn) Kind() OpKind
func (AlterColumn) NullabilityAdded ¶
func (o AlterColumn) NullabilityAdded() bool
func (AlterColumn) NullabilityDropped ¶
func (o AlterColumn) NullabilityDropped() bool
func (AlterColumn) RequiresBackfill ¶
func (o AlterColumn) RequiresBackfill() bool
func (AlterColumn) Reversible ¶
func (o AlterColumn) Reversible() bool
func (AlterColumn) Risk ¶
func (o AlterColumn) Risk() RiskClass
func (AlterColumn) String ¶
func (o AlterColumn) String() string
func (AlterColumn) TypeChanged ¶
func (o AlterColumn) TypeChanged() bool
type BaseType ¶
type BaseType int
BaseType is the canonical family of a column type (the alias map collapses every Postgres spelling — int4/int/integer, varchar/character varying, etc. — onto one of these).
const ( BaseUnknown BaseType = iota BaseSmallint BaseInteger BaseBigint BaseNumeric BaseReal // float4 BaseDouble // float8 BaseBool BaseText BaseVarchar BaseChar BaseUUID BaseTimestamptz BaseTimestamp BaseDate BaseTime BaseTimetz BaseJSON BaseJSONB BaseBytea BaseInterval BaseInet BaseUserDefined // enum / domain / anything not in the alias map (see Type.UserName) )
type Check ¶
Check is a CHECK constraint. Expression is the predicate text from pg_get_constraintdef (minus the leading "CHECK ").
type Column ¶
type Column struct {
Name string
// RenamedFrom is the EXPLICIT previous column name when the desired schema
// declares a rename (never a heuristic). Empty for an introspected column.
RenamedFrom string
Type Type
NotNull bool
// Default is the normalized DEFAULT expression, or nil when the column has
// none. A serial/identity column carries its generation in Identity, not here
// (its nextval default is implied), so Default is nil for those.
Default *Expr
// Identity is set for an auto-generated value: legacy serial, or an explicit
// GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY column. nil for a plain column.
Identity *Identity
}
Column is one column. Type is the canonical struct (never a string), so type comparison is exact and spelling-independent.
type Concern ¶
Concern flags an operation in a plan that a human/gate should review before applying: data loss, a backfill requirement, or a data rewrite.
func Validate ¶
Validate returns the concerns in a plan (destructive, backfill-requiring, or transformational ops) with clear messages — the data a future approval gate uses to refuse or warn BEFORE applying. The executor itself renders faithfully and fails loudly on truly-inapplicable DDL; Validate is the up-front signal.
type CreateTable ¶
type CreateTable struct{ Table *Table }
func (CreateTable) Kind ¶
func (CreateTable) Kind() OpKind
func (CreateTable) RequiresBackfill ¶
func (CreateTable) RequiresBackfill() bool
func (CreateTable) Reversible ¶
func (CreateTable) Reversible() bool
func (CreateTable) Risk ¶
func (CreateTable) Risk() RiskClass
func (CreateTable) String ¶
func (o CreateTable) String() string
type DropColumn ¶
func (DropColumn) Kind ¶
func (DropColumn) Kind() OpKind
func (DropColumn) RequiresBackfill ¶
func (DropColumn) RequiresBackfill() bool
func (DropColumn) Reversible ¶
func (DropColumn) Reversible() bool
func (DropColumn) Risk ¶
func (DropColumn) Risk() RiskClass
func (DropColumn) String ¶
func (o DropColumn) String() string
type DropForeignKey ¶
type DropForeignKey struct {
Table string
FK *ForeignKey
}
func (DropForeignKey) Kind ¶
func (DropForeignKey) Kind() OpKind
func (DropForeignKey) RequiresBackfill ¶
func (DropForeignKey) RequiresBackfill() bool
func (DropForeignKey) Reversible ¶
func (DropForeignKey) Reversible() bool
func (DropForeignKey) Risk ¶
func (DropForeignKey) Risk() RiskClass
func (DropForeignKey) String ¶
func (o DropForeignKey) String() string
type DropPrimaryKey ¶
type DropPrimaryKey struct {
Table string
PK *PrimaryKey
}
func (DropPrimaryKey) Kind ¶
func (DropPrimaryKey) Kind() OpKind
func (DropPrimaryKey) RequiresBackfill ¶
func (DropPrimaryKey) RequiresBackfill() bool
func (DropPrimaryKey) Reversible ¶
func (DropPrimaryKey) Reversible() bool
func (DropPrimaryKey) Risk ¶
func (DropPrimaryKey) Risk() RiskClass
func (DropPrimaryKey) String ¶
func (o DropPrimaryKey) String() string
type DropTable ¶
type DropTable struct{ Table *Table }
func (DropTable) RequiresBackfill ¶
func (DropTable) Reversible ¶
type DropUnique ¶
type DropUnique struct {
Table string
Unique *UniqueConstraint
}
func (DropUnique) Kind ¶
func (DropUnique) Kind() OpKind
func (DropUnique) RequiresBackfill ¶
func (DropUnique) RequiresBackfill() bool
func (DropUnique) Reversible ¶
func (DropUnique) Reversible() bool
func (DropUnique) Risk ¶
func (DropUnique) Risk() RiskClass
func (DropUnique) String ¶
func (o DropUnique) String() string
type EnumType ¶
EnumType is a Postgres enum type with its labels in sort order. Appximo' own converger does not create enums (enum fields are TEXT + app-layer validation), but the model captures them so an introspected schema that has them is faithful.
type Executor ¶
type Executor struct {
Pool *pgxpool.Pool
Schema string // tenant schema name (the migration's search_path)
LockTimeout time.Duration // per-statement lock wait before failing fast (default 5s)
MaxRetries int // retries after a lock-timeout failure (default 5)
BackoffBase time.Duration // first retry backoff, doubling each attempt (default 50ms)
// OnRetry, if set, is called before each retry with the attempt number (1-based)
// and the lock-timeout error that triggered it (used by tests/observability).
OnRetry func(attempt int, err error)
}
Executor applies a Plan to one tenant schema with production-safety guarantees. The zero value is unusable; set Pool and Schema. The timing knobs default when zero.
func (*Executor) Apply ¶
Apply orders the plan (dependency-safe), renders it to safe SQL, and executes it: transactional statements run in atomic batches (with lock_timeout + retry), and CONCURRENTLY statements run between batches on their own autocommit connection.
func (*Executor) Exec ¶
Exec runs ONE DDL statement in its own transaction with this executor's search_path, lock_timeout and retry-on-lock-timeout — the same safety wrapper Apply uses for a transactional batch. It is exposed so a caller can apply a single statement with those guarantees OUTSIDE a full Plan: e.g. the migration layer's transition-tolerant foreign-key policy, which commits an `ADD CONSTRAINT … NOT VALID` and then attempts `VALIDATE CONSTRAINT` as a separate, failure-tolerant step over pre-existing data.
func (*Executor) ExecBatch ¶
ExecBatch runs several DDL statements in ONE transaction, atomically: either all of them land or none does. Same safety wrapper as Exec (search_path, lock_timeout, retry-on-lock-timeout).
It exists for operations that are only correct as a PAIR — above all replacing a foreign key whose definition changed (ENG-13): dropping the old constraint and adding the new one must not be separable, or a failure of the second half would leave the table with no foreign key at all.
type Expr ¶
type Expr struct {
Raw string
}
Expr is a (default / check) expression captured verbatim from pg_get_expr. The text Postgres returns is already canonical and deterministic; deeper semantic normalization (e.g. stripping redundant `::type` casts so a desired default matches a real one) is a diff-session concern, intentionally not done here.
type ForeignKey ¶
type ForeignKey struct {
Symbol string
Columns []string
RefTable string
RefColumns []string
OnDelete RefAction
OnUpdate RefAction
}
ForeignKey is one foreign-key constraint, including its referential actions — which the current converger never emits (docs/MIGRATION_DIAG.md §2). Columns and RefColumns are ordered and may be composite.
type Identity ¶
type Identity struct {
// Generated is one of:
// "serial" — legacy serial / bigserial (integer + an owned sequence
// default). The implied nextval default is NOT stored on the
// column (it is fully captured by this field).
// "always" — GENERATED ALWAYS AS IDENTITY
// "by_default" — GENERATED BY DEFAULT AS IDENTITY
Generated string
}
Identity describes how a column's value is auto-generated.
type Index ¶
type Index struct {
Name string
Columns []string
Unique bool
Method string // access method: btree, gin, …
Predicate string
// Opclass is the operator class applied to every column when the index is
// CREATED (e.g. "jsonb_path_ops" on a GIN index). It is DELIBERATELY not part
// of indexKey: the introspector reads an index's key columns from
// pg_index.indkey, which carries no opclass, so an opclass can never be read
// back. Excluding it from the diff key is what keeps a declared opclass from
// churning (drop+recreate) on every migration. Rendered, never compared.
Opclass string
}
Index is a STANDALONE index — one NOT backing a primary-key or unique constraint (those live in PK / Uniques). Columns are in index order; an expression key is captured as "(expression)". Predicate is the partial-index WHERE text, empty for a total index.
type OpKind ¶
type OpKind int
OpKind enumerates the operation kinds (useful for switch-free grouping/tests).
type Operation ¶
type Operation interface {
Kind() OpKind
Risk() RiskClass
// Reversible reports whether rolling the operation back does not, by itself,
// lose pre-existing data (adding a column is reversible — dropping it only
// loses data that did not exist before; dropping a column is not).
Reversible() bool
// RequiresBackfill reports whether the operation needs a data backfill (or a
// table rewrite) to be safe over a non-empty table.
RequiresBackfill() bool
String() string
}
Operation is one typed entry in a migration Plan.
type Plan ¶
type Plan struct {
Ops []Operation
}
Plan is the ordered list of operations that turn the current schema into the desired one. Operations are emitted in a coarse, dependency-respecting PHASE order (renames → drops → creates → adds, FKs last) that is applicable for the common acyclic case; the FINE topological order (inter-table dependency sort + FK cycle breaking) is a later session that re-orders this same op list.
func Diff ¶
Diff computes the typed Plan that turns current into desired, in O(N+M) over the total number of schema objects (tables, columns, constraints, indexes) — every level is matched by NAME (or by STRUCTURE for auto-named constraints/indexes) through the model's indexed maps, never an O(N·M) pairwise scan.
RENAMES ARE BY EXPLICIT INTENT (the model's RenamedFrom), never a heuristic, and are resolved FIRST so a renamed table/column is matched to its old self and emitted as a RENAME — never the converger's drop+add that strands the data (docs/MIGRATION_DIAG.md case F). A malformed rename intent (RenamedFrom naming a table/column that does not exist in current) is an error, not a silent drop+add.
func OrderPlan ¶
OrderPlan reorders a diff Plan into a dependency-safe execution order:
- CreateTable ops are ordered topologically (a referenced table before a referencing one). Because FKs are deferred to AddForeignKey at the end, this is for cleanliness; create-side cycles are already broken by that deferral.
- DropTable ops are ordered in REVERSE (a referencing table before the one it references), so a plain DROP TABLE never hits a still-referencing table.
- Drop-side cycles (mutually-referencing tables both being dropped) are broken by emitting DropForeignKey for the cycle-internal constraints BEFORE the DropTables (DetachCycles).
Every other operation keeps its phase and relative order. The result is a Plan whose operations can be executed top-to-bottom without a dependency error.
type PrimaryKey ¶
PrimaryKey is the table's primary key (always exactly one, or nil).
type Querier ¶
Querier is the minimal read surface Introspect needs. *pgxpool.Pool, *pgxpool.Conn and pgx.Tx all satisfy it. Pass a transaction (REPEATABLE READ) when a snapshot consistent across the four catalog queries matters; a pool is fine for a quiescent schema.
type RefAction ¶
type RefAction int
RefAction is a foreign-key referential action (ON DELETE / ON UPDATE).
type RenameColumn ¶
RenameColumn preserves data (ALTER … RENAME COLUMN). Table is the CURRENT (post table-rename) table name. This is the fix for the diagnostic's worst bug.
func (RenameColumn) Kind ¶
func (RenameColumn) Kind() OpKind
func (RenameColumn) RequiresBackfill ¶
func (RenameColumn) RequiresBackfill() bool
func (RenameColumn) Reversible ¶
func (RenameColumn) Reversible() bool
func (RenameColumn) Risk ¶
func (RenameColumn) Risk() RiskClass
func (RenameColumn) String ¶
func (o RenameColumn) String() string
type RenameTable ¶
type RenameTable struct{ From, To string }
RenameTable preserves data (the whole point — the converger's drop+add loses it).
func (RenameTable) Kind ¶
func (RenameTable) Kind() OpKind
func (RenameTable) RequiresBackfill ¶
func (RenameTable) RequiresBackfill() bool
func (RenameTable) Reversible ¶
func (RenameTable) Reversible() bool
func (RenameTable) Risk ¶
func (RenameTable) Risk() RiskClass
func (RenameTable) String ¶
func (o RenameTable) String() string
type RiskClass ¶
type RiskClass int
RiskClass is how dangerous an operation is to apply over live data. It is a coarse, conservative classification — the detailed risk/backfill-strategy engine is a later session; these values exist so a Plan is introspectable now.
const ( // RiskSafe — additive / non-destructive, no table rewrite, no data loss // (create table, add nullable column, add/drop index, drop a constraint). RiskSafe RiskClass = iota // RiskBackfill — needs existing data to already satisfy it (or a backfill) to // apply cleanly: add NOT NULL without default, add PK/unique/check on data. RiskBackfill // RiskDestructive — loses data (drop table, drop column). RiskDestructive // RiskTransformational — rewrites the data representation (column type change). RiskTransformational )
type Schema ¶
type Schema struct {
Name string
Tables map[string]*Table // by table name
Enums map[string]*EnumType // by enum type name
}
Schema is the canonical, comparable model of one Postgres schema (namespace). Tables and Enums are indexed by name for O(1) lookup — the diff that compares two Schemas walks these maps, never a linear list. Two Schemas built by Introspect are reflect.DeepEqual iff they describe the same database state.
func Introspect ¶
Introspect reads the REAL state of the named Postgres schema from pg_catalog (never information_schema — pg_catalog is faster and exposes the referential actions, identity kind and partial-index predicates information_schema hides) and materializes it into the canonical model.
It runs a FIXED four queries regardless of table count — columns, constraints, indexes, enums — grouping by table in memory, never one-query-per-table (no N+1). Column references in constraints and indexes are resolved structurally via attribute numbers, not by parsing DDL text.
Introspect is deterministic: introspecting the same unchanged schema twice yields reflect.DeepEqual Schemas — the property the future diff relies on so that diff(Introspect(x), Introspect(x)) is empty.
type Statement ¶
type Statement struct {
SQL string
// Concurrent marks a statement that MUST run outside a transaction
// (CREATE/DROP INDEX CONCURRENTLY) — the executor runs it on its own connection.
Concurrent bool
// CleanupSQL, if set, is run (best-effort) before a retry of a failed Concurrent
// statement — e.g. dropping the invalid index a failed CONCURRENTLY build leaves.
CleanupSQL string
// Op is the source operation (for risk/diagnostics).
Op Operation
}
Statement is one rendered SQL statement plus how it must be executed.
type Table ¶
type Table struct {
Name string
// RenamedFrom is the EXPLICIT previous name of this table when the desired
// schema declares a rename. It is never inferred by a heuristic — the diff
// uses it to emit ALTER TABLE … RENAME instead of drop+create. Empty for an
// introspected (real) table.
RenamedFrom string
Columns map[string]*Column
ColumnOrder []string
PK *PrimaryKey
FKs map[string]*ForeignKey // by constraint symbol
Uniques map[string]*UniqueConstraint // by constraint symbol
Checks map[string]*Check // by constraint symbol
Indexes map[string]*Index // by index name (standalone indexes only)
}
Table is one relation with its columns and constraints. Columns are indexed by name (O(1)); ColumnOrder preserves the on-disk column order so DDL can be emitted stably. Constraints are split by kind — PK, FKs, Uniques and Checks are declarative integrity (each altered with distinct DDL), while Indexes holds the STANDALONE indexes only (those NOT backing a PK or unique constraint).
type Type ¶
type Type struct {
Base BaseType
Size int // length for varchar(N) / char(N); 0 if unspecified
Prec int // precision for numeric(P,S); fractional-seconds precision for time/timestamp
Scale int // scale for numeric(P,S)
Array bool
// UserName holds the raw type name when Base == BaseUserDefined (a Postgres
// enum, domain, or any type outside the alias map). Empty otherwise.
UserName string
}
Type is the canonical, comparable representation of a column type. It is a struct of comparable fields, so `==` and reflect.DeepEqual both work and two textual spellings of the same type normalize to the SAME Type — the whole point of not modeling types as strings (e.g. "varchar(255)" and "character varying(255)" both → {Base: BaseVarchar, Size: 255}).
func ParseType ¶
ParseType normalizes any Postgres type spelling — the output of format_type(atttypid, atttypmod) or a hand-written type string — into the canonical Type. Unknown types (enums, domains, extensions) become {Base: BaseUserDefined, UserName: <raw>}; nothing is lost.
It handles: short and long spellings (int4 vs integer, varchar vs character varying, float8 vs double precision), the time-zone phrase appearing after an optional precision ("timestamp(3) with time zone"), length/precision/scale arguments, and array suffixes ("integer[]").
func TypeForAPIType ¶
TypeForAPIType maps one Appximo schema-JSON field type to its canonical Postgres Type, mirroring exactly what the engine's converger lays down (pkg/migration.fieldTypeToPG): string/text → text, int → integer, int64 → bigint, float64 → double precision, bool → boolean, uuid → uuid, time → timestamptz, json → text (the engine stores json as TEXT), and anything else → text. This is the canonicalizer covering the desired (schema) side, the counterpart to ParseType covering the real (introspected) side — the two halves the future diff compares. It is an independent copy of the mapping (no import of the engine), kept deliberately in lockstep with it.
type UniqueConstraint ¶
UniqueConstraint is a UNIQUE constraint (distinct from a standalone UNIQUE INDEX: a constraint is altered via ALTER TABLE ADD/DROP CONSTRAINT and can be the target of a foreign key, so the diff must treat the two differently).