db

package
v1.27.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 11, 2026 License: Apache-2.0 Imports: 23 Imported by: 0

Documentation

Overview

Package db provides database connectivity for the Nucleus framework. The runtime is implemented directly on top of database/sql.

Package db: driver error classification.

Code that writes to the database needs to tell one failure apart from another, and the only portable signal database/sql offers is the error value itself. The tempting shortcut — matching substrings of the driver's message ("duplicate key", "unique constraint", …) — is wrong in a way that is invisible in development and fatal in production: PostgreSQL, MySQL, Oracle and SQL Server all TRANSLATE their messages when the server runs in another language. A PostgreSQL server started with lc_messages='es_ES.utf8' answers a rejected insert with 

llave duplicada viola restricción de unicidad «users_email_key»

in which no English substring appears. Every such check silently returns false, and the branch it guards becomes dead code on that deployment.

The predicates here match on the CODE the driver reports, through errors.As, so they are unaffected by the server's locale and by wording changes between driver releases. They walk the Unwrap chain, so an error the caller has wrapped still classifies.

Index

Constants

View Source 
const (
	// DriftKindMissingUpFile is reported when a migration ID is recorded as
	// applied in nucleus_schema_migrations but the corresponding .up.sql
	// file is absent from the migrations directory. Typical cause: someone
	// deleted a migration after applying it — the database remembers the
	// row, but no reproducible script exists to recreate that state.
	DriftKindMissingUpFile = "missing_up_file"

	// DriftKindChecksumMismatch is reported when the SHA-256 of the
	// `.up.sql` currently on disk does not match the checksum recorded in
	// nucleus_schema_migration_checksums when the migration was originally
	// applied. Typical cause: someone edited a migration file in place
	// after it had already been applied — the database state reflects the
	// pre-edit script, but the on-disk content claims something else.
	// Migrations applied before checksum tracking was introduced have no
	// recorded checksum and are not reported as drift on that basis.
	DriftKindChecksumMismatch = "checksum_mismatch"
)

Drift kinds reported by Migrator.Drift.

View Source 
const (
	// DriftKindSchemaMissingTable is reported when the caller declares
	// a table that does not exist in the live database. Typical
	// cause: a migration was forgotten or the deployment never ran
	// AutoMigrate after a model was added.
	DriftKindSchemaMissingTable = "schema_missing_table"

	// DriftKindSchemaMissingColumn is reported when the caller's
	// expected schema includes a column the live table is missing.
	// Typical cause: a column was added to the model after the initial
	// migration; AutoMigrate does not ALTER existing tables, so the
	// new column never made it into the DB.
	DriftKindSchemaMissingColumn = "schema_missing_column"

	// DriftKindSchemaExtraColumn is reported when the live table has a
	// column the caller's expected schema does not declare. Typical
	// cause: a column was added by an ad-hoc DDL (psql, a sidecar
	// migration, a manual fix) but never reflected back into the model
	// definition. Not always a bug — but always worth surfacing.
	DriftKindSchemaExtraColumn = "schema_extra_column"

	// DriftKindSchemaColumnNullability is reported when a column
	// exists on both sides but the expected schema says NOT NULL and
	// the live table says nullable (or vice versa). The detected
	// polarity is recorded in the entry's Expected/Actual fields so
	// the operator can read the row and know which side is wrong.
	DriftKindSchemaColumnNullability = "schema_column_nullability"
)

Drift kinds reported by Migrator.SchemaDrift. These complement the file-level kinds in migrate.go (DriftKindMissingUpFile, DriftKindChecksumMismatch).

Variables

View Source 
var (
	ErrUnsupportedEngine = errors.New("unsupported database engine")
	ErrSQLRequired       = errors.New("sql runtime is required")
	ErrAutoMigrate       = errors.New("automigrate is not supported; use SQL migrations")
)
View Source 
var ErrSchemaDriftUnsupported = errors.New("db.Migrator.SchemaDrift: schema-level introspection is not implemented for this database engine")

ErrSchemaDriftUnsupported is returned by Migrator.SchemaDrift when the underlying database dialect does not have an introspection implementation. Today this covers any engine outside the supported set (SQLite, PostgreSQL, MySQL, MSSQL, Oracle). Callers can `errors.Is` against this sentinel to distinguish "no drift detected" from "drift could not be checked on this engine".

Functions

func ExecScript 

func ExecScript(execer sqlExecer, system, script string) error

ExecScript executes a (possibly multi-statement) migration script on execer, splitting it into individually-executable units per the SQL dialect.

Oracle (`oracle`): the go-ora driver executes exactly one statement / PL/SQL block per Exec and raises ORA-06550 on a SQL*Plus `/` terminator. Framework scaffolds (and idiomatic hand-written Oracle migrations) therefore separate PL/SQL blocks with a `/` on its own line. ExecScript splits on those `/` lines, drops the marker, and Execs each block in order — so the `/` is a split directive that is never sent to the driver. (Oracle DDL auto-commits, so per-block execution is the only correct path regardless of any surrounding transaction.)

MySQL (`mysql`) and SQLite (`sqlite`): the go-sql-driver/mysql driver (unless the DSN sets multiStatements=true) and the pure-Go modernc SQLite driver execute exactly one statement per Exec and reject a multi-statement batch (MySQL fails the second statement with Error 1064). A scaffold for a model with a secondary index emits CREATE TABLE plus one CREATE INDEX per index, so ExecScript splits the script into its `;`-terminated statements and Execs each in order. The split is quote- and comment-aware (see splitSQLStatements) so a `;` inside a string literal or comment is not a false boundary.

PostgreSQL (`postgresql`) and SQL Server (`mssql`): the script is sent as-is in a single Exec — the pgx/lib-pq and go-mssqldb drivers accept multiple `;`-separated statements in one round trip, and this preserves the established behaviour exactly.

execer is satisfied by `*sql.DB` and `*sql.Tx` (the interface is unexported because those are the only intended arguments). Oracle splitting is line-oriented on the SQL*Plus convention: a `/` alone on its own line inside a multi-line PL/SQL string literal would be mistaken for a separator — the same constraint SQL*Plus itself has. Framework scaffolds never emit such a line.

func IsUniqueViolation 

func IsUniqueViolation(err error) bool

IsUniqueViolation reports whether err was caused by a unique or primary-key constraint. It is the signal to treat a rejected insert as "that value is already taken" rather than as an internal error: 

if err := insertUser(ctx, sqlDB, u); err != nil {
    if db.IsUniqueViolation(err) {
        http.Error(w, "email already registered", http.StatusConflict)
        return
    }
    return err
}

It does NOT report foreign-key, not-null or check violations: a caller acting on "unique" wants to point at one field, and widening the predicate later would silently change what that branch catches.

Coverage follows the driver modules linked into the binary. PostgreSQL is the exception and is classified here: any PostgreSQL driver exposes the SQLSTATE through a `SQLState() string` method, so the check costs no import and works for pgx and lib/pq alike. Every other engine is classified by the module that registers its driver, because recognising its error requires its error TYPE.

An engine whose driver is not linked in cannot produce an error to classify, so a build that omits a driver module loses nothing. The case that does lose something is a caller who registers a driver directly — importing github.com/go-sql-driver/mysql itself instead of the nucleus module — and never registers a classifier: this returns false for errors it has no way to recognise. Config.Open says so at startup rather than letting it surface as a wrong answer under load.

func SystemFromURL 

func SystemFromURL(raw string) string

SystemFromURL resolves a connection URL to the SQL system name a *DB would report from System() — "postgresql", "mysql", "sqlite", "mssql", "oracle", or "unknown" — without opening a connection. The CLI scaffolders use it to pick the migration dialect for the configured database (QCD-CLI-4).

Types

type AppliedMigration 

type AppliedMigration struct {
	ID        string    `json:"id"`
	Namespace string    `json:"namespace,omitempty"`
	AppliedAt time.Time `json:"applied_at"`
}

AppliedMigration is one row of the migration ledger as stored: ID is the storage key (`<module>/<id>` for a module-scoped Migrator's rows, the bare file ID for the host's own), Namespace the module name that key carries, empty for unscoped rows.

type Config 

type Config struct {
	Engine              Engine
	DatabaseURL         string
	DatabaseMaxOpen     int
	DatabaseMaxIdle     int
	DatabaseMaxLifetime time.Duration

	// StatementObserver, when non-nil, enables driver-level SQL
	// instrumentation: the database/sql driver is wrapped so every direct
	// db.QueryContext/ExecContext is reported to the observer AFTER the call
	// returns. Statements issued through model.CRUD are already observed at
	// the model layer and are suppressed here (see StatementObserver's godoc
	// and observe.CtxWithModelObserved). Nil (the default) leaves the stock
	// database/sql path untouched — no wrapping, no hot-path cost.
	StatementObserver StatementObserver
}

Config contains the database-specific settings needed to open a connection. It intentionally avoids depending on app.Config to keep packages decoupled.

type DB 

type DB struct {
	// contains filtered or unexported fields
}

DB wraps the SQL runtime.

func New 

func New(cfg Config, logger *slog.Logger) (*DB, error)

New opens a database connection based on config. Supported URL schemes: - postgres://, postgresql:// - mysql:// - sqlite:// (or .db/.sqlite path) - sqlserver://, mssql:// - oracle:// A plain file path ending in .db or .sqlite is treated as SQLite. Default engine is EngineSQL.

func (*DB) AutoMigrate 

func (d *DB) AutoMigrate(models ...interface{}) error

AutoMigrate is intentionally unsupported at the db.DB layer. Use explicit SQL migration files through Migrator, or call the application-level App.AutoMigrate (pkg/app), which builds a dialect-aware scaffold for SQLite, PostgreSQL, MySQL, MSSQL, and Oracle and applies it through the same Migrator pipeline.

func (*DB) Close 

func (d *DB) Close() error

Close closes the underlying sql.DB connection.

func (*DB) Engine 

func (d *DB) Engine() Engine

Engine returns the selected runtime engine.

func (*DB) Health 

func (d *DB) Health(ctx context.Context) error

Health verifies the database is reachable.

func (*DB) SqlDB 

func (d *DB) SqlDB() (*sql.DB, error)

SqlDB returns the underlying *sql.DB.

func (*DB) System 

func (d *DB) System() string

System returns the underlying SQL system name as resolved from the connection URL: one of "postgresql", "mysql", "sqlite", "mssql", "oracle", or "unknown". Callers can dispatch dialect-specific code off this value — `app.AutoMigrate` uses it to pick a migration scaffold builder.

func (*DB) Tx 

func (d *DB) Tx(ctx context.Context, fn func(tx *sql.Tx) error) error

Tx runs fn inside a SQL transaction.

type DriftEntry 

type DriftEntry struct {
	ID        string    `json:"id"`
	Kind      string    `json:"kind"`
	AppliedAt time.Time `json:"applied_at"`
	// ExpectedChecksum is the SHA-256 hex of the .up.sql file as it
	// existed when the migration was originally applied. Populated only
	// for `checksum_mismatch` drift entries.
	ExpectedChecksum string `json:"expected_checksum,omitempty"`
	// ActualChecksum is the SHA-256 hex of the .up.sql file currently on
	// disk. Populated only for `checksum_mismatch` drift entries.
	ActualChecksum string `json:"actual_checksum,omitempty"`
}

DriftEntry describes a divergence between the migrations recorded as applied in the database and the migration files on disk.

type Engine 

type Engine string

Engine identifies the SQL runtime used by DB. 

const (
	// EngineSQL is the native database/sql runtime.
	EngineSQL Engine = "sql"
)

type ExpectedColumn 

type ExpectedColumn struct {
	Name     string
	Nullable bool
}

ExpectedColumn is one row of an ExpectedTable. SchemaDrift compares existence and nullability today; column types are explicitly out of scope because cross-dialect type families (BIGINT vs INT vs BIGSERIAL vs NUMBER vs NVARCHAR vs VARCHAR vs TEXT) require a per-dialect compatibility table that is its own rabbit hole.

ExpectedColumn may grow additively (e.g. a future Type, Default, or CheckConstraint field) — never via positional replacement. Callers should always use field-named struct literals so the public surface stays forward-compatible.

type ExpectedTable 

type ExpectedTable struct {
	// Name is the SQL table name as it appears in the database. Case
	// matters for engines that fold identifiers (Oracle, MSSQL with
	// case-sensitive collation); for the supported engines (SQLite,
	// PostgreSQL, MySQL) lower-snake-case is the convention.
	Name string
	// Columns are the columns the caller expects on the table.
	// Indexes, constraints, and foreign keys are intentionally out of
	// scope for the initial SchemaDrift cut.
	Columns []ExpectedColumn
}

ExpectedTable is the caller's declaration of what a single table should look like, fed into Migrator.SchemaDrift for comparison against the live database. ExpectedTable is intentionally model-agnostic — `pkg/db` does not import `pkg/model` (that would cycle through model's tests, which use pkg/db). Callers wrap their own metadata source (the typical one is `model.ExtractMeta`, but anything that can produce a table name + column list works).

type MigrationStatus 

type MigrationStatus struct {
	ID        string     `json:"id"`
	Applied   bool       `json:"applied"`
	AppliedAt *time.Time `json:"applied_at,omitempty"`
	HasUp     bool       `json:"has_up"`
	HasDown   bool       `json:"has_down"`
}

MigrationStatus describes the migration state for one migration ID.

type Migrator 

type Migrator struct {
	// contains filtered or unexported fields
}

Migrator manages SQL-based database migrations using timestamped .up.sql and .down.sql files. For production use where schema changes must be explicit and reversible.

Module-scoped Migrators (constructed via `NewModuleMigrator`) namespace their applied-migrations and checksum rows under a `<moduleName>/` prefix in the framework's tracking tables. This prevents two modules that ship `001_init.up.sql` from colliding on a primary-key insert when they share a database alias. ADR-010 §16 / Phase 2d. The bare constructor `NewMigrator` keeps the legacy unscoped behaviour so host applications that pre-date the module pattern continue to work without re-applying their migration history.

func NewMigrator 

func NewMigrator(db *DB, migrationsPath string, logger *slog.Logger) *Migrator

NewMigrator creates an unscoped Migrator that reads migration files from the given directory. Applied migrations and checksums are stored under the migration file's bare ID — the legacy behaviour preserved for host applications that pre-date the module pattern.

func NewModuleFSMigrator 

func NewModuleFSMigrator(db *DB, fsys fs.FS, moduleName string, logger *slog.Logger) *Migrator

NewModuleFSMigrator creates a module-scoped Migrator that reads its migration files from an fs.FS instead of a disk directory — the reader for `Module.Migrations` (ADR-022, executing the ADR-013 §R1 follow-up). Ledger namespacing is identical to NewModuleMigrator: storage IDs are prefixed `<moduleName>/`, so an embedded `001_init.up.sql` cannot collide with another module's file of the same name on a shared alias.

Migration files are read from the FS root (`.`), flat, with the same `.up.sql`/`.down.sql` naming as the disk layout — pass `fs.Sub` for a nested layout (e.g. an `embed.FS` rooted at the package directory). Create is disk-only and returns an error on an FS-backed Migrator.

The same constructor-misuse rules as NewModuleMigrator apply, plus a nil fsys panics: all three are programming errors at construction time.

func NewModuleMigrator 

func NewModuleMigrator(db *DB, migrationsPath, moduleName string, logger *slog.Logger) *Migrator

NewModuleMigrator creates a Migrator scoped to a named module. The migration files are still read from `migrationsPath`, but the IDs recorded in the framework's tracking tables (`nucleus_schema_migrations` and `nucleus_schema_migration_checksums`) are prefixed `<moduleName>/`. ADR-010 §16: this prevents cross-module filename collisions when multiple modules share a database alias, while keeping the on-disk migration filenames module-author-friendly (`001_init.up.sql` rather than `articles_001_init.up.sql`).

`moduleName` must be non-empty and must not contain `/` (the namespace separator). The function reports an empty-name input as a `panic` because constructor-time misuse is a programming error the framework cannot recover from; non-`/` validation is enforced at storage time.

func (*Migrator) Applied 

func (m *Migrator) Applied() ([]AppliedMigration, error)

Applied returns every row of the migration ledger, sorted by ID, whoever wrote it: the host's unscoped migrations and every module's namespaced ones. Status answers for the files THIS Migrator reads; Applied is the complement the CLI needs to show what modules applied through their embedded migrations at start, which no directory on disk describes.

func (*Migrator) Create 

func (m *Migrator) Create(name string) error

Create generates a pair of empty migration files with a timestamp prefix. Files are created as {timestamp}_{name}.up.sql and {timestamp}_{name}.down.sql.

func (*Migrator) Down 

func (m *Migrator) Down() error

Down rolls back the latest applied migration.

func (*Migrator) Drift 

func (m *Migrator) Drift() ([]DriftEntry, error)

Drift returns entries that indicate the migrations log no longer matches the files on disk.

Two kinds are detected:

  • DriftKindMissingUpFile — the migration is recorded as applied but the `.up.sql` file is gone.
  • DriftKindChecksumMismatch — the migration is recorded as applied, the `.up.sql` is present, but its SHA-256 differs from the checksum stored at apply time.

Schema-level drift (actual `information_schema.columns` shape vs what the migration files would have produced) is provided by SchemaDrift, which compares a caller-supplied expected schema against the live database for SQLite, PostgreSQL, and MySQL.

func (*Migrator) SchemaDrift 

func (m *Migrator) SchemaDrift(ctx context.Context, expected []ExpectedTable) ([]SchemaDriftEntry, error)

SchemaDrift compares a caller-provided expected schema against the live database shape and returns entries describing the divergences.

The comparator is deliberately conservative:

  • It checks table existence (DriftKindSchemaMissingTable) and the full set of column names (DriftKindSchemaMissingColumn, DriftKindSchemaExtraColumn).
  • It checks nullability per column (DriftKindSchemaColumnNullability) because nullability is deterministic across dialects.
  • It does NOT compare column types — see ExpectedColumn's comment for the reasoning.

All five supported engines are covered: SQLite, PostgreSQL, MySQL, MSSQL, and Oracle. Unknown engines return ErrSchemaDriftUnsupported.

func (*Migrator) Status 

func (m *Migrator) Status() ([]MigrationStatus, error)

Status returns migration state for all discovered migration files.

func (*Migrator) Steps 

func (m *Migrator) Steps(n int) error

Steps applies n migrations (n>0) or rolls back n migrations (n<0).

The plan runs under a cluster-wide advisory lock on the engines that have one (NU-13): two replicas running `migrate up` at the same time used to both apply the same pending migration, and on MySQL, MariaDB and SQL Server — where DDL commits itself — the second left its DDL applied and failed on the ledger. SQLite has a single writer and Oracle's DBMS_LOCK needs a grant most schemas do not have: neither locks here.

func (*Migrator) Up 

func (m *Migrator) Up() error

Up applies all pending migrations.

type SchemaDriftEntry 

type SchemaDriftEntry struct {
	Kind     string `json:"kind"`
	Table    string `json:"table"`
	Column   string `json:"column,omitempty"`
	Expected string `json:"expected,omitempty"`
	Actual   string `json:"actual,omitempty"`
}

SchemaDriftEntry describes a single divergence between an ExpectedTable set and the live database shape, reported by SchemaDrift.

type StatementInfo 

type StatementInfo struct {
	// Operation is the SQL verb classified from Query (SELECT/INSERT/…),
	// lowercase, or "other"/"unknown".
	Operation string
	// Query is the SQL statement text as sent to the driver.
	Query string
	// Args are the driver arguments, unredacted. The observer MUST sanitize
	// them before exposing the statement anywhere — the driver layer does no
	// redaction.
	Args []any
	// Duration is the wall-clock time the driver call took.
	Duration time.Duration
	// Err is the driver error, if any.
	Err error
	// RowsAffected is the exec row count (0 for queries, and for execs whose
	// driver does not report it).
	RowsAffected int64
}

StatementInfo is a single observed database/sql statement handed to a StatementObserver. It is the driver-level counterpart to the model layer's SQLQueryEvent: it carries no model name (the driver only sees SQL text), so it exists to surface the traffic model.CRUD never sees — direct db.QueryContext/ExecContext calls from outbox dispatch, SQL session stores, migrations, schema drift checks, and any other code that runs raw SQL.

type StatementObserver 

type StatementObserver func(ctx context.Context, info StatementInfo)

StatementObserver is invoked after each direct database/sql statement when Config.StatementObserver is set. It runs synchronously on the caller's goroutine after the underlying driver call returns, so it MUST be cheap and non-blocking; the wiring in pkg/app gates it on the observability bus having subscribers before doing any real work.

Statements issued through model.CRUD are NOT delivered here: CRUD already observes them at the model layer (enriched with the model name) and marks the context so this layer skips them — see observe.CtxWithModelObserved.

Opt-in: when Config.StatementObserver is nil the driver is not wrapped at all, so the hot path is byte-for-byte the stock database/sql path.

Source Files

View all Source files

Directories

Path Synopsis
Package driver is the contract a database driver module implements to plug into pkg/db.
 Package driver is the contract a database driver module implements to plug into pkg/db.
drivertest
Package drivertest is a conformance kit for database driver modules.
 Package drivertest is a conformance kit for database driver modules.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.