database

package
v1.3.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 12, 2026 License: MIT Imports: 25 Imported by: 0

Documentation

Overview

Package database — drill.go: high-level DR drill orchestration.

The `nself backup drill` command wraps the existing RestoreDrill primitive (restore_drill.go) with three production concerns the bare drill primitive does not address:

  1. RTO measurement: total wall-clock from "drill start" to "smoke checks pass", broken into download / decrypt / restore / verify phases. The target per STRAT-11 is 4 hours; anything longer needs investigation.
  2. Critical-table smoke: row-count assertions on np_users, np_licenses, np_audit_log, np_plugins, np_billing — five tables whose presence is load-bearing for license validation + signup recovery.
  3. Drill-log emission: an entry in .nself/drill-log.json that the doctor check OPS-DRILL-01 reads to enforce "drill within last 7 days".

The drill never touches production. RestoreDrill restores into a scratch database named <db>_drilltest, runs the smoke suite, then drops the scratch DB. Failure at any phase is reported with the partial timing data so operators can see exactly which phase blew the budget.

This file lives in `database` (not `backup`) to avoid an import cycle — `backup` already imports `database`, so the high-level drill orchestrator must live next to RestoreDrill rather than alongside backup.Create.

Package database — drill_critical_tables.go: the drill's critical-table list and its per-project resolution. Split out of drill.go (file-size ratchet, internal/repoqa) — this is the self-contained piece of the drill smoke gate that needed its own home once it grew a resolution function and tests, per the same drill_*.go concern-split convention already used for restore_drill.go.

Package database provides database and Hasura metadata operations.

Package database provides SQL identifier sanitization helpers to prevent SQL injection in contexts where identifiers (database, schema, table, column names) must be interpolated into SQL statements (positional parameters do not bind identifiers in PostgreSQL).

Package database — restore_drill_verify.go: post-restore verification queries for RestoreDrill (VerifyRestoredDatabase, verifyCriticalTables). Split out of restore_drill.go (file-size ratchet, internal/repoqa) — this is the self-contained "what does the restored DB actually contain" half of the drill, following the same drill_*.go concern-split convention already used for drill_critical_tables.go. No behavior change.

Index

Constants

This section is empty.

Variables

View Source 
var DefaultCriticalTables = []string{
	"np_users",
	"np_licenses",
	"np_audit_log",
	"np_plugins",
	"np_billing",
}

DefaultCriticalTables is the canonical np_*-prefixed critical-table list used when a project has not set BACKUP_CRITICAL_TABLES. The five tables here are load-bearing for the most common recovery scenario on an np_-prefixed (multi-app-isolation, per the Multi-Tenant Convention Wall) schema: license validation + signup pipeline + audit log must survive a restore.

Decision (2026-09-01, closes the drill-critical-tables-naming bug note, option (c)): the list is project-configurable rather than a single hardcoded convention, because real deployed schemas disagree on naming — this project's own staging nself_web_db uses unprefixed names (users, licenses, audit_logs, plugins) with no np_ prefix and no billing-equivalent table at all. Hardcoding either convention would make the drill permanently wrong-shaped for whichever projects don't use it. ResolveCriticalTables reads the per-project override; this var stays the zero-config fallback so existing np_-prefixed deployments (and every test in this package) see no behavior change.

View Source 
var ErrInvalidIdentifier = errors.New("invalid SQL identifier")

ErrInvalidIdentifier is wrapped into every error SanitizeIdentifier returns. Callers several layers up (e.g. cmd/commands/start_containers.go) use errors.Is against this sentinel to distinguish "the configured name isn't a usable SQL identifier" from unrelated Postgres failures (auth, connectivity, etc.) so they can point the user at the actual problem instead of a generic credentials/connectivity checklist.

View Source 
var Theme25RequiredColumns = []RequiredColumn{
	{Name: "id", DataType: "uuid", Required: true, Description: "UUID primary key"},
	{Name: "created_at", DataType: "timestamp with time zone", Required: true, Description: "creation timestamp"},
	{Name: "updated_at", DataType: "timestamp with time zone", Required: true, Description: "last update timestamp"},
	{Name: "user_id", DataType: "uuid", Required: false, Description: "owner reference to auth.users"},
	{Name: "deleted_at", DataType: "timestamp with time zone", Required: false, Description: "soft-delete timestamp"},
}

Theme25RequiredColumns are the 5 columns mandated by Theme 25.

Functions

func ApplyDefaultRLSPolicies added in v1.0.9 

func ApplyDefaultRLSPolicies(ctx context.Context, cfg *config.Config, schema, table string) error

ApplyDefaultRLSPolicies creates standard Hasura-compatible RLS policies on the given table. Policies use current_setting('hasura.user', true) for user identity and current_setting('hasura.role', true) for admin bypass.

If the table has no user_id column the user-based predicates are omitted and only admin-bypass policies are applied.

Schema and table names must originate from AuditRLS (pg_catalog validated).

func ApplyDir added in v1.1.1 

func ApplyDir(ctx context.Context, cfg *config.Config, dirPath string) (int, error)

ApplyDir is an alias for MigrateUpDir with the name expected by the task spec (G-008). Both functions apply all .sql files in the given directory in lexicographic order, skipping files already recorded in schema_versions.

func ApplyFKIndexes added in v1.0.9 

func ApplyFKIndexes(ctx context.Context, cfg *config.Config, findings []FKIndexInfo) (created int, errs []error)

ApplyFKIndexes creates missing indexes for all FK columns lacking one. Returns count of indexes created and any errors.

func ApplyFile added in v1.1.1 

func ApplyFile(ctx context.Context, cfg *config.Config, filePath string) (skipped bool, err error)

ApplyFile applies a single external SQL migration file and records it in schema_versions by filename + SHA-256 checksum (G-008).

Double-apply protection: if a file with the same filename is already present in schema_versions, the function logs the skip and returns (true, nil). The checksum is stored in nself_ops.migrations for audit purposes.

This enables plugin-claw external RLS migrations to be applied via CLI without requiring 'nself db shell' as a workaround.

func ApplyMetadataFromGit added in v1.0.9 

func ApplyMetadataFromGit(ctx context.Context, cfg *config.Config, projectDir string, ref string) error

ApplyMetadataFromGit checks out the metadata files at the given git ref and then applies them via the standard metadata apply mechanism. ref can be a branch name, tag, or commit hash.

func ApplyRLSBatch added in v1.0.9 

func ApplyRLSBatch(ctx context.Context, cfg *config.Config, tables []RLSTableInfo) (applied int, skipped int, errs []error)

ApplyRLSBatch runs EnableRLSOnTable and ApplyDefaultRLSPolicies on each table in the provided list. Tables that already have RLS enabled and at least one policy are skipped. Returns counts of applied and skipped tables along with any non-fatal errors encountered.

func ApplySoftDelete added in v1.0.9 

func ApplySoftDelete(ctx context.Context, cfg *config.Config, schema, table string) error

ApplySoftDelete runs the soft-delete migration for a specific table.

func ApplyTenantRLS added in v1.0.9 

func ApplyTenantRLS(ctx context.Context, dbcfg *config.Config, rlsCfg TenantRLSConfig) error

ApplyTenantRLS generates the RLS SQL for the given TenantRLSConfig and pipes it to the postgres container via psql.

func Backup 

func Backup(ctx context.Context, cfg *config.Config, outputPath string) error

Backup runs pg_dump inside the project's postgres container and streams the output directly to a file on disk. The custom format (-Fc) is used so the dump can be restored with pg_restore.

If outputPath is empty, a timestamped filename is generated under a "backups/" directory relative to the current working directory.

func BaselineMigration added in v1.3.6 

func BaselineMigration(ctx context.Context, cfg *config.Config, filePath string) error

BaselineMigration records a single on-disk migration as applied WITHOUT executing its SQL: it creates schema_versions/nself_ops.migrations if absent, then writes the same two ledger INSERTs 'db migrate up' writes after successfully running a migration — skipping straight to the record step. Errors if the migration is already recorded (callers filter those out via BaselinePlan.AlreadyApplied first; this is a defensive re-check).

func CheckMigrationIdempotency added in v1.0.9 

func CheckMigrationIdempotency(sqlContent string) (bool, []string)

CheckMigrationIdempotency analyzes SQL content for idempotent patterns. Returns true if the migration appears safe to re-run. Idempotent indicators: IF NOT EXISTS, IF EXISTS, CREATE OR REPLACE Non-idempotent: bare CREATE TABLE, ALTER TABLE ... ADD COLUMN without IF NOT EXISTS

func ClassifyByPresence added in v1.3.6 

func ClassifyByPresence(objects []ObjectRef, existing map[string]bool) (class DetectClass, present, missing []ObjectRef)

ClassifyByPresence classifies a migration by how many of the objects it creates already exist (existing, keyed by ObjectRef.Key()). BASELINE is only reachable when every object is present and APPLY only when none are — any partial match is CONFLICT, which callers must never auto-resolve. UNKNOWN means extraction found no objects to check (e.g. a data-only migration) — callers should fall back to ledger-only status.

func CompareSnapshots added in v1.0.9 

func CompareSnapshots(before, after []byte) (string, error)

CompareSnapshots returns a human-readable diff between two metadata snapshots. It compares table lists extracted from the sources and reports additions and removals.

func EnableRLSOnTable added in v1.0.9 

func EnableRLSOnTable(ctx context.Context, cfg *config.Config, schema, table string) error

EnableRLSOnTable runs ALTER TABLE to enable and force row-level security on the given schema.table. Schema and table names must come from AuditRLS output (validated against pg_catalog) and are quoted to prevent SQL injection.

func ExportAndCommitMetadata added in v1.0.9 

func ExportAndCommitMetadata(ctx context.Context, cfg *config.Config, projectDir string, commitMsg string) (string, error)

ExportAndCommitMetadata exports Hasura metadata and creates a git commit. commitMsg is the commit message; if empty, a default message is generated. Returns the commit hash on success.

func GenerateCrossRegionCronScript added in v1.0.9 

func GenerateCrossRegionCronScript(cfg *config.Config, xrCfg CrossRegionBackupConfig) string

GenerateCrossRegionCronScript generates a shell script suitable for use in a Docker cron container for daily cross-region backups.

func GenerateDriftMigration added in v1.0.9 

func GenerateDriftMigration(result SchemaDriftResult) (upSQL, downSQL string)

GenerateDriftMigration generates SQL to add missing columns to a table. Returns up.sql and down.sql content.

func GenerateFKIndexSQL added in v1.0.9 

func GenerateFKIndexSQL(info FKIndexInfo) string

GenerateFKIndexSQL generates CREATE INDEX SQL for a missing FK index.

func GenerateIdempotentVersion added in v1.0.9 

func GenerateIdempotentVersion(sqlContent string) (converted string, changes []string)

GenerateIdempotentVersion takes migration SQL and attempts to convert it to an idempotent form by adding IF NOT EXISTS clauses where missing. Returns the converted SQL and a list of conversions made.

func GeneratePITRPostgresConf added in v1.0.9 

func GeneratePITRPostgresConf(cfg *config.Config, pitrCfg PITRConfig) string

GeneratePITRPostgresConf generates the postgresql.conf snippet needed for PITR. Returns the configuration as a string ready to append to postgresql.conf.

func GenerateRollbackSQL added in v1.0.9 

func GenerateRollbackSQL(schema, table string) string

GenerateRollbackSQL returns SQL that removes all four standard RLS policies and disables row-level security on the given schema.table. The schema and table names must originate from AuditRLS (pg_catalog validated).

func GenerateSoftDeleteMigration added in v1.0.9 

func GenerateSoftDeleteMigration(schema, table string) (upSQL, downSQL string)

GenerateSoftDeleteMigration generates the SQL to add soft-delete support to a table: adds deleted_at TIMESTAMPTZ column, index, and creates a {table}_active view that filters WHERE deleted_at IS NULL. Returns up.sql and down.sql content.

func GenerateTenantRLSSQL added in v1.0.9 

func GenerateTenantRLSSQL(cfg TenantRLSConfig) (string, error)

GenerateTenantRLSSQL generates the complete SQL block (ENABLE RLS + FORCE RLS + DROP/CREATE policies) for the given TenantRLSConfig. The returned SQL is safe to pipe directly to psql.

func GenerateTrackTableSQL added in v1.0.9 

func GenerateTrackTableSQL(schema, table string) string

GenerateTrackTableSQL generates a comment block describing how to track a new table in Hasura metadata. It returns a YAML snippet and CLI hint rather than raw SQL because table tracking is a Hasura metadata operation.

func HasuraApplyMetadata 

func HasuraApplyMetadata(ctx context.Context, cfg *config.Config, projectDir string) error

HasuraApplyMetadata applies Hasura metadata from projectDir.

It supports two metadata formats:

  1. YAML directory format (standard Hasura CLI): projectDir/hasura/metadata/ Tables use !include directives referencing individual YAML files. This format is detected first. If the hasura CLI binary is in PATH it is used directly (handles all YAML features). Otherwise nSelf resolves !include directives in Go and sends the result to the metadata API.

  2. JSON format (legacy nSelf): projectDir/hasura/metadata.json Sent directly to the replace_metadata API.

If neither exists, reload_metadata is called (safe/idempotent).

func HasuraDiffMetadata added in v1.0.6 

func HasuraDiffMetadata(ctx context.Context, cfg *config.Config, projectDir string) ([]string, error)

HasuraDiffMetadata compares live metadata against on-disk YAML files. Returns a list of keys that differ. Empty list means no drift.

func HasuraExportMetadata 

func HasuraExportMetadata(ctx context.Context, cfg *config.Config) ([]byte, error)

HasuraExportMetadata retrieves the current metadata from Hasura as raw JSON bytes.

func HasuraExportToYAML added in v1.0.6 

func HasuraExportToYAML(ctx context.Context, cfg *config.Config, projectDir string) (string, error)

HasuraExportToYAML exports live Hasura metadata to a git-friendly sorted YAML directory structure under {projectDir}/hasura/metadata/. Returns the output directory path.

func HasuraGetInconsistentMetadata added in v1.3.6 

func HasuraGetInconsistentMetadata(ctx context.Context, cfg *config.Config) ([]string, error)

HasuraGetInconsistentMetadata calls Hasura's get_inconsistent_metadata and returns the names of any inconsistent metadata objects (empty when consistent). Used after an apply to surface drift the apply itself can't detect (e.g. a tracked table whose underlying column was renamed).

func HasuraReloadMetadata 

func HasuraReloadMetadata(ctx context.Context, cfg *config.Config) error

HasuraReloadMetadata tells Hasura to reload its metadata from the internal catalog, picking up any changes to tracked tables, relationships, or permissions.

func HasuraValidatePermissions added in v1.0.6 

func HasuraValidatePermissions(ctx context.Context, cfg *config.Config) ([]string, error)

HasuraValidatePermissions checks that every tenant-scoped table tracked by Hasura has permissions for tenant_member and tenant_admin roles. Returns a list of tables missing required permissions.

func InitializeDatabase 

func InitializeDatabase(ctx context.Context, cfg *config.Config) error

InitializeDatabase waits for PostgreSQL to become ready, then creates the database, schemas, grants, and required extensions. This is Phase 3 of the nself startup sequence.

Steps:

  1. Wait for a stable streak of successful readiness probes (max 60s, see postgresReadinessStreak)
  2. CREATE DATABASE IF NOT EXISTS
  3. CREATE SCHEMA IF NOT EXISTS auth, storage, public
  4. GRANT ALL ON SCHEMA auth, storage, public TO user
  5. CREATE EXTENSION IF NOT EXISTS pgcrypto, citext

func ListMigrationFiles added in v1.3.6 

func ListMigrationFiles(cfg *config.Config, dir string) ([]string, error)

ListMigrationFiles returns the sorted on-disk migration files for dir (or the auto-detected directory when dir is ""). Exported so commands that need the raw file list (lint) don't have to reach into the DB-backed status/apply paths.

func MetadataSnapshot added in v1.0.9 

func MetadataSnapshot(ctx context.Context, cfg *config.Config) ([]byte, error)

MetadataSnapshot captures the current live metadata as a JSON snapshot. Useful for comparing before/after schema migrations.

func MigrateDown 

func MigrateDown(ctx context.Context, cfg *config.Config) error

MigrateDown reverts the most recently applied migration. It looks for a corresponding .down.sql file next to the original migration.

func MigrateUp 

func MigrateUp(ctx context.Context, cfg *config.Config, plugin string) (int, error)

MigrateUp applies all pending migrations from the migrations directory. If plugin is non-empty, only that plugin's migrations are applied. Uses advisory locks to prevent concurrent runs, records SHA-256 checksums, and detects non-transactional statements (CREATE INDEX CONCURRENTLY) to run them outside a transaction. Returns the count of migrations applied.

func MigrateUpDir added in v1.1.1 

func MigrateUpDir(ctx context.Context, cfg *config.Config, dir string) (int, error)

MigrateUpDir applies all .sql files in the given directory in lexicographic order, skipping any already recorded in schema_versions (idempotent). This is the --migration-dir companion to ApplyFile (G-008).

func MustSanitizeIdentifier added in v1.0.9 

func MustSanitizeIdentifier(s string) string

MustSanitizeIdentifier is SanitizeIdentifier that panics on invalid input. Only use for compile-time-constant identifiers where validation is a sanity check, not a user-input boundary.

func Open added in v1.0.9 

func Open(ctx context.Context, cfg *config.Config) (*sql.DB, error)

Open opens a database/sql connection to the project's PostgreSQL instance. The caller is responsible for calling db.Close() when done.

func PITRRestore added in v1.0.9 

func PITRRestore(ctx context.Context, cfg *config.Config, targetTime string) error

PITRRestore restores the database to a specific point in time using archived WALs. targetTime is in RFC3339 format (e.g. "2024-01-15T14:30:00Z").

func PendingMigrations added in v1.0.2 

func PendingMigrations(ctx context.Context, cfg *config.Config, plugin string) ([]string, error)

PendingMigrations returns the list of migration names that have not yet been applied.

func PgBouncerConfigFile added in v1.0.9 

func PgBouncerConfigFile(cfg *config.Config) string

PgBouncerConfigFile generates the pgbouncer.ini content for a project. Returns the INI file contents as a string.

func PgBouncerConnectionURL added in v1.0.9 

func PgBouncerConnectionURL(cfg *config.Config) string

PgBouncerConnectionURL returns the connection URL that points through PgBouncer instead of directly to PostgreSQL.

func PgBouncerHBAFile added in v1.0.9 

func PgBouncerHBAFile(cfg *config.Config) string

PgBouncerHBAFile generates the pg_hba.conf-style auth file content for PgBouncer.

func PgBouncerUserlistFile added in v1.0.9 

func PgBouncerUserlistFile(cfg *config.Config) string

PgBouncerUserlistFile generates the userlist.txt content for PgBouncer. Passwords are stored in plaintext here; PgBouncer supports md5 and plaintext.

func PruneRemoteBackups added in v1.0.9 

func PruneRemoteBackups(ctx context.Context, cfg *config.Config, xrCfg CrossRegionBackupConfig) (pruned int, err error)

PruneRemoteBackups removes remote backups older than RetentionDays.

func RecordDrillResult added in v1.0.9 

func RecordDrillResult(projectDir string, result RestoreDrillResult) error

RecordDrillResult appends the drill result to .nself/restore-drills.log in JSON format (one JSON object per line).

func ResetChecksum added in v1.0.6 

func ResetChecksum(ctx context.Context, cfg *config.Config, migrationID string) error

ResetChecksum updates the stored checksum for a migration to match the current file on disk.

func ResolveCriticalTables added in v1.3.6 

func ResolveCriticalTables(cfg *config.Config) []string

ResolveCriticalTables returns the critical-table list to smoke-check for this project: cfg.Backup.CriticalTables (BACKUP_CRITICAL_TABLES, comma-separated, whitespace-trimmed, empty entries dropped) when set, else DefaultCriticalTables. A nil cfg (unit tests exercising smokeCheck/ verifyCriticalTables directly) also falls back to the default.

func Restore 

func Restore(ctx context.Context, cfg *config.Config, inputPath string) error

Restore streams a pg_dump custom-format file into the project's postgres container via pg_restore. The file contents are piped to docker exec's stdin, avoiding the need to copy the dump into the container first.

func SanitizeIdentifier added in v1.0.9 

func SanitizeIdentifier(s string) (string, error)

SanitizeIdentifier validates a SQL identifier and returns a double-quoted, escape-safe form suitable for direct interpolation into SQL. Returns an error wrapping ErrInvalidIdentifier if the input does not match the safe identifier pattern or exceeds the PostgreSQL identifier byte limit (63 bytes).

Use this for database names, schema names, table names, column names, and any other SQL identifier that must appear in a statement string. NEVER interpolate a raw string into SQL without validation.

func Seed 

func Seed(ctx context.Context, cfg *config.Config, file string) error

Seed executes SQL seed files against the postgres database. If file is specified, only that file is run. If file is empty, all *.sql files in the seeds/ directory are executed in alphabetical order.

func SeedFilter 

func SeedFilter(seedFiles []string, dbEnvSeeds string, currentEnv string) []string

SeedFilter returns the subset of seedFiles that should run in currentEnv. DB_ENV_SEEDS format: "dev:file1.sql,staging:file2.sql" Files not mentioned in dbEnvSeeds run in all environments. Files mentioned in dbEnvSeeds only run when their env matches currentEnv.

func SyncBackupToRemote added in v1.0.9 

func SyncBackupToRemote(ctx context.Context, cfg *config.Config, xrCfg CrossRegionBackupConfig) error

SyncBackupToRemote syncs local backups to the remote location using rclone. Creates a local backup first if no recent backup exists (within 24h).

func TestWALArchive added in v1.0.9 

func TestWALArchive(ctx context.Context, cfg *config.Config) error

TestWALArchive creates a test WAL file and verifies it is archived correctly. Returns an error if archiving fails or times out.

func ValidateMigrationSQL added in v1.3.6 

func ValidateMigrationSQL(name, sql string) error

ValidateMigrationSQL reports constructs that no Postgres version accepts, so they surface before the migration is applied rather than as a syntax error partway through a batch.

It is deliberately narrow. It flags only forms that are invalid in every supported Postgres, never merely unusual ones: a lint that cries wolf on valid SQL would be worked around rather than fixed.

func VerifyRestoredDatabase added in v1.0.9 

func VerifyRestoredDatabase(ctx context.Context, cfg *config.Config, drillDB string) (tables int, rows int64, err error)

VerifyRestoredDatabase connects to a restored database and runs basic integrity checks: counts rows in key tables, verifies pg_catalog consistency.

func WritePITRConfig added in v1.0.9 

func WritePITRConfig(cfg *config.Config, projectDir string, pitrCfg PITRConfig) error

WritePITRConfig writes the PITR configuration to .nself/pitr/postgresql.conf.d/pitr.conf. This file should be mounted into the PostgreSQL container.

func WritePgBouncerConfig added in v1.0.9 

func WritePgBouncerConfig(ctx context.Context, cfg *config.Config, projectDir string) error

WritePgBouncerConfig writes the three PgBouncer config files to the project's .nself/pgbouncer/ directory.

Types

type BaselinePlan added in v1.3.6 

type BaselinePlan struct {
	Name           string // ledger key (migrationKey)
	MigrationID    string // nself_ops.migrations id (extractMigrationID)
	Checksum       string // SHA-256 hex of the on-disk file
	FilePath       string
	AlreadyApplied bool
}

BaselinePlan describes what baselining one migration would record.

func PlanBaseline added in v1.3.6 

func PlanBaseline(ctx context.Context, cfg *config.Config, dir string, names []string) ([]BaselinePlan, error)

PlanBaseline resolves each requested migration name against dir (or the auto-detected migrations directory) and computes what baselining it would record, without writing anything. If the schema_versions ledger does not exist yet, every migration is reported as not-yet-applied rather than erroring — this keeps --dry-run usable before any migration has ever run.

type ChecksumMismatch added in v1.0.6 

type ChecksumMismatch struct {
	ID       string
	Name     string
	Expected string // stored in DB
	Actual   string // computed from disk
}

ChecksumMismatch describes a migration whose on-disk checksum differs from the recorded value.

func VerifyChecksums added in v1.0.6 

func VerifyChecksums(ctx context.Context, cfg *config.Config, plugin string) ([]ChecksumMismatch, error)

VerifyChecksums compares on-disk migration checksums against stored values. Returns a list of mismatches (empty means all good).

type CrossRegionBackupConfig added in v1.0.9 

type CrossRegionBackupConfig struct {
	Enabled         bool
	RemotePath      string // rclone remote path (e.g. "s3:mybucket/backups")
	SecondaryRegion string
	Retention       int // days to keep remote backups
	Encrypted       bool
	AgeRecipients   string
}

CrossRegionBackupConfig holds the configuration for cross-region backup.

type CrossRegionBackupStatus added in v1.0.9 

type CrossRegionBackupStatus struct {
	LastSync         time.Time
	FilesTransferred int
	BytesTransferred int64
	Errors           []string
}

CrossRegionBackupStatus describes the last backup sync result.

func GetCrossRegionStatus added in v1.0.9 

func GetCrossRegionStatus(ctx context.Context, cfg *config.Config, xrCfg CrossRegionBackupConfig) (CrossRegionBackupStatus, error)

GetCrossRegionStatus checks the rclone remote for recent backups.

type DetectClass added in v1.3.6 

type DetectClass string

DetectClass is the honest three-way (plus unknown) classification of a pending migration against the live schema. 

const (
	DetectApply    DetectClass = "APPLY"
	DetectBaseline DetectClass = "BASELINE"
	DetectConflict DetectClass = "CONFLICT"
	DetectUnknown  DetectClass = "UNKNOWN"
)

type DetectResult added in v1.3.6 

type DetectResult struct {
	Name    string
	Class   DetectClass
	Present []ObjectRef
	Missing []ObjectRef
	Lint    *LintFinding
}

DetectResult is one pending migration's classification against the live schema, plus its bulk-unguarded-CREATE-TABLE lint finding (nil if clean).

func DetectMigrations added in v1.3.6 

func DetectMigrations(ctx context.Context, cfg *config.Config, dir string) ([]DetectResult, error)

DetectMigrations classifies every pending (not-yet-in-ledger) migration in dir (or the auto-detected directory when dir is "") against the live schema.

type DriftSummary added in v1.0.9 

type DriftSummary struct {
	TotalTables     int
	CompliantTables int
	DriftedTables   int
	MissingRequired int // tables missing at least one REQUIRED column
	OverallScore    int // 0-100
}

DriftSummary returns a high-level summary: total tables, compliant, drifted, score.

func SummarizeDrift added in v1.0.9 

func SummarizeDrift(results []SchemaDriftResult) DriftSummary

SummarizeDrift returns a high-level summary across all drift results.

type DrillOptions added in v1.1.0 

type DrillOptions struct {
	// BackupFile, when set, drills against a specific backup. Empty = most recent.
	BackupFile string
	// RTOTargetHours is the failure threshold for total drill wall-clock.
	// Default 4 hours per STRAT-11. Setting <= 0 disables the gate (the drill
	// still runs and reports duration; result.RTOTargetMet stays true).
	RTOTargetHours float64
	// DryRun, when true, validates the inputs and prints what would happen
	// without actually creating the scratch DB or touching pg_restore.
	DryRun bool
	// ProjectDir is where .nself/drill-log.json lives. Defaults to ".".
	ProjectDir string
}

DrillOptions captures the user-facing knobs for `nself backup drill`. All fields are optional; sensible defaults come from the project config.

type DrillResult added in v1.1.0 

type DrillResult struct {
	StartedAt      time.Time     `json:"started_at"`
	CompletedAt    time.Time     `json:"completed_at"`
	BackupFile     string        `json:"backup_file"`
	Success        bool          `json:"success"`
	Phase          string        `json:"phase,omitempty"` // last phase reached (init|restore|verify|smoke|done|dry-run)
	ErrorMessage   string        `json:"error_message,omitempty"`
	TotalDuration  time.Duration `json:"total_duration_ns"`
	RestoreSeconds float64       `json:"restore_seconds"`
	VerifySeconds  float64       `json:"verify_seconds"`
	SmokeSeconds   float64       `json:"smoke_seconds"`
	TablesChecked  int           `json:"tables_checked"`
	RowsObserved   int64         `json:"rows_observed"`
	RTOTargetMet   bool          `json:"rto_target_met"`
	// MissingCriticalTables lists CriticalTables entries not found by name in
	// the restored database. Informational, not a failure condition on its
	// own — see the smoke-check comment below.
	MissingCriticalTables []string `json:"missing_critical_tables,omitempty"`
}

DrillResult is the structured outcome of a single `nself backup drill` run. Phase timings are zero when a phase did not execute. All fields serialize to JSON for the drill log + doctor consumption.

func Drill added in v1.1.0 

func Drill(ctx context.Context, cfg *config.Config, opts DrillOptions) (DrillResult, error)

Drill runs the high-level drill orchestration: invokes RestoreDrill for the heavy lifting, layers RTO measurement + smoke checks on top, and emits a structured DrillResult to .nself/drill-log.json. The function never returns nil result — even on early failure the result captures whatever timing data was collected up to the failure point.

func LastSuccessfulDrill added in v1.1.0 

func LastSuccessfulDrill(projectDir string) (DrillResult, bool)

LastSuccessfulDrill returns the most recent successful DrillResult from the drill log, or (zero, false) when no successful drill has run yet. The doctor check OPS-DRILL-01 calls this to enforce "drill within last 7 days".

type FKIndexInfo added in v1.0.9 

type FKIndexInfo struct {
	TableSchema    string
	TableName      string
	ColumnName     string
	ForeignSchema  string
	ForeignTable   string
	ForeignColumn  string
	HasIndex       bool
	ConstraintName string
}

FKIndexInfo describes a foreign key constraint and whether it has an index.

func AuditFKIndexes added in v1.0.9 

func AuditFKIndexes(ctx context.Context, cfg *config.Config) ([]FKIndexInfo, error)

AuditFKIndexes finds all foreign key columns and reports whether each has a supporting index. Only tables in np_* and public schemas are checked.

type HasuraGitStatus added in v1.0.9 

type HasuraGitStatus struct {
	Branch       string
	CommitHash   string
	LastExportAt time.Time
	Modified     []string // metadata files with uncommitted changes
	IsClean      bool
}

HasuraGitStatus describes the git state of metadata files.

func GetGitStatus added in v1.0.9 

func GetGitStatus(projectDir string) (HasuraGitStatus, error)

GetGitStatus returns the git status of the hasura/metadata/ directory.

type LintFinding added in v1.3.6 

type LintFinding struct {
	Rule    string
	Message string
	Count   int
}

LintFinding describes one migration-file lint violation.

func LintMigrationFile added in v1.3.6 

func LintMigrationFile(path string) (*LintFinding, error)

LintMigrationFile reads a single migration file and lints it.

func LintUnguardedBulkCreateTables added in v1.3.6 

func LintUnguardedBulkCreateTables(sqlContent string) *LintFinding

LintUnguardedBulkCreateTables returns nil when sqlContent has fewer than bulkUnguardedCreateTableThreshold unguarded CREATE TABLE statements. Callers (the `db migrate lint` command, --detect) treat a non-nil finding as a refusal, never a warning to silently route around.

type MetadataDrift added in v1.0.9 

type MetadataDrift struct {
	// TablesAdded are tracked tables in live Hasura not in committed metadata.
	TablesAdded []string
	// TablesRemoved are tables in committed metadata not in live Hasura.
	TablesRemoved []string
	// PermissionsChanged are tables where permissions differ.
	PermissionsChanged []string
	// RelationshipsChanged are tables where relationships differ.
	RelationshipsChanged []string
	// IsClean is true when live matches committed exactly.
	IsClean bool
}

MetadataDrift describes the difference between live Hasura metadata and committed files.

func DetectMetadataDrift added in v1.0.9 

func DetectMetadataDrift(ctx context.Context, cfg *config.Config, projectDir string) (MetadataDrift, error)

DetectMetadataDrift compares live Hasura metadata against committed files in hasura/metadata/. It exports current metadata via the Hasura API and diffs against the on-disk files.

type MetadataDriftFinding added in v1.3.6 

type MetadataDriftFinding struct {
	Table  string // "schema.name"
	Role   string // permission role; empty for a table/relationship finding
	Kind   string // "table" | "relationship" | "insert" | "select" | "update" | "delete"
	Detail string
}

MetadataDriftFinding is one row of Hasura metadata drift: a specific table/role/kind combination where committed metadata and the live Hasura instance disagree.

func ScanMetadataDrift added in v1.3.6 

func ScanMetadataDrift(ctx context.Context, cfg *config.Config, projectDir string) ([]MetadataDriftFinding, error)

ScanMetadataDrift compares committed hasura/metadata/** against a live Hasura instance and returns every finding. An empty result means clean.

type MigrationAuditResult added in v1.0.9 

type MigrationAuditResult struct {
	Name          string
	Applied       bool
	Idempotent    bool     // uses IF NOT EXISTS, IF EXISTS, CREATE OR REPLACE
	HasRollback   bool     // a down.sql exists alongside up.sql
	ChecksumMatch bool     // file checksum matches stored checksum in nself_ops.migrations
	Issues        []string // list of idempotency problems found
}

MigrationAuditResult describes the audit result for a single migration.

func AuditMigrations added in v1.0.9 

func AuditMigrations(ctx context.Context, cfg *config.Config) ([]MigrationAuditResult, error)

AuditMigrations audits all migration files for idempotency and drift. It checks:

  • All applied migrations have matching checksums (no file edits after apply)
  • Pending migrations use idempotent SQL patterns (IF NOT EXISTS, etc.)
  • Each migration has a corresponding down/rollback file

type MigrationRecord added in v1.0.6 

type MigrationRecord struct {
	ID         string
	Name       string
	Checksum   string
	AppliedAt  string
	AppliedBy  string
	DurationMs int
	RolledBack bool
}

MigrationRecord holds the state of a migration from nself_ops.migrations.

type MigrationStatus 

type MigrationStatus struct {
	Name      string
	Applied   bool
	Timestamp time.Time
}

MigrationStatus describes the state of a single migration file.

func MigrateStatus 

func MigrateStatus(ctx context.Context, cfg *config.Config, dir string) ([]MigrationStatus, error)

MigrateStatus returns the status of all known migrations (applied and pending). It merges on-disk migration files with the schema_versions table, so orphaned migrations (applied but no longer on disk) are also reported.

dir overrides the auto-detected migrations directory (the --migration-dir companion to MigrateUpDir, G-008): repos with non-standard layouts, e.g. ntask's postgres/migrations, would otherwise report "No migrations found". Pass "" to auto-detect via migrationsDir.

type MissingPrerequisite added in v1.3.6 

type MissingPrerequisite struct {
	Object      ObjectRef
	MigrationID string // ledger key of the migration whose ALTER TABLE needs it
}

MissingPrerequisite is one ALTER TABLE target that will not exist by the time the migration needing it runs.

type ObjectKind added in v1.3.6 

type ObjectKind string

ObjectKind identifies the kind of schema object a CREATE statement makes. 

const (
	ObjectTable      ObjectKind = "table"
	ObjectIndex      ObjectKind = "index"
	ObjectType       ObjectKind = "type"
	ObjectView       ObjectKind = "view"
	ObjectMatView    ObjectKind = "materialized view"
	ObjectSequence   ObjectKind = "sequence"
	ObjectSchemaKind ObjectKind = "schema"
)

type ObjectRef added in v1.3.6 

type ObjectRef struct {
	Kind ObjectKind
	Name string // as written in the SQL; may be schema-qualified
}

ObjectRef is one schema object a migration file creates.

func ExtractAlteredObjects added in v1.3.6 

func ExtractAlteredObjects(sqlContent string) []ObjectRef

ExtractAlteredObjects returns every table sqlContent's ALTER TABLE statements target, skipping any guarded with IF EXISTS (those cannot fail on a missing table, so have no prerequisite to check).

func ExtractCreatedObjects added in v1.3.6 

func ExtractCreatedObjects(sqlContent string) []ObjectRef

ExtractCreatedObjects returns every schema object sqlContent creates.

func (ObjectRef) Key added in v1.3.6 

func (o ObjectRef) Key() string

Key returns the unique map key used to dedupe/look up presence for o.

type PITRConfig added in v1.0.9 

type PITRConfig struct {
	ArchiveDir  string
	WalInterval int
	RemotePath  string // rclone remote path for WAL shipping
}

PITRConfig holds the generated PostgreSQL settings for PITR.

type PITRStatus added in v1.0.9 

type PITRStatus struct {
	ArchiveMode     string // on, off, always
	ArchiveCommand  string
	WALLevel        string // replica, logical
	MaxWALSenders   int
	WALArchiveDir   string
	LastArchivedWAL string
	LastArchiveTime string
	ArchiveEnabled  bool
}

PITRStatus describes the current PITR configuration state.

func GetPITRStatus added in v1.0.9 

func GetPITRStatus(ctx context.Context, cfg *config.Config) (PITRStatus, error)

GetPITRStatus queries PostgreSQL for current WAL archiving settings.

type PgBouncerStatus added in v1.0.9 

type PgBouncerStatus struct {
	Running       bool
	PoolMode      string
	ActivePools   int
	IdleServers   int
	ActiveClients int
}

PgBouncerStatus holds runtime status of PgBouncer.

func GetPgBouncerStatus added in v1.0.9 

func GetPgBouncerStatus(ctx context.Context, cfg *config.Config) (*PgBouncerStatus, error)

GetPgBouncerStatus queries the PgBouncer admin database for runtime status. It connects via psql inside the pgbouncer container (named {project}_pgbouncer).

type RLSTableInfo added in v1.0.9 

type RLSTableInfo struct {
	TableSchema string `json:"table_schema"`
	TableName   string `json:"table_name"`
	RLSEnabled  bool   `json:"rls_enabled"`
	ForceRLS    bool   `json:"force_rls"`
	PolicyCount int    `json:"policy_count"`
}

RLSTableInfo holds the RLS state for a single table returned by AuditRLS.

func AuditRLS added in v1.0.9 

func AuditRLS(ctx context.Context, cfg *config.Config) ([]RLSTableInfo, error)

AuditRLS queries pg_catalog.pg_class and pg_policies to return the RLS state for every table in schemas starting with "np_" or in schema "public". It identifies tables that are missing RLS or have zero policies.

type ReconcileChange added in v1.3.6 

type ReconcileChange struct {
	Table       string
	Description string
}

ReconcileChange is one human-readable line of a reconcile plan.

type ReconcilePlan added in v1.3.6 

type ReconcilePlan struct {
	BulkOps []map[string]interface{}
	Changes []ReconcileChange
}

ReconcilePlan is dry-run output: the exact bulk operations Apply would send, plus a human-readable change list.

func BuildReconcilePlan added in v1.3.6 

func BuildReconcilePlan(ctx context.Context, cfg *config.Config, projectDir string) (ReconcilePlan, error)

BuildReconcilePlan loads repo + live metadata, refuses on either unsafe condition (see package doc), and returns the ordered, targeted plan to bring live into agreement with the repo. It never touches a live table the repo doesn't declare, and never removes anything.

func (ReconcilePlan) Apply added in v1.3.6 

func (p ReconcilePlan) Apply(ctx context.Context, cfg *config.Config) error

Apply executes the plan's bulk operations against the live instance as ONE atomic Hasura `bulk` metadata call, so a failure rolls back rather than leaving a partial apply.

type RequiredColumn added in v1.0.9 

type RequiredColumn struct {
	Name        string
	DataType    string
	Required    bool // if false, column is recommended but not required
	Description string
}

RequiredColumn defines a column that should exist on all np_* tables.

type RestoreDrillResult added in v1.0.9 

type RestoreDrillResult struct {
	StartedAt      time.Time `json:"started_at"`
	CompletedAt    time.Time `json:"completed_at"`
	BackupFile     string    `json:"backup_file"`
	Success        bool      `json:"success"`
	TablesVerified int       `json:"tables_verified"`
	RowsVerified   int64     `json:"rows_verified"`
	// MissingCriticalTables lists the entries of CriticalTables that were not
	// found (by name, any schema) in the restored database. Informational —
	// see the naming-mismatch note in drill.go: several drilled environments
	// do not use the np_ prefix, so this is reported rather than failing the
	// drill outright.
	MissingCriticalTables []string      `json:"missing_critical_tables,omitempty"`
	ErrorMessage          string        `json:"error_message,omitempty"`
	Duration              time.Duration `json:"duration_ns"`
}

RestoreDrillResult describes the outcome of a restore test.

func RestoreDrill added in v1.0.9 

func RestoreDrill(ctx context.Context, cfg *config.Config, backupFile string) (RestoreDrillResult, error)

RestoreDrill runs a restore drill using the most recent backup. It restores to a TEMPORARY database named {dbname}_drilltest, verifies data integrity, then drops the temporary database. This is non-destructive — it does not touch the production database.

type RoleReachability added in v1.3.6 

type RoleReachability struct {
	Role      string
	Queries   int
	Mutations int
}

RoleReachability is the count of top-level query/mutation fields visible to a role, from a role-scoped introspection query.

func VerifyRoleReachability added in v1.3.6 

func VerifyRoleReachability(ctx context.Context, cfg *config.Config, role string) (RoleReachability, error)

VerifyRoleReachability runs an introspection query against the live GraphQL endpoint impersonating the given role (admin secret + X-Hasura-Role — see package doc for why a bare role header does not work).

type SchemaDriftResult added in v1.0.9 

type SchemaDriftResult struct {
	TableSchema    string
	TableName      string
	MissingColumns []RequiredColumn
	PresentColumns []string
	DriftScore     int // 0-100, higher = more drift
}

SchemaDriftResult describes drift for a single table.

func ScanSchemaDrift added in v1.0.9 

func ScanSchemaDrift(ctx context.Context, cfg *config.Config) ([]SchemaDriftResult, error)

ScanSchemaDrift scans all np_* tables for the 5 required Theme 25 columns. Only tables in schemas starting with "np_" are checked.

type SoftDeleteInfo added in v1.0.9 

type SoftDeleteInfo struct {
	TableSchema  string
	TableName    string
	HasDeletedAt bool
	HasIndex     bool // index on deleted_at for performance
	HasView      bool // {table}_active view exists
}

SoftDeleteInfo describes a table's soft-delete status.

func AuditSoftDelete added in v1.0.9 

func AuditSoftDelete(ctx context.Context, cfg *config.Config) ([]SoftDeleteInfo, error)

AuditSoftDelete scans tables in schemas starting with "np_" or "public" and reports their soft-delete status.

type TenantRLSConfig added in v1.0.9 

type TenantRLSConfig struct {
	Pattern      TenantRLSPattern
	Schema       string
	Table        string
	UserColumn   string // column compared to hasura.user (default: "user_id")
	TenantColumn string // column compared to hasura.tenant (default: "tenant_id")
}

TenantRLSConfig specifies the parameters needed to generate and apply RLS policies for a single table.

type TenantRLSPattern added in v1.0.9 

type TenantRLSPattern string

TenantRLSPattern describes the access-control model for a table. 

const (
	// PatternUserOwned restricts access to rows whose user_id matches the
	// Hasura session variable hasura.user. Admins bypass via hasura.role.
	PatternUserOwned TenantRLSPattern = "user_owned"

	// PatternTenantScoped restricts access to rows whose tenant_id matches the
	// Hasura session variable hasura.tenant. Admins bypass via hasura.role.
	PatternTenantScoped TenantRLSPattern = "tenant_scoped"

	// PatternPublic makes every row visible to all sessions without restriction.
	PatternPublic TenantRLSPattern = "public"

	// PatternAdminOnly restricts all access to sessions where hasura.role = 'admin'.
	PatternAdminOnly TenantRLSPattern = "admin_only"
)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.