migrations

package

v0.2.2 Latest Latest Go to latest Published: Nov 10, 2026 License: AGPL-3.0 Imports: 1 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/Lutfifakee-Project/GSEER

Links

Open Source Insights

README ¶

GSEER migrations — authoring guidelines

The SQL files in this directory are embedded in the gseer binary (embed.go) and applied by the golang-migrate runner (internal/storage/postgres/migrate.go) on gseer setup --self-hosted and gseer server start. Users never hand-run SQL (SPEC-05 §1).

Rules (normative)

One atomic transaction per migration. The pgx/v5 driver runs with MultiStatementMode=false: every statement in a .up.sql file commits together or rolls back together. Author every file so it is safe to run inside a transaction.
Forbidden statements. These cannot run inside a PostgreSQL transaction and will fail the migration: CREATE DATABASE, DROP DATABASE, CREATE TABLESPACE, DROP TABLESPACE, VACUUM, CLUSTER, REINDEX, CREATE INDEX CONCURRENTLY, ALTER SYSTEM. If a future migration truly needs one, it must go in its own file and the runner must be extended deliberately (out of v0.0.1 scope).
Naming. NNNN_name.up.sql / NNNN_name.down.sql. The numeric prefix is the migration version (golang-migrate parses it). Never renumber or edit an applied migration; always add a new one.
schema_migrations is NOT ours. The golang-migrate driver creates and owns the schema_migrations table. Application migrations must never create, drop, or modify it. Recovery from a dirty state uses gseer db migrate force --version N (metadata-only, confirmation- gated) after manual database inspection via gseer db migrate diagnose.
Every .up.sql has a matching .down.sql that reverses it in FK-safe order, so migrate force recovery and test round-trips work.
Collector-side tables are v0.0.2 (owner decisions D-M3-1/D-M3-3): source_checkpoints, ingestion_dead_letter, scopes, collection_jobs, suppressions. suppressions MUST land before any first probe (v0.0.2 release gate).

Inventory (v0.2.0)

Version	Name	Purpose	Status
0001	`0001_core`	`assets`, `services`, `observations`, `data_sources` + indexes	shipped v0.1.0
0002	`0002_helper_views`	`latest_http`, `latest_tls`, `latest_fingerprints`	shipped v0.1.0
0003	`0003_search_index`	`search_index` union view (planner target)	shipped v0.1.0
0004	`0004_search_indexes`	evidence-driven supporting indexes	shipped v0.1.0
0005	`0005_suppressions`	suppression rules + audit surface + DB backstop v1	shipped v0.1.0
0006	`0006_collection`	`collection_jobs`, `source_checkpoints`, `ingestion_dead_letter`, `scopes`	shipped v0.1.0
0007–0015	—	permanently reserved (OD-4); no migration will ever exist	RESERVED
0016	`0016_suppression_policy_hardening`	`gseer_policy` enforcement layer (hash-pinned Rev 6.7.2 artifact; refusal-first guard; SECURITY DEFINER policy functions; `fn_check_asset_suppression` v2)	v0.2.0 — PINNED, do not edit
0017	`0017_search_safe_casts`	safe-cast helpers + rebuilt `latest_http`/`latest_tls` (GAP-1 mandate; hash-pinned)	v0.2.0 — PINNED

Pinned artifacts (0016 pair, 0017 pair, and the three deploy/sql scripts) carry registered SHA-256 hashes enforced by artifacts_test.go. Suppression writes go through the gseer_policy.suppression_* SECURITY DEFINER functions after 0016 — direct DML on public.suppressions is refused by the policy gate (ERRCODE GSUP2).

Documentation ¶

Overview ¶

Package migrations embeds the GSEER SQL migration files so the compiled binary is self-contained (no runtime file-path lookup).

The golang-migrate source driver `source/iofs` consumes the embed.FS directly: migrate.NewWithSourceInstance("iofs", migrations.FS, driver).

Migration set (owner-approved M3/M4 scope; SPEC-05 §2/§3/§5 + v0.0.2 Passive Intelligence milestone M13):

0001_core             assets, services, observations, data_sources
                      + L1/L2 uniqueness indexes + search indexes
0002_helper_views     latest_http, latest_tls, latest_fingerprints
0003_search_index     search_index union view (M4 planner target)
0004_search_indexes   evidence-driven supporting indexes (M4 STEP 2/3;
                      EXPLAIN ANALYZE baseline in
                      internal/storage/postgres/testdata/plan_baseline.md)
0005_suppressions     suppression policy rules + audit surface +
                      the DB backstop trigger (M13; fail-closed)
0006_collection       collector-side bookkeeping: collection_jobs,
                      source_checkpoints, ingestion_dead_letter,
                      scopes (M14; scopes itself stays P2-unused)
0016_suppression_policy_hardening
                      the pinned Rev 6.7.2 gseer_policy enforcement
                      layer (hash-pinned; versions 0007–0015 are
                      permanently RESERVED per OD-4 — they must
                      never be created)
0017_search_safe_casts
                      safe-cast helpers + rebuilt latest_http /
                      latest_tls views (GAP-1 mandate; malformed
                      evidence can no longer break search)

The schema_migrations table is owned by the golang-migrate driver and is NEVER created or modified by an application migration.