Documentation
¶
Overview ¶
Package db provides Postgres connection bootstrapping and schema migration for the Wardyn control plane. Postgres is the ONLY required dependency.
Index ¶
- Constants
- Variables
- func AppClockAgeMicros(t, now time.Time) int64
- func AppClockAgeSQL(placeholder string) string
- func AuditChainCanary(ctx context.Context, pool *pgxpool.Pool) error
- func AuditChainLockTimeoutSQL() string
- func AuditDDLBypassRoutes(ctx context.Context, pool *pgxpool.Pool) ([]string, error)
- func AuditDDLProtected(ctx context.Context, pool *pgxpool.Pool) (bool, error)
- func Connect(ctx context.Context, dsn string) (*pgxpool.Pool, error)
- func Migrate(ctx context.Context, pool *pgxpool.Pool) error
- func TryAdvisoryLock(ctx context.Context, pool *pgxpool.Pool, key int64) (release func(), ok bool, err error)
Constants ¶
const AuditChainLockKey int64 = 0x5741524459_434841 // ASCII "WARDYCHA"
AuditChainLockKey serializes appends to the audit_events hash chain (migration 0047). Unlike every key above it is taken with the TRANSACTION -scoped pg_advisory_xact_lock, never the session-scoped form: it is released by the commit that makes the new row visible, so the next writer's head read cannot miss it, and no code path can leak it by forgetting a release. SINCE 0056_audit_chain_serialize.sql THE TRIGGER TAKES IT TOO, and that is what binds writers this package does not know about. 0047 could not: the identity default had already assigned seq by the time a BEFORE INSERT trigger ran, so two racing writers could take the lock there in the opposite order to their seq allocation and invert chain order against seq order. 0056 removes that objection by allocating seq inside the trigger, under this lock, so position and chain link are decided together. Both in-tree insert paths still take it BEFORE their INSERT statement, on the inserting transaction (store.InsertAuditEvent and the broker's insertAuditEventTx): advisory locks are re-entrant within a transaction, so the trigger's acquisition is free for them, and holding it across the whole statement is what it always was. ponytail: ONE lock for the whole chain, so audit appends are globally serialized. That IS the feature (a chain has exactly one head), and audit write volume is nowhere near a contention regime. If it ever is, the upgrade is per-partition chains with a key per partition, not a finer lock over one.
const GroundTruthRotatorLockKey int64 = 0x5741524459_475452 // ASCII "WARDYGTR"
GroundTruthRotatorLockKey elects the ground-truth token rotator's leader across control planes (S2). Unlike ReaperAdvisoryLockKey (re-tried every tick via TryAdvisoryLock, release()d at the end of each one), this key is acquired ONCE before the rotator's loop starts and held for the process lifetime: only the holder mints/writes the shared token file, and every other replica parks on a backoff and retries, taking over automatically when the holder's Postgres session ends.
It buys AT MOST ONE STEADY-STATE leader, NOT mutual exclusion. An advisory lock dies with its SESSION, not with the process, and the holder never re-verifies it: a Postgres restart, a failover, pg_terminate_backend or an idle-session timeout releases it under a still-running leader, and a standby takes over within one backoff — two rotators, neither aware. Harmless for THIS workload only, because every write is an atomic rename of a stateless token (cmd/wardynd/gt_rotator.go). Work that needs genuine fencing must not reuse this key. Any stable value works, as long as it differs from every other key in this file.
const ReaperAdvisoryLockKey int64 = 0x5741524459_524541 // ASCII "WARDYREA"
ReaperAdvisoryLockKey makes the lifecycle reap tick single-flight across control planes. Unlike migrateAdvisoryLockKey (a BLOCKING lock — the second boot must still see the migrations applied), this one is only ever taken with TryAdvisoryLock: a replica that loses SKIPS the tick, because a queued second reap of the same runs is pure duplicate work and a duplicate run.autostop.
const SecretRekeyLockKey int64 = 0x5741524459_524B59 // ASCII "WARDYRKY"
SecretRekeyLockKey serializes the `wardynd -rotate-age-key` maintenance mode (cmd/wardynd's rotateAgeKeyMode): two concurrent rekeys of the same store would each re-encrypt from an old key the other has already replaced, so the second is refused rather than queued (TryAdvisoryLock, like ReaperAdvisoryLockKey).
HONEST CEILING — this does NOT detect a running wardynd. No wardynd holds a process-lifetime lock on THIS key: a serving daemon holds SingleInstanceLockKey (a different key), the reaper takes ReaperAdvisoryLockKey per tick and releases it, and GroundTruthRotatorLockKey is only taken when the rotator is configured — so a serving daemon is invisible to this check. "Stop the daemon first" is an operator procedure documented in docs/OPERATIONS.md, not something this lock enforces — a live daemon holds the OLD identity in memory and would write ciphertext under a key the rekey has already retired.
const SingleInstanceLockKey int64 = 0x5741524459_494E53 // ASCII "WARDYINS"
SingleInstanceLockKey is the RUNTIME half of the one-replica safety control (the Helm chart's `replicas > 1` render refusal is the other half). wardynd takes it ONCE at boot, with TryAdvisoryLock, and holds it for the process lifetime: a second instance against the same database refuses to start instead of quietly serving.
WHY IT IS A SAFETY CONTROL, not modesty: wardynd keeps state per-process that a second instance cannot see, the sharpest being the secret-masking registry (internal/secretmask) — an in-memory, process-local map that FAILS OPEN. Secrets are registered on whichever instance served the run's proxy injection; a session recording uploaded to any other instance finds an empty snapshot and is persisted VERBATIM, live credentials in cleartext, with a `success` audit event. The chart's refusal is render-time only, so `kubectl scale`, an HPA, or a non-Helm replica edit defeated it silently.
HONEST CEILING — AT MOST ONE STEADY-STATE INSTANCE, NOT MUTUAL EXCLUSION, exactly as GroundTruthRotatorLockKey documents for the same mechanism. An advisory lock dies with its SESSION, not with the process, and the holder never re-verifies it: a Postgres restart, a failover, pg_terminate_backend or an idle-session timeout releases it under a still-running daemon, and the next instance to boot takes it — two daemons, neither aware. It closes the silent-scale hole; it is not a fence. Any stable value works, as long as it differs from every other key in this file.
Variables ¶
var AuditChainLockTimeout = 5 * time.Second
AuditChainLockTimeout bounds how long ANY writer waits for AuditChainLockKey before giving up. Since 0056 the trigger takes that lock on every insert into audit_events, including inserts from outside this repo, so one transaction that inserted an audit row and stayed open holds up every audit write in the process - and that is reachable with no Wardyn bug at all: an operator's psql session, a seed script, a paused migration tool. AuditSpool.Drain already bounds its side of this at 15s per pass ("one idle psql transaction must not become a process-wide stall"); the SYNCHRONOUS side had no bound of any kind. A request-path audit write took the lock on the raw request context, against a server with lock_timeout = 0 and statement_timeout = 0 and an http.Server that deliberately sets no WriteTimeout, so it waited forever - pinning a request goroutine and a pool connection each time. With pool_max_conns at the documented minimum of 3, a handful of stuck audit writes exhausts the pool and every other query in the process starts blocking behind them.
BOTH DIRECTIONS OF THE CHOICE, because a bound on a synchronous path can fail either way. Too short and a healthy-but-loaded deployment refuses audit writes it could have completed; too long and the request path stalls exactly when the database is already in trouble. 5s is chosen against measured shapes rather than taste: a legitimate wait here is other audit writers queueing, each holding the lock for one nextval, one indexed head read, one sha256 and one insert - low single-digit milliseconds - so 5s absorbs a queue in the thousands before it ever refuses a write that would have completed. It is also deliberately well UNDER the drain's 15s pass bound, so the request path yields before the background drain does, which is the right order: the drain is the thing built to absorb a backlog.
WHAT HAPPENS TO THE WRITE THAT LOSES THE RACE decides whether this is a fix or a relocation of the failure, so it is stated here. On the request path, the error travels back through spoolingRecorder, which fsyncs the event to the local spool and logs AUDIT WRITE FAILED; the drain replays it once the lock clears. The event is not dropped - it takes exactly the degraded path C1 built for a failed durable write. On the broker's mint transaction the audit insert is in the same tx as the credential, so a timeout refuses the MINT: no credential is issued that could not be audited, which is the fail-closed direction a governance tool wants, and is what that path already did for every other audit failure.
lock_timeout rather than a context deadline, verified rather than assumed: it fires ONLY on a lock wait, never on a slow-but-progressing statement (so the "too short" direction cannot abort work that was making progress), it is enforced server-side, SET LOCAL scopes it to the transaction so nothing leaks onto a pooled connection, and it reports the distinguishable SQLSTATE 55P03. Measured on Postgres 17: it bounds the explicit pg_advisory_xact_lock AND the trigger's own acquisition during an ordinary INSERT, so it also covers a writer that never takes the lock explicitly.
Functions ¶
func AppClockAgeMicros ¶ added in v0.7.0
AppClockAgeMicros returns how long ago the app-clock instant t was, as of the app-clock instant now, in microseconds — clamped to [0, maxAppClockAge].
BOTH ARGUMENTS MUST COME FROM THE SAME CLOCK. That is the whole point: a difference of two readings of one clock is a duration, and a duration carries no skew, so it can be handed to a different clock without carrying an offset across. Passing a value that was read back from the DATABASE as t would put the skew straight back in, in the unsafe direction when the app is behind.
CLAMPED AT ZERO, so a value stamped in the future (the app clock stepped backwards between the stamp and this call) becomes "now" rather than a negative interval that would push a row's timestamp forward of the database's clock — which is the exact shape of the defect this function exists to close.
The ZERO TIME is deliberately NOT special-cased here: it means different things to different callers (an unstamped row, versus a pre-D16 cookie that must read as revoked), and a helper that guessed which would be wrong for one of them. Each caller states its own answer.
func AppClockAgeSQL ¶ added in v0.7.0
AppClockAgeSQL renders an app-measured age, in MICROSECONDS, as an interval to subtract from the database's own clock. The parameter placeholder is supplied by the caller because it differs per statement.
Microseconds, as a bigint, rather than a float or a pgx interval: it is timestamptz's own resolution, so the conversion is exact in both directions and there is no rounding to reason about.
func AuditChainCanary ¶ added in v0.7.0
auditChainCanary appends ONE synthetic audit row inside a transaction it always rolls back, and asserts the row came out chained: row_hash set, and prev_hash equal to the head read under the same lock. It is the FUNCTIONAL half of the boot check.
WHY CATALOG SHAPE IS NOT ENOUGH, and the release has the receipt. Everything above this asks the catalog: is the trigger there, is it enabled, is it the function we ship. 0057 shipped a trigger that was all three and did not work — a SECURITY DEFINER pinned to `pg_catalog, public` while its body named audit_events unqualified, so on a deployment whose objects live in another schema Migrate reported success and then EVERY audit insert failed from inside the trigger, or (quieter and worse, where public held a second Wardyn schema) linked the chain to the wrong table's head and never linked at all. 0058 repaired that particular body; nothing made the failure CLASS visible at boot. A single round trip does, and it converts the class from post-hoc to boot-time.
NEVER COMMITTED, so there is no synthetic row in anybody's audit log and no question about what an operator is looking at. The cost is one burned seq per boot — which the sweep already tolerates by design: auditChainWalk's own doc records that seq gaps below the chain come from rolled-back inserts and that seq is not hashed.
WHAT REFUSES AND WHAT ONLY REPORTS. A chain that demonstrably does not chain — the insert succeeded and the row came back with no row_hash, or with a prev_hash that is not the head — refuses the boot: that is the 0057 state, and a wardynd serving over it writes an audit log the verify sweep will report as broken for as long as it runs. A canary that could not be RUN because the chain lock was busy or the statement was cancelled reports at ERROR and lets the boot continue: those are bounded, transient and self-clearing (the whole subject of AuditChainLockTimeout), and bricking a boot over one is a failure this check would cause rather than one it would find. AuditChainCanary runs the boot canary against an arbitrary pool, for the ONE caller that needs it on a pool Migrate never touched.
WHY THE SPLIT-ROLE POSTURE NEEDS IT. Migrate — and so the canary at its tail — runs on the MIGRATE pool when WARDYN_PG_MIGRATE_DSN is set, and that is the posture the daemon's own boot log recommends. But the migrate role is not the role that writes audit rows: every real audit write goes through the APP pool, as a different role, with a different search_path and different privileges. A canary that only ever ran as the migrator therefore proved the chain works for a connection nothing audits on, and a split-role boot could start clean while the app pool's very next audit write came back unchained — the exact failure class this check exists to convert from post-hoc to boot-time.
SAME FUNCTION, not a second copy: the refusal rules, the rolled-back transaction, the READ COMMITTED pin and the transient-error treatment are the ones documented on auditChainCanary below, so the two boot paths cannot come to different conclusions about what a working chain is.
func AuditChainLockTimeoutSQL ¶ added in v0.7.0
func AuditChainLockTimeoutSQL() string
AuditChainLockTimeoutSQL is the statement that applies AuditChainLockTimeout to the current transaction. SET takes no bind parameters, so the value is formatted in - it is an integer from the variable above, never caller input. SET LOCAL, so it reverts at commit or rollback and the pooled connection is handed back exactly as it was found.
func AuditDDLBypassRoutes ¶ added in v0.7.0
AuditDDLBypassRoutes names EVERY route by which pool's role can reach past audit_events' append-only guard. An empty slice means protected, and is what AuditDDLProtected is defined as.
IT EXISTS SO THE BOOT LOG CAN NAME THE ROUTE. A bare bool made the daemon guess: its WARN told the operator the app role "still owns audit_events or is a superuser" and prescribed "connect wardynd as a distinct non-owner role", which is the wrong remedy for two of the four routes and actively misleading for the fourth — a role that owns nothing and is nobody's superuser, but holds GRANT SET ON PARAMETER session_replication_role, gets a warning naming two things it is not. Reporting the route is also the operator-facing half of the decision taken on this finding: report the replication-role route rather than arming ENABLE ALWAYS.
ONE PREDICATE, so the report and the verdict cannot disagree: the bool is len(routes) == 0 and there is no second query anywhere that answers it.
FAILS SAFE, exactly as the bool did: a missing table, or any error, is a bypass rather than a protection claim.
func AuditDDLProtected ¶
AuditDDLProtected reports whether the given (application) pool's role is UNABLE to bypass the audit_events append-only triggers — i.e. it is neither a superuser nor a MEMBER of the table's owner role (membership, not just direct ownership: a role GRANTed the owner role inherits DROP TRIGGER / ALTER ... DISABLE TRIGGER rights), it does not hold the TRIGGER privilege on the table, AND it cannot SET session_replication_role, which silences every simply-enabled trigger without touching DDL at all (the fourth leg, below). The N4 role-separation only protects the append-only guarantee when this is true, so the two-DSN deploy must be VERIFIED here rather than assumed (honesty: never log a protection claim stronger than the enforcing role setup). Fails safe: any ambiguity (missing table, error) reports NOT protected.
THE LEGS THEMSELVES LIVE IN AuditDDLBypassRoutes, which answers the same question and NAMES the routes that fired so the boot log can tell an operator which one to close. This function is that answer as a bool, and there is no second query anywhere that could disagree with it.
THE TRIGGER PRIVILEGE IS PART OF THE CLAIM, and it is the least obvious third of it. A role that is neither owner nor superuser but holds GRANT TRIGGER ON audit_events cannot drop the shipped triggers — it can do something quieter: CREATE its own row-level BEFORE INSERT trigger, which is handed NEW and whose changes are what Postgres stores — so it can rewrite any field, choose prev_hash/row_hash, or drop the row entirely, minting records that say whatever it wants while every shipped guard stays armed and every catalog check still finds them. Name order is NOT what makes that work: a trigger sorting after audit_events_chain (same-event row triggers fire in name order) runs last and can overwrite the hashes directly, but one sorting BEFORE it is easier still — it rewrites NEW and the shipped chain trigger then hashes the forgery for it. ensureAuditTriggers refuses the boot over either, keyed on the trigger's SHAPE rather than its name; this function is the PREVENTIVE half and only asks whether the privilege to create one is held. 0007_audit_least_privilege.sql revokes TRIGGER from PUBLIC precisely because of that, but a deploy is free to grant it back, so the claim has to be checked and not inferred.
ALL FOUR ROLE LEGS TEST MEMBERSHIP, not a role attribute. The superuser leg asks whether current_user is a member of ANY role with rolsuper — not whether current_user itself has rolsuper. Reading the attribute off the current_user row missed the ordinary managed-Postgres shape (GRANT some admin role TO the app role): that role has rolsuper = false, is not a member of the table's owner, and holds no TRIGGER privilege, so it was reported PROTECTED while it could SET ROLE to a superuser and ALTER TABLE ... DISABLE TRIGGER. pg_has_role with 'MEMBER' is what makes this honest: 'MEMBER' is the right to SET ROLE, so it follows the grant chain to any depth AND ignores INHERIT — a NOINHERIT role that can still SET ROLE is caught. A role is a member of itself, so a directly superuser role is reported exactly as it was before.
WHAT THIS DELIBERATELY DOES NOT MODEL, stated so the next reader does not widen it by guesswork. Membership in pg_write_all_data is NOT a bypass and is NOT tested for: it confers INSERT/UPDATE/DELETE rights, but the append-only triggers still fire and raise — measured in the probe beside this function, not assumed. Roles that own the HOST rather than the guard — pg_execute_server_program, pg_write_server_files — can escalate to superuser by documented PostgreSQL behaviour and are still reported protected here. The predicate stays a closed, catalog-derived test (rolsuper) rather than a list of role names that rots with every Postgres release: once the database host is compromised, no claim Wardyn makes about that database survives anyway.
func Connect ¶
Connect opens a pgxpool to dsn and performs a lightweight liveness check. Returns the pool; caller owns Close().
func Migrate ¶
Migrate applies all migrations in internal/db/migrations/*.sql in lexical order. Each migration runs inside its own transaction; already-applied filenames (tracked in schema_migrations) are skipped. Idempotent.
func TryAdvisoryLock ¶ added in v0.5.0
func TryAdvisoryLock(ctx context.Context, pool *pgxpool.Pool, key int64) (release func(), ok bool, err error)
TryAdvisoryLock takes session-level advisory lock key on a connection borrowed from pool WITHOUT waiting, reporting ok=false when another session already holds it. Call the returned release (deferred) to unlock and hand the connection back — skipping it strands a pooled conn for the life of the process. Only for work of BOUNDED duration: the borrowed conn is unavailable to everyone else until release — which also means the caller's own queries need a SECOND conn, so this requires pool_max_conns >= 2 (a 1-conn pool would self-deadlock: the lock holds the only conn while the guarded work blocks on Acquire; the reaper's per-tick deadline turns that into a failed tick, not a hang, but the lock is still wasted).
Types ¶
This section is empty.
Source Files