dump

package
v1.6.20 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 19, 2026 License: Apache-2.0 Imports: 16 Imported by: 0

Documentation

Overview

Package dump is the core of pg-autodump: it orchestrates one dump run, drives the pg boundary to stream a network pg_dump per database, verifies each dump locally, atomically replaces the previous good file, and reports a typed result per database. It defines the narrow interface it consumes (PGTool) so the logic is testable against fakes with no network, no daemon, and no real filesystem dependencies beyond a temp dir.

Index

Constants

View Source
const ProbeTimeoutCap = 10 * time.Second

ProbeTimeoutCap bounds the per-database reachability probe so a dead host classifies as connect_error quickly instead of consuming the dump budget. Exported so the trigger client's worst-case wait model (cmd/pg-autodump triggerTimeout) consumes the same constant the orchestrator enforces and the two can never drift.

Variables

This section is empty.

Functions

func DueForStartupDump added in v1.1.5

func DueForStartupDump(dumpDir string, interval time.Duration, now time.Time) bool

DueForStartupDump reports whether the built-in scheduler should run one dump at startup. It returns true when no dump artifact under dumpDir is newer than one interval (including when there are none at all). This closes the restart-starvation gap: the ticker's first fire is one full interval after start and its clock resets on every restart, so a container restarting more often than its interval could otherwise produce no backups indefinitely. Gating on recency means a restart that already has a fresh dump does not re-dump, so a crash/restart loop cannot become a dump loop. now is the wall clock (file mtimes are wall-clock); a dump with a future mtime (a backward clock step) reads as fresh and suppresses the startup dump, which is the safe direction (never destroys budget on a redundant run).

func ReclaimOrphans added in v1.3.0

func ReclaimOrphans(dumpDir string, log *slog.Logger)

ReclaimOrphans removes crash-orphaned temp files under dumpDir: every first-level per-server subdirectory — where dump temps stage (stageAndReplace targets <host>_<port>/ and atomicfile creates its temp in the target's own directory) — and the DUMP_DIR root itself, which holds exactly one class of the app's own artifact: the writability probe obs.Preflight leaves behind when the directory accepts a write but denies the unlink. Both scans reap only what atomicfile recognises as its own temp (".atomicfile-<digits>.tmp", regular files), so an operator's own files at the root — dumps, notes, anything merely prefix-alike — are never touched.

It MUST only be called while no dump can be in flight — at the start of a cycle with the cross-process cycle lock held, or at startup with the lock momentarily acquired — because every temp it sees is then a crash orphan (graceful failure paths run pending.Cleanup() themselves). A concurrently running preflight probe is the one benign exception: losing its own temp to this sweep reads as an already-removed file, which atomicfile counts as a clean removal. Best-effort: unreadable directories are skipped and per-file failures are handled inside CleanupStaleTemps; only the outcome is logged here.

Types

type Conn

type Conn struct {
	Host   string
	DBName string
	User   string
	Port   int
}

Conn is the network coordinates of one database. It carries no password: libpq resolves credentials from .pgpass keyed on host:port:dbname:user.

type FailKind

type FailKind int

FailKind is a typed classification the pg boundary returns from Probe (and from a connection-phase Dump failure) so the orchestrator can map a failure to a Reason structurally, never by substring-matching stderr.

const (
	// FailNone means no boundary-classified failure occurred.
	FailNone FailKind = iota
	// FailConnect means the host was unreachable or refused the connection.
	FailConnect
	// FailAuth means authentication failed (bad/missing .pgpass entry or role).
	FailAuth
	// FailVersion means the client pg_dump major is older than the server major.
	FailVersion
)

type Guard

type Guard struct {
	// contains filtered or unexported fields
}

Guard enforces one dump run at a time in-process: the resident HTTP server owns a single Guard shared by its HTTP, exec-trigger, and ticker paths, and it holds the in-flight run's cancel func so the shutdown path can reach and cancel it. Cross-process coordination (the server vs an exec'd `pg-autodump run`) is layered on top via scheduler.Exclusive's kernel-released flock — unlike the 1.x lock file, a crashed holder can never wedge it, and the Guard remains the only place that owns cancellation and drain.

func (*Guard) CancelInFlight

func (g *Guard) CancelInFlight()

CancelInFlight cancels the in-flight run's context if one is active, else is a no-op. The shutdown goroutine calls it after the drain budget expires; cancelling the run context cascades to every worker's per-database context.

func (*Guard) TryAcquire

func (g *Guard) TryAcquire(cancel context.CancelFunc) (release func(), ok bool)

TryAcquire marks a run in flight and returns a release closure if no run is active; it returns ok=false otherwise (the caller responds 429). The caller passes the run's cancel func so shutdown can reach it. The release closure clears both the flag and the cancel pointer and is safe to defer.

func (*Guard) WaitIdle added in v1.1.5

func (g *Guard) WaitIdle(ctx context.Context) bool

WaitIdle blocks until the in-flight run finishes or ctx is done. It returns true if no run was active or the run finished, false if ctx fired first. The shutdown path uses it to drain a ticker-triggered run that holds no HTTP connection for srv.Shutdown to see.

The wait is webhttp.AwaitDone's, for the recheck a bare two-case select does not do: webhttp.Run hands the teardown a context carrying whatever is LEFT of the one shutdown grace after the HTTP drain, so a drain that spent the whole budget calls this with an ALREADY-EXPIRED context, and a select whose cases are both ready picks pseudo-randomly — a dump that DID finish then reported false, and the caller logged "drain budget exceeded" and cancelled an idle guard. AwaitDone re-checks completion after ctx fires, so completion wins. (webhttp appears in this package for a pure context helper, not for server plumbing: the misreport is HERE, in the wait, not at the call site.)

type Orchestrator

type Orchestrator struct {
	// contains filtered or unexported fields
}

Orchestrator owns a dump run: split valid from invalid specs, drive the bounded worker pool, and return one Result per spec in spec order.

func New

func New(p *Params) *Orchestrator

New builds an Orchestrator from validated params.

func (*Orchestrator) Run

func (o *Orchestrator) Run(ctx context.Context) []Result

Run executes every spec and returns one Result per database in spec order. Invalid and duplicate specs yield a Result without being dispatched; valid specs run through the bounded worker pool. It never returns a partial slice.

Run assumes it is the only dump run in flight (its production callers hold the in-process guard and the cross-process cycle lock), so it first reclaims crash-orphaned temp files under DumpDir — every temp visible at cycle start is an orphan. On completion it emits a single "dump cycle complete" heartbeat (total/ok/failed tallied from the results); the README's alerting section documents the Loki absence rule that keys on it to catch a silent non-run of this backup-critical sidecar, not just a loud per-DB failure.

type PGTool

type PGTool interface {
	// Probe performs a TCP dial followed by an authenticated round-trip to
	// classify connect/auth/version failures and report the server major version.
	Probe(ctx context.Context, c Conn) (serverMajor int, kind FailKind, err error)

	// Dump streams a network custom-format pg_dump for c to w and returns the
	// process exit code and a bounded tail of stderr. pg_dump runs as a local
	// child process, so ctx cancellation terminates it directly.
	Dump(ctx context.Context, c Conn, w io.Writer) (exitCode int, stderrTail string, err error)

	// VerifyTOC runs a local `pg_restore --list` against the custom-format
	// file at path. It needs no network and no daemon. Returns nil iff the
	// table-of-contents header is readable.
	VerifyTOC(ctx context.Context, path string) error
}

PGTool is the external boundary the orchestrator depends on. The concrete implementation lives in internal/pg; tests supply a fake. Every method is context-bounded and the implementation enforces that a deadline is present.

type Params

type Params struct {
	PG          PGTool
	Logger      *slog.Logger
	Now         func() time.Time // injectable clock; defaults to time.Now
	DumpDir     string
	Specs       []spec.DBSpec
	DumpTimeout time.Duration
	Concurrency int
	Keep        int
	FreeKBWarn  int64
}

Params configures an Orchestrator. The pg boundary is injected; the rest are validated config values.

type Reason

type Reason string

Reason is the closed dump-result taxonomy.

const (
	ReasonOK              Reason = "ok"
	ReasonEmpty           Reason = "empty"
	ReasonTruncated       Reason = "truncated"
	ReasonTimeout         Reason = "timeout"
	ReasonKilled          Reason = "killed"
	ReasonPGError         Reason = "pg_error"
	ReasonConnectError    Reason = "connect_error"
	ReasonAuthError       Reason = "auth_error"
	ReasonVersionMismatch Reason = "version_mismatch"
	ReasonOther           Reason = "other"
	ReasonDuplicate       Reason = "duplicate"
	ReasonInvalid         Reason = "invalid"
	ReasonMkdirFailed     Reason = "mkdir_failed"
	ReasonRenameFailed    Reason = "rename_failed"
	ReasonSkipped         Reason = "skipped"
)

ReasonOK and the following constants are the valid Reason values for a dump result.

type Result

type Result struct {
	Host          string
	DBName        string
	Reason        Reason
	Detail        string // human line for the HTTP body, e.g. "ok (4823104 bytes)"
	Bytes         int64
	ServerVersion int // server major version from the probe (server_version_num/10000), 0 if unknown
	Duration      time.Duration
}

Result is one database's outcome from a dump run.

func (*Result) BodyDetail added in v1.1.5

func (r *Result) BodyDetail() string

BodyDetail is the operator-facing line for the POST /dump (and `trigger`) response body. For the execution-tool failure reasons whose Detail carries a raw pg_dump/pg_restore stderr tail (pg_error, truncated, other), it returns only the reason word: that stderr can echo schema/object/role names and error text, and the endpoint may run open (no AUTH_TOKEN), so the full detail is kept to the logs (the orchestrator's per-result log line records r.Detail) and out of a body any reachable client could read. Every other reason returns its Detail (e.g. "ok (4823104 bytes)", a validation reason, "connect_error"), falling back to the reason word when Detail is empty.

func (*Result) OK

func (r *Result) OK() bool

OK reports whether the dump succeeded.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL