backup

package
v0.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 12, 2026 License: Apache-2.0 Imports: 38 Imported by: 0

Documentation

Overview

Package backup drives a database backup end to end: dump a managed database container's data, stream it straight to an S3-compatible bucket, and record the attempt in store.BackupHistory. It composes two independent capabilities, each with its own file and its own real implementation:

  • Dumper (dump.go): produces a dump stream from a running database container.
  • Uploader (uploader.go): writes a stream to a configured destination bucket.
  • Deleter (deleter.go): removes a single object from a configured destination bucket, used by Scheduler's retention pruning.

Runner (runner.go) is the only piece that talks to store and internal/secrets; Dumper, Uploader, and Deleter implementations know nothing about either, so each is independently testable with a fake on the other side of its interface.

Index

Constants

View Source
const (
	CoverageVerified    = "verified"
	CoverageUnverified  = "unverified"
	CoverageNeverRan    = "never_ran"
	CoverageFailing     = "failing"
	CoverageUnprotected = "unprotected"
)

Coverage states, one per database or app volume. Only verified and unverified count as protected: a schedule that never ran protects nothing.

View Source
const (
	ActionProtect   = "protect"
	ActionBackupNow = "backup_now"
	ActionRestore   = "restore"
)

Primary actions a coverage row offers.

View Source
const (
	DrillStageObject   = "object"
	DrillStageDownload = "download"
	DrillStageRestore  = "restore"
	DrillStageContent  = "content"
	DrillStageDone     = "done"
)

Drill stages, recorded on store.BackupDrill.Stage.

View Source
const (
	ObjectOK        = "ok"
	ObjectMissing   = "missing"
	ObjectTruncated = "truncated"
)

Object states CheckObject reports.

View Source
const (
	ProtectionLocked  = "locked"
	ProtectionVersion = "versioned"
	ProtectionOpen    = "open"
)

Protection levels reported by Level.

View Source
const (
	HealthFailing     = "failing"
	HealthUnverified  = "unverified"
	HealthWarning     = "warning"
	HealthHealthy     = "healthy"
	HealthUnprotected = "unprotected"
)

Resource health states, worst first.

View Source
const (
	CodecZstd    = "zstd"
	CodecZstdAge = "zstd+age"
)

Backup object codecs recorded on store.BackupHistory.Codec. The empty string is a raw legacy object.

View Source
const (
	EnvBackupEncryption    = "APP_BACKUP_ENCRYPTION"
	EnvBackupEncryptionKey = "APP_BACKUP_ENCRYPTION_KEY_FILE"
	EnvBackupMaxSizeMB     = "APP_BACKUP_MAX_SIZE_MB"
	EnvBackupTimeoutMin    = "APP_BACKUP_TIMEOUT_MINUTES"
	EnvBackupUploadTries   = "APP_BACKUP_UPLOAD_ATTEMPTS"
	EnvBackupDrillInterval = "APP_BACKUP_DRILL_INTERVAL_HOURS"
	EnvBackupStaleMinutes  = "APP_BACKUP_STALE_RUNNING_MINUTES"
)

Environment variables that tune sealed volume backups.

View Source
const DefaultDrillInterval = 7 * 24 * time.Hour

DefaultDrillInterval is how often each resource is drilled when APP_BACKUP_DRILL_INTERVAL_HOURS is unset.

View Source
const DefaultStaleRunning = 6 * time.Hour

DefaultStaleRunning is how long a backup may stay running before the sweep examines it.

View Source
const ScheduledVerificationCheckedBy = "scheduler"

ScheduledVerificationCheckedBy is the CheckedBy value Scheduler records for its own automatic verification, distinguishing it from a manual one in the UI/CLI.

Variables

View Source
var ErrBackupTooLarge = errors.New("volume archive exceeds the backup size budget")

ErrBackupTooLarge means a volume archive exceeded the configured size budget.

View Source
var ErrNoBaseBackup = errors.New("backup: no succeeded base backup exists yet, point-in-time restore is not available")

ErrNoBaseBackup is returned by ValidatePITRTarget when win has no succeeded base backup at all: point-in-time restore genuinely has nothing to restore from yet, not merely a target outside its range.

View Source
var ErrNoDecryptionKey = errors.New("backup is encrypted and this control plane has no decryption key")

ErrNoDecryptionKey means an encrypted object was opened without identities.

Functions

func CheckObject added in v0.2.0

func CheckObject(info ObjectInfo, h store.BackupHistory) (state, reason string)

CheckObject compares a HEAD result with what was recorded at backup time. A recorded size of zero (legacy rows) only requires the object to exist.

func CheckWorkDirSpace added in v0.2.0

func CheckWorkDirSpace() error

CheckWorkDirSpace runs the same free-space preflight RunBackup does, so an API handler can refuse a manual backup with a visible error instead of accepting it and having the detached run fail without a history row.

func CodecSuffix added in v0.2.0

func CodecSuffix(codec string) string

CodecSuffix is the object key suffix for the codec, tar payload assumed.

func CopyVolume added in v0.2.0

func CopyVolume(ctx context.Context, rt docker.Runtime, src, dst string) error

CopyVolume copies every file of src into dst, keeping modes and owners. Both volumes must not be mounted by a running database.

func DeriveCoverage added in v0.2.0

func DeriveCoverage(resources []CoverageResource, health []ResourceHealth) ([]CoverageRow, CoverageSummary)

DeriveCoverage joins desired resources with their health rows. A health row whose resource no longer exists is ignored: its backups are a storage concern, not coverage.

func DrillIntervalFromEnv added in v0.2.0

func DrillIntervalFromEnv(lookup func(string) (string, bool)) time.Duration

DrillIntervalFromEnv parses APP_BACKUP_DRILL_INTERVAL_HOURS: unset means the default, 0 disables scheduled drills.

func ExpiredByPolicy added in v0.2.0

func ExpiredByPolicy(items []RetentionItem, p RetentionPolicy) []string

ExpiredByPolicy returns the IDs of items no bucket of p wants to keep. The newest item is always kept so a policy can never empty a history. Buckets count only periods that actually hold a backup, so a gap in backups does not shrink what is kept.

func MoveVolume added in v0.2.0

func MoveVolume(ctx context.Context, archiver VolumeArchiver, restorer VolumeRestorer, dockerVolumeName string) error

MoveVolume pipes archiver's Archive stream directly into restorer's Restore: no S3 target or store.BackupHistory row needed, since the data only ever exists transiently in flight. Callers must construct archiver against the volume's current node and restorer against its destination, and keep dockerVolumeName identical on both ends (see internal/api's handleMoveAppWithVolumes).

func NewDrillID added in v0.2.0

func NewDrillID() (string, error)

NewDrillID mints a drill identifier safe to embed in a Docker name.

func OpenSealed added in v0.2.0

func OpenSealed(r io.Reader, codec string, identities []age.Identity) (io.ReadCloser, error)

OpenSealed reverses Seal for the recorded codec. An empty codec returns r unchanged. The returned reader fails on a truncated or tampered object.

func PruneRemoteWAL added in v0.2.0

func PruneRemoteWAL(ctx context.Context, dest Destination, lister Lister, deleter Deleter, databaseName, startLSN string) (int, error)

PruneRemoteWAL deletes shipped segments older than the segment holding startLSN, mirroring PruneWALArchive for the copy in the backup target.

func PruneWALArchive added in v0.2.0

func PruneWALArchive(ctx context.Context, rt Runtime, containerName, startLSN string) error

PruneWALArchive removes archived WAL segments older than the one holding startLSN. Compares the 16 hex digit segment number so a timeline change (which only alters the 8 digit prefix) cannot delete segments a base backup on an older timeline still needs.

func RecordProtection added in v0.2.0

func RecordProtection(ctx context.Context, st ProtectionRecorder, secrets SecretsResolver, prober ProtectionProber, targetID string, at time.Time) store.BackupTargetProtection

RecordProtection probes one target's bucket and stores the result. A probe that cannot run is stored with its error so the dashboard can say so.

func RecoverableWindowEnd added in v0.2.0

func RecoverableWindowEnd(ctx context.Context, rt Runtime, containerName string) (time.Time, error)

RecoverableWindowEnd returns the latest timestamp a PITR restore of containerName can safely target: forces a WAL switch and waits for archive_command to actually land it, rather than trusting archive_timeout's own lag, so the bound is provably exact. A target beyond what's genuinely archived would hang recovery rather than fail it (ValidatePITRTarget).

func StaleRunningFromEnv added in v0.2.0

func StaleRunningFromEnv(lookup func(string) (string, bool)) time.Duration

StaleRunningFromEnv parses APP_BACKUP_STALE_RUNNING_MINUTES.

func StartLSN added in v0.2.0

func StartLSN(ctx context.Context, rt Runtime, containerName string) string

StartLSN best-effort queries the current WAL insert location for BaseBackupHistory.LSN's diagnostics-only purpose. Failures are swallowed: losing this one diagnostic string must never fail an otherwise-successful base backup.

func ValidatePITRTarget added in v0.2.0

func ValidatePITRTarget(win PITRWindow, target time.Time) error

ValidatePITRTarget checks that target actually falls inside win before RunPITRRestore ever attempts anything against a real container. This is the one guard standing between an operator-chosen timestamp and a restore that hangs forever rather than fails cleanly: Postgres recovery given a recovery_target_time beyond what's actually archived does not error, it sits retrying restore_command for a WAL segment that will never arrive (see RecoverableWindowEnd's own doc comment), so this function's job is making sure that situation is never reached in the first place.

func ValidateSqlitePath added in v0.2.0

func ValidateSqlitePath(relPath string) error

ValidateSqlitePath rejects an empty, absolute, or volume-escaping path.

func ValidateVolumeName added in v0.2.0

func ValidateVolumeName(name string) error

ValidateVolumeName reports whether name is a legal new Docker volume name.

func WALPrefix added in v0.2.0

func WALPrefix(databaseName string) string

WALPrefix is the key prefix under which a database's archived WAL lives in its backup target.

func WipeVolume added in v0.2.0

func WipeVolume(ctx context.Context, rt docker.Runtime, name string) error

WipeVolume deletes every file in a volume, dotfiles included.

Types

type AppContainerResolver added in v0.2.0

type AppContainerResolver func(ctx context.Context, serviceName string) (docker.Runtime, string, error)

AppContainerResolver finds the node runtime and running container of an app.

type BaseBackupHistoryStore added in v0.2.0

type BaseBackupHistoryStore interface {
	TargetGetter
	StartBaseBackupHistory(ctx context.Context, h store.BaseBackupHistory) error
	FinishBaseBackupHistory(ctx context.Context, id, status string, sizeBytes int64, lsn, errMsg, finishedAt string) error
}

BaseBackupHistoryStore is the store surface BaseBackupRunner needs, narrowed the same way HistoryStore is for the logical-dump direction (runner.go).

type BaseBackupMaintainer added in v0.2.0

type BaseBackupMaintainer interface {
	Prune(ctx context.Context, databaseName, containerName string) error
}

BaseBackupMaintainer is the retention hook PITRMaintainer implements.

type BaseBackupRunner added in v0.2.0

type BaseBackupRunner struct {
	Store        BaseBackupHistoryStore
	Secrets      SecretsResolver
	BaseBackuper BaseBackuper
	Uploader     Uploader
	// Runtime, if set, backs StartLSN's best-effort diagnostic lookup
	// (base_backup.go). nil is valid: RunBaseBackup simply records an
	// empty LSN, the same "optional capability" shape this package
	// already uses for Runner.VolumeArchiver.
	Runtime Runtime
	// Maintainer, if set, prunes old base backups and archived WAL after each
	// successful base backup. Failures are logged, never returned: the new
	// backup already succeeded.
	Maintainer BaseBackupMaintainer
	// WAL, when set, ships the local WAL archive to the target right after a
	// successful base backup, so the bucket holds what a restore replays.
	WAL WALShipping
	// WorkDir/Now match Runner's own identically-named fields exactly
	// (runner.go's own doc comments for each apply here unchanged).
	WorkDir string
	Now     func() time.Time
}

BaseBackupRunner ties a BaseBackuper and an Uploader to store and internal/secrets, the physical-backup counterpart of Runner (runner.go): the only piece in this package that knows how a base backup attempt gets recorded and how a target's live credentials get resolved.

func (*BaseBackupRunner) RunBaseBackup added in v0.2.0

func (r *BaseBackupRunner) RunBaseBackup(ctx context.Context, historyID, databaseName, containerName, targetID string) error

RunBaseBackup takes a physical base backup of databaseName (running in containerName) and uploads it to targetID, recording the attempt in store.BaseBackupHistory throughout: the same running-then-succeeded- or-failed lifecycle RunBackup (runner.go) already establishes for a logical dump, applied to a physical one. historyID is minted by the caller, the same "generate the ID before the first write" convention RunBackup's own doc comment describes; expected to run in a goroutine the caller has already detached.

type BaseBackuper added in v0.2.0

type BaseBackuper interface {
	BaseBackup(ctx context.Context, containerName string) (io.ReadCloser, error)
}

BaseBackuper produces a physical base backup as a tar stream, for a single PITR-enabled Postgres database container already running under containerName: the physical counterpart of Dumper (dump.go), which produces a logical dump.

type BucketProtection added in v0.2.0

type BucketProtection struct {
	ObjectLock bool
	LockMode   string
	Versioning string
	CanDelete  bool
}

BucketProtection is what a bucket does to protect stored backups.

func (BucketProtection) Level added in v0.2.0

func (p BucketProtection) Level() string

Level classifies the protection.

func (BucketProtection) Warning added in v0.2.0

func (p BucketProtection) Warning() string

Warning returns operator-facing advice, or "" when the bucket is locked.

type CloneRestoreRunner

type CloneRestoreRunner struct {
	Store      CloneRestoreStore
	Secrets    SecretsResolver
	Downloader Downloader
	Restorer   Restorer
	// Now returns the current time, the same "field, not time.Now called
	// directly" reasoning RestoreRunner.Now's own doc comment gives.
	Now func() time.Time
	// ReadyTimeout/PollInterval override defaultCloneRestoreReadyTimeout/
	// defaultCloneRestoreReadyPollInterval when non-zero, so tests don't
	// have to wait out the real default.
	ReadyTimeout time.Duration
	PollInterval time.Duration
	// SettleTimeout overrides defaultCloneRestoreSettleTimeout when non-zero.
	SettleTimeout time.Duration
}

CloneRestoreRunner restores a previously succeeded backup into a brand-new database rather than overwriting the one it came from: the non-destructive counterpart to RestoreRunner. internal/api creates the new database's desired state itself (reusing handleCreateDatabase's own path, see database_clone_restore.go's own doc comment for why) before ever calling RunCloneRestore; this type's only job is to wait for that database's reconciler to bring it up and then run the identical download-and-apply RestoreRunner already implements.

func (*CloneRestoreRunner) RunCloneRestore

func (r *CloneRestoreRunner) RunCloneRestore(ctx context.Context, historyID, sourceDatabaseName, newDatabaseName, backupHistoryID, engine, containerName, controllerName string) error

RunCloneRestore waits for newDatabaseName (identified to the reconciler by controllerName, "database/" + newDatabaseName) to report Ready, then downloads and restores backupHistoryID into it, recording the attempt as store.CloneRestore throughout: a running row before any work starts, then succeeded or failed once it's done. sourceDatabaseName is recorded for history only; RunCloneRestore never reads from or writes to it. Expected to run in a goroutine the caller has already detached, the same convention RestoreRunner.RunRestore's own doc comment describes.

type CloneRestoreStore

type CloneRestoreStore interface {
	StartCloneRestore(ctx context.Context, h store.CloneRestore) error
	FinishCloneRestore(ctx context.Context, id, status, errMsg, finishedAt string) error
	GetConditions(ctx context.Context, controllerName string) ([]reconcile.Condition, error)
	// contains filtered or unexported methods
}

CloneRestoreStore is the narrow store surface CloneRestoreRunner needs: backupResolver's two methods to resolve the source backup, the clone- restore bookkeeping pair, and GetConditions to detect when the newly created database has actually come up. A separate interface from RestoreHistoryStore, the same "narrow interface per real dependency" reasoning that type's own doc comment gives, since the two share no method beyond the backupResolver pair.

type ContainerBaseBackuper added in v0.2.0

type ContainerBaseBackuper struct {
	Runtime Runtime
}

ContainerBaseBackuper is the real BaseBackuper: it runs pg_basebackup inside the already-running container via docker.Runtime.Exec, the same "no separate copy of the tool, no exposed port" reasoning ContainerDumper's own doc comment gives.

func (*ContainerBaseBackuper) BaseBackup added in v0.2.0

func (b *ContainerBaseBackuper) BaseBackup(ctx context.Context, containerName string) (io.ReadCloser, error)

BaseBackup implements BaseBackuper. -X none skips pg_basebackup's own WAL streaming: the database already archives WAL continuously via archive_command, so a restore replays from the archive, not from anything bundled into the backup itself. See ADR 016.

type ContainerDumper

type ContainerDumper struct {
	Runtime docker.Runtime
}

ContainerDumper is the real Dumper: it runs each engine's own native dump tool inside the already-running database container via docker.Runtime.Exec, so this process never needs its own copy of pg_dump/mysqldump/etc. and the database never needs a network port exposed anywhere reachable.

func (*ContainerDumper) Dump

func (d *ContainerDumper) Dump(ctx context.Context, engine, containerName string) (io.ReadCloser, error)

Dump implements Dumper.

type ContainerPITRRestorer added in v0.2.0

type ContainerPITRRestorer struct {
	Runtime docker.Runtime
}

ContainerPITRRestorer is the real PITRRestorer: the wipe-then-extract- then-configure sequence inside a short-lived helper container, the same createVolumeHelper mechanism ContainerVolumeRestorer uses.

func (*ContainerPITRRestorer) Restore added in v0.2.0

func (r *ContainerPITRRestorer) Restore(ctx context.Context, dataVolumeName string, baseBackup io.Reader, targetTime time.Time) error

Restore implements PITRRestorer.

type ContainerQuiescer added in v0.2.0

type ContainerQuiescer struct {
	Resolve AppContainerResolver
}

ContainerQuiescer is the real Quiescer: hooks run with sh -c inside the app's container, pause freezes it for the duration of the archive.

func (*ContainerQuiescer) Begin added in v0.2.0

func (q *ContainerQuiescer) Begin(ctx context.Context, serviceName string, p store.VolumeBackupPolicy) (func(context.Context) error, error)

Begin implements Quiescer.

type ContainerRestorer

type ContainerRestorer struct {
	Runtime docker.Runtime
	// ReadyTimeout/ReadyInterval override the engine readiness wait defaults.
	ReadyTimeout  time.Duration
	ReadyInterval time.Duration
}

ContainerRestorer is the real Restorer: it runs each engine's own native restore path inside the running container via docker.Runtime.ExecWithInput, ContainerDumper's own reasoning plus a stdin stream.

func (*ContainerRestorer) Restore

func (r *ContainerRestorer) Restore(ctx context.Context, engine, containerName string, dump io.Reader) error

Restore implements Restorer.

type ContainerSqliteSnapshotter added in v0.2.0

type ContainerSqliteSnapshotter struct {
	Runtime docker.Runtime
}

ContainerSqliteSnapshotter is the real SqliteSnapshotter: a helper container mounts the volume read-only and runs sqlite3's own ".backup" command, safe under the app's own concurrent writes since it goes through SQLite's file locking rather than a filesystem copy.

func (*ContainerSqliteSnapshotter) Snapshot added in v0.2.0

func (s *ContainerSqliteSnapshotter) Snapshot(ctx context.Context, volumeName, relPath string) (io.ReadCloser, error)

Snapshot implements SqliteSnapshotter.

type ContainerVolumeArchiver

type ContainerVolumeArchiver struct {
	Runtime docker.Runtime
}

ContainerVolumeArchiver is the real VolumeArchiver: it creates a helper container with volumeName mounted read-only, tars its contents to stdout, and removes the container once the caller has read the stream to completion (or given up partway through).

func (*ContainerVolumeArchiver) Archive

func (a *ContainerVolumeArchiver) Archive(ctx context.Context, volumeName string) (io.ReadCloser, error)

Archive implements VolumeArchiver.

type ContainerVolumeRestorer

type ContainerVolumeRestorer struct {
	Runtime docker.Runtime
}

ContainerVolumeRestorer is the real VolumeRestorer: it runs the wipe-then-extract command inside a short-lived helper container with volumeName mounted read-write.

func (*ContainerVolumeRestorer) Restore

func (r *ContainerVolumeRestorer) Restore(ctx context.Context, volumeName string, archive io.Reader) error

Restore implements VolumeRestorer.

type CoverageResource added in v0.2.0

type CoverageResource struct {
	Kind     string
	AppName  string
	Name     string
	Schedule string
	TargetID string
	// Scratch marks preview and other disposable data, left out of the
	// protected totals.
	Scratch bool
}

CoverageResource is one database or app volume that exists in desired state, whether or not anything backs it up.

type CoverageRow added in v0.2.0

type CoverageRow struct {
	Kind            string
	AppName         string
	Name            string
	Scratch         bool
	State           string
	Reason          string
	PrimaryAction   string
	Schedule        string
	TargetID        string
	NextRun         string
	LastBackupAt    string
	LastBackupID    string
	LastVerifiedAt  string
	TotalBytes      int64
	BackupCount     int
	Encrypted       bool
	ProtectionLevel string
}

CoverageRow is the derived protection state of one CoverageResource.

type CoverageSummary added in v0.2.0

type CoverageSummary struct {
	Total      int
	Protected  int
	Verified   int
	Unverified int
	NeverRan   int
	Failing    int
	// Unprotected counts resources with no schedule or no target.
	Unprotected         int
	Scratch             int
	StorageBytes        int64
	LastVerifiedRestore string
	NextRun             string
}

CoverageSummary aggregates the non-scratch rows.

type DatabaseInfo added in v0.2.0

type DatabaseInfo func(ctx context.Context, databaseName string) (engine, version string, err error)

DatabaseInfo resolves a database's engine and version for a drill.

type DeleteHistoryStore added in v0.2.0

type DeleteHistoryStore interface {
	GetBackupHistory(ctx context.Context, id string) (store.BackupHistory, error)
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
	DeleteBackupHistory(ctx context.Context, id string) error
}

DeleteHistoryStore is the store.DB surface DeleteRunner needs: GetBackupHistory/GetBackupTarget resolve where a backup attempt's object actually lives, the identical pair DownloadHistoryStore's own doc comment describes; DeleteBackupHistory removes the row itself once the underlying object delete has been attempted.

type DeleteRunner added in v0.2.0

type DeleteRunner struct {
	Store   DeleteHistoryStore
	Secrets SecretsResolver
	// Deleter may be nil, the same optional-capability shape
	// Scheduler.Deleter's own doc comment establishes: DeleteBackup then
	// only removes the store row, never attempting the object delete.
	Deleter Deleter
	Logger  *slog.Logger
}

DeleteRunner deletes one backup attempt on demand: an operator-directed counterpart to Scheduler's own retention-driven deletePrunedObjects, sharing the same "best-effort object delete, the store row is the thing that must always go" contract (Scheduler.deletePrunedObjects's own doc comment). Unlike that batch path, DeleteBackup always finishes by removing the row, regardless of whether the object delete succeeded, failed, or couldn't even be attempted: an operator who explicitly named this one archive to delete should not be left with it still showing up in history because deleting the underlying object hit a storage-side problem, on top of Deleter's own contract that a missing object is already a no-op success (backup.go's own Deleter doc comment).

func (*DeleteRunner) DeleteBackup added in v0.2.0

func (d *DeleteRunner) DeleteBackup(ctx context.Context, historyID string) error

DeleteBackup removes historyID's backup_history row and best-effort deletes the stored object behind it first. A target that no longer resolves, a secret that fails to resolve, or a Deleter call that itself fails is logged and does not stop the row from being deleted: the object is already unreachable or already gone either way, and an operator asking to delete an archive should not be blocked by a storage-side problem with the very thing they're trying to get rid of.

type Deleter

type Deleter interface {
	Delete(ctx context.Context, dest Destination, key string) error
}

Deleter removes a single object from a Destination, the inverse of Uploader. Implementations must treat deleting a key that is already gone (or never existed) as success, not an error: S3-compatible DeleteObject calls are naturally idempotent this way.

type Destination

type Destination struct {
	// Provider is one of store.BackupProviderAWS / R2 / Custom. An
	// Uploader implementation may use it to pick defaults (e.g. AWS's
	// per-region default endpoint), but Endpoint below is always the
	// final authority when non-empty.
	Provider string
	// Endpoint is the S3-compatible API base URL. Empty for
	// BackupProviderAWS (the SDK's own region-based resolver applies);
	// required for R2 and Custom.
	Endpoint string
	Region   string
	Bucket   string

	AccessKeyID     string
	SecretAccessKey string
}

Destination is everything an Uploader needs to reach a specific bucket, credentials included. Built fresh per call from a store.BackupTarget plus its resolved internal/secrets values (see store.BackupTargetSecretsKey): never persisted or logged as a whole, since AccessKeyID and SecretAccessKey are live credentials.

type DownloadHistoryStore

type DownloadHistoryStore interface {
	GetBackupHistory(ctx context.Context, id string) (store.BackupHistory, error)
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
}

DownloadHistoryStore is the store.DB surface DownloadRunner needs: GetBackupHistory to resolve the attempt a download names, GetBackupTarget to resolve where its object actually lives, the same pair RestoreHistoryStore's own doc comment describes for the restore direction. No Start/Finish pair here: unlike a backup or a restore, a download has no attempt of its own worth recording in a history table, it only reads an object an earlier, already-recorded backup wrote.

type DownloadRunner

type DownloadRunner struct {
	Store      DownloadHistoryStore
	Secrets    SecretsResolver
	Downloader Downloader
}

DownloadRunner resolves a succeeded backup attempt to its live object stream, the read-only counterpart of Runner (dump-and-upload) and RestoreRunner (download-and-restore-in-place): given a store.BackupHistory ID, it resolves the row's target and credentials the identical way Runner.runDumpAndUpload and RestoreRunner.runDownloadAndRestore already do, then hands the caller (internal/api's handleDownloadBackup) the raw object stream to copy directly into an HTTP response, never buffering it here.

func (*DownloadRunner) Download

func (d *DownloadRunner) Download(ctx context.Context, historyID string) (io.ReadCloser, error)

Download resolves historyID's backup attempt to its object stream. Refuses (without ever resolving credentials or contacting the bucket) a backup whose Status is not store.BackupStatusSucceeded, the same safety property RunRestore's own doc comment describes for the restore path: a running attempt has no complete object at its recorded key yet, and a failed one may have none, or a partial one. internal/api's handleDownloadBackup does the identical check before ever calling this; this is the defense-in-depth backstop for a caller that doesn't.

type Downloader

type Downloader interface {
	Download(ctx context.Context, dest Destination, key string) (io.ReadCloser, error)
}

Downloader retrieves a single object from a Destination, the exact reverse of Uploader: given the same (Destination, key) an earlier Upload call was made with, it returns that object's bytes as a stream. Implementations are not required to know the object's size up front (see S3Downloader's own doc comment for why GetObject already makes this a non-issue, unlike Upload's own unknown-size handling), only to stream it back faithfully.

type DrillRunner added in v0.2.0

type DrillRunner struct {
	Store      DrillStore
	Secrets    SecretsResolver
	Downloader Downloader
	Prober     ObjectProber
	Identities []age.Identity

	// Volume drills.
	Restorer VolumeRestorer
	Archiver VolumeArchiver
	Volumes  VolumeCreator
	Remover  VolumeRemover

	// Database drills. Any nil leaves databases at integrity-only checks.
	Runtime    docker.Runtime
	DBRestorer Restorer
	DBInfo     DatabaseInfo
	ImageFor   func(engine, version string) string

	// OnFailure is called once for every failed drill.
	OnFailure func(ctx context.Context, d store.BackupDrill)
	Logger    *slog.Logger
	Now       func() time.Time
}

DrillRunner restores a stored backup into a scratch resource, validates it and destroys the scratch resource, recording every step.

func (*DrillRunner) RunDrill added in v0.2.0

func (r *DrillRunner) RunDrill(ctx context.Context, drillID, backupHistoryID, trigger string) (store.BackupDrill, error)

RunDrill executes one drill of backupHistoryID. The drill row is always finished; the returned drill carries the outcome. A failed drill returns a non-nil error describing it.

type DrillScheduler added in v0.2.0

type DrillScheduler struct {
	Store      DrillSchedulerStore
	Secrets    SecretsResolver
	Runner     *DrillRunner
	Prober     ObjectProber
	Downloader Downloader
	Protection ProtectionProber
	// Interval is the drill period per resource; zero disables drills.
	Interval time.Duration
	// StaleAfter is the age a running backup must reach before the sweep
	// finalizes it.
	StaleAfter time.Duration
	Logger     *slog.Logger
	Now        func() time.Time
	// contains filtered or unexported fields
}

DrillScheduler drills every backed up resource on an interval, finalizes backups the control plane lost track of, and refreshes bucket protection probes.

func (*DrillScheduler) Run added in v0.2.0

func (s *DrillScheduler) Run(ctx context.Context, every time.Duration) error

Run ticks until ctx is done.

func (*DrillScheduler) Tick added in v0.2.0

func (s *DrillScheduler) Tick(ctx context.Context) error

Tick runs one scheduling pass.

type DrillSchedulerStore added in v0.2.0

type DrillSchedulerStore interface {
	ListSucceededBackups(ctx context.Context) ([]store.BackupHistory, error)
	ListBackupDrills(ctx context.Context, serviceName, volumeName, databaseName string, limit int) ([]store.BackupDrill, error)
	ListRunningBackups(ctx context.Context, startedBefore time.Time) ([]store.BackupHistory, error)
	FinishBackupHistory(ctx context.Context, id, status string, sizeBytes int64, checksum, errMsg, finishedAt string) error
	ListBackupTargets(ctx context.Context) ([]store.BackupTarget, error)
	SetBackupTargetProtection(ctx context.Context, p store.BackupTargetProtection) error
	// contains filtered or unexported methods
}

DrillSchedulerStore is the store surface the drill scheduler needs.

type DrillStore added in v0.2.0

type DrillStore interface {
	StartBackupDrill(ctx context.Context, d store.BackupDrill) error
	FinishBackupDrill(ctx context.Context, d store.BackupDrill) error
	// contains filtered or unexported methods
}

DrillStore is the store surface a drill needs.

type DrillSummary added in v0.2.0

type DrillSummary struct {
	ID      string
	At      string
	Status  string
	Stage   string
	Error   string
	Trigger string
	Files   int64
}

DrillSummary is one drill in a health row.

type Dumper

type Dumper interface {
	Dump(ctx context.Context, engine, containerName string) (io.ReadCloser, error)
}

Dumper produces a database dump as a stream, for a single managed database container already running under containerName (the same name application.ContainerName / the database controller's own container-naming convention already produces; Dumper does not construct or look this up itself, Runner does).

The returned ReadCloser's Close must be called by every caller once done, success or failure, the same contract io.ReadCloser always carries; a Dumper backed by a subprocess or a docker exec stream depends on Close to release that underlying resource.

type EncryptionStatus added in v0.2.0

type EncryptionStatus struct {
	Enabled   bool
	Recipient string
	Codec     string
}

EncryptionStatus describes client side encryption of new volume backups.

type FailedDrill added in v0.2.0

type FailedDrill struct {
	Resource string
	Reason   string
	At       string
}

FailedDrill is a resource whose latest drill failed.

type GFSStore added in v0.2.0

type GFSStore interface {
	ListSucceededVolumeBackups(ctx context.Context, serviceName, volumeName string) ([]store.BackupHistory, error)
	DeleteBackupHistory(ctx context.Context, id string) error
}

GFSStore is the store surface grandfather-father-son pruning needs.

type HealthBackup added in v0.2.0

type HealthBackup struct {
	ID        string
	At        string
	Status    string
	SizeBytes int64
	Codec     string
	Error     string
}

HealthBackup is one backup in a health row.

type HistoryStore

type HistoryStore interface {
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
	StartBackupHistory(ctx context.Context, h store.BackupHistory) error
	FinishBackupHistory(ctx context.Context, id, status string, sizeBytes int64, checksum, errMsg, finishedAt string) error
}

HistoryStore is the store.DB surface Runner needs, narrowed the same way SecretsResolver is: a fake satisfying six methods is a lot less to stub than the full *store.DB.

type Lister added in v0.2.0

type Lister interface {
	List(ctx context.Context, dest Destination, prefix string) ([]RemoteObject, error)
}

Lister lists the objects under a key prefix in a backup target.

type MajorUpgradeRunner added in v0.2.0

type MajorUpgradeRunner struct {
	Store    MajorUpgradeStore
	Runtime  VolumeRuntime
	Dumper   Dumper
	Restorer Restorer
	Nudge    func()
	Now      func() time.Time
	WorkDir  string
	Logger   *slog.Logger
	// Wait bounds each wait for a container to stop, start or accept
	// connections; Poll is the interval between checks.
	Wait time.Duration
	Poll time.Duration
}

MajorUpgradeRunner upgrades a managed Postgres database across a major version by dump and restore. The old data volume is never rewritten in place: it is copied to a snapshot volume first, the dump is taken from the snapshot, and any failure after the live volume is wiped restores the snapshot and the old version.

func (*MajorUpgradeRunner) DiscardSnapshot added in v0.2.0

func (r *MajorUpgradeRunner) DiscardSnapshot(ctx context.Context, upgradeID string) error

DiscardSnapshot deletes an upgrade's rollback snapshot volume.

func (*MajorUpgradeRunner) RecoverInterrupted added in v0.2.0

func (r *MajorUpgradeRunner) RecoverInterrupted(ctx context.Context)

RecoverInterrupted settles upgrades left running by a control plane restart: before the wipe it unsuspends the database, after it it rolls back.

func (*MajorUpgradeRunner) Rollback added in v0.2.0

func (r *MajorUpgradeRunner) Rollback(ctx context.Context, upgradeID string) error

Rollback undoes a finished upgrade by restoring its snapshot. Writes made since the upgrade are lost.

func (*MajorUpgradeRunner) RunMajorUpgrade added in v0.2.0

func (r *MajorUpgradeRunner) RunMajorUpgrade(ctx context.Context, historyID, databaseName, toVersion string) error

RunMajorUpgrade runs the upgrade of databaseName to toVersion and records every phase under historyID. It returns the failure, if any, after the rollback has run.

type MajorUpgradeStore added in v0.2.0

type MajorUpgradeStore interface {
	GetDesiredDatabase(ctx context.Context, name string) (*store.DesiredDatabase, error)
	SaveDesiredDatabase(ctx context.Context, d store.DesiredDatabase) error
	UpdateDatabaseSuspended(ctx context.Context, name string, suspended bool) error
	StartMajorUpgrade(ctx context.Context, u store.MajorUpgrade) error
	UpdateMajorUpgradePhase(ctx context.Context, id, phase, snapshotVolume string) error
	FinishMajorUpgrade(ctx context.Context, id, status, errMsg, finishedAt string) error
	GetMajorUpgrade(ctx context.Context, id string) (store.MajorUpgrade, error)
	ListRunningMajorUpgrades(ctx context.Context) ([]store.MajorUpgrade, error)
	ClearMajorUpgradeSnapshot(ctx context.Context, id string) error
}

MajorUpgradeStore is the store surface MajorUpgradeRunner needs.

type ObjectInfo added in v0.2.0

type ObjectInfo struct {
	Exists bool
	Size   int64
}

ObjectInfo is what a HEAD request learned about a stored backup object.

type ObjectProber added in v0.2.0

type ObjectProber interface {
	Head(ctx context.Context, dest Destination, key string) (ObjectInfo, error)
}

ObjectProber checks an object without downloading it.

type PITRHistoryStore added in v0.2.0

type PITRHistoryStore interface {
	TargetGetter
	GetBaseBackupHistory(ctx context.Context, id string) (store.BaseBackupHistory, error)
	GetOldestSucceededBaseBackup(ctx context.Context, databaseName string) (store.BaseBackupHistory, bool, error)
	StartPITRRestoreHistory(ctx context.Context, h store.PITRRestoreHistory) error
	FinishPITRRestoreHistory(ctx context.Context, id, status, errMsg, finishedAt string) error
	UpdateDatabaseSuspended(ctx context.Context, name string, suspended bool) error
}

PITRHistoryStore is the store surface PITRRunner needs. See ADR 016 for why UpdateDatabaseSuspended is part of a restore's own store surface rather than a separate concern.

type PITRMaintainer added in v0.2.0

type PITRMaintainer struct {
	Store   PITRMaintenanceStore
	Resolve func(ctx context.Context, targetID string) (Destination, error)
	Deleter Deleter
	// Lister enables pruning the shipped WAL copy in the bucket too.
	Lister  Lister
	Runtime Runtime
	// Keep overrides the env default when positive.
	Keep   int
	Logger *slog.Logger
}

PITRMaintainer keeps a PITR-enabled database's disk and bucket bounded: without it the local WAL archive and the base backups grow forever.

func (*PITRMaintainer) Prune added in v0.2.0

func (m *PITRMaintainer) Prune(ctx context.Context, databaseName, containerName string) error

Prune deletes succeeded base backups beyond the newest keep(), then the archived WAL older than the oldest base backup that remains: a restore can never start before that backup, so earlier segments are dead weight.

type PITRMaintenanceStore added in v0.2.0

type PITRMaintenanceStore interface {
	ListBaseBackupHistory(ctx context.Context, databaseName string) ([]store.BaseBackupHistory, error)
	DeleteBaseBackupHistory(ctx context.Context, id string) error
}

PITRMaintenanceStore is the store surface PITRMaintainer needs.

type PITRRestorer added in v0.2.0

type PITRRestorer interface {
	Restore(ctx context.Context, dataVolumeName string, baseBackup io.Reader, targetTime time.Time) error
}

PITRRestorer applies a base backup to dataVolumeName in place, configuring Postgres to replay WAL to targetTime and promote: the point-in-time-restore counterpart of Restorer. Unlike Restorer, this never touches a running container; dataVolumeName's container must already be stopped (PITRRunner's job).

type PITRRunner added in v0.2.0

type PITRRunner struct {
	Store      PITRHistoryStore
	Secrets    SecretsResolver
	Downloader Downloader
	Restorer   PITRRestorer
	// WAL and WALVolume, when set, pull the WAL shipped to the target into the
	// database's archive volume before the destructive phase, so a restore works
	// after the local archive is lost. WALVolume maps a database name to it.
	WAL       WALFetching
	WALVolume func(databaseName string) string
	// Runtime backs the teardown/startup/promotion polls: Suspended and
	// Nudge only ever request a change, this verifies it happened.
	Runtime docker.Runtime
	// Nudge requests an immediate reconcile pass (*reconcile.Engine's
	// own Nudge); nil is valid, falling back to the resync cadence.
	Nudge func()
	// Now returns the current time for history timestamps; nil falls
	// back to time.Now.
	Now func() time.Time
	// TeardownTimeout/StartupTimeout/PromoteTimeout/PollInterval override
	// the package defaults above when non-zero, for tests.
	TeardownTimeout time.Duration
	StartupTimeout  time.Duration
	PromoteTimeout  time.Duration
	PollInterval    time.Duration
}

PITRRunner orchestrates one point-in-time restore end to end: download the named base backup, hand it to a PITRRestorer to wipe and re-populate the database's data volume with recovery configured for targetTime, then wait for Postgres to actually promote before calling the attempt succeeded. Unlike RestoreRunner, this needs the container down for the whole window; see ADR 016 for why Suspended is what keeps the reconciler from racing that, and why promotion is polled directly rather than trusted to the reconciler's Ready condition.

func (*PITRRunner) RunPITRRestore added in v0.2.0

func (r *PITRRunner) RunPITRRestore(ctx context.Context, historyID, databaseName, containerName, dataVolumeName, baseBackupHistoryID string, targetTime time.Time) error

RunPITRRestore restores databaseName to targetTime from the physical base backup baseBackupHistoryID, recording a running-then-succeeded- or-failed store.PITRRestoreHistory row. targetTime is assumed already validated by the caller (ValidateTarget). Expected to run in a goroutine the caller has already detached.

func (*PITRRunner) ValidateTarget added in v0.2.0

func (r *PITRRunner) ValidateTarget(ctx context.Context, databaseName, containerName string, target time.Time) error

ValidateTarget checks target falls inside databaseName's current window, the synchronous check RunPITRRestore itself never repeats.

func (*PITRRunner) Window added in v0.2.0

func (r *PITRRunner) Window(ctx context.Context, databaseName, containerName string) (PITRWindow, error)

Window computes databaseName's current PITRWindow: the oldest succeeded base backup plus how far WAL archiving has actually, provably reached right now (RecoverableWindowEnd). Returns ErrNoBaseBackup without ever reaching containerName when there is no succeeded base backup yet.

func (*PITRRunner) WindowBounds added in v0.2.0

func (r *PITRRunner) WindowBounds(ctx context.Context, databaseName, containerName string) (hasBaseBackup bool, start, end time.Time, err error)

WindowBounds is Window with its result destructured into plain values, since an interface method's return type must be identical across the internal/api package boundary (see pitr.go).

type PITRWindow added in v0.2.0

type PITRWindow struct {
	HasBaseBackup bool
	Start         time.Time
	End           time.Time
}

PITRWindow is one database's currently recoverable point-in-time restore range: Start is the oldest succeeded base backup's own started_at (nothing earlier is recoverable, since there is no base to replay WAL forward from), End is the latest instant WAL archiving has continuously and provably reached (RecoverableWindowEnd). HasBaseBackup is false whenever no base backup has ever succeeded yet, in which case Start/End are both zero and carry no meaning.

type Protection added in v0.2.0

type Protection struct {
	Store  ProtectionStore
	Drills *DrillRunner
	Prober ProtectionProber
	// Secrets resolves target credentials for probes and restores.
	Secrets    SecretsResolver
	Downloader Downloader
	// RestoreStore records restore-to-new-volume attempts.
	RestoreStore VolumeCloneRestoreStore
	Sealer       *Sealer
	// DrillInterval is the configured drill period, used to flag a resource
	// whose last verified restore is older than two periods.
	DrillInterval time.Duration
	Logger        *slog.Logger
	Now           func() time.Time
}

Protection reports backup health and starts drills.

func (*Protection) Encryption added in v0.2.0

func (p *Protection) Encryption() EncryptionStatus

Encryption reports whether new volume backups are encrypted.

func (*Protection) FailedDrills added in v0.2.0

func (p *Protection) FailedDrills(ctx context.Context) ([]FailedDrill, error)

FailedDrills returns one entry per resource whose most recent finished drill failed. A later passing drill clears it.

func (*Protection) Health added in v0.2.0

func (p *Protection) Health(ctx context.Context) ([]ResourceHealth, error)

Health builds one row per backed up or scheduled resource.

func (*Protection) RefreshProtection added in v0.2.0

func (p *Protection) RefreshProtection(ctx context.Context, targetIDs []string) ([]store.BackupTargetProtection, error)

RefreshProtection re-probes every named target's bucket (all when targetIDs is empty) and returns the stored results.

func (*Protection) RestoreVolumeTo added in v0.2.0

func (p *Protection) RestoreVolumeTo(ctx context.Context, rtm docker.Runtime, historyID, sourceService, sourceVolume, newVolume, backupID string) error

RestoreVolumeTo restores a volume backup into a brand-new volume on the node rtm drives. It never writes to an existing volume: the caller checks VolumeExists first.

func (*Protection) StartDrill added in v0.2.0

func (p *Protection) StartDrill(ctx context.Context, backupID string) (string, error)

StartDrill validates backupID and runs a manual drill in the background, returning the drill ID to poll.

func (*Protection) VolumeExists added in v0.2.0

func (p *Protection) VolumeExists(ctx context.Context, rtm docker.Runtime, name string) (exists, known bool, err error)

VolumeExists reports whether a volume of that name is already on the runtime's node. known is false when the node cannot list volumes, in which case only a freshly generated name may be used.

type ProtectionProber added in v0.2.0

type ProtectionProber interface {
	Probe(ctx context.Context, dest Destination) (BucketProtection, error)
}

ProtectionProber inspects a bucket's protection settings.

type ProtectionRecorder added in v0.2.0

type ProtectionRecorder interface {
	SetBackupTargetProtection(ctx context.Context, p store.BackupTargetProtection) error
	// contains filtered or unexported methods
}

ProtectionRecorder is the slice of the store RecordProtection writes to.

type ProtectionStore added in v0.2.0

type ProtectionStore interface {
	ListSucceededBackups(ctx context.Context) ([]store.BackupHistory, error)
	ListAllBackupHistory(ctx context.Context, limit int, before *time.Time) ([]store.BackupHistory, error)
	ListScheduledServiceVolumes(ctx context.Context) ([]store.ServiceVolumeBackupConfig, error)
	ListScheduledDatabases(ctx context.Context) ([]store.DesiredDatabase, error)
	ListBackupDrills(ctx context.Context, serviceName, volumeName, databaseName string, limit int) ([]store.BackupDrill, error)
	ListBackupTargetProtection(ctx context.Context) ([]store.BackupTargetProtection, error)
	SetBackupTargetProtection(ctx context.Context, p store.BackupTargetProtection) error
	GetVolumeBackupPolicy(ctx context.Context, serviceName, volumeName string) (store.VolumeBackupPolicy, error)
	SetVolumeBackupPolicy(ctx context.Context, p store.VolumeBackupPolicy) error
	// contains filtered or unexported methods
}

ProtectionStore is the store surface the protection service reads and writes.

type Quiescer added in v0.2.0

type Quiescer interface {
	Begin(ctx context.Context, serviceName string, p store.VolumeBackupPolicy) (end func(context.Context) error, err error)
}

Quiescer makes a volume consistent for the duration of its archive. It runs the policy's pre-hook and optionally pauses the app. The returned end function undoes the pause and runs the post-hook.

type RemoteObject added in v0.2.0

type RemoteObject struct {
	Key  string
	Size int64
}

RemoteObject is one object found under a bucket prefix.

type ResourceHealth added in v0.2.0

type ResourceHealth struct {
	Kind         string
	AppName      string
	ResourceName string
	TargetID     string
	Schedule     string
	NextRun      string
	BackupCount  int
	TotalBytes   int64
	LastAttempt  *HealthBackup
	LastBackup   *HealthBackup
	// LastVerifiedRestore is the newest passed drill: proof a restore
	// worked, not just that a file exists.
	LastVerifiedRestore *DrillSummary
	LastDrill           *DrillSummary
	// Encrypted is true when the newest backup is client side encrypted.
	Encrypted       bool
	State           string
	StateReason     string
	ProtectionLevel string
	Warning         string
}

ResourceHealth is the protection state of one database or app volume.

type RestoreHistoryStore

type RestoreHistoryStore interface {
	GetBackupHistory(ctx context.Context, id string) (store.BackupHistory, error)
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
	StartRestoreHistory(ctx context.Context, h store.RestoreHistory) error
	FinishRestoreHistory(ctx context.Context, id, status, errMsg, finishedAt string) error
}

RestoreHistoryStore is the store.DB surface RestoreRunner needs, narrowed the same way Runner's own HistoryStore is: GetBackupHistory to resolve the source backup a restore names, GetBackupTarget to resolve where that backup's object actually lives, and StartRestoreHistory/FinishRestoreHistory to record the restore attempt itself. Deliberately not HistoryStore plus new methods bolted on: a fake satisfying HistoryStore's forward-backup methods has no reason to also implement restore's, and the reverse is equally true, so keeping them as two separate narrow interfaces (which happen to overlap on GetBackupTarget, satisfied by the same *store.DB either way) matches this codebase's own "narrow interface per real dependency, not one grab-bag interface per package" convention.

type RestoreRunner

type RestoreRunner struct {
	Store      RestoreHistoryStore
	Secrets    SecretsResolver
	Downloader Downloader
	Restorer   Restorer
	// VolumeRestorer backs RunVolumeRestore the same way Restorer backs
	// RunRestore. nil is valid, the same "optional capability" shape
	// Runner.VolumeArchiver's own doc comment establishes.
	VolumeRestorer VolumeRestorer
	// Identities decrypt sealed volume backups; empty is valid for raw ones.
	Identities []age.Identity
	// WorkDir is RunRestore/RunVolumeRestore's disk-space preflight
	// target, matching Runner.WorkDir's own fallback-to-APP_DATA_DIR,
	// skip-if-still-empty behavior.
	WorkDir string
	// Now returns the current time, matching Runner.Now's own "field, not
	// time.Now called directly" reasoning for deterministic tests.
	Now func() time.Time
}

RestoreRunner ties a Downloader and a Restorer to store and internal/secrets, the restore counterpart of Runner: the only piece in this package that knows how a restore attempt gets recorded and how a target's live credentials get resolved. internal/api constructs one real RestoreRunner at startup, the same place it constructs Runner, and calls RunRestore per triggered restore.

A separate type from Runner rather than a second method on it: Runner is already a coherent "dump this, upload it" unit with its own field set (Dumper, Uploader); RestoreRunner's fields (Downloader, Restorer) point the opposite direction and share nothing with Runner's except Store's underlying *store.DB value and Secrets' underlying *secrets.Manager value, both already expressed as separate narrow interfaces above and in runner.go rather than a shared struct field, so there is no real duplication to unify by merging the two types, only two independently-testable units that happen to be wired from the same concrete dependencies at startup.

func (*RestoreRunner) RunRestore

func (r *RestoreRunner) RunRestore(ctx context.Context, historyID, databaseName, backupHistoryID, engine, containerName string) error

RunRestore downloads the object recorded by backupHistoryID and applies it to containerName (running engine) as databaseName's new contents, recording the attempt in store.RestoreHistory throughout: a running row before any work starts, then succeeded or failed once it's done. historyID is minted by the caller (internal/api), the same "generate the ID before the first write" convention RunBackup's own doc comment describes; RunRestore is expected to run in a goroutine the caller has already detached, for the same reason RunBackup is, see internal/api's handleTriggerRestore.

RunRestore refuses to proceed, failing the history row rather than attempting anything, if the named backup's own Status is not store.BackupStatusSucceeded: restoring from a backup that was still running (and so has no complete, consistent object at its recorded key yet) or that already failed (and so may have no object there at all, or a partial one) is not a smaller version of a real restore, it is a data-loss-shaped mistake this function refuses to make even if a caller's own validation (internal/api's handleTriggerRestore does the identical check before ever calling this, so a caller going through the ordinary API path never reaches this branch) somehow let it through.

func (*RestoreRunner) RunVolumeRestore

func (r *RestoreRunner) RunVolumeRestore(ctx context.Context, historyID, serviceName, volumeName, dockerVolumeName, backupHistoryID string) error

RunVolumeRestore downloads the object recorded by backupHistoryID and applies it to dockerVolumeName (serviceName's Docker volume backing its volumeName-named app.yaml volume) as its new, complete contents, recording the attempt in store.RestoreHistory exactly the way RunRestore does for a database. See RunRestore's own doc comment for the full reasoning this mirrors, including the identical refusal to proceed against a backup whose Status isn't store.BackupStatusSucceeded.

type Restorer

type Restorer interface {
	Restore(ctx context.Context, engine, containerName string, dump io.Reader) error
}

Restorer applies a previously taken dump back onto a live database container. Like Dumper, it takes no credentials beyond what the container's own environment already provides. Its contract is "the restore command exited zero," not "and the data was verified correct afterward."

type RetentionItem added in v0.2.0

type RetentionItem struct {
	ID string
	At time.Time
}

RetentionItem is one backup candidate for retention.

type RetentionPolicy added in v0.2.0

type RetentionPolicy struct {
	Daily   int
	Weekly  int
	Monthly int
}

RetentionPolicy keeps the newest backup of each of the last Daily days, Weekly ISO weeks and Monthly months that have one. A zero policy keeps everything (it expires nothing).

func (RetentionPolicy) Active added in v0.2.0

func (p RetentionPolicy) Active() bool

Active reports whether any bucket count is set.

type Runner

type Runner struct {
	Store    HistoryStore
	Secrets  SecretsResolver
	Dumper   Dumper
	Uploader Uploader
	// VolumeArchiver backs RunVolumeBackup the same way Dumper backs
	// RunBackup. nil is valid: a control plane with no volume-backup
	// callers configured (today, none do this by default) simply never
	// calls RunVolumeBackup, the same "optional capability" shape this
	// codebase already uses for Scheduler.Deleter/Verifier.
	VolumeArchiver VolumeArchiver
	// SqliteSnapshotter backs RunVolumeBackup when a volume's sqlite_path
	// is set; nil is valid the same way VolumeArchiver's nil is.
	SqliteSnapshotter SqliteSnapshotter
	// WorkDir is the directory RunBackup/RunVolumeBackup's disk-space
	// preflight checks (checkDiskSpace) before starting. Empty falls back
	// to APP_DATA_DIR (resolveWorkDir), and empty even then skips the
	// check entirely: see checkDiskSpace's own doc comment for why a
	// backup, which streams straight from its source to its destination
	// bucket, still has a real local directory worth guarding.
	WorkDir string
	// Volume holds the optional sealed-backup behavior: compression,
	// encryption, hooks, budgets. nil keeps the raw legacy encoding.
	Volume *VolumeBackupOptions
	// Now returns the current time. A field, not time.Now called
	// directly, so tests get deterministic timestamps without a real
	// clock dependency; production code leaves it nil and RunBackup
	// falls back to time.Now.
	Now func() time.Time
}

Runner ties a Dumper and an Uploader to store and internal/secrets: the only piece in this package that knows how a backup attempt gets recorded and how a target's live credentials get resolved. internal/api constructs one real Runner at startup (the same "wire concrete implementations once, in cmd/levelrail or internal/api's own constructor" pattern every other cross-package dependency in this codebase already follows) and calls RunBackup per triggered backup.

func (*Runner) ResolveDestination

func (r *Runner) ResolveDestination(ctx context.Context, targetID string) (Destination, error)

ResolveDestination resolves targetID's stored config and live secrets into a Destination, the same lookup runDumpAndUpload does before every Upload call. Exported so Scheduler can resolve the identical Destination for a Deleter when pruning old backups for the same target.

func (*Runner) RunBackup

func (r *Runner) RunBackup(ctx context.Context, historyID, databaseName, engine, containerName, targetID string) error

RunBackup dumps databaseName (running in containerName under engine) and uploads the result to targetID, recording the attempt in store.BackupHistory throughout: a running row before any work starts, then succeeded or failed once it's done. historyID is minted by the caller (internal/api, the same "generate the ID before the first write" convention store.BackupTarget's own doc comment describes) so the caller can return it in an HTTP response immediately without waiting for the backup to finish; RunBackup itself is expected to run in a goroutine the caller has already detached; see internal/api's handleTriggerBackup for how it starts, and this file's own tests for the exact happy-path and every-failure-point sequencing this describes.

Every failure path (target not found, secret resolution, dump, upload) finishes the history row as BackupStatusFailed with a real error message rather than leaving it stuck at BackupStatusRunning: the one exception is a failure to even write FinishBackupHistory itself, which RunBackup can't recover from and returns as-is, the same "give up honestly rather than mask it" the reconcilers elsewhere in this codebase already follow when their own condition-write fails.

func (*Runner) RunVolumeBackup

func (r *Runner) RunVolumeBackup(ctx context.Context, historyID, serviceName, volumeName, dockerVolumeName, targetID, sqlitePath string) error

RunVolumeBackup archives dockerVolumeName (serviceName's Docker volume backing its volumeName-named app.yaml volume) and uploads the result to targetID, recording the attempt in store.BackupHistory exactly the way RunBackup does for a database: the same running-then-succeeded-or- failed row, the same historyID-minted-by-the-caller contract, the same expectation of running in a goroutine the caller has already detached. See RunBackup's own doc comment for the reasoning this mirrors in full; this only differs in what it dumps (VolumeArchiver.Archive instead of Dumper.Dump) and what identifies the row (ServiceName/VolumeName/ResourceKind instead of DatabaseName).

type Runtime added in v0.2.0

type Runtime interface {
	Exec(ctx context.Context, containerID string, cmd []string) (io.ReadCloser, error)
	ExecWithInput(ctx context.Context, containerID string, cmd []string, stdin io.Reader) (io.ReadCloser, error)
}

Runtime is the narrow docker.Runtime surface (Exec/ExecWithInput) this file's functions and ContainerPITRRestorer need.

type S3Deleter

type S3Deleter struct{}

S3Deleter is the real Deleter, covering AWS/R2/custom endpoints the same way S3Uploader does, reusing newS3Client (uploader.go) rather than building its own client.

func (S3Deleter) Delete

func (S3Deleter) Delete(ctx context.Context, dest Destination, key string) error

Delete removes key from dest.Bucket. A DeleteObject call against a key that is already gone (or never existed) returns success, not an error, the same way real S3-compatible services answer it: no existence check happens here on purpose, see Deleter's own doc comment.

type S3Downloader

type S3Downloader struct{}

S3Downloader is the real Downloader: the download side of the same AWS/R2/custom-endpoint story S3Uploader already covers, one implementation for every provider this codebase supports, driven by the same Destination.Endpoint-is-the-final-authority rule.

Unlike S3Uploader, this has no multipart-vs-single-PUT decision to make: S3's GetObject API already streams a response body of whatever size the object actually is, with no equivalent to PutObject's Content-Length requirement on the request side, so there's no unknown-size problem here to work around with an upload-manager-style abstraction. A plain client.GetObject call is the entire implementation.

func (S3Downloader) Download

func (S3Downloader) Download(ctx context.Context, dest Destination, key string) (io.ReadCloser, error)

Download implements Downloader. The returned ReadCloser is GetObjectOutput.Body itself, not copied into a buffer first: a database dump can be large, and RunRestore already has to stream it straight into ContainerRestorer.Restore's stdin (see restore.go), so buffering the whole object in this process's memory first would cost real memory for no benefit; Close on the returned value releases the underlying HTTP response exactly the same way Dumper's own ReadCloser contract already requires callers to honor.

type S3Lister added in v0.2.0

type S3Lister struct{}

S3Lister is the real Lister, over ListObjectsV2.

func (S3Lister) List added in v0.2.0

func (S3Lister) List(ctx context.Context, dest Destination, prefix string) ([]RemoteObject, error)

List implements Lister, following pagination to the end.

type S3Prober added in v0.2.0

type S3Prober struct{}

S3Prober is the real ObjectProber.

func (S3Prober) Head added in v0.2.0

func (S3Prober) Head(ctx context.Context, dest Destination, key string) (ObjectInfo, error)

Head implements ObjectProber.

type S3ProtectionProber added in v0.2.0

type S3ProtectionProber struct{}

S3ProtectionProber is the real ProtectionProber. It writes and deletes one small canary object to learn whether the key can delete.

func (S3ProtectionProber) Probe added in v0.2.0

Probe implements ProtectionProber.

type S3Tester

type S3Tester struct{}

S3Tester is the real Tester, covering AWS/R2/custom endpoints the same way S3Uploader does, reusing newS3Client rather than building its own client.

func (S3Tester) Test

func (S3Tester) Test(ctx context.Context, dest Destination) error

Test issues a HeadBucket call against dest.Bucket: the cheapest S3 operation that proves both the endpoint is reachable and the credentials are accepted, without reading or writing any object.

type S3Uploader

type S3Uploader struct{}

S3Uploader is the real Uploader: every backup target this codebase supports (AWS, R2, and any other S3-compatible custom endpoint) speaks the same REST API, so one implementation covers all three, driven entirely by the Provider/Endpoint distinction already captured on Destination.

A new S3 client is built per Upload call rather than cached on the struct: Destination carries live, per-target credentials resolved fresh from internal/secrets on every backup run (see Runner), and a target's bucket, region, or endpoint can change between runs too. Backups are infrequent (the default schedule is once a day, see store.BackupTarget's own docs), so the cost of rebuilding a client each time is not worth the complexity of a cache that would need invalidating whenever a target's settings change.

func (S3Uploader) Upload

func (S3Uploader) Upload(ctx context.Context, dest Destination, key string, r io.Reader, _ int64) error

Upload writes r to dest.Bucket under key using the AWS SDK's S3 upload manager rather than a raw PutObject call, because size is frequently -1 here: Runner streams a database dump straight from a running pg_dump/mysqldump process into Upload while the dump is still in flight, so there is no Content-Length to give PutObject and no seekable body to compute one from. The upload manager handles this by buffering r into fixed-size parts and issuing an S3 multipart upload, which needs no upfront length. The size parameter goes unused (named _ below, the same convention this package's own test fakes already use for it): it is a hint for implementations that can act on it (see backup.go's own doc comment on Uploader), and this one cannot, since the manager sizes itself from r as it streams regardless of what is passed.

This intentionally uses feature/s3/manager rather than its replacement, feature/s3/transfermanager: as of this writing transfermanager's UploadObject takes a Body that must support Len() or Seek() to size itself, which a streaming pg_dump/mysqldump pipe cannot provide, and its own maintainers describe the unknown-size streaming case as unsupported. feature/s3/manager is marked deprecated in its doc comments but not removed or broken; it is the only one of the two that actually does what Runner needs. Revisit this once transfermanager grows real support for an unbounded io.Reader body.

type ScheduleStore

type ScheduleStore interface {
	ListScheduledDatabases(ctx context.Context) ([]store.DesiredDatabase, error)
	PruneBackupHistory(ctx context.Context, databaseName string, keep int, olderThan time.Time) ([]store.PrunedBackup, error)
	// ListScheduledServiceVolumes/PruneServiceVolumeBackupHistory/
	// ResolveServiceVolumeDockerName are the volume counterparts of the
	// two database methods above plus one extra resolution step a
	// database schedule doesn't need: store.ServiceVolumeBackupConfig
	// only carries a volume's logical name (what an operator wrote in
	// app.yaml), never its real Docker volume name, so RunVolumeBackup's
	// dockerVolumeName parameter has to be resolved fresh every tick, the
	// same "re-derive, never cache" principle this whole interface
	// already follows for which databases/volumes are even scheduled.
	ListScheduledServiceVolumes(ctx context.Context) ([]store.ServiceVolumeBackupConfig, error)
	PruneServiceVolumeBackupHistory(ctx context.Context, serviceName, volumeName string, keep int, olderThan time.Time) ([]store.PrunedBackup, error)
	ResolveServiceVolumeDockerName(ctx context.Context, serviceName, volumeName string) (string, error)
}

ScheduleStore is the narrow store surface Scheduler needs: which databases currently have a due-able cron schedule, re-derived fresh on every tick, and the retention prune a successful scheduled run triggers afterward. *store.DB satisfies this structurally, the same "consumer defines its own narrow interface" shape HistoryStore (runner.go) already establishes for this package's other store dependency.

type ScheduledBackupRunner

type ScheduledBackupRunner interface {
	RunBackup(ctx context.Context, historyID, databaseName, engine, containerName, targetID string) error
	// RunVolumeBackup matches api.ServiceVolumeBackupRunner's shape
	// (internal/api/app_volume_backups.go); *Runner satisfies both, the
	// volume counterpart of RunBackup's own doc comment.
	RunVolumeBackup(ctx context.Context, historyID, serviceName, volumeName, dockerVolumeName, targetID, sqlitePath string) error
	// ResolveDestination resolves a backup target ID into a live
	// Destination (*Runner.ResolveDestination, runner.go), the same
	// resolution RunBackup does internally before every upload. Needed
	// separately here because retention pruning deletes objects for a
	// pruned row's own TargetID after a run, not during one.
	ResolveDestination(ctx context.Context, targetID string) (Destination, error)
}

ScheduledBackupRunner is the surface Scheduler needs to actually run a backup once its schedule comes due: identical in shape to internal/api's own BackupRunner interface (internal/api/backups.go), redeclared here rather than imported across the package boundary, the same reasoning that file's own BackupRunner doc comment gives for why internal/api never imports internal/backup directly. *Runner satisfies this structurally, so cmd/levelrail/main.go wires the exact same *Runner instance into both api.WithBackupRunner (the manual trigger path) and NewScheduler (this one): one backup implementation, two ways to invoke it, exactly the design internal/spec.Backup's own doc comment implies by giving app.yaml a single databases[].backup block rather than separate manual/scheduled config shapes.

type ScheduledBaseBackupRunner added in v0.2.0

type ScheduledBaseBackupRunner interface {
	RunBaseBackup(ctx context.Context, historyID, databaseName, containerName, targetID string) error
}

ScheduledBaseBackupRunner is *BaseBackupRunner's RunBaseBackup.

type ScheduledVerifier

type ScheduledVerifier interface {
	VerifyBackup(ctx context.Context, verificationID, backupHistoryID, engine, checkedBy string) error
}

ScheduledVerifier matches api.BackupVerifier's shape (internal/api/backup_verify.go); *VerifyRunner satisfies both, so cmd/levelrail/main.go shares one instance.

type Scheduler

type Scheduler struct {
	Store  ScheduleStore
	Runner ScheduledBackupRunner
	// Deleter removes the bucket object behind each pruned backup_history
	// row after a successful retention prune. nil disables object
	// deletion entirely (store rows still get pruned), the same
	// optional-capability shape this codebase already uses elsewhere
	// (e.g. backupRunner itself being nil when no master key is
	// configured, cmd/levelrail/main.go).
	Deleter Deleter
	// Verifier, if set, re-checks a scheduled backup right after it
	// succeeds (runScheduled). nil disables auto-verification, same
	// optional-capability shape as Deleter above. Set directly like Now
	// below, not a constructor param, to keep NewScheduler's signature
	// stable for existing call sites.
	Verifier ScheduledVerifier
	// BaseBackups, if set, takes a physical base backup after each scheduled
	// logical backup of a PITR-enabled database, so point-in-time restore
	// stays fresh without a second schedule. nil disables it.
	BaseBackups ScheduledBaseBackupRunner
	// Policies and GFS enable per-volume daily/weekly/monthly retention;
	// both nil is valid and leaves only the count/age rules.
	Policies VolumePolicyLookup
	GFS      GFSStore
	Logger   *slog.Logger
	// Now returns the current time, the same testable-clock field
	// Runner.Now already establishes in runner.go: production code
	// leaves it nil and Tick falls back to time.Now, tests set it for
	// deterministic schedule matching.
	Now func() time.Time
	// contains filtered or unexported fields
}

Scheduler periodically checks which databases have a due cron schedule (store.DesiredDatabase's BackupSchedule/BackupTargetID/ BackupRetain, migrations/0023_scheduled_backups.sql) and runs a backup through the same Runner the manual trigger path (internal/api's handleTriggerBackup) uses, recording the attempt in store.BackupHistory exactly like a manual trigger does, since both paths call the identical Runner.RunBackup.

Tick's own shape (list what to evaluate, act, persist any resulting state, join and return errors rather than stopping at the first one) deliberately mirrors internal/alerting.Engine.Tick: that is this codebase's own established shape for "evaluate every enabled X on an interval, one broken X must never block the rest," and a cron schedule check is exactly that same shape applied to a different resource. Run's own ticker loop is the identical pattern internal/alerting.Engine.Run, internal/telemetry.Collector.Run, and every other periodic loop in cmd/levelrail/main.go already use.

Unlike alerting.Engine.Tick's own per-rule dispatch, a due backup runs synchronously inside Tick, not fired off into its own goroutine the way internal/api's handleTriggerBackup dispatches one: that handler's goroutine exists specifically because an HTTP response must return in milliseconds, a constraint that does not apply to a background ticker loop already running off the request path. Running sequentially here also means a database's backup finishing (or failing) is fully resolved, including its retention prune, before Tick moves on to evaluate the next database, which keeps this method's own control flow as a flat, easy-to-reason-about loop instead of needing a WaitGroup or a results channel to know when a tick's work is actually done.

Missed-schedule handling is a known, deliberate, and documented gap, not an oversight: Scheduler tracks each database's next fire time purely in memory (nextRun below), armed fresh the first tick a schedule is observed rather than reconstructed from store.BackupHistory on startup. A schedule that should have fired while the control plane was down does not "catch up" the moment it restarts; it simply waits for its next naturally occurring occurrence. This mirrors the honesty this codebase already extends to a related gap on the record-keeping side (store.BackupHistory's own status constants doc comment: "nothing currently sweeps a stuck running row back to failed... a known, honest gap for the eventual scheduler to close alongside its own retry logic, not invented speculatively here"). Building real catch-up/retry semantics is a materially bigger feature (it needs to decide how many missed runs to catch up on, how to avoid a thundering herd of backups firing at once after a long outage, and how that interacts with retention) than this task's scope asked for, so it is left as a follow-up rather than guessed at here.

func NewScheduler

func NewScheduler(store ScheduleStore, runner ScheduledBackupRunner, deleter Deleter, logger *slog.Logger) *Scheduler

NewScheduler builds a Scheduler ready to Tick or Run. logger defaults to slog.Default() if nil, the same convention alerting.NewEngine already establishes for its own logger parameter. deleter may be nil (see Scheduler.Deleter's own doc comment).

func (*Scheduler) Run

func (s *Scheduler) Run(ctx context.Context, interval time.Duration) error

Run calls Tick on interval until ctx is done, matching the shape of every other periodic loop in this codebase (alerting.Engine.Run, telemetry.Collector.Run, and cmd/levelrail/main.go's own retention sweeps all follow this identical select-on-ticker-or-ctx-Done shape).

func (*Scheduler) Tick

func (s *Scheduler) Tick(ctx context.Context) error

Tick evaluates every currently scheduled database once. See this struct's own doc comment for the overall shape and the missed-schedule gap this deliberately leaves open. Errors from an individual database (an invalid cron string, RunBackup itself failing, a retention prune failing) are collected and joined, never stopping evaluation of the remaining databases, the same "one broken resource must not block the rest" principle Tick's own doc comment traces back to alerting.Engine.Tick.

type SealRecorder added in v0.2.0

type SealRecorder interface {
	SetBackupSealInfo(ctx context.Context, id, codec, manifestDigest string, files, bytes int64) error
}

SealRecorder persists the codec and manifest of a sealed backup.

type Sealer added in v0.2.0

type Sealer struct {
	Recipients []age.Recipient
	Identities []age.Identity
}

Sealer compresses and optionally encrypts a stream. A nil Sealer means the legacy raw encoding.

func LoadSealer added in v0.2.0

func LoadSealer(lookup func(string) (string, bool), dataDir string) (*Sealer, error)

LoadSealer builds the Sealer from the environment. Encryption is on unless APP_BACKUP_ENCRYPTION is "off"; the age identity lives in APP_BACKUP_ENCRYPTION_KEY_FILE, default <dataDir>/backup-encryption.key, and is generated on first use. An empty dataDir with no key file disables encryption.

func (*Sealer) Codec added in v0.2.0

func (s *Sealer) Codec() string

Codec is the codec name this Sealer produces.

func (*Sealer) Seal added in v0.2.0

func (s *Sealer) Seal(src io.Reader) io.ReadCloser

Seal returns a reader producing src compressed (and encrypted when recipients are set). Closing it early stops the pipeline.

type SecretsResolver

type SecretsResolver interface {
	Resolve(ctx context.Context, serviceName, envKey string) (string, error)
}

SecretsResolver is the one internal/secrets.Manager method Runner needs, narrowed to keep this package's tests from needing a real Manager (Manager itself has no interface today; a struct-shaped fake satisfying this one-method interface is enough).

type SqliteSnapshotter added in v0.2.0

type SqliteSnapshotter interface {
	Snapshot(ctx context.Context, volumeName, relPath string) (io.ReadCloser, error)
}

SqliteSnapshotter produces a consistent snapshot of a SQLite .db file living inside a named Docker volume, the sqlite-path counterpart of VolumeArchiver.

type TargetGetter added in v0.2.0

type TargetGetter interface {
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
}

TargetGetter is the one-method surface resolveTargetDestination needs, shared by BaseBackupHistoryStore and PITRHistoryStore so both can resolve a target's live Destination the identical way without each repeating the lookup-plus-two-secret-resolves sequence Runner. ResolveDestination (runner.go) already establishes for the logical-dump direction.

type TargetTestStore

type TargetTestStore interface {
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
}

TargetTestStore is the store surface TargetTester needs, narrowed the same way HistoryStore/VerificationHistoryStore already are: only the one lookup a connection test actually requires.

type TargetTester

type TargetTester struct {
	Store   TargetTestStore
	Secrets SecretsResolver
	Tester  Tester
}

TargetTester resolves targetID's stored fields plus its live credentials from internal/secrets and runs Tester.Test against them, the "verify a stored credential by actually using it, once, on demand" shape this codebase's own RegistryAuthTester established first for registry credentials.

func (*TargetTester) TestTarget

func (t *TargetTester) TestTarget(ctx context.Context, targetID string) error

TestTarget probes targetID's configured bucket over its stored credentials, without uploading or deleting anything.

type Tester

type Tester interface {
	Test(ctx context.Context, dest Destination) error
}

Tester probes a Destination for a lightweight, non-destructive connectivity and credential check, the test-connection counterpart Uploader/Deleter above establish for real backup/prune work.

type TreeManifest added in v0.2.0

type TreeManifest struct {
	Files  int64
	Dirs   int64
	Bytes  int64
	Digest string
}

TreeManifest summarizes a volume archive's content: counts plus an order-independent digest over every entry's path, type, mode, owner, size and content hash. Two archives of the same tree produce the same digest whatever order the tar tool walked it in.

func ComputeManifest added in v0.2.0

func ComputeManifest(r io.Reader) (TreeManifest, error)

ComputeManifest reads a tar stream to its end. The volume root entry (".") is ignored because extraction legitimately rewrites its mode and times.

type Uploader

type Uploader interface {
	Upload(ctx context.Context, dest Destination, key string, r io.Reader, size int64) error
}

Uploader writes a single object to a Destination. Implementations must read r to completion (or return an error partway through, in which case the object must not be left partially visible under key, an S3-compatible multipart upload aborted rather than completed covers this for free) rather than assume a specific size up front: size is a hint for implementations that benefit from knowing it (choosing single-PUT vs. multipart), not a hard contract r will produce exactly that many bytes.

type VerificationHistoryStore

type VerificationHistoryStore interface {
	GetBackupHistory(ctx context.Context, id string) (store.BackupHistory, error)
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
	StartBackupVerification(ctx context.Context, v store.BackupVerification) error
	FinishBackupVerification(ctx context.Context, id, status string, checksumMatch, sizeMatch, formatValid bool, downloadedBytes int64, errMsg, finishedAt string) error
}

VerificationHistoryStore is the store.DB surface VerifyRunner needs: GetBackupHistory/GetBackupTarget resolve exactly what DownloadRunner.Download's own identical pair already resolve (the object a verification attempt re-downloads is the same object a real download would fetch), plus Start/FinishBackupVerification to record the attempt itself, the same shape RestoreHistoryStore's own doc comment describes for the restore direction.

type VerifyRunner

type VerifyRunner struct {
	Store      VerificationHistoryStore
	Secrets    SecretsResolver
	Downloader Downloader
	// Now returns the current time, matching Runner.Now and
	// RestoreRunner.Now's own "field, not time.Now called directly"
	// reasoning for deterministic tests.
	Now func() time.Time
}

VerifyRunner confirms a previously succeeded backup is still intact without ever attempting a live restore: re-download the object, re-hash it, and compare against what Runner recorded at backup time, the non-destructive counterpart RunRestore's own doc comment explicitly rules out for this feature (a live restore against a running system is real risk, not a safe thing to run unattended on a schedule or a button click).

func (*VerifyRunner) VerifyBackup

func (r *VerifyRunner) VerifyBackup(ctx context.Context, verificationID, backupHistoryID, engine, checkedBy string) error

VerifyBackup re-downloads backupHistoryID's stored object and checks it for corruption, recording the attempt in store.BackupVerification throughout: a running row before any work starts, then passed or failed once it's done. verificationID is minted by the caller (internal/api's handleVerifyBackup), the same "generate the ID before the first write" convention RunBackup's own doc comment describes; VerifyBackup is expected to run in a goroutine the caller has already detached, for the same reason RunBackup is.

Refuses (the same as RunRestore) to proceed against a backup whose Status is not store.BackupStatusSucceeded: a running or failed attempt has no complete object at its recorded key worth verifying.

engine is passed by the caller (internal/api resolves it from the database's own store.DesiredDatabase, the same "caller resolves engine, this package only consumes it" convention RunBackup's own engine parameter already establishes) rather than looked up here: it only sharpens the format check below (validateFormat), it is never required for the checksum/size checks that carry most of this function's real value, so an empty engine (e.g. the parent database was since deleted) still runs a meaningful, if slightly less specific, verification rather than failing outright.

type VolumeArchiver

type VolumeArchiver interface {
	Archive(ctx context.Context, volumeName string) (io.ReadCloser, error)
}

VolumeArchiver produces a tar archive of a named Docker volume's contents, the app-volume counterpart of Dumper for a database engine. The returned ReadCloser's Close must be called by every caller once done, the same contract Dumper's own doc comment establishes.

type VolumeBackupOptions added in v0.2.0

type VolumeBackupOptions struct {
	Sealer   *Sealer
	Policies VolumePolicyLookup
	Quiesce  Quiescer
	// MaxBytes aborts an archive larger than this many raw bytes; 0 is no limit.
	MaxBytes int64
	// Timeout bounds one whole backup; 0 is no limit.
	Timeout time.Duration
	// Attempts is how many times a failed archive-and-upload is retried
	// from a fresh archive; values below 1 mean one attempt.
	Attempts int
	Logger   *slog.Logger
}

VolumeBackupOptions configures sealed volume backups on a Runner.

func (*VolumeBackupOptions) OptionsFromEnv added in v0.2.0

func (o *VolumeBackupOptions) OptionsFromEnv(lookup func(string) (string, bool))

OptionsFromEnv fills the budget fields from the environment.

type VolumeCloneRestoreRunner

type VolumeCloneRestoreRunner struct {
	Store          VolumeCloneRestoreStore
	Secrets        SecretsResolver
	Downloader     Downloader
	VolumeRestorer VolumeRestorer
	Volumes        VolumeCreator
	// Identities decrypt sealed backups; empty is valid for raw ones.
	Identities []age.Identity
	// Now returns the current time, the same "field, not time.Now called
	// directly" reasoning CloneRestoreRunner.Now's own doc comment gives.
	Now func() time.Time
}

VolumeCloneRestoreRunner restores a previously succeeded volume backup into a brand-new, standalone Docker volume rather than overwriting the one it came from: the volume counterpart of CloneRestoreRunner. Simpler than CloneRestoreRunner because there is no reconciler to wait on (see VolumeCreator's own doc comment); creating the volume and restoring into it both happen synchronously, one after the other, inside RunVolumeCloneRestore itself.

func (*VolumeCloneRestoreRunner) RunVolumeCloneRestore

func (r *VolumeCloneRestoreRunner) RunVolumeCloneRestore(ctx context.Context, historyID, sourceServiceName, sourceVolumeName, newVolumeName, backupHistoryID string) error

RunVolumeCloneRestore creates newVolumeName as a brand-new, empty Docker volume, then downloads and restores backupHistoryID into it, recording the attempt as store.VolumeCloneRestore throughout: a running row before any work starts, then succeeded or failed once it's done. sourceServiceName/sourceVolumeName are recorded for history only; RunVolumeCloneRestore never reads from or writes to the volume they name. Expected to run in a goroutine the caller has already detached, the same convention RunCloneRestore's own doc comment describes.

type VolumeCloneRestoreStore

type VolumeCloneRestoreStore interface {
	StartVolumeCloneRestore(ctx context.Context, h store.VolumeCloneRestore) error
	FinishVolumeCloneRestore(ctx context.Context, id, status, errMsg, finishedAt string) error
	// contains filtered or unexported methods
}

VolumeCloneRestoreStore is the narrow store surface VolumeCloneRestoreRunner needs: backupResolver's two methods to resolve the source backup, plus the clone-restore bookkeeping pair, the app service volume counterpart of CloneRestoreStore.

type VolumeCreator

type VolumeCreator interface {
	EnsureVolume(ctx context.Context, name string) error
}

VolumeCreator is the narrow docker.Runtime surface VolumeCloneRestoreRunner needs to bring a brand-new, empty volume into existence before restoring into it: unlike CloneRestoreRunner's wait for a new database's own reconciler to report Ready, a bare Docker volume has no reconcile loop and exists the moment this call returns, so there is nothing to poll for here.

type VolumePolicyLookup added in v0.2.0

type VolumePolicyLookup interface {
	GetVolumeBackupPolicy(ctx context.Context, serviceName, volumeName string) (store.VolumeBackupPolicy, error)
}

VolumePolicyLookup reads a volume's backup policy.

type VolumeRemover added in v0.2.0

type VolumeRemover interface {
	RemoveVolume(ctx context.Context, name string) error
}

VolumeRemover deletes a Docker volume by name.

type VolumeRestorer

type VolumeRestorer interface {
	Restore(ctx context.Context, volumeName string, archive io.Reader) error
}

VolumeRestorer untars a previously archived stream back into a named Docker volume, in place: it wipes every existing file the volume holds first, so this is always a full replace, never a merge, matching Restorer's own "the restore command exited zero" contract.

type VolumeRuntime added in v0.2.0

type VolumeRuntime interface {
	docker.Runtime
	RemoveVolume(ctx context.Context, name string) error
}

VolumeRuntime is docker.Runtime plus volume removal.

type WALFetcher added in v0.2.0

type WALFetcher struct {
	Runtime    docker.Runtime
	Lister     Lister
	Downloader Downloader
}

WALFetcher restores shipped WAL from a backup target into a database's local archive volume, so recovery works after the local copy is lost.

func (*WALFetcher) Fetch added in v0.2.0

func (f *WALFetcher) Fetch(ctx context.Context, dest Destination, databaseName, walVolume, sinceLSN string) (int, error)

Fetch copies into walVolume every shipped file not older than the segment holding sinceLSN (empty fetches everything), overwriting same-named local files: after a host loss a re-initialised cluster reuses segment names, and the shipped copy is the lineage the base backup belongs to. It then fixes ownership for the postgres user and returns how many files it fetched.

type WALFetching added in v0.2.0

type WALFetching interface {
	Fetch(ctx context.Context, dest Destination, databaseName, walVolume, sinceLSN string) (int, error)
}

WALFetching is the WALFetcher surface PITRRunner needs.

type WALShipScheduler added in v0.2.0

type WALShipScheduler struct {
	Store         WALShipStore
	Shipper       WALShipping
	Resolve       func(ctx context.Context, targetID string) (Destination, error)
	ContainerName func(databaseName string) string
	// IsLocal reports whether a database's node is the one this control plane's
	// Docker client talks to; remote nodes are skipped.
	IsLocal func(nodeID string) bool
	Logger  *slog.Logger
	// contains filtered or unexported fields
}

WALShipScheduler ships the WAL of every local PITR-enabled database to the target of its newest base backup on a fixed interval, bounding the data lost if the database host's disk dies to one interval of WAL.

func (*WALShipScheduler) Run added in v0.2.0

func (s *WALShipScheduler) Run(ctx context.Context, interval time.Duration) error

Run ticks until ctx ends.

func (*WALShipScheduler) Tick added in v0.2.0

func (s *WALShipScheduler) Tick(ctx context.Context)

Tick runs one shipping pass over every eligible database.

func (*WALShipScheduler) WALShipStatus added in v0.2.0

func (s *WALShipScheduler) WALShipStatus(databaseName string) (WALShipStatus, bool)

WALShipStatus returns the last shipping outcome for a database.

type WALShipStatus added in v0.2.0

type WALShipStatus struct {
	LastAttemptAt time.Time
	LastSuccessAt time.Time
	LastError     string
	// Shipped counts segments uploaded since the control plane started.
	Shipped  int
	TargetID string
}

WALShipStatus is the outcome of the most recent shipping pass for one database, shown by "pitr status".

type WALShipStore added in v0.2.0

type WALShipStore interface {
	ListDesiredDatabases(ctx context.Context) ([]store.DesiredDatabase, error)
	ListBaseBackupHistory(ctx context.Context, databaseName string) ([]store.BaseBackupHistory, error)
}

WALShipStore is the store surface WALShipScheduler needs.

type WALShipper added in v0.2.0

type WALShipper struct {
	Runtime  Runtime
	Uploader Uploader
	Lister   Lister
	Logger   *slog.Logger
}

WALShipper copies a database's local WAL archive into its backup target so point-in-time restore survives losing the database host's disk.

func (*WALShipper) Ship added in v0.2.0

func (s *WALShipper) Ship(ctx context.Context, dest Destination, databaseName, containerName string) (int, error)

Ship uploads every complete archived segment the target does not already hold at the same size, and returns how many it uploaded. A segment still being copied by archive_command is skipped because it is not yet full size.

type WALShipping added in v0.2.0

type WALShipping interface {
	Ship(ctx context.Context, dest Destination, databaseName, containerName string) (int, error)
}

WALShipping is the WALShipper surface BaseBackupRunner and the ship scheduler need.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL