backup

type BaseBackupHistoryStore ¶ added in v0.2.0

type BaseBackupHistoryStore interface {
	TargetGetter
	StartBaseBackupHistory(ctx context.Context, h store.BaseBackupHistory) error
	FinishBaseBackupHistory(ctx context.Context, id, status string, sizeBytes int64, lsn, errMsg, finishedAt string) error
}

BaseBackupHistoryStore is the store surface BaseBackupRunner needs, narrowed the same way HistoryStore is for the logical-dump direction (runner.go).

type BaseBackupMaintainer ¶ added in v0.2.0

type BaseBackupMaintainer interface {
	Prune(ctx context.Context, databaseName, containerName string) error
}

BaseBackupMaintainer is the retention hook PITRMaintainer implements.

type BaseBackupRunner ¶ added in v0.2.0

type BaseBackupRunner struct {
	Store        BaseBackupHistoryStore
	Secrets      SecretsResolver
	BaseBackuper BaseBackuper
	Uploader     Uploader
	// Runtime, if set, backs StartLSN's best-effort diagnostic lookup
	// (base_backup.go). nil is valid: RunBaseBackup simply records an
	// empty LSN, the same "optional capability" shape this package
	// already uses for Runner.VolumeArchiver.
	Runtime Runtime
	// Maintainer, if set, prunes old base backups and archived WAL after each
	// successful base backup. Failures are logged, never returned: the new
	// backup already succeeded.
	Maintainer BaseBackupMaintainer
	// WAL, when set, ships the local WAL archive to the target right after a
	// successful base backup, so the bucket holds what a restore replays.
	WAL WALShipping
	// WorkDir/Now match Runner's own identically-named fields exactly
	// (runner.go's own doc comments for each apply here unchanged).
	WorkDir string
	Now     func() time.Time
}

BaseBackupRunner ties a BaseBackuper and an Uploader to store and internal/secrets, the physical-backup counterpart of Runner (runner.go): the only piece in this package that knows how a base backup attempt gets recorded and how a target's live credentials get resolved.

func (r *BaseBackupRunner) RunBaseBackup(ctx context.Context, historyID, databaseName, containerName, targetID string) error

type BaseBackuper ¶ added in v0.2.0

type BaseBackuper interface {
	BaseBackup(ctx context.Context, containerName string) (io.ReadCloser, error)
}

BaseBackuper produces a physical base backup as a tar stream, for a single PITR-enabled Postgres database container already running under containerName: the physical counterpart of Dumper (dump.go), which produces a logical dump.

type CloneRestoreRunner ¶

type CloneRestoreRunner struct {
	Store      CloneRestoreStore
	Secrets    SecretsResolver
	Downloader Downloader
	Restorer   Restorer
	// Now returns the current time, the same "field, not time.Now called
	// directly" reasoning RestoreRunner.Now's own doc comment gives.
	Now func() time.Time
	// ReadyTimeout/PollInterval override defaultCloneRestoreReadyTimeout/
	// defaultCloneRestoreReadyPollInterval when non-zero, so tests don't
	// have to wait out the real default.
	ReadyTimeout time.Duration
	PollInterval time.Duration
	// SettleTimeout overrides defaultCloneRestoreSettleTimeout when non-zero.
	SettleTimeout time.Duration
}

CloneRestoreRunner restores a previously succeeded backup into a brand-new database rather than overwriting the one it came from: the non-destructive counterpart to RestoreRunner. internal/api creates the new database's desired state itself (reusing handleCreateDatabase's own path, see database_clone_restore.go's own doc comment for why) before ever calling RunCloneRestore; this type's only job is to wait for that database's reconciler to bring it up and then run the identical download-and-apply RestoreRunner already implements.

func (r *CloneRestoreRunner) RunCloneRestore(ctx context.Context, historyID, sourceDatabaseName, newDatabaseName, backupHistoryID, engine, containerName, controllerName string) error

type CloneRestoreStore ¶

type CloneRestoreStore interface {
	StartCloneRestore(ctx context.Context, h store.CloneRestore) error
	FinishCloneRestore(ctx context.Context, id, status, errMsg, finishedAt string) error
	GetConditions(ctx context.Context, controllerName string) ([]reconcile.Condition, error)
	// contains filtered or unexported methods
}

CloneRestoreStore is the narrow store surface CloneRestoreRunner needs: backupResolver's two methods to resolve the source backup, the clone- restore bookkeeping pair, and GetConditions to detect when the newly created database has actually come up. A separate interface from RestoreHistoryStore, the same "narrow interface per real dependency" reasoning that type's own doc comment gives, since the two share no method beyond the backupResolver pair.

type ContainerBaseBackuper ¶ added in v0.2.0

type ContainerBaseBackuper struct {
	Runtime Runtime
}

ContainerBaseBackuper is the real BaseBackuper: it runs pg_basebackup inside the already-running container via docker.Runtime.Exec, the same "no separate copy of the tool, no exposed port" reasoning ContainerDumper's own doc comment gives.

func (b *ContainerBaseBackuper) BaseBackup(ctx context.Context, containerName string) (io.ReadCloser, error)

type ContainerDumper ¶

type ContainerDumper struct {
	Runtime docker.Runtime
}

ContainerDumper is the real Dumper: it runs each engine's own native dump tool inside the already-running database container via docker.Runtime.Exec, so this process never needs its own copy of pg_dump/mysqldump/etc. and the database never needs a network port exposed anywhere reachable.

func (d *ContainerDumper) Dump(ctx context.Context, engine, containerName string) (io.ReadCloser, error)

type ContainerPITRRestorer ¶ added in v0.2.0

type ContainerPITRRestorer struct {
	Runtime docker.Runtime
}

ContainerPITRRestorer is the real PITRRestorer: the wipe-then-extract- then-configure sequence inside a short-lived helper container, the same createVolumeHelper mechanism ContainerVolumeRestorer uses.

func (r *ContainerPITRRestorer) Restore(ctx context.Context, dataVolumeName string, baseBackup io.Reader, targetTime time.Time) error

type ContainerRestorer ¶

type ContainerRestorer struct {
	Runtime docker.Runtime
	// ReadyTimeout/ReadyInterval override the engine readiness wait defaults.
	ReadyTimeout  time.Duration
	ReadyInterval time.Duration
}

ContainerRestorer is the real Restorer: it runs each engine's own native restore path inside the running container via docker.Runtime.ExecWithInput, ContainerDumper's own reasoning plus a stdin stream.

func (r *ContainerRestorer) Restore(ctx context.Context, engine, containerName string, dump io.Reader) error

type ContainerSqliteSnapshotter ¶ added in v0.2.0

type ContainerSqliteSnapshotter struct {
	Runtime docker.Runtime
}

ContainerSqliteSnapshotter is the real SqliteSnapshotter: a helper container mounts the volume read-only and runs sqlite3's own ".backup" command, safe under the app's own concurrent writes since it goes through SQLite's file locking rather than a filesystem copy.

func (s *ContainerSqliteSnapshotter) Snapshot(ctx context.Context, volumeName, relPath string) (io.ReadCloser, error)

type ContainerVolumeArchiver ¶

type ContainerVolumeArchiver struct {
	Runtime docker.Runtime
}

ContainerVolumeArchiver is the real VolumeArchiver: it creates a helper container with volumeName mounted read-only, tars its contents to stdout, and removes the container once the caller has read the stream to completion (or given up partway through).

func (a *ContainerVolumeArchiver) Archive(ctx context.Context, volumeName string) (io.ReadCloser, error)

type ContainerVolumeRestorer ¶

type ContainerVolumeRestorer struct {
	Runtime docker.Runtime
}

ContainerVolumeRestorer is the real VolumeRestorer: it runs the wipe-then-extract command inside a short-lived helper container with volumeName mounted read-write.

func (r *ContainerVolumeRestorer) Restore(ctx context.Context, volumeName string, archive io.Reader) error

type DeleteHistoryStore ¶ added in v0.2.0

type DeleteHistoryStore interface {
	GetBackupHistory(ctx context.Context, id string) (store.BackupHistory, error)
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
	DeleteBackupHistory(ctx context.Context, id string) error
}

DeleteHistoryStore is the store.DB surface DeleteRunner needs: GetBackupHistory/GetBackupTarget resolve where a backup attempt's object actually lives, the identical pair DownloadHistoryStore's own doc comment describes; DeleteBackupHistory removes the row itself once the underlying object delete has been attempted.

type DeleteRunner ¶ added in v0.2.0

type DeleteRunner struct {
	Store   DeleteHistoryStore
	Secrets SecretsResolver
	// Deleter may be nil, the same optional-capability shape
	// Scheduler.Deleter's own doc comment establishes: DeleteBackup then
	// only removes the store row, never attempting the object delete.
	Deleter Deleter
	Logger  *slog.Logger
}

DeleteRunner deletes one backup attempt on demand: an operator-directed counterpart to Scheduler's own retention-driven deletePrunedObjects, sharing the same "best-effort object delete, the store row is the thing that must always go" contract (Scheduler.deletePrunedObjects's own doc comment). Unlike that batch path, DeleteBackup always finishes by removing the row, regardless of whether the object delete succeeded, failed, or couldn't even be attempted: an operator who explicitly named this one archive to delete should not be left with it still showing up in history because deleting the underlying object hit a storage-side problem, on top of Deleter's own contract that a missing object is already a no-op success (backup.go's own Deleter doc comment).

func (d *DeleteRunner) DeleteBackup(ctx context.Context, historyID string) error

type Deleter ¶

type Deleter interface {
	Delete(ctx context.Context, dest Destination, key string) error
}

Deleter removes a single object from a Destination, the inverse of Uploader. Implementations must treat deleting a key that is already gone (or never existed) as success, not an error: S3-compatible DeleteObject calls are naturally idempotent this way.

type Destination ¶

type Destination struct {
	// Provider is one of store.BackupProviderAWS / R2 / Custom. An
	// Uploader implementation may use it to pick defaults (e.g. AWS's
	// per-region default endpoint), but Endpoint below is always the
	// final authority when non-empty.
	Provider string
	// Endpoint is the S3-compatible API base URL. Empty for
	// BackupProviderAWS (the SDK's own region-based resolver applies);
	// required for R2 and Custom.
	Endpoint string
	Region   string
	Bucket   string

	AccessKeyID     string
	SecretAccessKey string
}

Destination is everything an Uploader needs to reach a specific bucket, credentials included. Built fresh per call from a store.BackupTarget plus its resolved internal/secrets values (see store.BackupTargetSecretsKey): never persisted or logged as a whole, since AccessKeyID and SecretAccessKey are live credentials.

type DownloadHistoryStore ¶

type DownloadHistoryStore interface {
	GetBackupHistory(ctx context.Context, id string) (store.BackupHistory, error)
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
}

DownloadHistoryStore is the store.DB surface DownloadRunner needs: GetBackupHistory to resolve the attempt a download names, GetBackupTarget to resolve where its object actually lives, the same pair RestoreHistoryStore's own doc comment describes for the restore direction. No Start/Finish pair here: unlike a backup or a restore, a download has no attempt of its own worth recording in a history table, it only reads an object an earlier, already-recorded backup wrote.

type DownloadRunner ¶

type DownloadRunner struct {
	Store      DownloadHistoryStore
	Secrets    SecretsResolver
	Downloader Downloader
}

DownloadRunner resolves a succeeded backup attempt to its live object stream, the read-only counterpart of Runner (dump-and-upload) and RestoreRunner (download-and-restore-in-place): given a store.BackupHistory ID, it resolves the row's target and credentials the identical way Runner.runDumpAndUpload and RestoreRunner.runDownloadAndRestore already do, then hands the caller (internal/api's handleDownloadBackup) the raw object stream to copy directly into an HTTP response, never buffering it here.

func (d *DownloadRunner) Download(ctx context.Context, historyID string) (io.ReadCloser, error)

type Downloader ¶

type Downloader interface {
	Download(ctx context.Context, dest Destination, key string) (io.ReadCloser, error)
}

Downloader retrieves a single object from a Destination, the exact reverse of Uploader: given the same (Destination, key) an earlier Upload call was made with, it returns that object's bytes as a stream. Implementations are not required to know the object's size up front (see S3Downloader's own doc comment for why GetObject already makes this a non-issue, unlike Upload's own unknown-size handling), only to stream it back faithfully.

type Dumper ¶

type Dumper interface {
	Dump(ctx context.Context, engine, containerName string) (io.ReadCloser, error)
}

Dumper produces a database dump as a stream, for a single managed database container already running under containerName (the same name application.ContainerName / the database controller's own container-naming convention already produces; Dumper does not construct or look this up itself, Runner does).

The returned ReadCloser's Close must be called by every caller once done, success or failure, the same contract io.ReadCloser always carries; a Dumper backed by a subprocess or a docker exec stream depends on Close to release that underlying resource.

type HistoryStore ¶

type HistoryStore interface {
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
	StartBackupHistory(ctx context.Context, h store.BackupHistory) error
	FinishBackupHistory(ctx context.Context, id, status string, sizeBytes int64, checksum, errMsg, finishedAt string) error
}

HistoryStore is the store.DB surface Runner needs, narrowed the same way SecretsResolver is: a fake satisfying six methods is a lot less to stub than the full *store.DB.

type Lister ¶ added in v0.2.0

type Lister interface {
	List(ctx context.Context, dest Destination, prefix string) ([]RemoteObject, error)
}

Lister lists the objects under a key prefix in a backup target.

type MajorUpgradeRunner ¶ added in v0.2.0

type MajorUpgradeRunner struct {
	Store    MajorUpgradeStore
	Runtime  VolumeRuntime
	Dumper   Dumper
	Restorer Restorer
	Nudge    func()
	Now      func() time.Time
	WorkDir  string
	Logger   *slog.Logger
	// Wait bounds each wait for a container to stop, start or accept
	// connections; Poll is the interval between checks.
	Wait time.Duration
	Poll time.Duration
}

MajorUpgradeRunner upgrades a managed Postgres database across a major version by dump and restore. The old data volume is never rewritten in place: it is copied to a snapshot volume first, the dump is taken from the snapshot, and any failure after the live volume is wiped restores the snapshot and the old version.

func (r *MajorUpgradeRunner) DiscardSnapshot(ctx context.Context, upgradeID string) error

func (r *MajorUpgradeRunner) RecoverInterrupted(ctx context.Context)

func (r *MajorUpgradeRunner) Rollback(ctx context.Context, upgradeID string) error

func (r *MajorUpgradeRunner) RunMajorUpgrade(ctx context.Context, historyID, databaseName, toVersion string) error

type MajorUpgradeStore ¶ added in v0.2.0

type MajorUpgradeStore interface {
	GetDesiredDatabase(ctx context.Context, name string) (*store.DesiredDatabase, error)
	SaveDesiredDatabase(ctx context.Context, d store.DesiredDatabase) error
	UpdateDatabaseSuspended(ctx context.Context, name string, suspended bool) error
	StartMajorUpgrade(ctx context.Context, u store.MajorUpgrade) error
	UpdateMajorUpgradePhase(ctx context.Context, id, phase, snapshotVolume string) error
	FinishMajorUpgrade(ctx context.Context, id, status, errMsg, finishedAt string) error
	GetMajorUpgrade(ctx context.Context, id string) (store.MajorUpgrade, error)
	ListRunningMajorUpgrades(ctx context.Context) ([]store.MajorUpgrade, error)
	ClearMajorUpgradeSnapshot(ctx context.Context, id string) error
}

MajorUpgradeStore is the store surface MajorUpgradeRunner needs.

type PITRHistoryStore ¶ added in v0.2.0

type PITRHistoryStore interface {
	TargetGetter
	GetBaseBackupHistory(ctx context.Context, id string) (store.BaseBackupHistory, error)
	GetOldestSucceededBaseBackup(ctx context.Context, databaseName string) (store.BaseBackupHistory, bool, error)
	StartPITRRestoreHistory(ctx context.Context, h store.PITRRestoreHistory) error
	FinishPITRRestoreHistory(ctx context.Context, id, status, errMsg, finishedAt string) error
	UpdateDatabaseSuspended(ctx context.Context, name string, suspended bool) error
}

PITRHistoryStore is the store surface PITRRunner needs. See ADR 016 for why UpdateDatabaseSuspended is part of a restore's own store surface rather than a separate concern.

type PITRMaintainer ¶ added in v0.2.0

type PITRMaintainer struct {
	Store   PITRMaintenanceStore
	Resolve func(ctx context.Context, targetID string) (Destination, error)
	Deleter Deleter
	// Lister enables pruning the shipped WAL copy in the bucket too.
	Lister  Lister
	Runtime Runtime
	// Keep overrides the env default when positive.
	Keep   int
	Logger *slog.Logger
}

PITRMaintainer keeps a PITR-enabled database's disk and bucket bounded: without it the local WAL archive and the base backups grow forever.

func (m *PITRMaintainer) Prune(ctx context.Context, databaseName, containerName string) error

type PITRMaintenanceStore ¶ added in v0.2.0

type PITRMaintenanceStore interface {
	ListBaseBackupHistory(ctx context.Context, databaseName string) ([]store.BaseBackupHistory, error)
	DeleteBaseBackupHistory(ctx context.Context, id string) error
}

PITRMaintenanceStore is the store surface PITRMaintainer needs.

type PITRRestorer ¶ added in v0.2.0

type PITRRestorer interface {
	Restore(ctx context.Context, dataVolumeName string, baseBackup io.Reader, targetTime time.Time) error
}

PITRRestorer applies a base backup to dataVolumeName in place, configuring Postgres to replay WAL to targetTime and promote: the point-in-time-restore counterpart of Restorer. Unlike Restorer, this never touches a running container; dataVolumeName's container must already be stopped (PITRRunner's job).

type PITRRunner ¶ added in v0.2.0

type PITRRunner struct {
	Store      PITRHistoryStore
	Secrets    SecretsResolver
	Downloader Downloader
	Restorer   PITRRestorer
	// WAL and WALVolume, when set, pull the WAL shipped to the target into the
	// database's archive volume before the destructive phase, so a restore works
	// after the local archive is lost. WALVolume maps a database name to it.
	WAL       WALFetching
	WALVolume func(databaseName string) string
	// Runtime backs the teardown/startup/promotion polls: Suspended and
	// Nudge only ever request a change, this verifies it happened.
	Runtime docker.Runtime
	// Nudge requests an immediate reconcile pass (*reconcile.Engine's
	// own Nudge); nil is valid, falling back to the resync cadence.
	Nudge func()
	// Now returns the current time for history timestamps; nil falls
	// back to time.Now.
	Now func() time.Time
	// TeardownTimeout/StartupTimeout/PromoteTimeout/PollInterval override
	// the package defaults above when non-zero, for tests.
	TeardownTimeout time.Duration
	StartupTimeout  time.Duration
	PromoteTimeout  time.Duration
	PollInterval    time.Duration
}

PITRRunner orchestrates one point-in-time restore end to end: download the named base backup, hand it to a PITRRestorer to wipe and re-populate the database's data volume with recovery configured for targetTime, then wait for Postgres to actually promote before calling the attempt succeeded. Unlike RestoreRunner, this needs the container down for the whole window; see ADR 016 for why Suspended is what keeps the reconciler from racing that, and why promotion is polled directly rather than trusted to the reconciler's Ready condition.

func (r *PITRRunner) RunPITRRestore(ctx context.Context, historyID, databaseName, containerName, dataVolumeName, baseBackupHistoryID string, targetTime time.Time) error

func (r *PITRRunner) ValidateTarget(ctx context.Context, databaseName, containerName string, target time.Time) error

func (r *PITRRunner) Window(ctx context.Context, databaseName, containerName string) (PITRWindow, error)

func (r *PITRRunner) WindowBounds(ctx context.Context, databaseName, containerName string) (hasBaseBackup bool, start, end time.Time, err error)

type PITRWindow ¶ added in v0.2.0

type PITRWindow struct {
	HasBaseBackup bool
	Start         time.Time
	End           time.Time
}

PITRWindow is one database's currently recoverable point-in-time restore range: Start is the oldest succeeded base backup's own started_at (nothing earlier is recoverable, since there is no base to replay WAL forward from), End is the latest instant WAL archiving has continuously and provably reached (RecoverableWindowEnd). HasBaseBackup is false whenever no base backup has ever succeeded yet, in which case Start/End are both zero and carry no meaning.

type RemoteObject ¶ added in v0.2.0

type RemoteObject struct {
	Key  string
	Size int64
}

RemoteObject is one object found under a bucket prefix.

type RestoreHistoryStore ¶

type RestoreHistoryStore interface {
	GetBackupHistory(ctx context.Context, id string) (store.BackupHistory, error)
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
	StartRestoreHistory(ctx context.Context, h store.RestoreHistory) error
	FinishRestoreHistory(ctx context.Context, id, status, errMsg, finishedAt string) error
}

RestoreHistoryStore is the store.DB surface RestoreRunner needs, narrowed the same way Runner's own HistoryStore is: GetBackupHistory to resolve the source backup a restore names, GetBackupTarget to resolve where that backup's object actually lives, and StartRestoreHistory/FinishRestoreHistory to record the restore attempt itself. Deliberately not HistoryStore plus new methods bolted on: a fake satisfying HistoryStore's forward-backup methods has no reason to also implement restore's, and the reverse is equally true, so keeping them as two separate narrow interfaces (which happen to overlap on GetBackupTarget, satisfied by the same *store.DB either way) matches this codebase's own "narrow interface per real dependency, not one grab-bag interface per package" convention.

type RestoreRunner ¶

type RestoreRunner struct {
	Store      RestoreHistoryStore
	Secrets    SecretsResolver
	Downloader Downloader
	Restorer   Restorer
	// VolumeRestorer backs RunVolumeRestore the same way Restorer backs
	// RunRestore. nil is valid, the same "optional capability" shape
	// Runner.VolumeArchiver's own doc comment establishes.
	VolumeRestorer VolumeRestorer
	// WorkDir is RunRestore/RunVolumeRestore's disk-space preflight
	// target, matching Runner.WorkDir's own fallback-to-APP_DATA_DIR,
	// skip-if-still-empty behavior.
	WorkDir string
	// Now returns the current time, matching Runner.Now's own "field, not
	// time.Now called directly" reasoning for deterministic tests.
	Now func() time.Time
}

RestoreRunner ties a Downloader and a Restorer to store and internal/secrets, the restore counterpart of Runner: the only piece in this package that knows how a restore attempt gets recorded and how a target's live credentials get resolved. internal/api constructs one real RestoreRunner at startup, the same place it constructs Runner, and calls RunRestore per triggered restore.

A separate type from Runner rather than a second method on it: Runner is already a coherent "dump this, upload it" unit with its own field set (Dumper, Uploader); RestoreRunner's fields (Downloader, Restorer) point the opposite direction and share nothing with Runner's except Store's underlying *store.DB value and Secrets' underlying *secrets.Manager value, both already expressed as separate narrow interfaces above and in runner.go rather than a shared struct field, so there is no real duplication to unify by merging the two types, only two independently-testable units that happen to be wired from the same concrete dependencies at startup.

func (r *RestoreRunner) RunRestore(ctx context.Context, historyID, databaseName, backupHistoryID, engine, containerName string) error

func (r *RestoreRunner) RunVolumeRestore(ctx context.Context, historyID, serviceName, volumeName, dockerVolumeName, backupHistoryID string) error

type Restorer ¶

type Restorer interface {
	Restore(ctx context.Context, engine, containerName string, dump io.Reader) error
}

Restorer applies a previously taken dump back onto a live database container. Like Dumper, it takes no credentials beyond what the container's own environment already provides. Its contract is "the restore command exited zero," not "and the data was verified correct afterward."

type Runner ¶

type Runner struct {
	Store    HistoryStore
	Secrets  SecretsResolver
	Dumper   Dumper
	Uploader Uploader
	// VolumeArchiver backs RunVolumeBackup the same way Dumper backs
	// RunBackup. nil is valid: a control plane with no volume-backup
	// callers configured (today, none do this by default) simply never
	// calls RunVolumeBackup, the same "optional capability" shape this
	// codebase already uses for Scheduler.Deleter/Verifier.
	VolumeArchiver VolumeArchiver
	// SqliteSnapshotter backs RunVolumeBackup when a volume's sqlite_path
	// is set; nil is valid the same way VolumeArchiver's nil is.
	SqliteSnapshotter SqliteSnapshotter
	// WorkDir is the directory RunBackup/RunVolumeBackup's disk-space
	// preflight checks (checkDiskSpace) before starting. Empty falls back
	// to APP_DATA_DIR (resolveWorkDir), and empty even then skips the
	// check entirely: see checkDiskSpace's own doc comment for why a
	// backup, which streams straight from its source to its destination
	// bucket, still has a real local directory worth guarding.
	WorkDir string
	// Now returns the current time. A field, not time.Now called
	// directly, so tests get deterministic timestamps without a real
	// clock dependency; production code leaves it nil and RunBackup
	// falls back to time.Now.
	Now func() time.Time
}

Runner ties a Dumper and an Uploader to store and internal/secrets: the only piece in this package that knows how a backup attempt gets recorded and how a target's live credentials get resolved. internal/api constructs one real Runner at startup (the same "wire concrete implementations once, in cmd/levelrail or internal/api's own constructor" pattern every other cross-package dependency in this codebase already follows) and calls RunBackup per triggered backup.

func (r *Runner) ResolveDestination(ctx context.Context, targetID string) (Destination, error)

func (r *Runner) RunBackup(ctx context.Context, historyID, databaseName, engine, containerName, targetID string) error

func (r *Runner) RunVolumeBackup(ctx context.Context, historyID, serviceName, volumeName, dockerVolumeName, targetID, sqlitePath string) error

type Runtime ¶ added in v0.2.0

type Runtime interface {
	Exec(ctx context.Context, containerID string, cmd []string) (io.ReadCloser, error)
	ExecWithInput(ctx context.Context, containerID string, cmd []string, stdin io.Reader) (io.ReadCloser, error)
}

Runtime is the narrow docker.Runtime surface (Exec/ExecWithInput) this file's functions and ContainerPITRRestorer need.

type S3Deleter ¶

type S3Deleter struct{}

S3Deleter is the real Deleter, covering AWS/R2/custom endpoints the same way S3Uploader does, reusing newS3Client (uploader.go) rather than building its own client.

func (S3Deleter) Delete(ctx context.Context, dest Destination, key string) error

type S3Downloader ¶

type S3Downloader struct{}

S3Downloader is the real Downloader: the download side of the same AWS/R2/custom-endpoint story S3Uploader already covers, one implementation for every provider this codebase supports, driven by the same Destination.Endpoint-is-the-final-authority rule.

Unlike S3Uploader, this has no multipart-vs-single-PUT decision to make: S3's GetObject API already streams a response body of whatever size the object actually is, with no equivalent to PutObject's Content-Length requirement on the request side, so there's no unknown-size problem here to work around with an upload-manager-style abstraction. A plain client.GetObject call is the entire implementation.

func (S3Downloader) Download(ctx context.Context, dest Destination, key string) (io.ReadCloser, error)

type S3Lister ¶ added in v0.2.0

type S3Lister struct{}

S3Lister is the real Lister, over ListObjectsV2.

func (S3Lister) List(ctx context.Context, dest Destination, prefix string) ([]RemoteObject, error)

type S3Tester ¶

type S3Tester struct{}

S3Tester is the real Tester, covering AWS/R2/custom endpoints the same way S3Uploader does, reusing newS3Client rather than building its own client.

func (S3Tester) Test(ctx context.Context, dest Destination) error

type S3Uploader ¶

type S3Uploader struct{}

S3Uploader is the real Uploader: every backup target this codebase supports (AWS, R2, and any other S3-compatible custom endpoint) speaks the same REST API, so one implementation covers all three, driven entirely by the Provider/Endpoint distinction already captured on Destination.

A new S3 client is built per Upload call rather than cached on the struct: Destination carries live, per-target credentials resolved fresh from internal/secrets on every backup run (see Runner), and a target's bucket, region, or endpoint can change between runs too. Backups are infrequent (the default schedule is once a day, see store.BackupTarget's own docs), so the cost of rebuilding a client each time is not worth the complexity of a cache that would need invalidating whenever a target's settings change.

func (S3Uploader) Upload(ctx context.Context, dest Destination, key string, r io.Reader, _ int64) error

type ScheduleStore ¶

type ScheduleStore interface {
	ListScheduledDatabases(ctx context.Context) ([]store.DesiredDatabase, error)
	PruneBackupHistory(ctx context.Context, databaseName string, keep int, olderThan time.Time) ([]store.PrunedBackup, error)
	// ListScheduledServiceVolumes/PruneServiceVolumeBackupHistory/
	// ResolveServiceVolumeDockerName are the volume counterparts of the
	// two database methods above plus one extra resolution step a
	// database schedule doesn't need: store.ServiceVolumeBackupConfig
	// only carries a volume's logical name (what an operator wrote in
	// app.yaml), never its real Docker volume name, so RunVolumeBackup's
	// dockerVolumeName parameter has to be resolved fresh every tick, the
	// same "re-derive, never cache" principle this whole interface
	// already follows for which databases/volumes are even scheduled.
	ListScheduledServiceVolumes(ctx context.Context) ([]store.ServiceVolumeBackupConfig, error)
	PruneServiceVolumeBackupHistory(ctx context.Context, serviceName, volumeName string, keep int, olderThan time.Time) ([]store.PrunedBackup, error)
	ResolveServiceVolumeDockerName(ctx context.Context, serviceName, volumeName string) (string, error)
}

ScheduleStore is the narrow store surface Scheduler needs: which databases currently have a due-able cron schedule, re-derived fresh on every tick, and the retention prune a successful scheduled run triggers afterward. *store.DB satisfies this structurally, the same "consumer defines its own narrow interface" shape HistoryStore (runner.go) already establishes for this package's other store dependency.

type ScheduledBackupRunner ¶

type ScheduledBackupRunner interface {
	RunBackup(ctx context.Context, historyID, databaseName, engine, containerName, targetID string) error
	// RunVolumeBackup matches api.ServiceVolumeBackupRunner's shape
	// (internal/api/app_volume_backups.go); *Runner satisfies both, the
	// volume counterpart of RunBackup's own doc comment.
	RunVolumeBackup(ctx context.Context, historyID, serviceName, volumeName, dockerVolumeName, targetID, sqlitePath string) error
	// ResolveDestination resolves a backup target ID into a live
	// Destination (*Runner.ResolveDestination, runner.go), the same
	// resolution RunBackup does internally before every upload. Needed
	// separately here because retention pruning deletes objects for a
	// pruned row's own TargetID after a run, not during one.
	ResolveDestination(ctx context.Context, targetID string) (Destination, error)
}

ScheduledBackupRunner is the surface Scheduler needs to actually run a backup once its schedule comes due: identical in shape to internal/api's own BackupRunner interface (internal/api/backups.go), redeclared here rather than imported across the package boundary, the same reasoning that file's own BackupRunner doc comment gives for why internal/api never imports internal/backup directly. *Runner satisfies this structurally, so cmd/levelrail/main.go wires the exact same *Runner instance into both api.WithBackupRunner (the manual trigger path) and NewScheduler (this one): one backup implementation, two ways to invoke it, exactly the design internal/spec.Backup's own doc comment implies by giving app.yaml a single databases[].backup block rather than separate manual/scheduled config shapes.

type ScheduledBaseBackupRunner ¶ added in v0.2.0

type ScheduledBaseBackupRunner interface {
	RunBaseBackup(ctx context.Context, historyID, databaseName, containerName, targetID string) error
}

ScheduledBaseBackupRunner is *BaseBackupRunner's RunBaseBackup.

type ScheduledVerifier ¶

type ScheduledVerifier interface {
	VerifyBackup(ctx context.Context, verificationID, backupHistoryID, engine, checkedBy string) error
}

ScheduledVerifier matches api.BackupVerifier's shape (internal/api/backup_verify.go); *VerifyRunner satisfies both, so cmd/levelrail/main.go shares one instance.

type Scheduler ¶

type Scheduler struct {
	Store  ScheduleStore
	Runner ScheduledBackupRunner
	// Deleter removes the bucket object behind each pruned backup_history
	// row after a successful retention prune. nil disables object
	// deletion entirely (store rows still get pruned), the same
	// optional-capability shape this codebase already uses elsewhere
	// (e.g. backupRunner itself being nil when no master key is
	// configured, cmd/levelrail/main.go).
	Deleter Deleter
	// Verifier, if set, re-checks a scheduled backup right after it
	// succeeds (runScheduled). nil disables auto-verification, same
	// optional-capability shape as Deleter above. Set directly like Now
	// below, not a constructor param, to keep NewScheduler's signature
	// stable for existing call sites.
	Verifier ScheduledVerifier
	// BaseBackups, if set, takes a physical base backup after each scheduled
	// logical backup of a PITR-enabled database, so point-in-time restore
	// stays fresh without a second schedule. nil disables it.
	BaseBackups ScheduledBaseBackupRunner
	Logger      *slog.Logger
	// Now returns the current time, the same testable-clock field
	// Runner.Now already establishes in runner.go: production code
	// leaves it nil and Tick falls back to time.Now, tests set it for
	// deterministic schedule matching.
	Now func() time.Time
	// contains filtered or unexported fields
}

Scheduler periodically checks which databases have a due cron schedule (store.DesiredDatabase's BackupSchedule/BackupTargetID/ BackupRetain, migrations/0023_scheduled_backups.sql) and runs a backup through the same Runner the manual trigger path (internal/api's handleTriggerBackup) uses, recording the attempt in store.BackupHistory exactly like a manual trigger does, since both paths call the identical Runner.RunBackup.

Tick's own shape (list what to evaluate, act, persist any resulting state, join and return errors rather than stopping at the first one) deliberately mirrors internal/alerting.Engine.Tick: that is this codebase's own established shape for "evaluate every enabled X on an interval, one broken X must never block the rest," and a cron schedule check is exactly that same shape applied to a different resource. Run's own ticker loop is the identical pattern internal/alerting.Engine.Run, internal/telemetry.Collector.Run, and every other periodic loop in cmd/levelrail/main.go already use.

Unlike alerting.Engine.Tick's own per-rule dispatch, a due backup runs synchronously inside Tick, not fired off into its own goroutine the way internal/api's handleTriggerBackup dispatches one: that handler's goroutine exists specifically because an HTTP response must return in milliseconds, a constraint that does not apply to a background ticker loop already running off the request path. Running sequentially here also means a database's backup finishing (or failing) is fully resolved, including its retention prune, before Tick moves on to evaluate the next database, which keeps this method's own control flow as a flat, easy-to-reason-about loop instead of needing a WaitGroup or a results channel to know when a tick's work is actually done.

Missed-schedule handling is a known, deliberate, and documented gap, not an oversight: Scheduler tracks each database's next fire time purely in memory (nextRun below), armed fresh the first tick a schedule is observed rather than reconstructed from store.BackupHistory on startup. A schedule that should have fired while the control plane was down does not "catch up" the moment it restarts; it simply waits for its next naturally occurring occurrence. This mirrors the honesty this codebase already extends to a related gap on the record-keeping side (store.BackupHistory's own status constants doc comment: "nothing currently sweeps a stuck running row back to failed... a known, honest gap for the eventual scheduler to close alongside its own retry logic, not invented speculatively here"). Building real catch-up/retry semantics is a materially bigger feature (it needs to decide how many missed runs to catch up on, how to avoid a thundering herd of backups firing at once after a long outage, and how that interacts with retention) than this task's scope asked for, so it is left as a follow-up rather than guessed at here.

func NewScheduler(store ScheduleStore, runner ScheduledBackupRunner, deleter Deleter, logger *slog.Logger) *Scheduler

func (s *Scheduler) Run(ctx context.Context, interval time.Duration) error

func (s *Scheduler) Tick(ctx context.Context) error

type SecretsResolver ¶

type SecretsResolver interface {
	Resolve(ctx context.Context, serviceName, envKey string) (string, error)
}

SecretsResolver is the one internal/secrets.Manager method Runner needs, narrowed to keep this package's tests from needing a real Manager (Manager itself has no interface today; a struct-shaped fake satisfying this one-method interface is enough).

type SqliteSnapshotter ¶ added in v0.2.0

type SqliteSnapshotter interface {
	Snapshot(ctx context.Context, volumeName, relPath string) (io.ReadCloser, error)
}

SqliteSnapshotter produces a consistent snapshot of a SQLite .db file living inside a named Docker volume, the sqlite-path counterpart of VolumeArchiver.

type TargetGetter ¶ added in v0.2.0

type TargetGetter interface {
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
}

TargetGetter is the one-method surface resolveTargetDestination needs, shared by BaseBackupHistoryStore and PITRHistoryStore so both can resolve a target's live Destination the identical way without each repeating the lookup-plus-two-secret-resolves sequence Runner. ResolveDestination (runner.go) already establishes for the logical-dump direction.

type TargetTestStore ¶

type TargetTestStore interface {
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
}

TargetTestStore is the store surface TargetTester needs, narrowed the same way HistoryStore/VerificationHistoryStore already are: only the one lookup a connection test actually requires.

type TargetTester ¶

type TargetTester struct {
	Store   TargetTestStore
	Secrets SecretsResolver
	Tester  Tester
}

TargetTester resolves targetID's stored fields plus its live credentials from internal/secrets and runs Tester.Test against them, the "verify a stored credential by actually using it, once, on demand" shape this codebase's own RegistryAuthTester established first for registry credentials.

func (t *TargetTester) TestTarget(ctx context.Context, targetID string) error

type Tester ¶

type Tester interface {
	Test(ctx context.Context, dest Destination) error
}

Tester probes a Destination for a lightweight, non-destructive connectivity and credential check, the test-connection counterpart Uploader/Deleter above establish for real backup/prune work.

type Uploader ¶

type Uploader interface {
	Upload(ctx context.Context, dest Destination, key string, r io.Reader, size int64) error
}

Uploader writes a single object to a Destination. Implementations must read r to completion (or return an error partway through, in which case the object must not be left partially visible under key, an S3-compatible multipart upload aborted rather than completed covers this for free) rather than assume a specific size up front: size is a hint for implementations that benefit from knowing it (choosing single-PUT vs. multipart), not a hard contract r will produce exactly that many bytes.

type VerificationHistoryStore ¶

type VerificationHistoryStore interface {
	GetBackupHistory(ctx context.Context, id string) (store.BackupHistory, error)
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
	StartBackupVerification(ctx context.Context, v store.BackupVerification) error
	FinishBackupVerification(ctx context.Context, id, status string, checksumMatch, sizeMatch, formatValid bool, downloadedBytes int64, errMsg, finishedAt string) error
}

VerificationHistoryStore is the store.DB surface VerifyRunner needs: GetBackupHistory/GetBackupTarget resolve exactly what DownloadRunner.Download's own identical pair already resolve (the object a verification attempt re-downloads is the same object a real download would fetch), plus Start/FinishBackupVerification to record the attempt itself, the same shape RestoreHistoryStore's own doc comment describes for the restore direction.

type VerifyRunner ¶

type VerifyRunner struct {
	Store      VerificationHistoryStore
	Secrets    SecretsResolver
	Downloader Downloader
	// Now returns the current time, matching Runner.Now and
	// RestoreRunner.Now's own "field, not time.Now called directly"
	// reasoning for deterministic tests.
	Now func() time.Time
}

VerifyRunner confirms a previously succeeded backup is still intact without ever attempting a live restore: re-download the object, re-hash it, and compare against what Runner recorded at backup time, the non-destructive counterpart RunRestore's own doc comment explicitly rules out for this feature (a live restore against a running system is real risk, not a safe thing to run unattended on a schedule or a button click).

func (r *VerifyRunner) VerifyBackup(ctx context.Context, verificationID, backupHistoryID, engine, checkedBy string) error

type VolumeArchiver ¶

type VolumeArchiver interface {
	Archive(ctx context.Context, volumeName string) (io.ReadCloser, error)
}

VolumeArchiver produces a tar archive of a named Docker volume's contents, the app-volume counterpart of Dumper for a database engine. The returned ReadCloser's Close must be called by every caller once done, the same contract Dumper's own doc comment establishes.

type VolumeCloneRestoreRunner ¶

type VolumeCloneRestoreRunner struct {
	Store          VolumeCloneRestoreStore
	Secrets        SecretsResolver
	Downloader     Downloader
	VolumeRestorer VolumeRestorer
	Volumes        VolumeCreator
	// Now returns the current time, the same "field, not time.Now called
	// directly" reasoning CloneRestoreRunner.Now's own doc comment gives.
	Now func() time.Time
}

VolumeCloneRestoreRunner restores a previously succeeded volume backup into a brand-new, standalone Docker volume rather than overwriting the one it came from: the volume counterpart of CloneRestoreRunner. Simpler than CloneRestoreRunner because there is no reconciler to wait on (see VolumeCreator's own doc comment); creating the volume and restoring into it both happen synchronously, one after the other, inside RunVolumeCloneRestore itself.

func (r *VolumeCloneRestoreRunner) RunVolumeCloneRestore(ctx context.Context, historyID, sourceServiceName, sourceVolumeName, newVolumeName, backupHistoryID string) error

type VolumeCloneRestoreStore ¶

type VolumeCloneRestoreStore interface {
	StartVolumeCloneRestore(ctx context.Context, h store.VolumeCloneRestore) error
	FinishVolumeCloneRestore(ctx context.Context, id, status, errMsg, finishedAt string) error
	// contains filtered or unexported methods
}

VolumeCloneRestoreStore is the narrow store surface VolumeCloneRestoreRunner needs: backupResolver's two methods to resolve the source backup, plus the clone-restore bookkeeping pair, the app service volume counterpart of CloneRestoreStore.

type VolumeCreator ¶

type VolumeCreator interface {
	EnsureVolume(ctx context.Context, name string) error
}

VolumeCreator is the narrow docker.Runtime surface VolumeCloneRestoreRunner needs to bring a brand-new, empty volume into existence before restoring into it: unlike CloneRestoreRunner's wait for a new database's own reconciler to report Ready, a bare Docker volume has no reconcile loop and exists the moment this call returns, so there is nothing to poll for here.

type VolumeRestorer ¶

type VolumeRestorer interface {
	Restore(ctx context.Context, volumeName string, archive io.Reader) error
}

VolumeRestorer untars a previously archived stream back into a named Docker volume, in place: it wipes every existing file the volume holds first, so this is always a full replace, never a merge, matching Restorer's own "the restore command exited zero" contract.

type VolumeRuntime ¶ added in v0.2.0

type VolumeRuntime interface {
	docker.Runtime
	RemoveVolume(ctx context.Context, name string) error
}

VolumeRuntime is docker.Runtime plus volume removal.

type WALFetcher ¶ added in v0.2.0

type WALFetcher struct {
	Runtime    docker.Runtime
	Lister     Lister
	Downloader Downloader
}

WALFetcher restores shipped WAL from a backup target into a database's local archive volume, so recovery works after the local copy is lost.

func (f *WALFetcher) Fetch(ctx context.Context, dest Destination, databaseName, walVolume, sinceLSN string) (int, error)

type WALFetching ¶ added in v0.2.0

type WALFetching interface {
	Fetch(ctx context.Context, dest Destination, databaseName, walVolume, sinceLSN string) (int, error)
}

WALFetching is the WALFetcher surface PITRRunner needs.

type WALShipScheduler ¶ added in v0.2.0

type WALShipScheduler struct {
	Store         WALShipStore
	Shipper       WALShipping
	Resolve       func(ctx context.Context, targetID string) (Destination, error)
	ContainerName func(databaseName string) string
	// IsLocal reports whether a database's node is the one this control plane's
	// Docker client talks to; remote nodes are skipped.
	IsLocal func(nodeID string) bool
	Logger  *slog.Logger
	// contains filtered or unexported fields
}

WALShipScheduler ships the WAL of every local PITR-enabled database to the target of its newest base backup on a fixed interval, bounding the data lost if the database host's disk dies to one interval of WAL.

func (s *WALShipScheduler) Run(ctx context.Context, interval time.Duration) error

func (s *WALShipScheduler) Tick(ctx context.Context)

func (s *WALShipScheduler) WALShipStatus(databaseName string) (WALShipStatus, bool)

type WALShipStatus ¶ added in v0.2.0

type WALShipStatus struct {
	LastAttemptAt time.Time
	LastSuccessAt time.Time
	LastError     string
	// Shipped counts segments uploaded since the control plane started.
	Shipped  int
	TargetID string
}

WALShipStatus is the outcome of the most recent shipping pass for one database, shown by "pitr status".

type WALShipStore ¶ added in v0.2.0

type WALShipStore interface {
	ListDesiredDatabases(ctx context.Context) ([]store.DesiredDatabase, error)
	ListBaseBackupHistory(ctx context.Context, databaseName string) ([]store.BaseBackupHistory, error)
}

WALShipStore is the store surface WALShipScheduler needs.

type WALShipper ¶ added in v0.2.0

type WALShipper struct {
	Runtime  Runtime
	Uploader Uploader
	Lister   Lister
	Logger   *slog.Logger
}

WALShipper copies a database's local WAL archive into its backup target so point-in-time restore survives losing the database host's disk.

func (s *WALShipper) Ship(ctx context.Context, dest Destination, databaseName, containerName string) (int, error)

type WALShipping ¶ added in v0.2.0

type WALShipping interface {
	Ship(ctx context.Context, dest Destination, databaseName, containerName string) (int, error)
}

WALShipping is the WALShipper surface BaseBackupRunner and the ship scheduler need.