backup

type AppContainerResolver ¶ added in v0.2.0

type AppContainerResolver func(ctx context.Context, serviceName string) (docker.Runtime, string, error)

AppContainerResolver finds the node runtime and running container of an app.

type BaseBackupHistoryStore ¶ added in v0.2.0

type BaseBackupHistoryStore interface {
	TargetGetter
	StartBaseBackupHistory(ctx context.Context, h store.BaseBackupHistory) error
	FinishBaseBackupHistory(ctx context.Context, id, status string, sizeBytes int64, lsn, errMsg, finishedAt string) error
}

BaseBackupHistoryStore is the store surface BaseBackupRunner needs, narrowed the same way HistoryStore is for the logical-dump direction (runner.go).

type BaseBackupMaintainer ¶ added in v0.2.0

type BaseBackupMaintainer interface {
	Prune(ctx context.Context, databaseName, containerName string) error
}

BaseBackupMaintainer is the retention hook PITRMaintainer implements.

type BaseBackupRunner ¶ added in v0.2.0

type BaseBackupRunner struct {
	Store        BaseBackupHistoryStore
	Secrets      SecretsResolver
	BaseBackuper BaseBackuper
	Uploader     Uploader
	// Runtime, if set, backs StartLSN's best-effort diagnostic lookup
	// (base_backup.go). nil is valid: RunBaseBackup simply records an
	// empty LSN, the same "optional capability" shape this package
	// already uses for Runner.VolumeArchiver.
	Runtime Runtime
	// Maintainer, if set, prunes old base backups and archived WAL after each
	// successful base backup. Failures are logged, never returned: the new
	// backup already succeeded.
	Maintainer BaseBackupMaintainer
	// WAL, when set, ships the local WAL archive to the target right after a
	// successful base backup, so the bucket holds what a restore replays.
	WAL WALShipping
	// WorkDir/Now match Runner's own identically-named fields exactly
	// (runner.go's own doc comments for each apply here unchanged).
	WorkDir string
	Now     func() time.Time
}

BaseBackupRunner ties a BaseBackuper and an Uploader to store and internal/secrets, the physical-backup counterpart of Runner (runner.go): the only piece in this package that knows how a base backup attempt gets recorded and how a target's live credentials get resolved.

func (r *BaseBackupRunner) RunBaseBackup(ctx context.Context, historyID, databaseName, containerName, targetID string) error

type BaseBackuper ¶ added in v0.2.0

type BaseBackuper interface {
	BaseBackup(ctx context.Context, containerName string) (io.ReadCloser, error)
}

BaseBackuper produces a physical base backup as a tar stream, for a single PITR-enabled Postgres database container already running under containerName: the physical counterpart of Dumper (dump.go), which produces a logical dump.

type BucketProtection ¶ added in v0.2.0

type BucketProtection struct {
	ObjectLock bool
	LockMode   string
	Versioning string
	CanDelete  bool
}

BucketProtection is what a bucket does to protect stored backups.

func (p BucketProtection) Level() string

func (p BucketProtection) Warning() string

type CloneRestoreRunner ¶

type CloneRestoreRunner struct {
	Store      CloneRestoreStore
	Secrets    SecretsResolver
	Downloader Downloader
	Restorer   Restorer
	// Identities decrypt sealed database dumps; empty is valid for raw ones.
	Identities []age.Identity
	// Now returns the current time, the same "field, not time.Now called
	// directly" reasoning RestoreRunner.Now's own doc comment gives.
	Now func() time.Time
	// ReadyTimeout/PollInterval override defaultCloneRestoreReadyTimeout/
	// defaultCloneRestoreReadyPollInterval when non-zero, so tests don't
	// have to wait out the real default.
	ReadyTimeout time.Duration
	PollInterval time.Duration
	// SettleTimeout overrides defaultCloneRestoreSettleTimeout when non-zero.
	SettleTimeout time.Duration
}

CloneRestoreRunner restores a previously succeeded backup into a brand-new database rather than overwriting the one it came from: the non-destructive counterpart to RestoreRunner. internal/api creates the new database's desired state itself (reusing handleCreateDatabase's own path, see database_clone_restore.go's own doc comment for why) before ever calling RunCloneRestore; this type's only job is to wait for that database's reconciler to bring it up and then run the identical download-and-apply RestoreRunner already implements.

func (r *CloneRestoreRunner) RunCloneRestore(ctx context.Context, historyID, sourceDatabaseName, newDatabaseName, backupHistoryID, engine, containerName, controllerName string) error

type CloneRestoreStore ¶

type CloneRestoreStore interface {
	StartCloneRestore(ctx context.Context, h store.CloneRestore) error
	FinishCloneRestore(ctx context.Context, id, status, errMsg, finishedAt string) error
	GetConditions(ctx context.Context, controllerName string) ([]reconcile.Condition, error)
	// contains filtered or unexported methods
}

CloneRestoreStore is the narrow store surface CloneRestoreRunner needs: backupResolver's two methods to resolve the source backup, the clone- restore bookkeeping pair, and GetConditions to detect when the newly created database has actually come up. A separate interface from RestoreHistoryStore, the same "narrow interface per real dependency" reasoning that type's own doc comment gives, since the two share no method beyond the backupResolver pair.

type ContainerBaseBackuper ¶ added in v0.2.0

type ContainerBaseBackuper struct {
	Runtime Runtime
}

ContainerBaseBackuper is the real BaseBackuper: it runs pg_basebackup inside the already-running container via docker.Runtime.Exec, the same "no separate copy of the tool, no exposed port" reasoning ContainerDumper's own doc comment gives.

func (b *ContainerBaseBackuper) BaseBackup(ctx context.Context, containerName string) (io.ReadCloser, error)

type ContainerDumper ¶

type ContainerDumper struct {
	Runtime docker.Runtime
	// SearchTaskTimeout/SearchPollInterval override the snapshot wait for
	// the HTTP API search engines.
	SearchTaskTimeout  time.Duration
	SearchPollInterval time.Duration
}

ContainerDumper is the real Dumper: it runs each engine's own native dump tool inside the already-running database container via docker.Runtime.Exec, so this process never needs its own copy of pg_dump/mysqldump/etc. and the database never needs a network port exposed anywhere reachable.

func (d *ContainerDumper) Dump(ctx context.Context, engine, containerName string) (io.ReadCloser, error)

type ContainerPITRRestorer ¶ added in v0.2.0

type ContainerPITRRestorer struct {
	Runtime docker.Runtime
}

ContainerPITRRestorer is the real PITRRestorer: the wipe-then-extract- then-configure sequence inside a short-lived helper container, the same createVolumeHelper mechanism ContainerVolumeRestorer uses.

func (r *ContainerPITRRestorer) Restore(ctx context.Context, dataVolumeName string, baseBackup io.Reader, targetTime time.Time) error

type ContainerQuiescer ¶ added in v0.2.0

type ContainerQuiescer struct {
	Resolve AppContainerResolver
}

ContainerQuiescer is the real Quiescer: hooks run with sh -c inside the app's container, pause freezes it for the duration of the archive.

func (q *ContainerQuiescer) Begin(ctx context.Context, serviceName string, p store.VolumeBackupPolicy) (func(context.Context) error, error)

type ContainerRestorer ¶

type ContainerRestorer struct {
	Runtime docker.Runtime
	// ReadyTimeout/ReadyInterval override the engine readiness wait defaults.
	ReadyTimeout  time.Duration
	ReadyInterval time.Duration
}

ContainerRestorer is the real Restorer: it runs each engine's own native restore path inside the running container via docker.Runtime.ExecWithInput, ContainerDumper's own reasoning plus a stdin stream.

func (r *ContainerRestorer) Restore(ctx context.Context, engine, containerName string, dump io.Reader) error

type ContainerSqliteSnapshotter ¶ added in v0.2.0

type ContainerSqliteSnapshotter struct {
	Runtime docker.Runtime
}

ContainerSqliteSnapshotter is the real SqliteSnapshotter: a helper container mounts the volume read-only and runs sqlite3's own ".backup" command, safe under the app's own concurrent writes since it goes through SQLite's file locking rather than a filesystem copy.

func (s *ContainerSqliteSnapshotter) Snapshot(ctx context.Context, volumeName, relPath string) (io.ReadCloser, error)

type ContainerVolumeArchiver ¶

type ContainerVolumeArchiver struct {
	Runtime docker.Runtime
}

ContainerVolumeArchiver is the real VolumeArchiver: it creates a helper container with volumeName mounted read-only, tars its contents to stdout, and removes the container once the caller has read the stream to completion (or given up partway through).

func (a *ContainerVolumeArchiver) Archive(ctx context.Context, volumeName string) (io.ReadCloser, error)

type ContainerVolumeRestorer ¶

type ContainerVolumeRestorer struct {
	Runtime docker.Runtime
}

ContainerVolumeRestorer is the real VolumeRestorer: it runs the wipe-then-extract command inside a short-lived helper container with volumeName mounted read-write.

func (r *ContainerVolumeRestorer) Restore(ctx context.Context, volumeName string, archive io.Reader) error

type CoverageResource ¶ added in v0.2.0

type CoverageResource struct {
	Kind     string
	AppName  string
	Name     string
	Schedule string
	TargetID string
	// Scratch marks preview and other disposable data, left out of the
	// protected totals.
	Scratch bool
}

CoverageResource is one database or app volume that exists in desired state, whether or not anything backs it up.

type CoverageRow ¶ added in v0.2.0

type CoverageRow struct {
	Kind            string
	AppName         string
	Name            string
	Scratch         bool
	State           string
	Reason          string
	PrimaryAction   string
	Schedule        string
	TargetID        string
	NextRun         string
	LastBackupAt    string
	LastBackupID    string
	LastVerifiedAt  string
	TotalBytes      int64
	BackupCount     int
	Encrypted       bool
	ProtectionLevel string
}

CoverageRow is the derived protection state of one CoverageResource.

type CoverageSummary ¶ added in v0.2.0

type CoverageSummary struct {
	Total      int
	Protected  int
	Verified   int
	Unverified int
	NeverRan   int
	Failing    int
	// Unprotected counts resources with no schedule or no target.
	Unprotected         int
	Scratch             int
	StorageBytes        int64
	LastVerifiedRestore string
	NextRun             string
}

CoverageSummary aggregates the non-scratch rows.

type DatabaseInfo ¶ added in v0.2.0

type DatabaseInfo func(ctx context.Context, databaseName string) (engine, version string, err error)

DatabaseInfo resolves a database's engine and version for a drill.

type DeleteHistoryStore ¶ added in v0.2.0

type DeleteHistoryStore interface {
	GetBackupHistory(ctx context.Context, id string) (store.BackupHistory, error)
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
	DeleteBackupHistory(ctx context.Context, id string) error
}

DeleteHistoryStore is the store.DB surface DeleteRunner needs: GetBackupHistory/GetBackupTarget resolve where a backup attempt's object actually lives, the identical pair DownloadHistoryStore's own doc comment describes; DeleteBackupHistory removes the row itself once the underlying object delete has been attempted.

type DeleteRunner ¶ added in v0.2.0

type DeleteRunner struct {
	Store   DeleteHistoryStore
	Secrets SecretsResolver
	// Deleter may be nil, the same optional-capability shape
	// Scheduler.Deleter's own doc comment establishes: DeleteBackup then
	// only removes the store row, never attempting the object delete.
	Deleter Deleter
	// Tombstones, if set, keeps an object whose delete failed so the scheduler sweep retries it after the row is gone.
	Tombstones TombstoneStore
	Logger     *slog.Logger
}

DeleteRunner deletes one backup attempt on demand: an operator-directed counterpart to Scheduler's own retention-driven deletePrunedObjects, sharing the same "best-effort object delete, the store row is the thing that must always go" contract (Scheduler.deletePrunedObjects's own doc comment). Unlike that batch path, DeleteBackup always finishes by removing the row, regardless of whether the object delete succeeded, failed, or couldn't even be attempted: an operator who explicitly named this one archive to delete should not be left with it still showing up in history because deleting the underlying object hit a storage-side problem, on top of Deleter's own contract that a missing object is already a no-op success (backup.go's own Deleter doc comment).

func (d *DeleteRunner) DeleteBackup(ctx context.Context, historyID string) error

type Deleter ¶

type Deleter interface {
	Delete(ctx context.Context, dest Destination, key string) error
}

Deleter removes a single object from a Destination, the inverse of Uploader. Implementations must treat deleting a key that is already gone (or never existed) as success, not an error: S3-compatible DeleteObject calls are naturally idempotent this way.

type Destination ¶

type Destination struct {
	// Provider is one of store.BackupProviderAWS / R2 / Custom. An
	// Uploader implementation may use it to pick defaults (e.g. AWS's
	// per-region default endpoint), but Endpoint below is always the
	// final authority when non-empty.
	Provider string
	// Endpoint is the S3-compatible API base URL. Empty for
	// BackupProviderAWS (the SDK's own region-based resolver applies);
	// required for R2 and Custom.
	Endpoint string
	Region   string
	Bucket   string

	AccessKeyID     string
	SecretAccessKey string
}

Destination is everything an Uploader needs to reach a specific bucket, credentials included. Built fresh per call from a store.BackupTarget plus its resolved internal/secrets values (see store.BackupTargetSecretsKey): never persisted or logged as a whole, since AccessKeyID and SecretAccessKey are live credentials.

func (d Destination) IsAgent() bool

type DownloadHistoryStore ¶

type DownloadHistoryStore interface {
	GetBackupHistory(ctx context.Context, id string) (store.BackupHistory, error)
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
}

DownloadHistoryStore is the store.DB surface DownloadRunner needs: GetBackupHistory to resolve the attempt a download names, GetBackupTarget to resolve where its object actually lives, the same pair RestoreHistoryStore's own doc comment describes for the restore direction. No Start/Finish pair here: unlike a backup or a restore, a download has no attempt of its own worth recording in a history table, it only reads an object an earlier, already-recorded backup wrote.

type DownloadRunner ¶

type DownloadRunner struct {
	Store      DownloadHistoryStore
	Secrets    SecretsResolver
	Downloader Downloader
}

DownloadRunner resolves a succeeded backup attempt to its live object stream, the read-only counterpart of Runner (dump-and-upload) and RestoreRunner (download-and-restore-in-place): given a store.BackupHistory ID, it resolves the row's target and credentials the identical way Runner.runDumpAndUpload and RestoreRunner.runDownloadAndRestore already do, then hands the caller (internal/api's handleDownloadBackup) the raw object stream to copy directly into an HTTP response, never buffering it here.

func (d *DownloadRunner) Download(ctx context.Context, historyID string) (io.ReadCloser, error)

type Downloader ¶

type Downloader interface {
	Download(ctx context.Context, dest Destination, key string) (io.ReadCloser, error)
}

Downloader retrieves a single object from a Destination, the exact reverse of Uploader: given the same (Destination, key) an earlier Upload call was made with, it returns that object's bytes as a stream. Implementations are not required to know the object's size up front (see S3Downloader's own doc comment for why GetObject already makes this a non-issue, unlike Upload's own unknown-size handling), only to stream it back faithfully.

type DrillFailureClass ¶

type DrillFailureClass struct {
	Kind     string
	Guidance string
}

DrillFailureClass names why a drill failed and what the operator should do.

func ClassifyDrillFailure(errText string) DrillFailureClass

type DrillRunner ¶ added in v0.2.0

type DrillRunner struct {
	Store      DrillStore
	Secrets    SecretsResolver
	Downloader Downloader
	Prober     ObjectProber
	Identities []age.Identity

	// Volume drills.
	Restorer VolumeRestorer
	Archiver VolumeArchiver
	Volumes  VolumeCreator
	Remover  VolumeRemover

	// Database drills. Any nil leaves databases at integrity-only checks.
	Runtime    docker.Runtime
	DBRestorer Restorer
	DBInfo     DatabaseInfo
	ImageFor   func(engine, version string) string

	// ScratchMemoryBytes and ScratchNanoCPUs cap scratch containers; zero uses the defaults.
	ScratchMemoryBytes int64
	ScratchNanoCPUs    int64

	// OnFailure is called once for every failed drill.
	OnFailure func(ctx context.Context, d store.BackupDrill)
	Logger    *slog.Logger
	Now       func() time.Time
}

DrillRunner restores a stored backup into a scratch resource, validates it and destroys the scratch resource, recording every step.

func (r *DrillRunner) RunDrill(ctx context.Context, drillID, backupHistoryID, trigger string) (store.BackupDrill, error)

func (r *DrillRunner) SweepOrphans(ctx context.Context) (int, error)

type DrillScheduler ¶ added in v0.2.0

type DrillScheduler struct {
	Store      DrillSchedulerStore
	Secrets    SecretsResolver
	Runner     *DrillRunner
	Prober     ObjectProber
	Downloader Downloader
	Protection ProtectionProber
	// Interval is the drill period per resource; zero disables drills.
	Interval time.Duration
	// StaleAfter is the age a running backup must reach before the sweep
	// finalizes it.
	StaleAfter time.Duration
	Logger     *slog.Logger
	Now        func() time.Time
	// contains filtered or unexported fields
}

DrillScheduler drills every backed up resource on an interval, finalizes backups the control plane lost track of, and refreshes bucket protection probes.

func (s *DrillScheduler) Run(ctx context.Context, every time.Duration) error

func (s *DrillScheduler) Tick(ctx context.Context) error

type DrillSchedulerStore ¶ added in v0.2.0

type DrillSchedulerStore interface {
	ListSucceededBackups(ctx context.Context) ([]store.BackupHistory, error)
	ListBackupDrills(ctx context.Context, serviceName, volumeName, databaseName string, limit int) ([]store.BackupDrill, error)
	ListRunningBackups(ctx context.Context, startedBefore time.Time) ([]store.BackupHistory, error)
	FinishBackupHistory(ctx context.Context, id, status string, sizeBytes int64, checksum, errMsg, finishedAt string) error
	ListBackupTargets(ctx context.Context) ([]store.BackupTarget, error)
	SetBackupTargetProtection(ctx context.Context, p store.BackupTargetProtection) error
	// contains filtered or unexported methods
}

DrillSchedulerStore is the store surface the drill scheduler needs.

type DrillStore ¶ added in v0.2.0

type DrillStore interface {
	StartBackupDrill(ctx context.Context, d store.BackupDrill) error
	FinishBackupDrill(ctx context.Context, d store.BackupDrill) error
	// contains filtered or unexported methods
}

DrillStore is the store surface a drill needs.

type DrillSummary ¶ added in v0.2.0

type DrillSummary struct {
	ID      string
	At      string
	Status  string
	Stage   string
	Error   string
	Trigger string
	Files   int64
}

DrillSummary is one drill in a health row.

type Dumper ¶

type Dumper interface {
	Dump(ctx context.Context, engine, containerName string) (io.ReadCloser, error)
}

Dumper produces a database dump as a stream, for a single managed database container already running under containerName (the same name application.ContainerName / the database controller's own container-naming convention already produces; Dumper does not construct or look this up itself, Runner does).

The returned ReadCloser's Close must be called by every caller once done, success or failure, the same contract io.ReadCloser always carries; a Dumper backed by a subprocess or a docker exec stream depends on Close to release that underlying resource.

type EncryptionStatus ¶ added in v0.2.0

type EncryptionStatus struct {
	Enabled   bool
	Recipient string
	Codec     string
}

EncryptionStatus describes client side encryption of new volume backups.

type FailedDrill ¶ added in v0.2.0

type FailedDrill struct {
	Resource string
	Reason   string
	At       string
}

FailedDrill is a resource whose latest drill failed.

type GFSStore ¶ added in v0.2.0

type GFSStore interface {
	ListSucceededVolumeBackups(ctx context.Context, serviceName, volumeName string) ([]store.BackupHistory, error)
	DeleteBackupHistory(ctx context.Context, id string) error
}

GFSStore is the store surface grandfather-father-son pruning needs.

type HealthBackup ¶ added in v0.2.0

type HealthBackup struct {
	ID        string
	At        string
	Status    string
	SizeBytes int64
	Codec     string
	Error     string
}

HealthBackup is one backup in a health row.

type HistoryStore ¶

type HistoryStore interface {
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
	StartBackupHistory(ctx context.Context, h store.BackupHistory) error
	FinishBackupHistory(ctx context.Context, id, status string, sizeBytes int64, checksum, errMsg, finishedAt string) error
}

HistoryStore is the store.DB surface Runner needs, narrowed the same way SecretsResolver is: a fake satisfying six methods is a lot less to stub than the full *store.DB.

type Lister ¶ added in v0.2.0

type Lister interface {
	List(ctx context.Context, dest Destination, prefix string) ([]RemoteObject, error)
}

Lister lists the objects under a key prefix in a backup target.

type MajorUpgradeRunner ¶ added in v0.2.0

type MajorUpgradeRunner struct {
	Store    MajorUpgradeStore
	Runtime  VolumeRuntime
	Dumper   Dumper
	Restorer Restorer
	Nudge    func()
	Now      func() time.Time
	WorkDir  string
	Logger   *slog.Logger
	// Wait bounds each wait for a container to stop, start or accept
	// connections; Poll is the interval between checks.
	Wait time.Duration
	Poll time.Duration
}

MajorUpgradeRunner upgrades a managed Postgres database across a major version by dump and restore. The old data volume is never rewritten in place: it is copied to a snapshot volume first, the dump is taken from the snapshot, and any failure after the live volume is wiped restores the snapshot and the old version.

func (r *MajorUpgradeRunner) DiscardSnapshot(ctx context.Context, upgradeID string) error

func (r *MajorUpgradeRunner) RecoverInterrupted(ctx context.Context)

func (r *MajorUpgradeRunner) Rollback(ctx context.Context, upgradeID string) error

func (r *MajorUpgradeRunner) RunMajorUpgrade(ctx context.Context, historyID, databaseName, toVersion string) error

type MajorUpgradeStore ¶ added in v0.2.0

type MajorUpgradeStore interface {
	GetDesiredDatabase(ctx context.Context, name string) (*store.DesiredDatabase, error)
	SaveDesiredDatabase(ctx context.Context, d store.DesiredDatabase) error
	UpdateDatabaseSuspended(ctx context.Context, name string, suspended bool) error
	StartMajorUpgrade(ctx context.Context, u store.MajorUpgrade) error
	UpdateMajorUpgradePhase(ctx context.Context, id, phase, snapshotVolume string) error
	FinishMajorUpgrade(ctx context.Context, id, status, errMsg, finishedAt string) error
	GetMajorUpgrade(ctx context.Context, id string) (store.MajorUpgrade, error)
	ListRunningMajorUpgrades(ctx context.Context) ([]store.MajorUpgrade, error)
	ClearMajorUpgradeSnapshot(ctx context.Context, id string) error
}

MajorUpgradeStore is the store surface MajorUpgradeRunner needs.

type NodeFiles ¶

type NodeFiles interface {
	Put(ctx context.Context, nodeID, path string, r io.Reader) error
	Get(ctx context.Context, nodeID, path string) (io.ReadCloser, error)
	Delete(ctx context.Context, nodeID, path string) error
	List(ctx context.Context, nodeID, prefix string) ([]RemoteObject, error)
}

NodeFiles is the control plane view of a node's confined storage directory, reached over the existing agent session.

type ObjectInfo ¶ added in v0.2.0

type ObjectInfo struct {
	Exists bool
	Size   int64
}

ObjectInfo is what a HEAD request learned about a stored backup object.

type ObjectProber ¶ added in v0.2.0

type ObjectProber interface {
	Head(ctx context.Context, dest Destination, key string) (ObjectInfo, error)
}

ObjectProber checks an object without downloading it.

type PITRHistoryStore ¶ added in v0.2.0

type PITRHistoryStore interface {
	TargetGetter
	GetBaseBackupHistory(ctx context.Context, id string) (store.BaseBackupHistory, error)
	GetOldestSucceededBaseBackup(ctx context.Context, databaseName string) (store.BaseBackupHistory, bool, error)
	StartPITRRestoreHistory(ctx context.Context, h store.PITRRestoreHistory) error
	FinishPITRRestoreHistory(ctx context.Context, id, status, errMsg, finishedAt string) error
	UpdateDatabaseSuspended(ctx context.Context, name string, suspended bool) error
}

PITRHistoryStore is the store surface PITRRunner needs. See ADR 016 for why UpdateDatabaseSuspended is part of a restore's own store surface rather than a separate concern.

type PITRMaintainer ¶ added in v0.2.0

type PITRMaintainer struct {
	Store   PITRMaintenanceStore
	Resolve func(ctx context.Context, targetID string) (Destination, error)
	Deleter Deleter
	// Lister enables pruning the shipped WAL copy in the bucket too.
	Lister  Lister
	Runtime Runtime
	// Keep overrides the env default when positive.
	Keep   int
	Logger *slog.Logger
}

PITRMaintainer keeps a PITR-enabled database's disk and bucket bounded: without it the local WAL archive and the base backups grow forever.

func (m *PITRMaintainer) Prune(ctx context.Context, databaseName, containerName string) error

type PITRMaintenanceStore ¶ added in v0.2.0

type PITRMaintenanceStore interface {
	ListBaseBackupHistory(ctx context.Context, databaseName string) ([]store.BaseBackupHistory, error)
	DeleteBaseBackupHistory(ctx context.Context, id string) error
}

PITRMaintenanceStore is the store surface PITRMaintainer needs.

type PITRRestorer ¶ added in v0.2.0

type PITRRestorer interface {
	Restore(ctx context.Context, dataVolumeName string, baseBackup io.Reader, targetTime time.Time) error
}

PITRRestorer applies a base backup to dataVolumeName in place, configuring Postgres to replay WAL to targetTime and promote: the point-in-time-restore counterpart of Restorer. Unlike Restorer, this never touches a running container; dataVolumeName's container must already be stopped (PITRRunner's job).

type PITRRunner ¶ added in v0.2.0

type PITRRunner struct {
	Store      PITRHistoryStore
	Secrets    SecretsResolver
	Downloader Downloader
	Restorer   PITRRestorer
	// WAL and WALVolume, when set, pull the WAL shipped to the target into the
	// database's archive volume before the destructive phase, so a restore works
	// after the local archive is lost. WALVolume maps a database name to it.
	WAL       WALFetching
	WALVolume func(databaseName string) string
	// Runtime backs the teardown/startup/promotion polls: Suspended and
	// Nudge only ever request a change, this verifies it happened.
	Runtime docker.Runtime
	// Nudge requests an immediate reconcile pass (*reconcile.Engine's
	// own Nudge); nil is valid, falling back to the resync cadence.
	Nudge func()
	// Now returns the current time for history timestamps; nil falls
	// back to time.Now.
	Now func() time.Time
	// TeardownTimeout/StartupTimeout/PromoteTimeout/PollInterval override
	// the package defaults above when non-zero, for tests.
	TeardownTimeout time.Duration
	StartupTimeout  time.Duration
	PromoteTimeout  time.Duration
	PollInterval    time.Duration
}

PITRRunner orchestrates one point-in-time restore end to end: download the named base backup, hand it to a PITRRestorer to wipe and re-populate the database's data volume with recovery configured for targetTime, then wait for Postgres to actually promote before calling the attempt succeeded. Unlike RestoreRunner, this needs the container down for the whole window; see ADR 016 for why Suspended is what keeps the reconciler from racing that, and why promotion is polled directly rather than trusted to the reconciler's Ready condition.

func (r *PITRRunner) RunPITRRestore(ctx context.Context, historyID, databaseName, containerName, dataVolumeName, baseBackupHistoryID string, targetTime time.Time) error

func (r *PITRRunner) ValidateTarget(ctx context.Context, databaseName, containerName string, target time.Time) error

func (r *PITRRunner) Window(ctx context.Context, databaseName, containerName string) (PITRWindow, error)

func (r *PITRRunner) WindowBounds(ctx context.Context, databaseName, containerName string) (hasBaseBackup bool, start, end time.Time, err error)

type PITRWindow ¶ added in v0.2.0

type PITRWindow struct {
	HasBaseBackup bool
	Start         time.Time
	End           time.Time
}

PITRWindow is one database's currently recoverable point-in-time restore range: Start is the oldest succeeded base backup's own started_at (nothing earlier is recoverable, since there is no base to replay WAL forward from), End is the latest instant WAL archiving has continuously and provably reached (RecoverableWindowEnd). HasBaseBackup is false whenever no base backup has ever succeeded yet, in which case Start/End are both zero and carry no meaning.

type Protection ¶ added in v0.2.0

type Protection struct {
	Store  ProtectionStore
	Drills *DrillRunner
	Prober ProtectionProber
	// Secrets resolves target credentials for probes and restores.
	Secrets    SecretsResolver
	Downloader Downloader
	// RestoreStore records restore-to-new-volume attempts.
	RestoreStore VolumeCloneRestoreStore
	Sealer       *Sealer
	// DrillInterval is the configured drill period, used to flag a resource
	// whose last verified restore is older than two periods.
	DrillInterval time.Duration
	Logger        *slog.Logger
	Now           func() time.Time
}

Protection reports backup health and starts drills.

func (p *Protection) Encryption() EncryptionStatus

func (p *Protection) FailedDrills(ctx context.Context) ([]FailedDrill, error)

func (p *Protection) Health(ctx context.Context) ([]ResourceHealth, error)

func (p *Protection) RefreshProtection(ctx context.Context, targetIDs []string) ([]store.BackupTargetProtection, error)

func (p *Protection) RestoreVolumeTo(ctx context.Context, rtm docker.Runtime, historyID, sourceService, sourceVolume, newVolume, backupID string) error

func (p *Protection) StartDrill(ctx context.Context, backupID string) (string, error)

func (p *Protection) VolumeExists(ctx context.Context, rtm docker.Runtime, name string) (exists, known bool, err error)

type ProtectionProber ¶ added in v0.2.0

type ProtectionProber interface {
	Probe(ctx context.Context, dest Destination) (BucketProtection, error)
}

ProtectionProber inspects a bucket's protection settings.

type ProtectionRecorder ¶ added in v0.2.0

type ProtectionRecorder interface {
	SetBackupTargetProtection(ctx context.Context, p store.BackupTargetProtection) error
	// contains filtered or unexported methods
}

ProtectionRecorder is the slice of the store RecordProtection writes to.

type ProtectionStore ¶ added in v0.2.0

type ProtectionStore interface {
	ListSucceededBackups(ctx context.Context) ([]store.BackupHistory, error)
	ListAllBackupHistory(ctx context.Context, limit int, before *time.Time) ([]store.BackupHistory, error)
	ListScheduledServiceVolumes(ctx context.Context) ([]store.ServiceVolumeBackupConfig, error)
	ListScheduledDatabases(ctx context.Context) ([]store.DesiredDatabase, error)
	ListBackupDrills(ctx context.Context, serviceName, volumeName, databaseName string, limit int) ([]store.BackupDrill, error)
	ListBackupTargetProtection(ctx context.Context) ([]store.BackupTargetProtection, error)
	SetBackupTargetProtection(ctx context.Context, p store.BackupTargetProtection) error
	GetVolumeBackupPolicy(ctx context.Context, serviceName, volumeName string) (store.VolumeBackupPolicy, error)
	SetVolumeBackupPolicy(ctx context.Context, p store.VolumeBackupPolicy) error
	// contains filtered or unexported methods
}

ProtectionStore is the store surface the protection service reads and writes.

type Quiescer ¶ added in v0.2.0

type Quiescer interface {
	Begin(ctx context.Context, serviceName string, p store.VolumeBackupPolicy) (end func(context.Context) error, err error)
}

Quiescer makes a volume consistent for the duration of its archive. It runs the policy's pre-hook and optionally pauses the app. The returned end function undoes the pause and runs the post-hook.

type RemoteObject ¶ added in v0.2.0

type RemoteObject struct {
	Key  string
	Size int64
}

RemoteObject is one object found under a bucket prefix.

type ResourceHealth ¶ added in v0.2.0

type ResourceHealth struct {
	Kind         string
	AppName      string
	ResourceName string
	TargetID     string
	Schedule     string
	NextRun      string
	BackupCount  int
	TotalBytes   int64
	LastAttempt  *HealthBackup
	LastBackup   *HealthBackup
	// LastVerifiedRestore is the newest passed drill: proof a restore
	// worked, not just that a file exists.
	LastVerifiedRestore *DrillSummary
	LastDrill           *DrillSummary
	// Encrypted is true when the newest backup is client side encrypted.
	Encrypted       bool
	State           string
	StateReason     string
	ProtectionLevel string
	Warning         string
}

ResourceHealth is the protection state of one database or app volume.

type RestoreHistoryStore ¶

type RestoreHistoryStore interface {
	GetBackupHistory(ctx context.Context, id string) (store.BackupHistory, error)
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
	StartRestoreHistory(ctx context.Context, h store.RestoreHistory) error
	FinishRestoreHistory(ctx context.Context, id, status, errMsg, finishedAt string) error
}

RestoreHistoryStore is the store.DB surface RestoreRunner needs, narrowed the same way Runner's own HistoryStore is: GetBackupHistory to resolve the source backup a restore names, GetBackupTarget to resolve where that backup's object actually lives, and StartRestoreHistory/FinishRestoreHistory to record the restore attempt itself. Deliberately not HistoryStore plus new methods bolted on: a fake satisfying HistoryStore's forward-backup methods has no reason to also implement restore's, and the reverse is equally true, so keeping them as two separate narrow interfaces (which happen to overlap on GetBackupTarget, satisfied by the same *store.DB either way) matches this codebase's own "narrow interface per real dependency, not one grab-bag interface per package" convention.

type RestoreRunner ¶

type RestoreRunner struct {
	Store      RestoreHistoryStore
	Secrets    SecretsResolver
	Downloader Downloader
	Restorer   Restorer
	// VolumeRestorer backs RunVolumeRestore the same way Restorer backs
	// RunRestore. nil is valid, the same "optional capability" shape
	// Runner.VolumeArchiver's own doc comment establishes.
	VolumeRestorer VolumeRestorer
	// Identities decrypt sealed volume backups; empty is valid for raw ones.
	Identities []age.Identity
	// WorkDir is RunRestore/RunVolumeRestore's disk-space preflight
	// target, matching Runner.WorkDir's own fallback-to-APP_DATA_DIR,
	// skip-if-still-empty behavior.
	WorkDir string
	// Now returns the current time, matching Runner.Now's own "field, not
	// time.Now called directly" reasoning for deterministic tests.
	Now func() time.Time
}

RestoreRunner ties a Downloader and a Restorer to store and internal/secrets, the restore counterpart of Runner: the only piece in this package that knows how a restore attempt gets recorded and how a target's live credentials get resolved. internal/api constructs one real RestoreRunner at startup, the same place it constructs Runner, and calls RunRestore per triggered restore.

A separate type from Runner rather than a second method on it: Runner is already a coherent "dump this, upload it" unit with its own field set (Dumper, Uploader); RestoreRunner's fields (Downloader, Restorer) point the opposite direction and share nothing with Runner's except Store's underlying *store.DB value and Secrets' underlying *secrets.Manager value, both already expressed as separate narrow interfaces above and in runner.go rather than a shared struct field, so there is no real duplication to unify by merging the two types, only two independently-testable units that happen to be wired from the same concrete dependencies at startup.

func (r *RestoreRunner) RunRestore(ctx context.Context, historyID, databaseName, backupHistoryID, engine, containerName string) error

func (r *RestoreRunner) RunVolumeRestore(ctx context.Context, historyID, serviceName, volumeName, dockerVolumeName, backupHistoryID string) error

type Restorer ¶

type Restorer interface {
	Restore(ctx context.Context, engine, containerName string, dump io.Reader) error
}

Restorer applies a previously taken dump back onto a live database container. Like Dumper, it takes no credentials beyond what the container's own environment already provides. Its contract is "the restore command exited zero," not "and the data was verified correct afterward."

type RetentionItem ¶ added in v0.2.0

type RetentionItem struct {
	ID string
	At time.Time
}

RetentionItem is one backup candidate for retention.

type RetentionPolicy ¶ added in v0.2.0

type RetentionPolicy struct {
	Daily   int
	Weekly  int
	Monthly int
}

RetentionPolicy keeps the newest backup of each of the last Daily days, Weekly ISO weeks and Monthly months that have one. A zero policy keeps everything (it expires nothing).

func (p RetentionPolicy) Active() bool

type Routed ¶

type Routed struct {
	Files NodeFiles
}

Routed sends operations for an agent destination to the node and everything else to the S3 implementations.

func (r Routed) Delete(ctx context.Context, dest Destination, key string) error

func (r Routed) Download(ctx context.Context, dest Destination, key string) (io.ReadCloser, error)

func (r Routed) List(ctx context.Context, dest Destination, prefix string) ([]RemoteObject, error)

func (r Routed) Test(ctx context.Context, dest Destination) error

func (r Routed) Upload(ctx context.Context, dest Destination, key string, rd io.Reader, size int64) error

type Runner ¶

type Runner struct {
	Store    HistoryStore
	Secrets  SecretsResolver
	Dumper   Dumper
	Uploader Uploader
	// VolumeArchiver backs RunVolumeBackup the same way Dumper backs
	// RunBackup. nil is valid: a control plane with no volume-backup
	// callers configured (today, none do this by default) simply never
	// calls RunVolumeBackup, the same "optional capability" shape this
	// codebase already uses for Scheduler.Deleter/Verifier.
	VolumeArchiver VolumeArchiver
	// SqliteSnapshotter backs RunVolumeBackup when a volume's sqlite_path
	// is set; nil is valid the same way VolumeArchiver's nil is.
	SqliteSnapshotter SqliteSnapshotter
	// WorkDir is the directory RunBackup/RunVolumeBackup's disk-space
	// preflight checks (checkDiskSpace) before starting. Empty falls back
	// to APP_DATA_DIR (resolveWorkDir), and empty even then skips the
	// check entirely: see checkDiskSpace's own doc comment for why a
	// backup, which streams straight from its source to its destination
	// bucket, still has a real local directory worth guarding.
	WorkDir string
	// Volume holds the optional sealed-backup behavior: compression,
	// encryption, hooks, budgets. nil keeps the raw legacy encoding.
	Volume *VolumeBackupOptions
	// Now returns the current time. A field, not time.Now called
	// directly, so tests get deterministic timestamps without a real
	// clock dependency; production code leaves it nil and RunBackup
	// falls back to time.Now.
	Now func() time.Time
}

Runner ties a Dumper and an Uploader to store and internal/secrets: the only piece in this package that knows how a backup attempt gets recorded and how a target's live credentials get resolved. internal/api constructs one real Runner at startup (the same "wire concrete implementations once, in cmd/levelrail or internal/api's own constructor" pattern every other cross-package dependency in this codebase already follows) and calls RunBackup per triggered backup.

func (r *Runner) ResolveDestination(ctx context.Context, targetID string) (Destination, error)

func (r *Runner) RunBackup(ctx context.Context, historyID, databaseName, engine, containerName, targetID string) error

func (r *Runner) RunVolumeBackup(ctx context.Context, historyID, serviceName, volumeName, dockerVolumeName, targetID, sqlitePath string) error

type Runtime ¶ added in v0.2.0

type Runtime interface {
	Exec(ctx context.Context, containerID string, cmd []string) (io.ReadCloser, error)
	ExecWithInput(ctx context.Context, containerID string, cmd []string, stdin io.Reader) (io.ReadCloser, error)
}

Runtime is the narrow docker.Runtime surface (Exec/ExecWithInput) this file's functions and ContainerPITRRestorer need.

type S3Deleter ¶

type S3Deleter struct{}

S3Deleter is the real Deleter, covering AWS/R2/custom endpoints the same way S3Uploader does, reusing newS3Client (uploader.go) rather than building its own client.

func (S3Deleter) Delete(ctx context.Context, dest Destination, key string) error

type S3Downloader ¶

type S3Downloader struct{}

S3Downloader is the real Downloader: the download side of the same AWS/R2/custom-endpoint story S3Uploader already covers, one implementation for every provider this codebase supports, driven by the same Destination.Endpoint-is-the-final-authority rule.

Unlike S3Uploader, this has no multipart-vs-single-PUT decision to make: S3's GetObject API already streams a response body of whatever size the object actually is, with no equivalent to PutObject's Content-Length requirement on the request side, so there's no unknown-size problem here to work around with an upload-manager-style abstraction. A plain client.GetObject call is the entire implementation.

func (S3Downloader) Download(ctx context.Context, dest Destination, key string) (io.ReadCloser, error)

type S3Lister ¶ added in v0.2.0

type S3Lister struct{}

S3Lister is the real Lister, over ListObjectsV2.

func (S3Lister) List(ctx context.Context, dest Destination, prefix string) ([]RemoteObject, error)

type S3Prober ¶ added in v0.2.0

type S3Prober struct{}

S3Prober is the real ObjectProber.

func (S3Prober) Head(ctx context.Context, dest Destination, key string) (ObjectInfo, error)

type S3ProtectionProber ¶ added in v0.2.0

type S3ProtectionProber struct{}

S3ProtectionProber is the real ProtectionProber. It writes and deletes one small canary object to learn whether the key can delete.

func (S3ProtectionProber) Probe(ctx context.Context, dest Destination) (BucketProtection, error)

type S3Tester ¶

type S3Tester struct{}

S3Tester is the real Tester, covering AWS/R2/custom endpoints the same way S3Uploader does, reusing newS3Client rather than building its own client.

func (S3Tester) Test(ctx context.Context, dest Destination) error

type S3Uploader ¶

type S3Uploader struct{}

S3Uploader is the real Uploader: every backup target this codebase supports (AWS, R2, and any other S3-compatible custom endpoint) speaks the same REST API, so one implementation covers all three, driven entirely by the Provider/Endpoint distinction already captured on Destination.

A new S3 client is built per Upload call rather than cached on the struct: Destination carries live, per-target credentials resolved fresh from internal/secrets on every backup run (see Runner), and a target's bucket, region, or endpoint can change between runs too. Backups are infrequent (the default schedule is once a day, see store.BackupTarget's own docs), so the cost of rebuilding a client each time is not worth the complexity of a cache that would need invalidating whenever a target's settings change.

func (S3Uploader) Upload(ctx context.Context, dest Destination, key string, r io.Reader, _ int64) error

type ScheduleStore ¶

type ScheduleStore interface {
	ListScheduledDatabases(ctx context.Context) ([]store.DesiredDatabase, error)
	PruneBackupHistory(ctx context.Context, databaseName string, keep int, olderThan time.Time) ([]store.PrunedBackup, error)
	// ListScheduledServiceVolumes/PruneServiceVolumeBackupHistory/
	// ResolveServiceVolumeDockerName are the volume counterparts of the
	// two database methods above plus one extra resolution step a
	// database schedule doesn't need: store.ServiceVolumeBackupConfig
	// only carries a volume's logical name (what an operator wrote in
	// app.yaml), never its real Docker volume name, so RunVolumeBackup's
	// dockerVolumeName parameter has to be resolved fresh every tick, the
	// same "re-derive, never cache" principle this whole interface
	// already follows for which databases/volumes are even scheduled.
	ListScheduledServiceVolumes(ctx context.Context) ([]store.ServiceVolumeBackupConfig, error)
	PruneServiceVolumeBackupHistory(ctx context.Context, serviceName, volumeName string, keep int, olderThan time.Time) ([]store.PrunedBackup, error)
	ResolveServiceVolumeDockerName(ctx context.Context, serviceName, volumeName string) (string, error)
}

ScheduleStore is the narrow store surface Scheduler needs: which databases currently have a due-able cron schedule, re-derived fresh on every tick, and the retention prune a successful scheduled run triggers afterward. *store.DB satisfies this structurally, the same "consumer defines its own narrow interface" shape HistoryStore (runner.go) already establishes for this package's other store dependency.

type ScheduledBackupRunner ¶

type ScheduledBackupRunner interface {
	RunBackup(ctx context.Context, historyID, databaseName, engine, containerName, targetID string) error
	// RunVolumeBackup matches api.ServiceVolumeBackupRunner's shape
	// (internal/api/app_volume_backups.go); *Runner satisfies both, the
	// volume counterpart of RunBackup's own doc comment.
	RunVolumeBackup(ctx context.Context, historyID, serviceName, volumeName, dockerVolumeName, targetID, sqlitePath string) error
	// ResolveDestination resolves a backup target ID into a live
	// Destination (*Runner.ResolveDestination, runner.go), the same
	// resolution RunBackup does internally before every upload. Needed
	// separately here because retention pruning deletes objects for a
	// pruned row's own TargetID after a run, not during one.
	ResolveDestination(ctx context.Context, targetID string) (Destination, error)
}

ScheduledBackupRunner is the surface Scheduler needs to actually run a backup once its schedule comes due: identical in shape to internal/api's own BackupRunner interface (internal/api/backups.go), redeclared here rather than imported across the package boundary, the same reasoning that file's own BackupRunner doc comment gives for why internal/api never imports internal/backup directly. *Runner satisfies this structurally, so cmd/levelrail/main.go wires the exact same *Runner instance into both api.WithBackupRunner (the manual trigger path) and NewScheduler (this one): one backup implementation, two ways to invoke it, exactly the design internal/spec.Backup's own doc comment implies by giving app.yaml a single databases[].backup block rather than separate manual/scheduled config shapes.

type ScheduledBaseBackupRunner ¶ added in v0.2.0

type ScheduledBaseBackupRunner interface {
	RunBaseBackup(ctx context.Context, historyID, databaseName, containerName, targetID string) error
}

ScheduledBaseBackupRunner is *BaseBackupRunner's RunBaseBackup.

type ScheduledVerifier ¶

type ScheduledVerifier interface {
	VerifyBackup(ctx context.Context, verificationID, backupHistoryID, engine, checkedBy string) error
}

ScheduledVerifier matches api.BackupVerifier's shape (internal/api/backup_verify.go); *VerifyRunner satisfies both, so cmd/levelrail/main.go shares one instance.

type Scheduler ¶

type Scheduler struct {
	Store  ScheduleStore
	Runner ScheduledBackupRunner
	// Deleter removes the bucket object behind each pruned backup_history
	// row after a successful retention prune. nil disables object
	// deletion entirely (store rows still get pruned), the same
	// optional-capability shape this codebase already uses elsewhere
	// (e.g. backupRunner itself being nil when no master key is
	// configured, cmd/levelrail/main.go).
	Deleter Deleter
	// Verifier, if set, re-checks a scheduled backup right after it
	// succeeds (runScheduled). nil disables auto-verification, same
	// optional-capability shape as Deleter above. Set directly like Now
	// below, not a constructor param, to keep NewScheduler's signature
	// stable for existing call sites.
	Verifier ScheduledVerifier
	// BaseBackups, if set, takes a physical base backup after each scheduled
	// logical backup of a PITR-enabled database, so point-in-time restore
	// stays fresh without a second schedule. nil disables it.
	BaseBackups ScheduledBaseBackupRunner
	// Policies and GFS enable per-volume daily/weekly/monthly retention;
	// both nil is valid and leaves only the count/age rules.
	Policies VolumePolicyLookup
	GFS      GFSStore
	// Tombstones, if set, keeps failed object deletes for retry instead of only logging them.
	Tombstones TombstoneStore
	Logger     *slog.Logger
	// Now returns the current time, the same testable-clock field
	// Runner.Now already establishes in runner.go: production code
	// leaves it nil and Tick falls back to time.Now, tests set it for
	// deterministic schedule matching.
	Now func() time.Time
	// contains filtered or unexported fields
}

Scheduler periodically checks which databases have a due cron schedule (store.DesiredDatabase's BackupSchedule/BackupTargetID/ BackupRetain, migrations/0023_scheduled_backups.sql) and runs a backup through the same Runner the manual trigger path (internal/api's handleTriggerBackup) uses, recording the attempt in store.BackupHistory exactly like a manual trigger does, since both paths call the identical Runner.RunBackup.

Tick's own shape (list what to evaluate, act, persist any resulting state, join and return errors rather than stopping at the first one) deliberately mirrors internal/alerting.Engine.Tick: that is this codebase's own established shape for "evaluate every enabled X on an interval, one broken X must never block the rest," and a cron schedule check is exactly that same shape applied to a different resource. Run's own ticker loop is the identical pattern internal/alerting.Engine.Run, internal/telemetry.Collector.Run, and every other periodic loop in cmd/levelrail/main.go already use.

Unlike alerting.Engine.Tick's own per-rule dispatch, a due backup runs synchronously inside Tick, not fired off into its own goroutine the way internal/api's handleTriggerBackup dispatches one: that handler's goroutine exists specifically because an HTTP response must return in milliseconds, a constraint that does not apply to a background ticker loop already running off the request path. Running sequentially here also means a database's backup finishing (or failing) is fully resolved, including its retention prune, before Tick moves on to evaluate the next database, which keeps this method's own control flow as a flat, easy-to-reason-about loop instead of needing a WaitGroup or a results channel to know when a tick's work is actually done.

Missed-schedule handling is a known, deliberate, and documented gap, not an oversight: Scheduler tracks each database's next fire time purely in memory (nextRun below), armed fresh the first tick a schedule is observed rather than reconstructed from store.BackupHistory on startup. A schedule that should have fired while the control plane was down does not "catch up" the moment it restarts; it simply waits for its next naturally occurring occurrence. This mirrors the honesty this codebase already extends to a related gap on the record-keeping side (store.BackupHistory's own status constants doc comment: "nothing currently sweeps a stuck running row back to failed... a known, honest gap for the eventual scheduler to close alongside its own retry logic, not invented speculatively here"). Building real catch-up/retry semantics is a materially bigger feature (it needs to decide how many missed runs to catch up on, how to avoid a thundering herd of backups firing at once after a long outage, and how that interacts with retention) than this task's scope asked for, so it is left as a follow-up rather than guessed at here.

func NewScheduler(store ScheduleStore, runner ScheduledBackupRunner, deleter Deleter, logger *slog.Logger) *Scheduler

func (s *Scheduler) Run(ctx context.Context, interval time.Duration) error

func (s *Scheduler) Tick(ctx context.Context) error

type SealRecorder ¶ added in v0.2.0

type SealRecorder interface {
	SetBackupSealInfo(ctx context.Context, id, codec, manifestDigest string, files, bytes int64) error
}

SealRecorder persists the codec and manifest of a sealed backup.

type Sealer ¶ added in v0.2.0

type Sealer struct {
	Recipients []age.Recipient
	Identities []age.Identity
}

Sealer compresses and optionally encrypts a stream. A nil Sealer means the legacy raw encoding.

func LoadSealer(lookup func(string) (string, bool), dataDir string) (*Sealer, error)

func (s *Sealer) Codec() string

func (s *Sealer) Seal(src io.Reader) io.ReadCloser

type SearchContent ¶

type SearchContent map[string]int64

SearchContent maps each index or collection to its document or point count, as the engine's own API reports it.

func DecodeSearchContent(s string) (SearchContent, error)

func ReadSearchContent(ctx context.Context, rt docker.Runtime, engine, containerName string) (SearchContent, error)

func (c SearchContent) Diff(got SearchContent) string

func (c SearchContent) Encode() string

func (c SearchContent) Total() int64

type SecretsResolver ¶

type SecretsResolver interface {
	Resolve(ctx context.Context, serviceName, envKey string) (string, error)
}

SecretsResolver is the one internal/secrets.Manager method Runner needs, narrowed to keep this package's tests from needing a real Manager (Manager itself has no interface today; a struct-shaped fake satisfying this one-method interface is enough).

type SqliteSnapshotter ¶ added in v0.2.0

type SqliteSnapshotter interface {
	Snapshot(ctx context.Context, volumeName, relPath string) (io.ReadCloser, error)
}

SqliteSnapshotter produces a consistent snapshot of a SQLite .db file living inside a named Docker volume, the sqlite-path counterpart of VolumeArchiver.

type TargetGetter ¶ added in v0.2.0

type TargetGetter interface {
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
}

TargetGetter is the one-method surface resolveTargetDestination needs, shared by BaseBackupHistoryStore and PITRHistoryStore so both can resolve a target's live Destination the identical way without each repeating the lookup-plus-two-secret-resolves sequence Runner. ResolveDestination (runner.go) already establishes for the logical-dump direction.

type TargetTestStore ¶

type TargetTestStore interface {
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
}

TargetTestStore is the store surface TargetTester needs, narrowed the same way HistoryStore/VerificationHistoryStore already are: only the one lookup a connection test actually requires.

type TargetTester ¶

type TargetTester struct {
	Store   TargetTestStore
	Secrets SecretsResolver
	Tester  Tester
}

TargetTester resolves targetID's stored fields plus its live credentials from internal/secrets and runs Tester.Test against them, the "verify a stored credential by actually using it, once, on demand" shape this codebase's own RegistryAuthTester established first for registry credentials.

func (t *TargetTester) TestTarget(ctx context.Context, targetID string) error

type Tester ¶

type Tester interface {
	Test(ctx context.Context, dest Destination) error
}

Tester probes a Destination for a lightweight, non-destructive connectivity and credential check, the test-connection counterpart Uploader/Deleter above establish for real backup/prune work.

type TombstoneStore ¶

type TombstoneStore interface {
	AddBackupTombstone(ctx context.Context, targetID, objectKey, reason string) error
	ListBackupTombstones(ctx context.Context, limit int) ([]store.BackupTombstone, error)
	ResolveBackupTombstone(ctx context.Context, id int64) error
	BumpBackupTombstone(ctx context.Context, id int64, reason string) error
}

TombstoneStore remembers objects whose delete failed after their row was removed.

type TreeManifest ¶ added in v0.2.0

type TreeManifest struct {
	Files  int64
	Dirs   int64
	Bytes  int64
	Digest string
}

TreeManifest summarizes a volume archive's content: counts plus an order-independent digest over every entry's path, type, mode, owner, size and content hash. Two archives of the same tree produce the same digest whatever order the tar tool walked it in.

func ComputeManifest(r io.Reader) (TreeManifest, error)

type Uploader ¶

type Uploader interface {
	Upload(ctx context.Context, dest Destination, key string, r io.Reader, size int64) error
}

Uploader writes a single object to a Destination. Implementations must read r to completion (or return an error partway through, in which case the object must not be left partially visible under key, an S3-compatible multipart upload aborted rather than completed covers this for free) rather than assume a specific size up front: size is a hint for implementations that benefit from knowing it (choosing single-PUT vs. multipart), not a hard contract r will produce exactly that many bytes.

type VerificationHistoryStore ¶

type VerificationHistoryStore interface {
	GetBackupHistory(ctx context.Context, id string) (store.BackupHistory, error)
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
	StartBackupVerification(ctx context.Context, v store.BackupVerification) error
	FinishBackupVerification(ctx context.Context, id, status string, checksumMatch, sizeMatch, formatValid bool, downloadedBytes int64, errMsg, finishedAt string) error
}

VerificationHistoryStore is the store.DB surface VerifyRunner needs: GetBackupHistory/GetBackupTarget resolve exactly what DownloadRunner.Download's own identical pair already resolve (the object a verification attempt re-downloads is the same object a real download would fetch), plus Start/FinishBackupVerification to record the attempt itself, the same shape RestoreHistoryStore's own doc comment describes for the restore direction.

type VerifyRunner ¶

type VerifyRunner struct {
	Store      VerificationHistoryStore
	Secrets    SecretsResolver
	Downloader Downloader
	// Identities decrypt sealed dumps before the format check; empty is valid for raw ones.
	Identities []age.Identity
	// Now returns the current time, matching Runner.Now and
	// RestoreRunner.Now's own "field, not time.Now called directly"
	// reasoning for deterministic tests.
	Now func() time.Time
}

VerifyRunner confirms a previously succeeded backup is still intact without ever attempting a live restore: re-download the object, re-hash it, and compare against what Runner recorded at backup time, the non-destructive counterpart RunRestore's own doc comment explicitly rules out for this feature (a live restore against a running system is real risk, not a safe thing to run unattended on a schedule or a button click).

func (r *VerifyRunner) VerifyBackup(ctx context.Context, verificationID, backupHistoryID, engine, checkedBy string) error

type VolumeArchiver ¶

type VolumeArchiver interface {
	Archive(ctx context.Context, volumeName string) (io.ReadCloser, error)
}

VolumeArchiver produces a tar archive of a named Docker volume's contents, the app-volume counterpart of Dumper for a database engine. The returned ReadCloser's Close must be called by every caller once done, the same contract Dumper's own doc comment establishes.

type VolumeBackupOptions ¶ added in v0.2.0

type VolumeBackupOptions struct {
	Sealer   *Sealer
	Policies VolumePolicyLookup
	Quiesce  Quiescer
	// MaxBytes aborts an archive larger than this many raw bytes; 0 is no limit.
	MaxBytes int64
	// Timeout bounds one whole backup; 0 is no limit.
	Timeout time.Duration
	// Attempts is how many times a failed archive-and-upload is retried
	// from a fresh archive; values below 1 mean one attempt.
	Attempts int
	Logger   *slog.Logger
}

VolumeBackupOptions configures sealed volume backups on a Runner.

func (o *VolumeBackupOptions) OptionsFromEnv(lookup func(string) (string, bool))

type VolumeCloneRestoreRunner ¶

type VolumeCloneRestoreRunner struct {
	Store          VolumeCloneRestoreStore
	Secrets        SecretsResolver
	Downloader     Downloader
	VolumeRestorer VolumeRestorer
	Volumes        VolumeCreator
	// Identities decrypt sealed backups; empty is valid for raw ones.
	Identities []age.Identity
	// Now returns the current time, the same "field, not time.Now called
	// directly" reasoning CloneRestoreRunner.Now's own doc comment gives.
	Now func() time.Time
}

VolumeCloneRestoreRunner restores a previously succeeded volume backup into a brand-new, standalone Docker volume rather than overwriting the one it came from: the volume counterpart of CloneRestoreRunner. Simpler than CloneRestoreRunner because there is no reconciler to wait on (see VolumeCreator's own doc comment); creating the volume and restoring into it both happen synchronously, one after the other, inside RunVolumeCloneRestore itself.

func (r *VolumeCloneRestoreRunner) RunVolumeCloneRestore(ctx context.Context, historyID, sourceServiceName, sourceVolumeName, newVolumeName, backupHistoryID string) error

type VolumeCloneRestoreStore ¶

type VolumeCloneRestoreStore interface {
	StartVolumeCloneRestore(ctx context.Context, h store.VolumeCloneRestore) error
	FinishVolumeCloneRestore(ctx context.Context, id, status, errMsg, finishedAt string) error
	// contains filtered or unexported methods
}

VolumeCloneRestoreStore is the narrow store surface VolumeCloneRestoreRunner needs: backupResolver's two methods to resolve the source backup, plus the clone-restore bookkeeping pair, the app service volume counterpart of CloneRestoreStore.

type VolumeCreator ¶

type VolumeCreator interface {
	EnsureVolume(ctx context.Context, name string) error
}

VolumeCreator is the narrow docker.Runtime surface VolumeCloneRestoreRunner needs to bring a brand-new, empty volume into existence before restoring into it: unlike CloneRestoreRunner's wait for a new database's own reconciler to report Ready, a bare Docker volume has no reconcile loop and exists the moment this call returns, so there is nothing to poll for here.

type VolumePolicyLookup ¶ added in v0.2.0

type VolumePolicyLookup interface {
	GetVolumeBackupPolicy(ctx context.Context, serviceName, volumeName string) (store.VolumeBackupPolicy, error)
}

VolumePolicyLookup reads a volume's backup policy.

type VolumeRemover ¶ added in v0.2.0

type VolumeRemover interface {
	RemoveVolume(ctx context.Context, name string) error
}

VolumeRemover deletes a Docker volume by name.

type VolumeRestorer ¶

type VolumeRestorer interface {
	Restore(ctx context.Context, volumeName string, archive io.Reader) error
}

VolumeRestorer untars a previously archived stream back into a named Docker volume, in place: it wipes every existing file the volume holds first, so this is always a full replace, never a merge, matching Restorer's own "the restore command exited zero" contract.

type VolumeRuntime ¶ added in v0.2.0

type VolumeRuntime interface {
	docker.Runtime
	RemoveVolume(ctx context.Context, name string) error
}

VolumeRuntime is docker.Runtime plus volume removal.

type WALFetcher ¶ added in v0.2.0

type WALFetcher struct {
	Runtime    docker.Runtime
	Lister     Lister
	Downloader Downloader
}

WALFetcher restores shipped WAL from a backup target into a database's local archive volume, so recovery works after the local copy is lost.

func (f *WALFetcher) Fetch(ctx context.Context, dest Destination, databaseName, walVolume, sinceLSN string) (int, error)

type WALFetching ¶ added in v0.2.0

type WALFetching interface {
	Fetch(ctx context.Context, dest Destination, databaseName, walVolume, sinceLSN string) (int, error)
}

WALFetching is the WALFetcher surface PITRRunner needs.

type WALShipScheduler ¶ added in v0.2.0

type WALShipScheduler struct {
	Store         WALShipStore
	Shipper       WALShipping
	Resolve       func(ctx context.Context, targetID string) (Destination, error)
	ContainerName func(databaseName string) string
	// IsLocal reports whether a database's node is the one this control plane's
	// Docker client talks to; remote nodes are skipped.
	IsLocal func(nodeID string) bool
	Logger  *slog.Logger
	// contains filtered or unexported fields
}

WALShipScheduler ships the WAL of every local PITR-enabled database to the target of its newest base backup on a fixed interval, bounding the data lost if the database host's disk dies to one interval of WAL.

func (s *WALShipScheduler) Run(ctx context.Context, interval time.Duration) error

func (s *WALShipScheduler) Tick(ctx context.Context)

func (s *WALShipScheduler) WALShipStatus(databaseName string) (WALShipStatus, bool)

type WALShipStatus ¶ added in v0.2.0

type WALShipStatus struct {
	LastAttemptAt time.Time
	LastSuccessAt time.Time
	LastError     string
	// Shipped counts segments uploaded since the control plane started.
	Shipped  int
	TargetID string
}

WALShipStatus is the outcome of the most recent shipping pass for one database, shown by "pitr status".

type WALShipStore ¶ added in v0.2.0

type WALShipStore interface {
	ListDesiredDatabases(ctx context.Context) ([]store.DesiredDatabase, error)
	ListBaseBackupHistory(ctx context.Context, databaseName string) ([]store.BaseBackupHistory, error)
}

WALShipStore is the store surface WALShipScheduler needs.

type WALShipper ¶ added in v0.2.0

type WALShipper struct {
	Runtime  Runtime
	Uploader Uploader
	Lister   Lister
	Logger   *slog.Logger
}

WALShipper copies a database's local WAL archive into its backup target so point-in-time restore survives losing the database host's disk.

func (s *WALShipper) Ship(ctx context.Context, dest Destination, databaseName, containerName string) (int, error)

type WALShipping ¶ added in v0.2.0

type WALShipping interface {
	Ship(ctx context.Context, dest Destination, databaseName, containerName string) (int, error)
}

WALShipping is the WALShipper surface BaseBackupRunner and the ship scheduler need.