backup

package
v0.2.0-beta Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 12, 2026 License: Apache-2.0 Imports: 21 Imported by: 0

Documentation

Overview

Package backup drives a database backup end to end: dump a managed database container's data, stream it straight to an S3-compatible bucket, and record the attempt in store.BackupHistory. It composes two independent capabilities, each with its own file and its own real implementation:

  • Dumper (dump.go): produces a dump stream from a running database container.
  • Uploader (uploader.go): writes a stream to a configured destination bucket.
  • Deleter (deleter.go): removes a single object from a configured destination bucket, used by Scheduler's retention pruning.

Runner (runner.go) is the only piece that talks to store and internal/secrets; Dumper, Uploader, and Deleter implementations know nothing about either, so each is independently testable with a fake on the other side of its interface.

Index

Constants

View Source
const ScheduledVerificationCheckedBy = "scheduler"

ScheduledVerificationCheckedBy is the CheckedBy value Scheduler records for its own automatic verification, distinguishing it from a manual one in the UI/CLI.

Variables

This section is empty.

Functions

This section is empty.

Types

type CloneRestoreRunner

type CloneRestoreRunner struct {
	Store      CloneRestoreStore
	Secrets    SecretsResolver
	Downloader Downloader
	Restorer   Restorer
	// Now returns the current time, the same "field, not time.Now called
	// directly" reasoning RestoreRunner.Now's own doc comment gives.
	Now func() time.Time
	// ReadyTimeout/PollInterval override defaultCloneRestoreReadyTimeout/
	// defaultCloneRestoreReadyPollInterval when non-zero, so tests don't
	// have to wait out the real default.
	ReadyTimeout time.Duration
	PollInterval time.Duration
}

CloneRestoreRunner restores a previously succeeded backup into a brand-new database rather than overwriting the one it came from: the non-destructive counterpart to RestoreRunner. internal/api creates the new database's desired state itself (reusing handleCreateDatabase's own path, see database_clone_restore.go's own doc comment for why) before ever calling RunCloneRestore; this type's only job is to wait for that database's reconciler to bring it up and then run the identical download-and-apply RestoreRunner already implements.

func (*CloneRestoreRunner) RunCloneRestore

func (r *CloneRestoreRunner) RunCloneRestore(ctx context.Context, historyID, sourceDatabaseName, newDatabaseName, backupHistoryID, engine, containerName, controllerName string) error

RunCloneRestore waits for newDatabaseName (identified to the reconciler by controllerName, "database/" + newDatabaseName) to report Ready, then downloads and restores backupHistoryID into it, recording the attempt as store.CloneRestore throughout: a running row before any work starts, then succeeded or failed once it's done. sourceDatabaseName is recorded for history only; RunCloneRestore never reads from or writes to it. Expected to run in a goroutine the caller has already detached, the same convention RestoreRunner.RunRestore's own doc comment describes.

type CloneRestoreStore

type CloneRestoreStore interface {
	StartCloneRestore(ctx context.Context, h store.CloneRestore) error
	FinishCloneRestore(ctx context.Context, id, status, errMsg, finishedAt string) error
	GetConditions(ctx context.Context, controllerName string) ([]reconcile.Condition, error)
	// contains filtered or unexported methods
}

CloneRestoreStore is the narrow store surface CloneRestoreRunner needs: backupResolver's two methods to resolve the source backup, the clone- restore bookkeeping pair, and GetConditions to detect when the newly created database has actually come up. A separate interface from RestoreHistoryStore, the same "narrow interface per real dependency" reasoning that type's own doc comment gives, since the two share no method beyond the backupResolver pair.

type ContainerDumper

type ContainerDumper struct {
	Runtime docker.Runtime
}

ContainerDumper is the real Dumper: it runs each engine's own native dump tool inside the already-running database container via docker.Runtime.Exec, so this process never needs its own copy of pg_dump/mysqldump/etc. and the database never needs a network port exposed anywhere reachable.

func (*ContainerDumper) Dump

func (d *ContainerDumper) Dump(ctx context.Context, engine, containerName string) (io.ReadCloser, error)

Dump implements Dumper.

type ContainerRestorer

type ContainerRestorer struct {
	Runtime docker.Runtime
}

ContainerRestorer is the real Restorer: it runs each engine's own native restore path inside the running container via docker.Runtime.ExecWithInput, ContainerDumper's own reasoning plus a stdin stream.

func (*ContainerRestorer) Restore

func (r *ContainerRestorer) Restore(ctx context.Context, engine, containerName string, dump io.Reader) error

Restore implements Restorer.

type ContainerVolumeArchiver

type ContainerVolumeArchiver struct {
	Runtime docker.Runtime
}

ContainerVolumeArchiver is the real VolumeArchiver: it creates a helper container with volumeName mounted read-only, tars its contents to stdout, and removes the container once the caller has read the stream to completion (or given up partway through).

func (*ContainerVolumeArchiver) Archive

func (a *ContainerVolumeArchiver) Archive(ctx context.Context, volumeName string) (io.ReadCloser, error)

Archive implements VolumeArchiver.

type ContainerVolumeRestorer

type ContainerVolumeRestorer struct {
	Runtime docker.Runtime
}

ContainerVolumeRestorer is the real VolumeRestorer: it runs the wipe-then-extract command inside a short-lived helper container with volumeName mounted read-write.

func (*ContainerVolumeRestorer) Restore

func (r *ContainerVolumeRestorer) Restore(ctx context.Context, volumeName string, archive io.Reader) error

Restore implements VolumeRestorer.

type Deleter

type Deleter interface {
	Delete(ctx context.Context, dest Destination, key string) error
}

Deleter removes a single object from a Destination, the inverse of Uploader. Implementations must treat deleting a key that is already gone (or never existed) as success, not an error: S3-compatible DeleteObject calls are naturally idempotent this way.

type Destination

type Destination struct {
	// Provider is one of store.BackupProviderAWS / R2 / Custom. An
	// Uploader implementation may use it to pick defaults (e.g. AWS's
	// per-region default endpoint), but Endpoint below is always the
	// final authority when non-empty.
	Provider string
	// Endpoint is the S3-compatible API base URL. Empty for
	// BackupProviderAWS (the SDK's own region-based resolver applies);
	// required for R2 and Custom.
	Endpoint string
	Region   string
	Bucket   string

	AccessKeyID     string
	SecretAccessKey string
}

Destination is everything an Uploader needs to reach a specific bucket, credentials included. Built fresh per call from a store.BackupTarget plus its resolved internal/secrets values (see store.BackupTargetSecretsKey): never persisted or logged as a whole, since AccessKeyID and SecretAccessKey are live credentials.

type DownloadHistoryStore

type DownloadHistoryStore interface {
	GetBackupHistory(ctx context.Context, id string) (store.BackupHistory, error)
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
}

DownloadHistoryStore is the store.DB surface DownloadRunner needs: GetBackupHistory to resolve the attempt a download names, GetBackupTarget to resolve where its object actually lives, the same pair RestoreHistoryStore's own doc comment describes for the restore direction. No Start/Finish pair here: unlike a backup or a restore, a download has no attempt of its own worth recording in a history table, it only reads an object an earlier, already-recorded backup wrote.

type DownloadRunner

type DownloadRunner struct {
	Store      DownloadHistoryStore
	Secrets    SecretsResolver
	Downloader Downloader
}

DownloadRunner resolves a succeeded backup attempt to its live object stream, the read-only counterpart of Runner (dump-and-upload) and RestoreRunner (download-and-restore-in-place): given a store.BackupHistory ID, it resolves the row's target and credentials the identical way Runner.runDumpAndUpload and RestoreRunner.runDownloadAndRestore already do, then hands the caller (internal/api's handleDownloadBackup) the raw object stream to copy directly into an HTTP response, never buffering it here.

func (*DownloadRunner) Download

func (d *DownloadRunner) Download(ctx context.Context, historyID string) (io.ReadCloser, error)

Download resolves historyID's backup attempt to its object stream. Refuses (without ever resolving credentials or contacting the bucket) a backup whose Status is not store.BackupStatusSucceeded, the same safety property RunRestore's own doc comment describes for the restore path: a running attempt has no complete object at its recorded key yet, and a failed one may have none, or a partial one. internal/api's handleDownloadBackup does the identical check before ever calling this; this is the defense-in-depth backstop for a caller that doesn't.

type Downloader

type Downloader interface {
	Download(ctx context.Context, dest Destination, key string) (io.ReadCloser, error)
}

Downloader retrieves a single object from a Destination, the exact reverse of Uploader: given the same (Destination, key) an earlier Upload call was made with, it returns that object's bytes as a stream. Implementations are not required to know the object's size up front (see S3Downloader's own doc comment for why GetObject already makes this a non-issue, unlike Upload's own unknown-size handling), only to stream it back faithfully.

type Dumper

type Dumper interface {
	Dump(ctx context.Context, engine, containerName string) (io.ReadCloser, error)
}

Dumper produces a database dump as a stream, for a single managed database container already running under containerName (the same name application.ContainerName / the database controller's own container-naming convention already produces; Dumper does not construct or look this up itself, Runner does).

The returned ReadCloser's Close must be called by every caller once done, success or failure, the same contract io.ReadCloser always carries; a Dumper backed by a subprocess or a docker exec stream depends on Close to release that underlying resource.

type HistoryStore

type HistoryStore interface {
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
	StartBackupHistory(ctx context.Context, h store.BackupHistory) error
	FinishBackupHistory(ctx context.Context, id, status string, sizeBytes int64, checksum, errMsg, finishedAt string) error
}

HistoryStore is the store.DB surface Runner needs, narrowed the same way SecretsResolver is: a fake satisfying six methods is a lot less to stub than the full *store.DB.

type RestoreHistoryStore

type RestoreHistoryStore interface {
	GetBackupHistory(ctx context.Context, id string) (store.BackupHistory, error)
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
	StartRestoreHistory(ctx context.Context, h store.RestoreHistory) error
	FinishRestoreHistory(ctx context.Context, id, status, errMsg, finishedAt string) error
}

RestoreHistoryStore is the store.DB surface RestoreRunner needs, narrowed the same way Runner's own HistoryStore is: GetBackupHistory to resolve the source backup a restore names, GetBackupTarget to resolve where that backup's object actually lives, and StartRestoreHistory/FinishRestoreHistory to record the restore attempt itself. Deliberately not HistoryStore plus new methods bolted on: a fake satisfying HistoryStore's forward-backup methods has no reason to also implement restore's, and the reverse is equally true, so keeping them as two separate narrow interfaces (which happen to overlap on GetBackupTarget, satisfied by the same *store.DB either way) matches this codebase's own "narrow interface per real dependency, not one grab-bag interface per package" convention.

type RestoreRunner

type RestoreRunner struct {
	Store      RestoreHistoryStore
	Secrets    SecretsResolver
	Downloader Downloader
	Restorer   Restorer
	// VolumeRestorer backs RunVolumeRestore the same way Restorer backs
	// RunRestore. nil is valid, the same "optional capability" shape
	// Runner.VolumeArchiver's own doc comment establishes.
	VolumeRestorer VolumeRestorer
	// Now returns the current time, matching Runner.Now's own "field, not
	// time.Now called directly" reasoning for deterministic tests.
	Now func() time.Time
}

RestoreRunner ties a Downloader and a Restorer to store and internal/secrets, the restore counterpart of Runner: the only piece in this package that knows how a restore attempt gets recorded and how a target's live credentials get resolved. internal/api constructs one real RestoreRunner at startup, the same place it constructs Runner, and calls RunRestore per triggered restore.

A separate type from Runner rather than a second method on it: Runner is already a coherent "dump this, upload it" unit with its own field set (Dumper, Uploader); RestoreRunner's fields (Downloader, Restorer) point the opposite direction and share nothing with Runner's except Store's underlying *store.DB value and Secrets' underlying *secrets.Manager value, both already expressed as separate narrow interfaces above and in runner.go rather than a shared struct field, so there is no real duplication to unify by merging the two types, only two independently-testable units that happen to be wired from the same concrete dependencies at startup.

func (*RestoreRunner) RunRestore

func (r *RestoreRunner) RunRestore(ctx context.Context, historyID, databaseName, backupHistoryID, engine, containerName string) error

RunRestore downloads the object recorded by backupHistoryID and applies it to containerName (running engine) as databaseName's new contents, recording the attempt in store.RestoreHistory throughout: a running row before any work starts, then succeeded or failed once it's done. historyID is minted by the caller (internal/api), the same "generate the ID before the first write" convention RunBackup's own doc comment describes; RunRestore is expected to run in a goroutine the caller has already detached, for the same reason RunBackup is, see internal/api's handleTriggerRestore.

RunRestore refuses to proceed, failing the history row rather than attempting anything, if the named backup's own Status is not store.BackupStatusSucceeded: restoring from a backup that was still running (and so has no complete, consistent object at its recorded key yet) or that already failed (and so may have no object there at all, or a partial one) is not a smaller version of a real restore, it is a data-loss-shaped mistake this function refuses to make even if a caller's own validation (internal/api's handleTriggerRestore does the identical check before ever calling this, so a caller going through the ordinary API path never reaches this branch) somehow let it through.

func (*RestoreRunner) RunVolumeRestore

func (r *RestoreRunner) RunVolumeRestore(ctx context.Context, historyID, serviceName, volumeName, dockerVolumeName, backupHistoryID string) error

RunVolumeRestore downloads the object recorded by backupHistoryID and applies it to dockerVolumeName (serviceName's Docker volume backing its volumeName-named app.yaml volume) as its new, complete contents, recording the attempt in store.RestoreHistory exactly the way RunRestore does for a database. See RunRestore's own doc comment for the full reasoning this mirrors, including the identical refusal to proceed against a backup whose Status isn't store.BackupStatusSucceeded.

type Restorer

type Restorer interface {
	Restore(ctx context.Context, engine, containerName string, dump io.Reader) error
}

Restorer applies a previously taken dump back onto a live database container. Like Dumper, it takes no credentials beyond what the container's own environment already provides. Its contract is "the restore command exited zero," not "and the data was verified correct afterward."

type Runner

type Runner struct {
	Store    HistoryStore
	Secrets  SecretsResolver
	Dumper   Dumper
	Uploader Uploader
	// VolumeArchiver backs RunVolumeBackup the same way Dumper backs
	// RunBackup. nil is valid: a control plane with no volume-backup
	// callers configured (today, none do this by default) simply never
	// calls RunVolumeBackup, the same "optional capability" shape this
	// codebase already uses for Scheduler.Deleter/Verifier.
	VolumeArchiver VolumeArchiver
	// Now returns the current time. A field, not time.Now called
	// directly, so tests get deterministic timestamps without a real
	// clock dependency; production code leaves it nil and RunBackup
	// falls back to time.Now.
	Now func() time.Time
}

Runner ties a Dumper and an Uploader to store and internal/secrets: the only piece in this package that knows how a backup attempt gets recorded and how a target's live credentials get resolved. internal/api constructs one real Runner at startup (the same "wire concrete implementations once, in cmd/levelrail or internal/api's own constructor" pattern every other cross-package dependency in this codebase already follows) and calls RunBackup per triggered backup.

func (*Runner) ResolveDestination

func (r *Runner) ResolveDestination(ctx context.Context, targetID string) (Destination, error)

ResolveDestination resolves targetID's stored config and live secrets into a Destination, the same lookup runDumpAndUpload does before every Upload call. Exported so Scheduler can resolve the identical Destination for a Deleter when pruning old backups for the same target.

func (*Runner) RunBackup

func (r *Runner) RunBackup(ctx context.Context, historyID, databaseName, engine, containerName, targetID string) error

RunBackup dumps databaseName (running in containerName under engine) and uploads the result to targetID, recording the attempt in store.BackupHistory throughout: a running row before any work starts, then succeeded or failed once it's done. historyID is minted by the caller (internal/api, the same "generate the ID before the first write" convention store.BackupTarget's own doc comment describes) so the caller can return it in an HTTP response immediately without waiting for the backup to finish; RunBackup itself is expected to run in a goroutine the caller has already detached; see internal/api's handleTriggerBackup for how it starts, and this file's own tests for the exact happy-path and every-failure-point sequencing this describes.

Every failure path (target not found, secret resolution, dump, upload) finishes the history row as BackupStatusFailed with a real error message rather than leaving it stuck at BackupStatusRunning: the one exception is a failure to even write FinishBackupHistory itself, which RunBackup can't recover from and returns as-is, the same "give up honestly rather than mask it" the reconcilers elsewhere in this codebase already follow when their own condition-write fails.

func (*Runner) RunVolumeBackup

func (r *Runner) RunVolumeBackup(ctx context.Context, historyID, serviceName, volumeName, dockerVolumeName, targetID string) error

RunVolumeBackup archives dockerVolumeName (serviceName's Docker volume backing its volumeName-named app.yaml volume) and uploads the result to targetID, recording the attempt in store.BackupHistory exactly the way RunBackup does for a database: the same running-then-succeeded-or- failed row, the same historyID-minted-by-the-caller contract, the same expectation of running in a goroutine the caller has already detached. See RunBackup's own doc comment for the reasoning this mirrors in full; this only differs in what it dumps (VolumeArchiver.Archive instead of Dumper.Dump) and what identifies the row (ServiceName/VolumeName/ResourceKind instead of DatabaseName).

type S3Deleter

type S3Deleter struct{}

S3Deleter is the real Deleter, covering AWS/R2/custom endpoints the same way S3Uploader does, reusing newS3Client (uploader.go) rather than building its own client.

func (S3Deleter) Delete

func (S3Deleter) Delete(ctx context.Context, dest Destination, key string) error

Delete removes key from dest.Bucket. A DeleteObject call against a key that is already gone (or never existed) returns success, not an error, the same way real S3-compatible services answer it: no existence check happens here on purpose, see Deleter's own doc comment.

type S3Downloader

type S3Downloader struct{}

S3Downloader is the real Downloader: the download side of the same AWS/R2/custom-endpoint story S3Uploader already covers, one implementation for every provider this codebase supports, driven by the same Destination.Endpoint-is-the-final-authority rule.

Unlike S3Uploader, this has no multipart-vs-single-PUT decision to make: S3's GetObject API already streams a response body of whatever size the object actually is, with no equivalent to PutObject's Content-Length requirement on the request side, so there's no unknown-size problem here to work around with an upload-manager-style abstraction. A plain client.GetObject call is the entire implementation.

func (S3Downloader) Download

func (S3Downloader) Download(ctx context.Context, dest Destination, key string) (io.ReadCloser, error)

Download implements Downloader. The returned ReadCloser is GetObjectOutput.Body itself, not copied into a buffer first: a database dump can be large, and RunRestore already has to stream it straight into ContainerRestorer.Restore's stdin (see restore.go), so buffering the whole object in this process's memory first would cost real memory for no benefit; Close on the returned value releases the underlying HTTP response exactly the same way Dumper's own ReadCloser contract already requires callers to honor.

type S3Tester

type S3Tester struct{}

S3Tester is the real Tester, covering AWS/R2/custom endpoints the same way S3Uploader does, reusing newS3Client rather than building its own client.

func (S3Tester) Test

func (S3Tester) Test(ctx context.Context, dest Destination) error

Test issues a HeadBucket call against dest.Bucket: the cheapest S3 operation that proves both the endpoint is reachable and the credentials are accepted, without reading or writing any object.

type S3Uploader

type S3Uploader struct{}

S3Uploader is the real Uploader: every backup target this codebase supports (AWS, R2, and any other S3-compatible custom endpoint) speaks the same REST API, so one implementation covers all three, driven entirely by the Provider/Endpoint distinction already captured on Destination.

A new S3 client is built per Upload call rather than cached on the struct: Destination carries live, per-target credentials resolved fresh from internal/secrets on every backup run (see Runner), and a target's bucket, region, or endpoint can change between runs too. Backups are infrequent (the default schedule is once a day, see store.BackupTarget's own docs), so the cost of rebuilding a client each time is not worth the complexity of a cache that would need invalidating whenever a target's settings change.

func (S3Uploader) Upload

func (S3Uploader) Upload(ctx context.Context, dest Destination, key string, r io.Reader, _ int64) error

Upload writes r to dest.Bucket under key using the AWS SDK's S3 upload manager rather than a raw PutObject call, because size is frequently -1 here: Runner streams a database dump straight from a running pg_dump/mysqldump process into Upload while the dump is still in flight, so there is no Content-Length to give PutObject and no seekable body to compute one from. The upload manager handles this by buffering r into fixed-size parts and issuing an S3 multipart upload, which needs no upfront length. The size parameter goes unused (named _ below, the same convention this package's own test fakes already use for it): it is a hint for implementations that can act on it (see backup.go's own doc comment on Uploader), and this one cannot, since the manager sizes itself from r as it streams regardless of what is passed.

This intentionally uses feature/s3/manager rather than its replacement, feature/s3/transfermanager: as of this writing transfermanager's UploadObject takes a Body that must support Len() or Seek() to size itself, which a streaming pg_dump/mysqldump pipe cannot provide, and its own maintainers describe the unknown-size streaming case as unsupported. feature/s3/manager is marked deprecated in its doc comments but not removed or broken; it is the only one of the two that actually does what Runner needs. Revisit this once transfermanager grows real support for an unbounded io.Reader body.

type ScheduleStore

type ScheduleStore interface {
	ListScheduledDatabases(ctx context.Context) ([]store.DesiredDatabase, error)
	PruneBackupHistory(ctx context.Context, databaseName string, keep int, olderThan time.Time) ([]store.PrunedBackup, error)
	// ListScheduledServiceVolumes/PruneServiceVolumeBackupHistory/
	// ResolveServiceVolumeDockerName are the volume counterparts of the
	// two database methods above plus one extra resolution step a
	// database schedule doesn't need: store.ServiceVolumeBackupConfig
	// only carries a volume's logical name (what an operator wrote in
	// app.yaml), never its real Docker volume name, so RunVolumeBackup's
	// dockerVolumeName parameter has to be resolved fresh every tick, the
	// same "re-derive, never cache" principle this whole interface
	// already follows for which databases/volumes are even scheduled.
	ListScheduledServiceVolumes(ctx context.Context) ([]store.ServiceVolumeBackupConfig, error)
	PruneServiceVolumeBackupHistory(ctx context.Context, serviceName, volumeName string, keep int, olderThan time.Time) ([]store.PrunedBackup, error)
	ResolveServiceVolumeDockerName(ctx context.Context, serviceName, volumeName string) (string, error)
}

ScheduleStore is the narrow store surface Scheduler needs: which databases currently have a due-able cron schedule, re-derived fresh on every tick, and the retention prune a successful scheduled run triggers afterward. *store.DB satisfies this structurally, the same "consumer defines its own narrow interface" shape HistoryStore (runner.go) already establishes for this package's other store dependency.

type ScheduledBackupRunner

type ScheduledBackupRunner interface {
	RunBackup(ctx context.Context, historyID, databaseName, engine, containerName, targetID string) error
	// RunVolumeBackup matches api.ServiceVolumeBackupRunner's shape
	// (internal/api/app_volume_backups.go); *Runner satisfies both, the
	// volume counterpart of RunBackup's own doc comment.
	RunVolumeBackup(ctx context.Context, historyID, serviceName, volumeName, dockerVolumeName, targetID string) error
	// ResolveDestination resolves a backup target ID into a live
	// Destination (*Runner.ResolveDestination, runner.go), the same
	// resolution RunBackup does internally before every upload. Needed
	// separately here because retention pruning deletes objects for a
	// pruned row's own TargetID after a run, not during one.
	ResolveDestination(ctx context.Context, targetID string) (Destination, error)
}

ScheduledBackupRunner is the surface Scheduler needs to actually run a backup once its schedule comes due: identical in shape to internal/api's own BackupRunner interface (internal/api/backups.go), redeclared here rather than imported across the package boundary, the same reasoning that file's own BackupRunner doc comment gives for why internal/api never imports internal/backup directly. *Runner satisfies this structurally, so cmd/levelrail/main.go wires the exact same *Runner instance into both api.WithBackupRunner (the manual trigger path) and NewScheduler (this one): one backup implementation, two ways to invoke it, exactly the design internal/spec.Backup's own doc comment implies by giving app.yaml a single databases[].backup block rather than separate manual/scheduled config shapes.

type ScheduledVerifier

type ScheduledVerifier interface {
	VerifyBackup(ctx context.Context, verificationID, backupHistoryID, engine, checkedBy string) error
}

ScheduledVerifier matches api.BackupVerifier's shape (internal/api/backup_verify.go); *VerifyRunner satisfies both, so cmd/levelrail/main.go shares one instance.

type Scheduler

type Scheduler struct {
	Store  ScheduleStore
	Runner ScheduledBackupRunner
	// Deleter removes the bucket object behind each pruned backup_history
	// row after a successful retention prune. nil disables object
	// deletion entirely (store rows still get pruned), the same
	// optional-capability shape this codebase already uses elsewhere
	// (e.g. backupRunner itself being nil when no master key is
	// configured, cmd/levelrail/main.go).
	Deleter Deleter
	// Verifier, if set, re-checks a scheduled backup right after it
	// succeeds (runScheduled). nil disables auto-verification, same
	// optional-capability shape as Deleter above. Set directly like Now
	// below, not a constructor param, to keep NewScheduler's signature
	// stable for existing call sites.
	Verifier ScheduledVerifier
	Logger   *slog.Logger
	// Now returns the current time, the same testable-clock field
	// Runner.Now already establishes in runner.go: production code
	// leaves it nil and Tick falls back to time.Now, tests set it for
	// deterministic schedule matching.
	Now func() time.Time
	// contains filtered or unexported fields
}

Scheduler periodically checks which databases have a due cron schedule (store.DesiredDatabase's BackupSchedule/BackupTargetID/ BackupRetain, migrations/0023_scheduled_backups.sql) and runs a backup through the same Runner the manual trigger path (internal/api's handleTriggerBackup) uses, recording the attempt in store.BackupHistory exactly like a manual trigger does, since both paths call the identical Runner.RunBackup.

Tick's own shape (list what to evaluate, act, persist any resulting state, join and return errors rather than stopping at the first one) deliberately mirrors internal/alerting.Engine.Tick: that is this codebase's own established shape for "evaluate every enabled X on an interval, one broken X must never block the rest," and a cron schedule check is exactly that same shape applied to a different resource. Run's own ticker loop is the identical pattern internal/alerting.Engine.Run, internal/telemetry.Collector.Run, and every other periodic loop in cmd/levelrail/main.go already use.

Unlike alerting.Engine.Tick's own per-rule dispatch, a due backup runs synchronously inside Tick, not fired off into its own goroutine the way internal/api's handleTriggerBackup dispatches one: that handler's goroutine exists specifically because an HTTP response must return in milliseconds, a constraint that does not apply to a background ticker loop already running off the request path. Running sequentially here also means a database's backup finishing (or failing) is fully resolved, including its retention prune, before Tick moves on to evaluate the next database, which keeps this method's own control flow as a flat, easy-to-reason-about loop instead of needing a WaitGroup or a results channel to know when a tick's work is actually done.

Missed-schedule handling is a known, deliberate, and documented gap, not an oversight: Scheduler tracks each database's next fire time purely in memory (nextRun below), armed fresh the first tick a schedule is observed rather than reconstructed from store.BackupHistory on startup. A schedule that should have fired while the control plane was down does not "catch up" the moment it restarts; it simply waits for its next naturally occurring occurrence. This mirrors the honesty this codebase already extends to a related gap on the record-keeping side (store.BackupHistory's own status constants doc comment: "nothing currently sweeps a stuck running row back to failed... a known, honest gap for the eventual scheduler to close alongside its own retry logic, not invented speculatively here"). Building real catch-up/retry semantics is a materially bigger feature (it needs to decide how many missed runs to catch up on, how to avoid a thundering herd of backups firing at once after a long outage, and how that interacts with retention) than this task's scope asked for, so it is left as a follow-up rather than guessed at here.

func NewScheduler

func NewScheduler(store ScheduleStore, runner ScheduledBackupRunner, deleter Deleter, logger *slog.Logger) *Scheduler

NewScheduler builds a Scheduler ready to Tick or Run. logger defaults to slog.Default() if nil, the same convention alerting.NewEngine already establishes for its own logger parameter. deleter may be nil (see Scheduler.Deleter's own doc comment).

func (*Scheduler) Run

func (s *Scheduler) Run(ctx context.Context, interval time.Duration) error

Run calls Tick on interval until ctx is done, matching the shape of every other periodic loop in this codebase (alerting.Engine.Run, telemetry.Collector.Run, and cmd/levelrail/main.go's own retention sweeps all follow this identical select-on-ticker-or-ctx-Done shape).

func (*Scheduler) Tick

func (s *Scheduler) Tick(ctx context.Context) error

Tick evaluates every currently scheduled database once. See this struct's own doc comment for the overall shape and the missed-schedule gap this deliberately leaves open. Errors from an individual database (an invalid cron string, RunBackup itself failing, a retention prune failing) are collected and joined, never stopping evaluation of the remaining databases, the same "one broken resource must not block the rest" principle Tick's own doc comment traces back to alerting.Engine.Tick.

type SecretsResolver

type SecretsResolver interface {
	Resolve(ctx context.Context, serviceName, envKey string) (string, error)
}

SecretsResolver is the one internal/secrets.Manager method Runner needs, narrowed to keep this package's tests from needing a real Manager (Manager itself has no interface today; a struct-shaped fake satisfying this one-method interface is enough).

type TargetTestStore

type TargetTestStore interface {
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
}

TargetTestStore is the store surface TargetTester needs, narrowed the same way HistoryStore/VerificationHistoryStore already are: only the one lookup a connection test actually requires.

type TargetTester

type TargetTester struct {
	Store   TargetTestStore
	Secrets SecretsResolver
	Tester  Tester
}

TargetTester resolves targetID's stored fields plus its live credentials from internal/secrets and runs Tester.Test against them, the "verify a stored credential by actually using it, once, on demand" shape this codebase's own RegistryAuthTester established first for registry credentials.

func (*TargetTester) TestTarget

func (t *TargetTester) TestTarget(ctx context.Context, targetID string) error

TestTarget probes targetID's configured bucket over its stored credentials, without uploading or deleting anything.

type Tester

type Tester interface {
	Test(ctx context.Context, dest Destination) error
}

Tester probes a Destination for a lightweight, non-destructive connectivity and credential check, the test-connection counterpart Uploader/Deleter above establish for real backup/prune work.

type Uploader

type Uploader interface {
	Upload(ctx context.Context, dest Destination, key string, r io.Reader, size int64) error
}

Uploader writes a single object to a Destination. Implementations must read r to completion (or return an error partway through, in which case the object must not be left partially visible under key, an S3-compatible multipart upload aborted rather than completed covers this for free) rather than assume a specific size up front: size is a hint for implementations that benefit from knowing it (choosing single-PUT vs. multipart), not a hard contract r will produce exactly that many bytes.

type VerificationHistoryStore

type VerificationHistoryStore interface {
	GetBackupHistory(ctx context.Context, id string) (store.BackupHistory, error)
	GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
	StartBackupVerification(ctx context.Context, v store.BackupVerification) error
	FinishBackupVerification(ctx context.Context, id, status string, checksumMatch, sizeMatch, formatValid bool, downloadedBytes int64, errMsg, finishedAt string) error
}

VerificationHistoryStore is the store.DB surface VerifyRunner needs: GetBackupHistory/GetBackupTarget resolve exactly what DownloadRunner.Download's own identical pair already resolve (the object a verification attempt re-downloads is the same object a real download would fetch), plus Start/FinishBackupVerification to record the attempt itself, the same shape RestoreHistoryStore's own doc comment describes for the restore direction.

type VerifyRunner

type VerifyRunner struct {
	Store      VerificationHistoryStore
	Secrets    SecretsResolver
	Downloader Downloader
	// Now returns the current time, matching Runner.Now and
	// RestoreRunner.Now's own "field, not time.Now called directly"
	// reasoning for deterministic tests.
	Now func() time.Time
}

VerifyRunner confirms a previously succeeded backup is still intact without ever attempting a live restore: re-download the object, re-hash it, and compare against what Runner recorded at backup time, the non-destructive counterpart RunRestore's own doc comment explicitly rules out for this feature (a live restore against a running system is real risk, not a safe thing to run unattended on a schedule or a button click).

func (*VerifyRunner) VerifyBackup

func (r *VerifyRunner) VerifyBackup(ctx context.Context, verificationID, backupHistoryID, engine, checkedBy string) error

VerifyBackup re-downloads backupHistoryID's stored object and checks it for corruption, recording the attempt in store.BackupVerification throughout: a running row before any work starts, then passed or failed once it's done. verificationID is minted by the caller (internal/api's handleVerifyBackup), the same "generate the ID before the first write" convention RunBackup's own doc comment describes; VerifyBackup is expected to run in a goroutine the caller has already detached, for the same reason RunBackup is.

Refuses (the same as RunRestore) to proceed against a backup whose Status is not store.BackupStatusSucceeded: a running or failed attempt has no complete object at its recorded key worth verifying.

engine is passed by the caller (internal/api resolves it from the database's own store.DesiredDatabase, the same "caller resolves engine, this package only consumes it" convention RunBackup's own engine parameter already establishes) rather than looked up here: it only sharpens the format check below (validateFormat), it is never required for the checksum/size checks that carry most of this function's real value, so an empty engine (e.g. the parent database was since deleted) still runs a meaningful, if slightly less specific, verification rather than failing outright.

type VolumeArchiver

type VolumeArchiver interface {
	Archive(ctx context.Context, volumeName string) (io.ReadCloser, error)
}

VolumeArchiver produces a tar archive of a named Docker volume's contents, the app-volume counterpart of Dumper for a database engine. The returned ReadCloser's Close must be called by every caller once done, the same contract Dumper's own doc comment establishes.

type VolumeCloneRestoreRunner

type VolumeCloneRestoreRunner struct {
	Store          VolumeCloneRestoreStore
	Secrets        SecretsResolver
	Downloader     Downloader
	VolumeRestorer VolumeRestorer
	Volumes        VolumeCreator
	// Now returns the current time, the same "field, not time.Now called
	// directly" reasoning CloneRestoreRunner.Now's own doc comment gives.
	Now func() time.Time
}

VolumeCloneRestoreRunner restores a previously succeeded volume backup into a brand-new, standalone Docker volume rather than overwriting the one it came from: the volume counterpart of CloneRestoreRunner. Simpler than CloneRestoreRunner because there is no reconciler to wait on (see VolumeCreator's own doc comment); creating the volume and restoring into it both happen synchronously, one after the other, inside RunVolumeCloneRestore itself.

func (*VolumeCloneRestoreRunner) RunVolumeCloneRestore

func (r *VolumeCloneRestoreRunner) RunVolumeCloneRestore(ctx context.Context, historyID, sourceServiceName, sourceVolumeName, newVolumeName, backupHistoryID string) error

RunVolumeCloneRestore creates newVolumeName as a brand-new, empty Docker volume, then downloads and restores backupHistoryID into it, recording the attempt as store.VolumeCloneRestore throughout: a running row before any work starts, then succeeded or failed once it's done. sourceServiceName/sourceVolumeName are recorded for history only; RunVolumeCloneRestore never reads from or writes to the volume they name. Expected to run in a goroutine the caller has already detached, the same convention RunCloneRestore's own doc comment describes.

type VolumeCloneRestoreStore

type VolumeCloneRestoreStore interface {
	StartVolumeCloneRestore(ctx context.Context, h store.VolumeCloneRestore) error
	FinishVolumeCloneRestore(ctx context.Context, id, status, errMsg, finishedAt string) error
	// contains filtered or unexported methods
}

VolumeCloneRestoreStore is the narrow store surface VolumeCloneRestoreRunner needs: backupResolver's two methods to resolve the source backup, plus the clone-restore bookkeeping pair, the app service volume counterpart of CloneRestoreStore.

type VolumeCreator

type VolumeCreator interface {
	EnsureVolume(ctx context.Context, name string) error
}

VolumeCreator is the narrow docker.Runtime surface VolumeCloneRestoreRunner needs to bring a brand-new, empty volume into existence before restoring into it: unlike CloneRestoreRunner's wait for a new database's own reconciler to report Ready, a bare Docker volume has no reconcile loop and exists the moment this call returns, so there is nothing to poll for here.

type VolumeRestorer

type VolumeRestorer interface {
	Restore(ctx context.Context, volumeName string, archive io.Reader) error
}

VolumeRestorer untars a previously archived stream back into a named Docker volume, in place: it wipes every existing file the volume holds first, so this is always a full replace, never a merge, matching Restorer's own "the restore command exited zero" contract.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL