Documentation
¶
Overview ¶
Package backup runs database backups and restores using the ecosystem pg-bkup / mysql-bkup tools as one-shot containers, to a per-workspace backup volume (local) or an S3-compatible bucket.
Index ¶
- Constants
- Variables
- func ArtifactName(out string, re *regexp.Regexp) (name string, encrypted bool, err error)
- func S3Env(c *S3Config) []string
- func SetPrefix(base, instance, ref string) string
- type AdoptResult
- type AdoptSkip
- type BackupAlerter
- type BackupReport
- type BackupReporter
- type DDLRunner
- type Destination
- type DiscoveredArtifact
- type DiscoveredSet
- type EventRecorder
- type ImageResolver
- type NodeDocker
- type RestoreSpec
- type RunOptions
- type S3Config
- type Service
- func (s *Service) AdoptSet(ctx context.Context, cfg *S3Config, basePath string, ...) (*AdoptResult, error)
- func (s *Service) Annotate(b *models.Backup, comment *string, pinned *bool) error
- func (s *Service) Delete(ctx context.Context, b *models.Backup) error
- func (s *Service) DeleteSet(ctx context.Context, set *models.DatabaseBackupSet) error
- func (s *Service) DiscoverSets(ctx context.Context, cfg *S3Config, basePath, passphrase string) ([]DiscoveredSet, error)
- func (s *Service) Download(ctx context.Context, b *models.Backup) (io.ReadCloser, int64, string, error)
- func (s *Service) Get(workspaceID, id uint) (*models.Backup, error)
- func (s *Service) GetSet(workspaceID, id uint) (*models.DatabaseBackupSet, error)
- func (s *Service) List(databaseID uint) ([]models.Backup, error)
- func (s *Service) ListSets(instanceID uint) ([]models.DatabaseBackupSet, error)
- func (s *Service) Prune(ctx context.Context, databaseID uint, maxBackups, retentionDays int) (int, error)
- func (s *Service) PruneSets(ctx context.Context, instanceID uint, maxSets, retentionDays int) (int, error)
- func (s *Service) RecoveryKit(set *models.DatabaseBackupSet) (string, []byte, error)
- func (s *Service) Restore(ctx context.Context, inst *models.DatabaseInstance, db *models.Database, ...) error
- func (s *Service) RestoreFromBackup(ctx context.Context, inst *models.DatabaseInstance, db *models.Database, ...) error
- func (s *Service) RestoreSet(ctx context.Context, inst *models.DatabaseInstance, ...) (*SetRestoreResult, error)
- func (s *Service) RestoreUpload(ctx context.Context, inst *models.DatabaseInstance, db *models.Database, ...) error
- func (s *Service) RewrapSets(workspaceID uint, oldPassphrase, newPassphrase string) (int, error)
- func (s *Service) Run(ctx context.Context, inst *models.DatabaseInstance, db *models.Database, ...) (*models.Backup, error)
- func (s *Service) RunSet(ctx context.Context, inst *models.DatabaseInstance, opts SetOptions, ...) (*models.DatabaseBackupSet, error)
- func (s *Service) SealedSetCount(workspaceID uint) (int, error)
- func (s *Service) SetAlerter(a BackupAlerter)
- func (s *Service) SetDDLRunner(r DDLRunner)
- func (s *Service) SetEventRecorder(r EventRecorder)
- func (s *Service) SetImageResolver(r ImageResolver)
- func (s *Service) SetLogStore(store *logstore.Store)
- func (s *Service) SetReporter(r BackupReporter)
- func (s *Service) SetSetRepository(r *repositories.DatabaseBackupSetRepository)
- func (s *Service) VerifySet(ctx context.Context, cfg *S3Config, set *models.DatabaseBackupSet, ...) (*VerifyResult, error)
- type SetInfo
- type SetInfoArtifact
- type SetOptions
- type SetRestoreResult
- type VerifyResult
Constants ¶
const ( VerifyOK = "ok" VerifyFailed = "failed" )
Verification outcomes recorded on a set.
const SetInfoObject = "info.json"
SetInfoObject is the descriptor's name within a set's prefix.
const SetInfoSchema = 1
SetInfoSchema is the descriptor's payload version. A reader refuses a schema it does not know rather than guessing at fields.
Variables ¶
var ( // ErrSetNotInBucket means no descriptor under the workspace's backup path names // the requested recovery point. ErrSetNotInBucket = errors.New("no recovery point with that ref is in the bucket") // ErrEngineMismatch refuses to adopt a set onto an instance running a different // engine. A PostgreSQL dump is not a MySQL one, and finding that out at restore // time is finding it out too late. ErrEngineMismatch = errors.New("the recovery point was taken from a different engine than this instance runs") )
var ( ErrUnsupportedEngine = errors.New("backups are not supported for this engine yet") ErrNoBackupFile = errors.New("backup has no artifact to restore") ErrDownloadRemote = errors.New("download is only available for local backups; fetch s3 backups from your bucket") )
var ( ErrSetsUnavailable = errors.New("backup sets are not available") // ErrNoDatabases means the instance has no logical database to back up. Redis // reaches here too: its data rides on the volume, so it has no logical // databases and is covered by a volume backup instead. ErrNoDatabases = errors.New("the instance has no database to back up") // ErrS3Required means the workspace has no object-storage target. A recovery // point exists to survive the loss of the host it was taken from, and the local // backup volume dies with that host — so a local set would promise something it // cannot deliver. Rediscovering sets from a bucket depends on this too. ErrS3Required = errors.New("backup sets require the workspace S3 backup target") )
var ErrRestoreVersionTooNew = errors.New("the backup was taken from a newer engine than this instance runs")
ErrRestoreVersionTooNew refuses to load a dump into an engine older than the one it came from. A PostgreSQL 17 dump does not load into 16, and the failure lands mid-restore over a database that has already been dropped on a force run.
Functions ¶
func ArtifactName ¶ added in v1.7.0
ArtifactName extracts the artifact the helper actually uploaded from its output and reports whether it is encrypted. It takes the LAST match, not the first: with encryption on the tools narrate the plain dump before the encrypted one they upload, so the first match names a file that was never written.
func S3Env ¶
S3Env builds the S3 environment for the ecosystem backup tools (pg-bkup, mysql-bkup, volume-bkup), which all read the same variable names. Exported so the volume backup service can target the same bucket without duplicating the var-name contract.
func SetPrefix ¶ added in v1.9.6
SetPrefix is where a recovery point's objects live: one directory per set, under one per instance, beneath the workspace's database backup path. Grouping matters because discovery reads a set as a unit, and a flat prefix shared by every set would make that a filename-parsing exercise.
Types ¶
type AdoptResult ¶ added in v1.9.6
type AdoptResult struct {
Ref string `json:"ref"`
SetID uint `json:"set_id"`
// Adopted is the number of artifacts now restorable from the console.
Adopted int `json:"adopted"`
// AlreadyKnown means this workspace already had the set; adopting is a no-op
// rather than a duplicate, so re-running it after a partial failure is safe.
AlreadyKnown bool `json:"already_known"`
// Skipped names the artifacts that could not be attached, and why. They stay in
// the bucket untouched — recreate the database and adopt again to pick them up.
Skipped []AdoptSkip `json:"skipped,omitempty"`
}
AdoptResult reports what adopting a recovery point produced.
type AdoptSkip ¶ added in v1.9.6
type AdoptSkip struct {
Database string `json:"database"`
Filename string `json:"filename"`
Reason string `json:"reason"`
}
AdoptSkip is one artifact that could not be attached to a local database.
type BackupAlerter ¶ added in v1.6.0
type BackupAlerter interface {
BackupFailed(workspaceID, databaseID uint, dbName, errMsg string)
BackupSucceeded(workspaceID, databaseID uint)
BackupSetFailed(workspaceID, instanceID uint, instanceName, ref, errMsg string)
BackupSetSucceeded(workspaceID, instanceID uint)
}
BackupAlerter receives database-backup outcomes so the alert engine can raise a "backup failed" alert and auto-resolve it on the next success. Kept as a local interface so the backup service stays decoupled from the alerting package; the wiring bridges it to the engine.
The set methods are deliberately separate rather than reusing the per-database pair with an instance id in the database slot. That is what the code did for verification failures, and it gave a set and a logical database that happen to share a numeric id the same dedup key — so a retried database silently resolved a still-broken recovery point.
type BackupReport ¶ added in v1.10.5
type BackupReport struct {
// Subject is the instance name for a set, the logical database's name otherwise.
Subject string
// Ref names the recovery point. Empty for a single-database backup.
Ref string
Link string
OK bool
// Databases is how many artifacts a set carries. Zero for a single-database backup.
Databases int
SizeBytes int64
Err string
}
BackupReport is one finished scheduled run — a single database's, or a whole recovery point's.
type BackupReporter ¶ added in v1.10.5
type BackupReporter interface {
ScheduledBackupFinished(workspaceID uint, rep BackupReport)
}
BackupReporter posts a finished SCHEDULED run to the workspace inbox.
Scheduled only, and that is the point: a 03:00 run has nobody watching it, which is the whole reason its outcome has to go somewhere. A manual backup reports itself in the row the operator who clicked it is already looking at, so notifying them as well would be noise.
Distinct from BackupAlerter because the two carry different things. A failure is a condition that stays open until a later run clears it; a completion is a fact with nothing to clear.
type DDLRunner ¶
type DDLRunner interface {
RecreateDatabase(ctx context.Context, inst *models.DatabaseInstance, db *models.Database) error
}
DDLRunner recreates a logical database for a force restore. Implemented by the database service; injected after construction (optional).
type Destination ¶
type Destination struct {
Type string // "local" | "s3"
S3 *S3Config
// GPGPassphrase encrypts the artifact: the *-bkup tools write "<name>.gpg" when it is set and decrypt
// transparently on restore with the same value. Used by portable workspace bundles and by platform
// disaster recovery, which protect every artifact they carry with one passphrase.
GPGPassphrase string
}
Destination selects where a backup is stored.
type DiscoveredArtifact ¶ added in v1.9.6
type DiscoveredArtifact struct {
Database string `json:"database"`
Filename string `json:"filename"`
SizeBytes int64 `json:"size_bytes,omitempty"`
Encrypted bool `json:"encrypted"`
// Present reports whether the object is actually in the bucket. A descriptor
// records what happened when the backup ran; retention and lifecycle rules act
// on the bucket afterwards.
Present bool `json:"present"`
}
DiscoveredArtifact is one database's dump within a discovered set.
type DiscoveredSet ¶ added in v1.9.6
type DiscoveredSet struct {
Ref string `json:"ref"`
Instance string `json:"instance"`
Engine string `json:"engine"`
Version string `json:"version,omitempty"`
CreatedAt string `json:"created_at"`
SizeBytes int64 `json:"size_bytes"`
Encrypted bool `json:"encrypted"`
// Prefix is where the set's objects live, so a restore knows where to look.
Prefix string `json:"prefix"`
Artifacts []DiscoveredArtifact `json:"artifacts"`
// Known is true when this workspace already has the set in its own database.
// Those need no adopting; they are listed so the view is the whole bucket
// rather than only the unfamiliar parts of it.
Known bool `json:"known"`
SetID uint `json:"set_id,omitempty"`
// Openable reports that the workspace's current passphrase opens this set's
// envelope. False on a set sealed with a different secret — which is worth
// knowing before a restore is attempted, not during one.
Openable bool `json:"openable"`
// Reason says why the set cannot be used, when it cannot.
Reason string `json:"reason,omitempty"`
}
DiscoveredSet is a recovery point found in the bucket.
The bucket is the authority, not this workspace's database: after losing a workspace — or moving to a new install — the sets Miabi knows about are whatever its own rows contain, while everything ever taken is still in object storage. Discovery makes those reachable again.
type EventRecorder ¶ added in v1.9.5
type EventRecorder interface {
EmitDatabase(workspaceID, databaseID uint, name string, t models.AppEventType, sev models.AppEventSeverity, message string, meta map[string]string, actorID *uint)
}
EventRecorder writes backup and restore outcomes to the owning instance's timeline. The subject is the instance, not the logical database, so one timeline covers the whole instance; the logical database name rides along in the metadata.
type ImageResolver ¶
ImageResolver resolves a deployment-config catalog key to an image ref.
type NodeDocker ¶
NodeDocker resolves the Docker client for a node id (0 = local).
type RestoreSpec ¶
type RestoreSpec struct {
Filename string
Destination string // "local" | "s3"
VolumeName string
S3 *S3Config
S3Path string // remote folder prefix the artifact was stored under
Force bool
// GPGPassphrase decrypts a ".gpg" artifact (see Destination.GPGPassphrase).
GPGPassphrase string
}
RestoreSpec describes a restore source. Filename is the artifact on the local backup volume or the S3 key; Force drops & recreates the database first.
type RunOptions ¶ added in v1.9.5
type RunOptions struct {
Trigger string // manual | scheduled | upgrade | bundle | platform-dr
Comment string
// SetID marks this backup as one item of a recovery point. Nil for an ordinary
// single-database backup.
SetID *uint
}
RunOptions carries the metadata of a backup run: why it was taken, and any note the operator attached. It replaces a bare trigger string so the two adjacent free-text values cannot be passed the wrong way round.
type S3Config ¶
type S3Config struct {
Endpoint string
Bucket string
Region string
AccessKey string
SecretKey string
Path string
UseSSL bool
ForcePathStyle bool
}
S3Config targets an S3-compatible bucket (endpoint empty => AWS).
type Service ¶
type Service struct {
// contains filtered or unexported fields
}
func NewService ¶
func NewService(repo *repositories.BackupRepository, dbs *repositories.DatabaseRepository, clients NodeDocker) *Service
func (*Service) AdoptSet ¶ added in v1.9.6
func (s *Service) AdoptSet(ctx context.Context, cfg *S3Config, basePath string, inst *models.DatabaseInstance, ref string) (*AdoptResult, error)
AdoptSet writes a recovery point found in the bucket into this workspace's history, so the console can list and restore it.
It writes rows and touches no data: adopting is how a fresh workspace pointed at an old bucket gets its backup history back, and it must be safe to run while the instance it names is serving traffic.
Artifacts are matched to the instance's logical databases BY NAME. An artifact whose database no longer exists is reported rather than invented: creating an empty database to hang a backup off would produce a row that looks restorable and restores into something nobody asked for.
func (*Service) Annotate ¶ added in v1.9.5
Annotate updates a backup's note and pin. Both are nil-able so a caller can change one without restating the other. Editing after the fact is the point: scheduled and pre-upgrade backups have no operator present when they run, and what a backup was for is often only clear once the change it guarded went wrong.
func (*Service) Delete ¶
Delete removes a backup record and, for local backups, its artifact file from the workspace backup volume (best-effort). S3 artifacts are left in place.
func (*Service) DeleteSet ¶ added in v1.9.6
DeleteSet removes a recovery point and every artifact it carries. The items' rows cascade with the set, but their artifacts are objects in a bucket, which only this service can remove.
func (*Service) DiscoverSets ¶ added in v1.9.6
func (s *Service) DiscoverSets(ctx context.Context, cfg *S3Config, basePath, passphrase string) ([]DiscoveredSet, error)
DiscoverSets lists the recovery points under a workspace's database backup path. It reads only the cleartext descriptors, so no passphrase is needed to see what exists — which is the whole point of keeping them readable.
passphrase is optional: supplied, it is used to report whether each set can actually be opened, without which the list says what is there but not what is usable.
func (*Service) Download ¶
func (s *Service) Download(ctx context.Context, b *models.Backup) (io.ReadCloser, int64, string, error)
Download streams a local backup artifact. The returned reader must be closed by the caller. S3 backups are not downloadable through Miabi.
func (*Service) GetSet ¶ added in v1.9.6
func (s *Service) GetSet(workspaceID, id uint) (*models.DatabaseBackupSet, error)
GetSet loads one recovery point with its items.
func (*Service) ListSets ¶ added in v1.9.6
func (s *Service) ListSets(instanceID uint) ([]models.DatabaseBackupSet, error)
ListSets returns an instance's recovery points, newest first.
func (*Service) Prune ¶
func (s *Service) Prune(ctx context.Context, databaseID uint, maxBackups, retentionDays int) (int, error)
Prune enforces a retention policy on a database's backups: keep at most maxBackups most-recent, and delete any older than retentionDays. A zero bound is ignored. Returns the number of backups removed.
Pinned backups are never pruned, and do not occupy a maxBackups slot: pinning a backup must not silently shrink how much rolling history the policy keeps.
func (*Service) PruneSets ¶ added in v1.9.6
func (s *Service) PruneSets(ctx context.Context, instanceID uint, maxSets, retentionDays int) (int, error)
PruneSets enforces retention over an instance's recovery points: keep at most maxSets most-recent, and delete any older than retentionDays. A zero bound is ignored. Returns the number of sets removed.
Three rules the per-database prune does not have:
- Whole sets only. Pruning items out of a set would leave the half recovery point the set exists to prevent.
- A set with any pinned member is never pruned, and does not occupy a maxSets slot, mirroring how a pinned backup behaves.
- The newest completed set always survives, whatever the policy says. A mistyped retention must not be able to leave an instance with nothing.
func (*Service) RecoveryKit ¶ added in v1.9.6
RecoveryKit renders everything needed to read a recovery point back without Miabi: where the artifacts are, the sealed envelope, the exact envelope format, and the commands that get from a passphrase to a restorable dump.
The kit deliberately does NOT contain the data key — only the envelope it is sealed in. A kit is therefore useless without the passphrase, which is the property that makes it safe to keep beside the backups it describes.
func (*Service) Restore ¶
func (s *Service) Restore(ctx context.Context, inst *models.DatabaseInstance, db *models.Database, spec RestoreSpec) error
Restore restores a logical database from the given source. When spec.Force is set the database is dropped & recreated first (clean slate). It wraps restore so that every one of its early returns still reports an outcome to the instance timeline.
func (*Service) RestoreFromBackup ¶
func (s *Service) RestoreFromBackup(ctx context.Context, inst *models.DatabaseInstance, db *models.Database, b *models.Backup, dest Destination, force, allowVersionMismatch bool) error
RestoreFromBackup restores a logical database from a stored backup record. For an S3 backup, dest.S3 must supply the bucket credentials again.
func (*Service) RestoreSet ¶ added in v1.9.6
func (s *Service) RestoreSet(ctx context.Context, inst *models.DatabaseInstance, set *models.DatabaseBackupSet, dest Destination, force, allowVersionMismatch bool) (*SetRestoreResult, error)
RestoreSet restores every database in a recovery point, in one operation.
Unlike a backup, this does NOT stop at the first failure. A restore is run when something has already gone wrong, and abandoning the remaining databases because one of them failed would turn a partial recovery into a smaller one. Every outcome is reported and the caller decides.
func (*Service) RestoreUpload ¶
func (s *Service) RestoreUpload(ctx context.Context, inst *models.DatabaseInstance, db *models.Database, origName string, content io.Reader, size int64, force bool) error
RestoreUpload streams an uploaded dump onto the workspace backup volume and restores the database from it (force drops & recreates first). The temp artifact is removed afterward (best-effort).
func (*Service) RewrapSets ¶ added in v1.9.6
RewrapSets re-seals every sealed recovery point in a workspace from one passphrase to the next. This is the whole reason the artifacts are encrypted with a per-set data key: rotation rewrites a few hundred bytes per set and never re-reads a dump.
It stops at the first envelope that will not open, because that means the old passphrase is wrong and continuing would rewrap the rest under a mismatched secret. Sets already rewrapped stay rewrapped; the operation is resumable.
func (*Service) Run ¶
func (s *Service) Run(ctx context.Context, inst *models.DatabaseInstance, db *models.Database, opts RunOptions, dest Destination) (*models.Backup, error)
Run performs a backup of a logical database to the given destination and records the result. inst is the server hosting db.
func (*Service) RunSet ¶ added in v1.9.6
func (s *Service) RunSet(ctx context.Context, inst *models.DatabaseInstance, opts SetOptions, dest Destination) (*models.DatabaseBackupSet, error)
RunSet backs up every logical database on an instance as one recovery point. It requires the workspace S3 target and returns ErrS3Required without one.
The set is marked failed if any item fails, and the error names the first failure: a recovery point that is missing a database is not one, and reporting it as a success is how people discover the gap during an incident instead of after the run.
func (*Service) SealedSetCount ¶ added in v1.9.6
SealedSetCount reports how many recovery points in a workspace are sealed, so a caller can refuse to discard the passphrase that opens them.
func (*Service) SetAlerter ¶ added in v1.6.0
func (s *Service) SetAlerter(a BackupAlerter)
SetAlerter wires backup-outcome alerting (optional; nil = no alerts).
func (*Service) SetDDLRunner ¶
SetDDLRunner wires the database service used to drop+recreate a database on a force restore.
func (*Service) SetEventRecorder ¶ added in v1.9.5
func (s *Service) SetEventRecorder(r EventRecorder)
SetEventRecorder wires the instance timeline recorder (nil-safe: unset means no events).
func (*Service) SetImageResolver ¶
func (s *Service) SetImageResolver(r ImageResolver)
SetImageResolver wires the deployment-config resolver for backup tool images.
func (*Service) SetLogStore ¶
SetLogStore wires the shared execution-log store. When set, a backup run's full output is externalized to the store on terminal state and the DB row keeps only a bounded tail + a reference. nil keeps DB-tail-only.
func (*Service) SetReporter ¶ added in v1.10.5
func (s *Service) SetReporter(r BackupReporter)
SetReporter wires scheduled-run inbox reporting (optional; nil = no inbox items).
func (*Service) SetSetRepository ¶ added in v1.9.6
func (s *Service) SetSetRepository(r *repositories.DatabaseBackupSetRepository)
SetSetRepository wires recovery-point storage (nil-safe: unset means RunSet and the prune below report ErrSetsUnavailable rather than panicking).
func (*Service) VerifySet ¶ added in v1.9.6
func (s *Service) VerifySet(ctx context.Context, cfg *S3Config, set *models.DatabaseBackupSet, passphrase string) (*VerifyResult, error)
VerifySet checks a recovery point against the bucket: every artifact is still there, still the size it was, and the envelope still opens.
It deliberately does not read the dumps back. That costs the size of the backup every time, which is how verification becomes the thing people switch off. Proving a dump actually loads is a drill, and a drill needs a live engine.
type SetInfo ¶ added in v1.9.6
type SetInfo struct {
Schema int `json:"schema"`
Ref string `json:"ref"`
Instance string `json:"instance"`
Engine string `json:"engine"`
Version string `json:"version,omitempty"`
Trigger string `json:"trigger,omitempty"`
CreatedAt time.Time `json:"created_at"`
SizeBytes int64 `json:"size_bytes"`
Encrypted bool `json:"encrypted"`
// Envelope seals this set's data key under the workspace backup passphrase.
// Ciphertext: useless to a reader who does not hold the passphrase.
Envelope string `json:"envelope,omitempty"`
Artifacts []SetInfoArtifact `json:"artifacts"`
}
SetInfo is the CLEARTEXT descriptor written beside a recovery point's artifacts.
Cleartext is the point, copied from platformbackup's discovery reader: you must be able to list what a bucket holds without the passphrase, or a user who has lost their secret cannot even see what they have lost. It carries the shape of the data — never any of it.
The sealed envelope rides along because it is the one piece a restore cannot reconstruct and cannot read without the passphrase anyway. Without it here, a set would be unreadable the moment the platform that took it was gone, which is precisely the case this exists for.
type SetInfoArtifact ¶ added in v1.9.6
type SetInfoArtifact struct {
Database string `json:"database"`
Filename string `json:"filename"`
SizeBytes int64 `json:"size_bytes,omitempty"`
Encrypted bool `json:"encrypted"`
}
SetInfoArtifact is one database's dump within the set.
type SetOptions ¶ added in v1.9.6
SetOptions tunes a set run.
type SetRestoreResult ¶ added in v1.9.6
type SetRestoreResult struct {
Ref string `json:"ref"`
Restored []string `json:"restored"`
Failed []string `json:"failed,omitempty"`
}
SetRestoreResult reports what a set-level restore did.
type VerifyResult ¶ added in v1.9.6
type VerifyResult struct {
Ref string `json:"ref"`
OK bool `json:"ok"`
Checked int `json:"checked"`
// Missing are artifacts the set recorded that are no longer in the bucket —
// usually a lifecycle rule, occasionally an upload nobody noticed had failed.
Missing []string `json:"missing,omitempty"`
// Resized are artifacts whose stored size no longer matches what was recorded,
// which means the object is not the one this set took.
Resized []string `json:"resized,omitempty"`
// EnvelopeOK reports that the workspace passphrase still opens the data key. A
// set can be perfectly intact in the bucket and unreadable all the same.
EnvelopeOK bool `json:"envelope_ok"`
Error string `json:"error,omitempty"`
}
VerifyResult is what a check found. It names artifacts rather than only counting them: "one file is missing" is not actionable, "billing's dump is gone" is.