backup

package
v1.10.21 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 25, 2026 License: AGPL-3.0 Imports: 24 Imported by: 0

Documentation

Overview

Package backup runs database backups and restores using the ecosystem pg-bkup / mysql-bkup tools as one-shot containers, to a per-workspace backup volume (local) or an S3-compatible bucket.

Index

Constants

View Source
const (
	VerifyOK     = "ok"
	VerifyFailed = "failed"
)

Verification outcomes recorded on a set.

View Source
const SetInfoObject = "info.json"

SetInfoObject is the descriptor's name within a set's prefix.

View Source
const SetInfoSchema = 1

SetInfoSchema is the descriptor's payload version. A reader refuses a schema it does not know rather than guessing at fields.

Variables

View Source
var (
	// ErrSetNotInBucket means no descriptor under the workspace's backup path names
	// the requested recovery point.
	ErrSetNotInBucket = errors.New("no recovery point with that ref is in the bucket")
	// ErrEngineMismatch refuses to adopt a set onto an instance running a different
	// engine. A PostgreSQL dump is not a MySQL one, and finding that out at restore
	// time is finding it out too late.
	ErrEngineMismatch = errors.New("the recovery point was taken from a different engine than this instance runs")
)
View Source
var (
	ErrUnsupportedEngine = errors.New("backups are not supported for this engine yet")
	ErrNoBackupFile      = errors.New("backup has no artifact to restore")
	ErrDownloadRemote    = errors.New("download is only available for local backups; fetch s3 backups from your bucket")
)
View Source
var (
	// ErrSetsUnavailable means the set repository was never wired.
	ErrSetsUnavailable = errors.New("backup sets are not available")
	// ErrNoDatabases means the instance has no logical database to back up. Redis
	// reaches here too: its data rides on the volume, so it has no logical
	// databases and is covered by a volume backup instead.
	ErrNoDatabases = errors.New("the instance has no database to back up")
	// ErrS3Required means the workspace has no object-storage target. A recovery
	// point exists to survive the loss of the host it was taken from, and the local
	// backup volume dies with that host — so a local set would promise something it
	// cannot deliver. Rediscovering sets from a bucket depends on this too.
	ErrS3Required = errors.New("backup sets require the workspace S3 backup target")
)
View Source
var ErrRestoreVersionTooNew = errors.New("the backup was taken from a newer engine than this instance runs")

ErrRestoreVersionTooNew refuses to load a dump into an engine older than the one it came from. A PostgreSQL 17 dump does not load into 16, and the failure lands mid-restore over a database that has already been dropped on a force run.

Functions

func ArtifactName added in v1.7.0

func ArtifactName(out string, re *regexp.Regexp) (name string, encrypted bool, err error)

ArtifactName extracts the artifact the helper actually uploaded from its output and reports whether it is encrypted. It takes the LAST match, not the first: with encryption on the tools narrate the plain dump before the encrypted one they upload, so the first match names a file that was never written.

func S3Env

func S3Env(c *S3Config) []string

S3Env builds the S3 environment for the ecosystem backup tools (pg-bkup, mysql-bkup, volume-bkup), which all read the same variable names. Exported so the volume backup service can target the same bucket without duplicating the var-name contract.

func SetPrefix added in v1.9.6

func SetPrefix(base, instance, ref string) string

SetPrefix is where a recovery point's objects live: one directory per set, under one per instance, beneath the workspace's database backup path. Grouping matters because discovery reads a set as a unit, and a flat prefix shared by every set would make that a filename-parsing exercise.

Types

type AdoptResult added in v1.9.6

type AdoptResult struct {
	Ref   string `json:"ref"`
	SetID uint   `json:"set_id"`
	// Adopted is the number of artifacts now restorable from the console.
	Adopted int `json:"adopted"`
	// AlreadyKnown means this workspace already had the set; adopting is a no-op
	// rather than a duplicate, so re-running it after a partial failure is safe.
	AlreadyKnown bool `json:"already_known"`
	// Skipped names the artifacts that could not be attached, and why. They stay in
	// the bucket untouched — recreate the database and adopt again to pick them up.
	Skipped []AdoptSkip `json:"skipped,omitempty"`
}

AdoptResult reports what adopting a recovery point produced.

type AdoptSkip added in v1.9.6

type AdoptSkip struct {
	Database string `json:"database"`
	Filename string `json:"filename"`
	Reason   string `json:"reason"`
}

AdoptSkip is one artifact that could not be attached to a local database.

type BackupAlerter added in v1.6.0

type BackupAlerter interface {
	BackupFailed(workspaceID, databaseID uint, dbName, errMsg string)
	BackupSucceeded(workspaceID, databaseID uint)
	BackupSetFailed(workspaceID, instanceID uint, instanceName, ref, errMsg string)
	BackupSetSucceeded(workspaceID, instanceID uint)
}

BackupAlerter receives database-backup outcomes so the alert engine can raise a "backup failed" alert and auto-resolve it on the next success. Kept as a local interface so the backup service stays decoupled from the alerting package; the wiring bridges it to the engine.

The set methods are deliberately separate rather than reusing the per-database pair with an instance id in the database slot. That is what the code did for verification failures, and it gave a set and a logical database that happen to share a numeric id the same dedup key — so a retried database silently resolved a still-broken recovery point.

type BackupReport added in v1.10.5

type BackupReport struct {
	// Subject is the instance name for a set, the logical database's name otherwise.
	Subject string
	// Ref names the recovery point. Empty for a single-database backup.
	Ref  string
	Link string
	OK   bool
	// Databases is how many artifacts a set carries. Zero for a single-database backup.
	Databases int
	SizeBytes int64
	Err       string
}

BackupReport is one finished scheduled run — a single database's, or a whole recovery point's.

type BackupReporter added in v1.10.5

type BackupReporter interface {
	ScheduledBackupFinished(workspaceID uint, rep BackupReport)
}

BackupReporter posts a finished SCHEDULED run to the workspace inbox.

Scheduled only, and that is the point: a 03:00 run has nobody watching it, which is the whole reason its outcome has to go somewhere. A manual backup reports itself in the row the operator who clicked it is already looking at, so notifying them as well would be noise.

Distinct from BackupAlerter because the two carry different things. A failure is a condition that stays open until a later run clears it; a completion is a fact with nothing to clear.

type DDLRunner

type DDLRunner interface {
	RecreateDatabase(ctx context.Context, inst *models.DatabaseInstance, db *models.Database) error
}

DDLRunner recreates a logical database for a force restore. Implemented by the database service; injected after construction (optional).

type Destination

type Destination struct {
	Type string // "local" | "s3"
	S3   *S3Config
	// GPGPassphrase encrypts the artifact: the *-bkup tools write "<name>.gpg" when it is set and decrypt
	// transparently on restore with the same value. Used by portable workspace bundles and by platform
	// disaster recovery, which protect every artifact they carry with one passphrase.
	GPGPassphrase string
}

Destination selects where a backup is stored.

type DiscoveredArtifact added in v1.9.6

type DiscoveredArtifact struct {
	Database  string `json:"database"`
	Filename  string `json:"filename"`
	SizeBytes int64  `json:"size_bytes,omitempty"`
	Encrypted bool   `json:"encrypted"`
	// Present reports whether the object is actually in the bucket. A descriptor
	// records what happened when the backup ran; retention and lifecycle rules act
	// on the bucket afterwards.
	Present bool `json:"present"`
}

DiscoveredArtifact is one database's dump within a discovered set.

type DiscoveredSet added in v1.9.6

type DiscoveredSet struct {
	Ref       string `json:"ref"`
	Instance  string `json:"instance"`
	Engine    string `json:"engine"`
	Version   string `json:"version,omitempty"`
	CreatedAt string `json:"created_at"`
	SizeBytes int64  `json:"size_bytes"`
	Encrypted bool   `json:"encrypted"`
	// Prefix is where the set's objects live, so a restore knows where to look.
	Prefix string `json:"prefix"`

	Artifacts []DiscoveredArtifact `json:"artifacts"`

	// Known is true when this workspace already has the set in its own database.
	// Those need no adopting; they are listed so the view is the whole bucket
	// rather than only the unfamiliar parts of it.
	Known bool `json:"known"`
	SetID uint `json:"set_id,omitempty"`

	// Openable reports that the workspace's current passphrase opens this set's
	// envelope. False on a set sealed with a different secret — which is worth
	// knowing before a restore is attempted, not during one.
	Openable bool `json:"openable"`
	// Reason says why the set cannot be used, when it cannot.
	Reason string `json:"reason,omitempty"`
}

DiscoveredSet is a recovery point found in the bucket.

The bucket is the authority, not this workspace's database: after losing a workspace — or moving to a new install — the sets Miabi knows about are whatever its own rows contain, while everything ever taken is still in object storage. Discovery makes those reachable again.

type EventRecorder added in v1.9.5

type EventRecorder interface {
	EmitDatabase(workspaceID, databaseID uint, name string, t models.AppEventType, sev models.AppEventSeverity, message string, meta map[string]string, actorID *uint)
}

EventRecorder writes backup and restore outcomes to the owning instance's timeline. The subject is the instance, not the logical database, so one timeline covers the whole instance; the logical database name rides along in the metadata.

type ImageResolver

type ImageResolver interface {
	Ref(key string) string
}

ImageResolver resolves a deployment-config catalog key to an image ref.

type NodeDocker

type NodeDocker interface {
	For(serverID uint) (docker.Client, error)
	LocalID() uint
}

NodeDocker resolves the Docker client for a node id (0 = local).

type RestoreSpec

type RestoreSpec struct {
	Filename    string
	Destination string // "local" | "s3"
	VolumeName  string
	S3          *S3Config
	S3Path      string // remote folder prefix the artifact was stored under
	Force       bool
	// GPGPassphrase decrypts a ".gpg" artifact (see Destination.GPGPassphrase).
	GPGPassphrase string
}

RestoreSpec describes a restore source. Filename is the artifact on the local backup volume or the S3 key; Force drops & recreates the database first.

type RunOptions added in v1.9.5

type RunOptions struct {
	Trigger string // manual | scheduled | upgrade | bundle | platform-dr
	Comment string
	// SetID marks this backup as one item of a recovery point. Nil for an ordinary
	// single-database backup.
	SetID *uint
}

RunOptions carries the metadata of a backup run: why it was taken, and any note the operator attached. It replaces a bare trigger string so the two adjacent free-text values cannot be passed the wrong way round.

type S3Config

type S3Config struct {
	Endpoint       string
	Bucket         string
	Region         string
	AccessKey      string
	SecretKey      string
	Path           string
	UseSSL         bool
	ForcePathStyle bool
}

S3Config targets an S3-compatible bucket (endpoint empty => AWS).

type Service

type Service struct {
	// contains filtered or unexported fields
}

func (*Service) AdoptSet added in v1.9.6

func (s *Service) AdoptSet(ctx context.Context, cfg *S3Config, basePath string, inst *models.DatabaseInstance, ref string) (*AdoptResult, error)

AdoptSet writes a recovery point found in the bucket into this workspace's history, so the console can list and restore it.

It writes rows and touches no data: adopting is how a fresh workspace pointed at an old bucket gets its backup history back, and it must be safe to run while the instance it names is serving traffic.

Artifacts are matched to the instance's logical databases BY NAME. An artifact whose database no longer exists is reported rather than invented: creating an empty database to hang a backup off would produce a row that looks restorable and restores into something nobody asked for.

func (*Service) Annotate added in v1.9.5

func (s *Service) Annotate(b *models.Backup, comment *string, pinned *bool) error

Annotate updates a backup's note and pin. Both are nil-able so a caller can change one without restating the other. Editing after the fact is the point: scheduled and pre-upgrade backups have no operator present when they run, and what a backup was for is often only clear once the change it guarded went wrong.

func (*Service) Delete

func (s *Service) Delete(ctx context.Context, b *models.Backup) error

Delete removes a backup record and, for local backups, its artifact file from the workspace backup volume (best-effort). S3 artifacts are left in place.

func (*Service) DeleteSet added in v1.9.6

func (s *Service) DeleteSet(ctx context.Context, set *models.DatabaseBackupSet) error

DeleteSet removes a recovery point and every artifact it carries. The items' rows cascade with the set, but their artifacts are objects in a bucket, which only this service can remove.

func (*Service) DiscoverSets added in v1.9.6

func (s *Service) DiscoverSets(ctx context.Context, cfg *S3Config, basePath, passphrase string) ([]DiscoveredSet, error)

DiscoverSets lists the recovery points under a workspace's database backup path. It reads only the cleartext descriptors, so no passphrase is needed to see what exists — which is the whole point of keeping them readable.

passphrase is optional: supplied, it is used to report whether each set can actually be opened, without which the list says what is there but not what is usable.

func (*Service) Download

func (s *Service) Download(ctx context.Context, b *models.Backup) (io.ReadCloser, int64, string, error)

Download streams a local backup artifact. The returned reader must be closed by the caller. S3 backups are not downloadable through Miabi.

func (*Service) Get

func (s *Service) Get(workspaceID, id uint) (*models.Backup, error)

func (*Service) GetSet added in v1.9.6

func (s *Service) GetSet(workspaceID, id uint) (*models.DatabaseBackupSet, error)

GetSet loads one recovery point with its items.

func (*Service) List

func (s *Service) List(databaseID uint) ([]models.Backup, error)

func (*Service) ListSets added in v1.9.6

func (s *Service) ListSets(instanceID uint) ([]models.DatabaseBackupSet, error)

ListSets returns an instance's recovery points, newest first.

func (*Service) Prune

func (s *Service) Prune(ctx context.Context, databaseID uint, maxBackups, retentionDays int) (int, error)

Prune enforces a retention policy on a database's backups: keep at most maxBackups most-recent, and delete any older than retentionDays. A zero bound is ignored. Returns the number of backups removed.

Pinned backups are never pruned, and do not occupy a maxBackups slot: pinning a backup must not silently shrink how much rolling history the policy keeps.

func (*Service) PruneSets added in v1.9.6

func (s *Service) PruneSets(ctx context.Context, instanceID uint, maxSets, retentionDays int) (int, error)

PruneSets enforces retention over an instance's recovery points: keep at most maxSets most-recent, and delete any older than retentionDays. A zero bound is ignored. Returns the number of sets removed.

Three rules the per-database prune does not have:

  • Whole sets only. Pruning items out of a set would leave the half recovery point the set exists to prevent.
  • A set with any pinned member is never pruned, and does not occupy a maxSets slot, mirroring how a pinned backup behaves.
  • The newest completed set always survives, whatever the policy says. A mistyped retention must not be able to leave an instance with nothing.

func (*Service) RecoveryKit added in v1.9.6

func (s *Service) RecoveryKit(set *models.DatabaseBackupSet) (string, []byte, error)

RecoveryKit renders everything needed to read a recovery point back without Miabi: where the artifacts are, the sealed envelope, the exact envelope format, and the commands that get from a passphrase to a restorable dump.

The kit deliberately does NOT contain the data key — only the envelope it is sealed in. A kit is therefore useless without the passphrase, which is the property that makes it safe to keep beside the backups it describes.

func (*Service) Restore

func (s *Service) Restore(ctx context.Context, inst *models.DatabaseInstance, db *models.Database, spec RestoreSpec) error

Restore restores a logical database from the given source. When spec.Force is set the database is dropped & recreated first (clean slate). It wraps restore so that every one of its early returns still reports an outcome to the instance timeline.

func (*Service) RestoreFromBackup

func (s *Service) RestoreFromBackup(ctx context.Context, inst *models.DatabaseInstance, db *models.Database, b *models.Backup, dest Destination, force, allowVersionMismatch bool) error

RestoreFromBackup restores a logical database from a stored backup record. For an S3 backup, dest.S3 must supply the bucket credentials again.

func (*Service) RestoreSet added in v1.9.6

func (s *Service) RestoreSet(ctx context.Context, inst *models.DatabaseInstance, set *models.DatabaseBackupSet,
	dest Destination, force, allowVersionMismatch bool) (*SetRestoreResult, error)

RestoreSet restores every database in a recovery point, in one operation.

Unlike a backup, this does NOT stop at the first failure. A restore is run when something has already gone wrong, and abandoning the remaining databases because one of them failed would turn a partial recovery into a smaller one. Every outcome is reported and the caller decides.

func (*Service) RestoreUpload

func (s *Service) RestoreUpload(ctx context.Context, inst *models.DatabaseInstance, db *models.Database, origName string, content io.Reader, size int64, force bool) error

RestoreUpload streams an uploaded dump onto the workspace backup volume and restores the database from it (force drops & recreates first). The temp artifact is removed afterward (best-effort).

func (*Service) RewrapSets added in v1.9.6

func (s *Service) RewrapSets(workspaceID uint, oldPassphrase, newPassphrase string) (int, error)

RewrapSets re-seals every sealed recovery point in a workspace from one passphrase to the next. This is the whole reason the artifacts are encrypted with a per-set data key: rotation rewrites a few hundred bytes per set and never re-reads a dump.

It stops at the first envelope that will not open, because that means the old passphrase is wrong and continuing would rewrap the rest under a mismatched secret. Sets already rewrapped stay rewrapped; the operation is resumable.

func (*Service) Run

Run performs a backup of a logical database to the given destination and records the result. inst is the server hosting db.

func (*Service) RunSet added in v1.9.6

RunSet backs up every logical database on an instance as one recovery point. It requires the workspace S3 target and returns ErrS3Required without one.

The set is marked failed if any item fails, and the error names the first failure: a recovery point that is missing a database is not one, and reporting it as a success is how people discover the gap during an incident instead of after the run.

func (*Service) SealedSetCount added in v1.9.6

func (s *Service) SealedSetCount(workspaceID uint) (int, error)

SealedSetCount reports how many recovery points in a workspace are sealed, so a caller can refuse to discard the passphrase that opens them.

func (*Service) SetAlerter added in v1.6.0

func (s *Service) SetAlerter(a BackupAlerter)

SetAlerter wires backup-outcome alerting (optional; nil = no alerts).

func (*Service) SetDDLRunner

func (s *Service) SetDDLRunner(r DDLRunner)

SetDDLRunner wires the database service used to drop+recreate a database on a force restore.

func (*Service) SetEventRecorder added in v1.9.5

func (s *Service) SetEventRecorder(r EventRecorder)

SetEventRecorder wires the instance timeline recorder (nil-safe: unset means no events).

func (*Service) SetImageResolver

func (s *Service) SetImageResolver(r ImageResolver)

SetImageResolver wires the deployment-config resolver for backup tool images.

func (*Service) SetLogStore

func (s *Service) SetLogStore(store *logstore.Store)

SetLogStore wires the shared execution-log store. When set, a backup run's full output is externalized to the store on terminal state and the DB row keeps only a bounded tail + a reference. nil keeps DB-tail-only.

func (*Service) SetReporter added in v1.10.5

func (s *Service) SetReporter(r BackupReporter)

SetReporter wires scheduled-run inbox reporting (optional; nil = no inbox items).

func (*Service) SetSetRepository added in v1.9.6

func (s *Service) SetSetRepository(r *repositories.DatabaseBackupSetRepository)

SetSetRepository wires recovery-point storage (nil-safe: unset means RunSet and the prune below report ErrSetsUnavailable rather than panicking).

func (*Service) VerifySet added in v1.9.6

func (s *Service) VerifySet(ctx context.Context, cfg *S3Config, set *models.DatabaseBackupSet, passphrase string) (*VerifyResult, error)

VerifySet checks a recovery point against the bucket: every artifact is still there, still the size it was, and the envelope still opens.

It deliberately does not read the dumps back. That costs the size of the backup every time, which is how verification becomes the thing people switch off. Proving a dump actually loads is a drill, and a drill needs a live engine.

type SetInfo added in v1.9.6

type SetInfo struct {
	Schema int `json:"schema"`

	Ref       string    `json:"ref"`
	Instance  string    `json:"instance"`
	Engine    string    `json:"engine"`
	Version   string    `json:"version,omitempty"`
	Trigger   string    `json:"trigger,omitempty"`
	CreatedAt time.Time `json:"created_at"`
	SizeBytes int64     `json:"size_bytes"`

	Encrypted bool `json:"encrypted"`
	// Envelope seals this set's data key under the workspace backup passphrase.
	// Ciphertext: useless to a reader who does not hold the passphrase.
	Envelope string `json:"envelope,omitempty"`

	Artifacts []SetInfoArtifact `json:"artifacts"`
}

SetInfo is the CLEARTEXT descriptor written beside a recovery point's artifacts.

Cleartext is the point, copied from platformbackup's discovery reader: you must be able to list what a bucket holds without the passphrase, or a user who has lost their secret cannot even see what they have lost. It carries the shape of the data — never any of it.

The sealed envelope rides along because it is the one piece a restore cannot reconstruct and cannot read without the passphrase anyway. Without it here, a set would be unreadable the moment the platform that took it was gone, which is precisely the case this exists for.

type SetInfoArtifact added in v1.9.6

type SetInfoArtifact struct {
	Database  string `json:"database"`
	Filename  string `json:"filename"`
	SizeBytes int64  `json:"size_bytes,omitempty"`
	Encrypted bool   `json:"encrypted"`
}

SetInfoArtifact is one database's dump within the set.

type SetOptions added in v1.9.6

type SetOptions struct {
	Trigger     string // manual | scheduled
	Comment     string
	Concurrency int
}

SetOptions tunes a set run.

type SetRestoreResult added in v1.9.6

type SetRestoreResult struct {
	Ref      string   `json:"ref"`
	Restored []string `json:"restored"`
	Failed   []string `json:"failed,omitempty"`
}

SetRestoreResult reports what a set-level restore did.

type VerifyResult added in v1.9.6

type VerifyResult struct {
	Ref     string `json:"ref"`
	OK      bool   `json:"ok"`
	Checked int    `json:"checked"`
	// Missing are artifacts the set recorded that are no longer in the bucket —
	// usually a lifecycle rule, occasionally an upload nobody noticed had failed.
	Missing []string `json:"missing,omitempty"`
	// Resized are artifacts whose stored size no longer matches what was recorded,
	// which means the object is not the one this set took.
	Resized []string `json:"resized,omitempty"`
	// EnvelopeOK reports that the workspace passphrase still opens the data key. A
	// set can be perfectly intact in the bucket and unreadable all the same.
	EnvelopeOK bool   `json:"envelope_ok"`
	Error      string `json:"error,omitempty"`
}

VerifyResult is what a check found. It names artifacts rather than only counting them: "one file is missing" is not actionable, "billing's dump is gone" is.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL