Help us shape the future of Go! Take the 2026 Go Developer Survey

backup

package
v0.3.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 3, 2026 License: GPL-3.0 Imports: 36 Imported by: 0

Documentation

Overview

Package backup creates integrity-checked, portable snapshots without copying live WAL files.

On SQLite a snapshot holds a VACUUM INTO copy (ether.db). On PostgreSQL, where file copies do not apply, it holds a logical dump of every table (database.jsonl, see store.ExportLogical) and the manifest records dialect "postgres"; a restored dump is imported into an empty database on the next start (ImportPending).

Index

Constants

View Source
const (
	// PendingFile records a requested restore in the data directory.
	PendingFile = "restore-pending.json"
	// ReportFile is the last restore's report, kept until dismissed.
	ReportFile = "restore-report.json"
	// RollbackDir holds the files each restore replaced, one folder per
	// restore (rollbacks/restore-20261031T101500Z).
	RollbackDir = "rollbacks"

	// StagedPrefix starts every staging folder's name.
	StagedPrefix = "restored-"
)

A guided restore runs in three steps. The running server validates a bundle into a staging folder in the data directory and records a pending restore (RequestRestore), then restarts. The next start applies it (ApplyPending) before anything opens the database or the settings: the current files move into a timestamped rollback folder and the staged ones take their place. The restore only counts once the restored server has opened its database and is about to serve (ConfirmRestore); a start that fails before then puts the original files back (RollbackApplied, or ApplyPending at the next start), and a report tells the owner what happened either way.

Every file move is journalled in the pending marker before and after it happens, and the marker, the files and their directories are fsynced, so a crash or power loss at any point leaves enough on disk to put exactly the original files back.

View Source
const EncryptedExtension = ".etherbackup"

EncryptedExtension names exported bundles.

View Source
const LogicalDumpFile = "database.jsonl"

LogicalDumpFile is the PostgreSQL bundle's database entry. Left in a data directory by a restore, it is imported into the (empty) database at start.

Variables

View Source
var (
	// ErrPasswordPolicy rejects an export password that is too short.
	ErrPasswordPolicy = errors.New("backup password must be at least 12 characters")
	// ErrBadPassword is a wrong password or a damaged first chunk; the
	// two cannot be told apart.
	ErrBadPassword = errors.New("wrong password, or the backup is damaged")
	// ErrDamaged is an encrypted backup that fails authentication after
	// its first chunk, or is truncated or extended.
	ErrDamaged = errors.New("encrypted backup is damaged or incomplete")
	// ErrNotEncrypted is a file without the encrypted backup header.
	ErrNotEncrypted = errors.New("not an encrypted Ether backup")
)
View Source
var ErrNoStaged = errors.New("the prepared backup is no longer available; choose it again")

ErrNoStaged means the staging folder named in a request does not exist.

View Source
var ErrRestoreInProgress = errors.New("a restore is already being applied; wait for Ether to start")

ErrRestoreInProgress means a restore has started (or its record is damaged): nothing else may be requested until the next start has finished or undone it.

Functions

func CancelPending added in v0.3.0

func CancelPending(dataDir string) error

CancelPending forgets a requested restore that has not started. A restore being applied (or a damaged record) is left alone.

func CleanStaged added in v0.3.0

func CleanStaged(dataDir string, age time.Duration)

CleanStaged removes staging folders older than age that no restore is waiting for: each holds a whole database.

func ConfirmRestore added in v0.3.0

func ConfirmRestore(dataDir string, rep *Report) error

ConfirmRestore records that the server started on the applied restore. Removing the restore record is the commit point and comes first: if it fails, the restore is not confirmed (the caller must not serve, and rolls back). Then the "restored" report is saved (with the integrity check and import result filled in by the caller) and the rollback folder is kept for undo.

func DiscardStaged added in v0.3.0

func DiscardStaged(dataDir, id string) error

DiscardStaged removes a staging folder the owner decided not to restore.

func DismissReport added in v0.3.0

func DismissReport(dataDir string) error

DismissReport removes the report; the rollback folder stays.

func FilesystemWarning

func FilesystemWarning(ctx context.Context, dataDir string) string

FilesystemWarning detects the mount containing dataDir on Linux and macOS.

func ImportPending

func ImportPending(ctx context.Context, db *sql.DB, dataDir string) (bool, error)

ImportPending imports a restored logical dump (LogicalDumpFile in dataDir) into db, which must be freshly migrated and empty, then renames the dump so it is imported only once. It reports whether it imported.

func IsEncrypted added in v0.2.4

func IsEncrypted(b []byte) bool

IsEncrypted reports whether b starts with the encrypted backup magic.

func NewDecryptReader added in v0.2.4

func NewDecryptReader(r io.Reader, password string) (io.Reader, error)

NewDecryptReader checks the header of an encrypted bundle, derives the key from password and authenticates the first chunk, so a wrong password fails here (ErrBadPassword) before anything is read from the bundle.

func NewEncryptWriter added in v0.2.4

func NewEncryptWriter(w io.Writer, password string) (io.WriteCloser, error)

NewEncryptWriter returns a writer that encrypts a bundle to w under password. Close must be called to write the final chunk; it does not close w.

func PrepareStaged added in v0.3.0

func PrepareStaged(dataDir, id string, current config.Server) error

PrepareStaged makes a staged bundle's settings fit this server before it is restored here: the listening port and address stay this server's (so the console reconnects after the restart), and the database setting is cleared, because a guided restore only replaces a standalone server's own database (a cluster's dump is imported into it).

func RequestRestore added in v0.3.0

func RequestRestore(dataDir string, p Pending) error

RequestRestore records that the staging folder staged (a name under dataDir) is to replace the current data at the next start. It refuses while a restore is being applied: its record must stay until that restore is finished or undone.

func RequestStaged added in v0.3.0

func RequestStaged(dataDir, id string) error

RequestStaged records a pending restore of a staging folder.

func RequestUndo added in v0.3.0

func RequestUndo(dataDir string) error

RequestUndo asks the next start to put back the files the last restore replaced (its rollback folder), as a restore of their own: the restored data moves into a new rollback folder, so an undo can itself be undone.

func RestoreBundle

func RestoreBundle(ctx context.Context, r io.Reader, destination string) error

RestoreBundle validates all content before atomically installing into a new directory. It deliberately refuses existing destinations and never replaces a running database.

func RestoreEncrypted added in v0.2.4

func RestoreEncrypted(ctx context.Context, r io.Reader, password, destination string) error

RestoreEncrypted is RestoreBundle for a password-encrypted bundle (NewEncryptWriter). Nothing is installed unless the whole file authenticates, including its final chunk and the absence of anything after it.

func UpdateReport added in v0.3.0

func UpdateReport(dataDir string, fn func(*Report)) error

UpdateReport changes the saved report, if there is one.

func WriteJSONDurable added in v0.3.0

func WriteJSONDurable(path string, v any) error

WriteJSONDurable writes v as JSON to path atomically and durably, owner-only (the same write the restore journal uses).

Types

type Manifest

type Manifest struct {
	Version int               `json:"version"`
	Created time.Time         `json:"created"`
	SHA256  map[string]string `json:"sha256"`
	// Dialect is "postgres" for a logical dump; empty means SQLite.
	Dialect string `json:"dialect,omitempty"`
}

type Node

type Node struct {
	ID, Name string
}

Node identifies the cluster node a Service runs on.

type Pending added in v0.3.0

type Pending struct {
	// Staged is the staging folder, relative to the data directory.
	Staged string `json:"staged"`
	// Kind is "snapshot" (a bundle kept on this server), "upload" (a
	// downloaded bundle) or "undo" (a previous restore's rollback).
	Kind string `json:"kind"`
	// Source names what is restored: a bundle or file name.
	Source        string    `json:"source"`
	BackupCreated time.Time `json:"backupCreated,omitzero"`
	Requested     time.Time `json:"requested"`
	// Rollback is set once the swap has started: the folder (under
	// rollbacks/) the current files move to.
	Rollback string `json:"rollback,omitempty"`
	// Moved are the original files now in the rollback folder, Installed
	// the staged files now in the data directory, and Step the move in
	// flight ("aside:", "install:", "return:" or "restore:" and a file
	// name), all written before and after each move.
	Moved     []string `json:"moved,omitempty"`
	Installed []string `json:"installed,omitempty"`
	Step      string   `json:"step,omitempty"`
	// Applied is set once every file is in place: the restored server is
	// starting and the restore is confirmed when it is ready.
	Applied bool `json:"applied,omitempty"`
	// RollingBack replaces Applied once an applied restore's undo has
	// swept the restored server's files into Installed: from then on only
	// the journal says what to move, so a retried undo never mistakes an
	// original already put back for a file the restored server made.
	RollingBack bool `json:"rollingBack,omitempty"`
	// Failure is why an applied restore is being rolled back.
	Failure string `json:"failure,omitempty"`
}

Pending is a restore waiting for (or in the middle of) a start.

func ReadPending added in v0.3.0

func ReadPending(dataDir string) (*Pending, error)

ReadPending returns the requested restore, or nil.

type Report added in v0.3.0

type Report struct {
	// Status is "restored", or "failed" when the original files are back
	// (the restore could not be applied, or the restored server could not
	// start: RolledBack).
	Status        string    `json:"status"`
	Kind          string    `json:"kind"`
	Source        string    `json:"source"`
	BackupCreated time.Time `json:"backupCreated,omitzero"`
	RestoredAt    time.Time `json:"restoredAt"`
	// RolledBack is set when the backup was in place but the server could
	// not start on it, so the original data was put back automatically.
	RolledBack bool `json:"rolledBack,omitempty"`
	// Rollback is the folder (under rollbacks/) holding the files this
	// restore replaced; Undo puts them back.
	Rollback     string `json:"rollback,omitempty"`
	RollbackPath string `json:"rollbackPath,omitempty"`
	// Files are the data-directory files that were replaced.
	Files []string `json:"files,omitempty"`
	// Logical is set for a PostgreSQL-style dump imported at start;
	// Imported once the import finished.
	Logical  bool `json:"logical,omitempty"`
	Imported bool `json:"imported,omitempty"`
	// Integrity is the restored database's check.
	Integrity *store.Integrity `json:"integrity,omitempty"`
	Error     string           `json:"error,omitempty"`
}

Report is the outcome of the last restore, shown in the console until the owner dismisses it.

func ApplyPending added in v0.3.0

func ApplyPending(dataDir string, now func() time.Time) (*Report, error)

ApplyPending applies a requested restore, or finishes undoing one a previous start left behind. It must run before anything opens the settings or the database. It returns nil when nothing was pending, a report with Status "applied" when the backup is now in place (call ConfirmRestore once the server is ready, or RollbackApplied if it cannot start), and otherwise the "failed" report it also saved, with every original file back in place. The error is reserved for the cases where the originals cannot be put back or the restore record is damaged; the server must then not start.

func ReadReport added in v0.3.0

func ReadReport(dataDir string) (*Report, error)

ReadReport returns the last restore's report, or nil.

func RollbackApplied added in v0.3.0

func RollbackApplied(dataDir string, now func() time.Time, reason string) (*Report, error)

RollbackApplied puts the original files back after the server could not start on an applied restore (reason says why), and saves the report. The settings and database must be closed.

type Rollback added in v0.3.0

type Rollback struct {
	Name       string    `json:"name"`
	Created    time.Time `json:"created"`
	ReplacedBy string    `json:"replacedBy,omitempty"`
	Files      []string  `json:"files,omitempty"`
}

Rollback is a folder of files a restore replaced.

type Service

type Service struct {
	// contains filtered or unexported fields
}

func New

func New(db *sql.DB, configPath, dir string, retain int) *Service

func (*Service) Configure

func (s *Service) Configure(ctx context.Context, v Settings) error

func (*Service) Create

func (s *Service) Create(ctx context.Context) (Snapshot, error)

func (*Service) Handler

func (s *Service) Handler() http.Handler

Handler is the owner's backup API, mounted behind the admin API's requireAuth (which also enforces the CSRF header):

GET  /               bundles, schedule and a filesystem warning
POST /settings       change the schedule              owner, setup complete
POST /               create a bundle now              owner, setup complete
POST /export/{name}  {"password"} → .etherbackup      owner, setup complete
GET  /restore              guided restore: report, pending, one-click?
POST /restore              stage an uploaded bundle (below)
POST /restore/snapshot/{n} stage a bundle kept here     owner, setup complete
POST /restore/apply/{id}   restore a staged bundle and restart
POST /restore/undo         put back what the last restore replaced
DELETE /restore/staged/{id}, DELETE /restore/report

A bundle holds config.json, the database and the credential key that unseals them, so it leaves the server only encrypted with a password the owner chooses (encrypted.go). Bundles kept in the data directory stay owner-only (0600) files.

Restore takes the raw file (application/octet-stream). An encrypted bundle needs its password in X-Ether-Backup-Password (base64 of the UTF-8 password) and is accepted wherever the admin API is: from the owner, or before setup completes from the local network (a fresh server being restored from backup). An unencrypted bundle from before encryption needs the signed-in owner and X-Ether-Backup-Legacy: plaintext. Either way the bundle is only validated and staged in a new directory. On a standalone server apply then records a pending restore and restarts: the next start swaps the staged files in (pending.go), keeping the replaced ones for undo. A cluster node only stages: its shared database is restored through a guided procedure.

func (*Service) List

func (s *Service) List() ([]Snapshot, error)

List is this node's own bundles (its data directory's backups).

func (*Service) OneClick added in v0.3.0

func (s *Service) OneClick() bool

OneClick reports whether this server can restore in place: only a standalone server, whose database is a file in its own data directory. A cluster node shares a PostgreSQL database with the other nodes, so replacing it is a guided, cluster-wide procedure.

func (*Service) Run

func (s *Service) Run(ctx context.Context, onResult func(error))

Run takes scheduled backups until ctx ends. onResult hears every scheduled attempt: its error, or nil once one succeeds. A failing backup is retried with backoff (15 minutes, doubling, at most the interval), not every minute.

func (*Service) SetCluster

func (s *Service) SetCluster(n Node)

SetCluster marks the server as clustered: every bundle is recorded in the shared database with the node that holds it (so any node lists the whole cluster's backups), and the shared backup directory setting is honoured. A standalone server never calls it and keeps its bundles in its data directory only.

func (*Service) SetRestart added in v0.3.0

func (s *Service) SetRestart(fn func(reason string, whenIdle bool) error)

SetRestart wires the server restart a guided restore needs.

func (*Service) Settings

func (s *Service) Settings(ctx context.Context) (Settings, error)

func (*Service) Snapshots

func (s *Service) Snapshots(ctx context.Context) ([]Snapshot, error)

Snapshots lists backups for the owner: this node's bundles on a standalone server, every node's (with where each is) on a clustered one.

func (*Service) StageSnapshot added in v0.3.0

func (s *Service) StageSnapshot(ctx context.Context, name string) (Staged, error)

StageSnapshot validates a bundle kept on this server into a staging folder.

type Settings

type Settings struct {
	IntervalHours int   `json:"intervalHours"`
	Retain        int   `json:"retain"`
	LastRun       int64 `json:"lastRun"`
	// SharedDir (clustered servers) is a directory every node can reach,
	// at the same path, where scheduled and manual backups are written
	// instead of the writing node's data directory. Empty keeps them on
	// the node that made them.
	SharedDir string `json:"sharedDir,omitempty"`
}

type Snapshot

type Snapshot struct {
	Name    string    `json:"name"`
	Size    int64     `json:"size"`
	Created time.Time `json:"created"`
	// Clustered servers only: the node that wrote the bundle and holds it
	// (unless Shared), whether it is in the shared backup directory, and
	// Remote when it is on another node's disk (download it there).
	Node   string `json:"node,omitempty"`
	NodeID string `json:"nodeId,omitempty"`
	Shared bool   `json:"shared,omitempty"`
	Remote bool   `json:"remote,omitempty"`
}

func LocalSnapshots added in v0.3.0

func LocalSnapshots(dataDir string) ([]Snapshot, error)

LocalSnapshots lists the bundles kept in a standalone server's data directory, read without its database (a start that failed).

type Staged added in v0.3.0

type Staged struct {
	// ID is the staging folder's name in the data directory.
	ID     string `json:"id"`
	Kind   string `json:"kind"`
	Source string `json:"source"`
	// Created is when the backup was taken.
	Created time.Time `json:"created"`
	// Logical is set for a PostgreSQL-style dump: it is imported into
	// this server's own database when it starts.
	Logical bool  `json:"logical,omitempty"`
	Size    int64 `json:"size"`
	// Path is the staging folder, for the guided restore on a cluster.
	Path string `json:"path"`
}

Staged is a validated bundle waiting in a staging folder.

func StageLocal added in v0.3.0

func StageLocal(ctx context.Context, dataDir, name string) (Staged, error)

StageLocal validates a bundle kept in a standalone server's data directory (LocalSnapshots) into a staging folder.

func StageUpload added in v0.3.0

func StageUpload(ctx context.Context, dataDir string, r io.Reader, password, source string) (Staged, error)

StageUpload validates an uploaded bundle (encrypted with password, or a legacy plaintext bundle when password is empty) into a new staging folder in dataDir.

func StagedInfo added in v0.3.0

func StagedInfo(dataDir, id string) (Staged, error)

StagedInfo returns the summary of a staging folder.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL