dbbackup

package
v0.7.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 4, 2026 License: AGPL-3.0 Imports: 8 Imported by: 0

Documentation

Overview

Package dbbackup holds the database backup job specification, agent-side credential profiles and the run manifest. It is shared by the server (validation, redaction), the agent (execution) and the configuration loader, so it must stay free of process-execution and platform dependencies.

Index

Constants

View Source
const (
	DefaultMinFreeBytes     = int64(1 << 30)
	DefaultSizeSafetyFactor = 1.2
	DefaultKillGrace        = 20 * time.Second
)

Defaults

View Source
const (
	ManifestFileName = "fallbck-db.json"
	ManifestBlobName = "fallbck-db.json"
	// ArchiveName is the pxar archive holding the spool directory
	ArchiveName = "db"
)

ManifestFileName is the manifest inside the archive; the blob is ManifestBlobName + ".blob"

View Source
const (
	EnginePostgres = "postgres"
	EngineMySQL    = "mysql"
	EngineMongoDB  = "mongodb"
)

Supported engines

View Source
const (
	TLSDisable    = "disable"
	TLSPrefer     = "prefer"
	TLSRequire    = "require"
	TLSVerifyCA   = "verify-ca"
	TLSVerifyFull = "verify-full"
)

TLS modes (libpq spelling; mapped per engine by the agent)

View Source
const (
	CompressionNone = "none"
	CompressionGzip = "gzip"
)

Compression settings for the spooled dumps. PBS deduplicates and compresses chunks itself, so the default is uncompressed dumps.

View Source
const (
	PostgresModeDump       = "dump"       // pg_dumpall --globals-only + pg_dump per database
	PostgresModeBasebackup = "basebackup" // pg_basebackup with streamed WAL
	PostgresModeBoth       = "both"
)

PostgreSQL backup modes

View Source
const ManifestSchemaVersion = 1

ManifestSchemaVersion is bumped on incompatible manifest changes

Variables

Engines lists the supported engines in display order

View Source
var ErrInvalidSpec = errors.New("invalid database job configuration")

ErrInvalidSpec wraps every validation failure

Functions

func Bool

func Bool(v *bool, def bool) bool

Bool returns the value of an optional boolean with a default

func DefaultBackupID

func DefaultBackupID(s *Spec) string

DefaultBackupID is the PBS backup group id for a database job: db-<engine>-<host>, keeping database snapshots apart from the host's filesystem backups.

func DefaultPort

func DefaultPort(engine string) int

DefaultPort returns the engine's default TCP port

func NormalizeEngine

func NormalizeEngine(name string) (string, bool)

NormalizeEngine maps aliases (postgresql, mariadb, mongo) to the canonical engine name

Types

type BinlogPosition

type BinlogPosition struct {
	File     string `json:"file"`
	Position int64  `json:"position"`
}

type BinlogPrevious

type BinlogPrevious struct {
	BinlogPosition
	RunID    string `json:"run_id,omitempty"`
	Snapshot string `json:"snapshot,omitempty"`
}

type Config

type Config struct {
	// SpoolDir receives the dumps before upload (default <data_dir>/spool)
	SpoolDir string `yaml:"spool_dir,omitempty" json:"spool_dir,omitempty"`
	// StateDir keeps per-job state such as the last MySQL binlog position (default <data_dir>/state/db)
	StateDir string `yaml:"state_dir,omitempty" json:"state_dir,omitempty"`
	// MinFreeBytes must be free on the spool filesystem before and during a run (default 1 GiB)
	MinFreeBytes int64 `yaml:"min_free_bytes,omitempty" json:"min_free_bytes,omitempty"`
	// SizeSafetyFactor scales the server's size estimate for the disk check (default 1.2)
	SizeSafetyFactor float64 `yaml:"size_safety_factor,omitempty" json:"size_safety_factor,omitempty"`
	// KillGrace is the SIGTERM → SIGKILL grace period for client tools (default 20s)
	KillGrace time.Duration `yaml:"kill_grace,omitempty" json:"kill_grace,omitempty"`
	// KeepFailedSpool leaves the spool directory in place when a run fails (debugging)
	KeepFailedSpool bool `yaml:"keep_failed_spool,omitempty" json:"keep_failed_spool,omitempty"`
	// Checksums records a sha256 per spooled file in the manifest (default true)
	Checksums *bool                        `yaml:"checksums,omitempty" json:"checksums,omitempty"`
	Tools     ToolConfig                   `yaml:"tools,omitempty" json:"tools"`
	Profiles  map[string]CredentialProfile `yaml:"profiles,omitempty" json:"profiles,omitempty"`
}

Config is the agent-side database backup configuration (db_backup: in agent.yaml)

func (*Config) ApplyDefaults

func (c *Config) ApplyDefaults(dataDir string)

ApplyDefaults fills unset values; dataDir anchors the spool and state directories

func (*Config) ChecksumsEnabled

func (c *Config) ChecksumsEnabled() bool

ChecksumsEnabled reports whether per-file checksums are recorded

func (*Config) Validate

func (c *Config) Validate() error

Validate checks the profiles (engine names) and normalises them

type CredentialProfile

type CredentialProfile struct {
	Engine   string `yaml:"engine" json:"engine"`
	User     string `yaml:"user,omitempty" json:"user,omitempty"`
	Password string `yaml:"password,omitempty" json:"-"`
	// PasswordFile is read at run time (trailing newline trimmed)
	PasswordFile string  `yaml:"password_file,omitempty" json:"password_file,omitempty"`
	Host         string  `yaml:"host,omitempty" json:"host,omitempty"`
	Port         int     `yaml:"port,omitempty" json:"port,omitempty"`
	TLS          TLSSpec `yaml:"tls,omitempty" json:"tls"`
}

CredentialProfile is a named set of database credentials kept on the agent host (db_backup.profiles in the agent configuration). A job that leaves its password blank is resolved against the profile it names (default: the engine name).

type Credentials

type Credentials struct {
	User     string
	Password string
	Host     string
	Port     int
	TLS      TLSSpec
	// Source records where the password came from ("job", "profile:<name>", "none")
	Source string
}

Credentials are the resolved connection settings for one run

func ResolveCredentials

func ResolveCredentials(spec *Spec, password string, profiles map[string]CredentialProfile) (Credentials, error)

ResolveCredentials merges the job spec with the agent profile. password is the server-stored password for this run ("" when the job has none). The profile is consulted only when the job carries no password; it may also fill in user, host, port and TLS settings the job left blank.

type DatabaseEntry

type DatabaseEntry struct {
	Name       string `json:"name"`
	File       string `json:"file,omitempty"`
	Format     string `json:"format,omitempty"`
	SizeBytes  int64  `json:"size_bytes"`
	SHA256     string `json:"sha256,omitempty"`
	DurationMS int64  `json:"duration_ms"`
}

type FileEntry

type FileEntry struct {
	Path      string `json:"path"`
	SizeBytes int64  `json:"size_bytes"`
	SHA256    string `json:"sha256,omitempty"`
	Kind      string `json:"kind,omitempty"`
}

type JobState

type JobState struct {
	JobID       string          `json:"job_id"`
	Engine      string          `json:"engine"`
	LastSuccess *JobStateRecord `json:"last_success,omitempty"`
}

JobState is persisted per job on the agent after every successful run

type JobStateRecord

type JobStateRecord struct {
	RunID       string         `json:"run_id"`
	Snapshot    string         `json:"snapshot"`
	CompletedAt time.Time      `json:"completed_at"`
	MySQL       *MySQLState    `json:"mysql,omitempty"`
	Postgres    *PostgresState `json:"postgres,omitempty"`
}

type Manifest

type Manifest struct {
	SchemaVersion int               `json:"schema_version"`
	Generator     ManifestGenerator `json:"generator"`
	Job           ManifestJob       `json:"job"`
	Snapshot      ManifestSnapshot  `json:"snapshot"`
	Engine        string            `json:"engine"`
	StartedAt     time.Time         `json:"started_at"`
	FinishedAt    time.Time         `json:"finished_at"`
	// ConsistentAsOf is the point in time the backup represents
	ConsistentAsOf time.Time         `json:"consistent_as_of"`
	Server         ManifestServer    `json:"server"`
	Tools          map[string]Tool   `json:"tools"`
	Spec           *Spec             `json:"spec"`
	Databases      []DatabaseEntry   `json:"databases"`
	Files          []FileEntry       `json:"files"`
	Warnings       []string          `json:"warnings,omitempty"`
	Notes          []string          `json:"notes,omitempty"`
	Postgres       *PostgresManifest `json:"postgres,omitempty"`
	MySQL          *MySQLManifest    `json:"mysql,omitempty"`
	MongoDB        *MongoManifest    `json:"mongodb,omitempty"`
	Restore        Restore           `json:"restore"`
}

Manifest describes one database backup run; it is written into the archive root and uploaded as a separate blob so restores can start from it.

func ParseManifest

func ParseManifest(data []byte) (*Manifest, error)

ParseManifest decodes a manifest

func (*Manifest) Marshal

func (m *Manifest) Marshal() ([]byte, error)

Marshal renders the manifest as indented JSON

type ManifestGenerator

type ManifestGenerator struct {
	Agent    string `json:"agent"`
	Version  string `json:"version"`
	Hostname string `json:"hostname"`
}

type ManifestJob

type ManifestJob struct {
	ID          string `json:"id"`
	Name        string `json:"name,omitempty"`
	RunID       string `json:"run_id,omitempty"`
	BackupRunID string `json:"backup_run_id"`
}

type ManifestServer

type ManifestServer struct {
	Host       string `json:"host"`
	Port       int    `json:"port"`
	User       string `json:"user,omitempty"`
	Version    string `json:"version,omitempty"`
	VersionNum int    `json:"version_num,omitempty"`
	Flavor     string `json:"flavor,omitempty"`
	IsReplica  bool   `json:"is_replica,omitempty"`
}

type ManifestSnapshot

type ManifestSnapshot struct {
	Type    string    `json:"type"`
	ID      string    `json:"id"`
	Time    time.Time `json:"time"`
	Archive string    `json:"archive"`
}

type MongoManifest

type MongoManifest struct {
	Dir            string   `json:"dir"`
	Oplog          bool     `json:"oplog"`
	OplogFile      string   `json:"oplog_file,omitempty"`
	Gzip           bool     `json:"gzip"`
	ReadPreference string   `json:"read_preference,omitempty"`
	Databases      []string `json:"databases,omitempty"`
	PerDatabase    bool     `json:"per_database"`
}

type MongoOptions

type MongoOptions struct {
	// Oplog captures the oplog during the dump (point-in-time, requires a replica set and
	// dumping all databases); default true when no database list is given
	Oplog                  *bool    `json:"oplog,omitempty"`
	ReadPreference         string   `json:"read_preference,omitempty"`
	AuthSource             string   `json:"auth_source,omitempty"` // default admin
	ReplicaSet             string   `json:"replica_set,omitempty"`
	NumParallelCollections int      `json:"num_parallel_collections,omitempty"`
	ExtraArgs              []string `json:"extra_args,omitempty"`
}

MongoOptions are the MongoDB-specific job settings

type MySQLBinlog

type MySQLBinlog struct {
	Enabled        bool            `json:"enabled"`
	PositionSource string          `json:"position_source,omitempty"`
	File           string          `json:"file,omitempty"`
	Position       int64           `json:"position,omitempty"`
	GTID           string          `json:"gtid,omitempty"`
	GTIDMode       string          `json:"gtid_mode,omitempty"`
	ServerID       int64           `json:"server_id,omitempty"`
	ServerUUID     string          `json:"server_uuid,omitempty"`
	Previous       *BinlogPrevious `json:"previous,omitempty"`
	Files          []FileEntry     `json:"files,omitempty"`
	CoversFrom     *BinlogPosition `json:"covers_from,omitempty"`
	CoversTo       *BinlogPosition `json:"covers_to,omitempty"`
	ChainReset     bool            `json:"chain_reset"`
	ChainGap       bool            `json:"chain_gap"`
}

type MySQLManifest

type MySQLManifest struct {
	Flavor                 string       `json:"flavor"`
	DumpFiles              []string     `json:"dump_files"`
	AllDatabases           bool         `json:"all_databases"`
	SingleTransaction      bool         `json:"single_transaction"`
	FlushLogs              bool         `json:"flush_logs"`
	NonTransactionalTables int          `json:"non_transactional_tables"`
	Binlog                 *MySQLBinlog `json:"binlog,omitempty"`
}

type MySQLOptions

type MySQLOptions struct {
	SingleTransaction *bool `json:"single_transaction,omitempty"` // default true
	// IncludeBinlogs fetches the binary logs written since the previous successful run
	IncludeBinlogs bool  `json:"include_binlogs,omitempty"`
	FlushLogs      *bool `json:"flush_logs,omitempty"` // default = IncludeBinlogs
	// IncludeSystemSchemas also dumps the mysql schema when an explicit list is given
	IncludeSystemSchemas bool     `json:"include_system_schemas,omitempty"`
	Routines             *bool    `json:"routines,omitempty"` // default true
	Triggers             *bool    `json:"triggers,omitempty"` // default true
	Events               *bool    `json:"events,omitempty"`   // default true
	ExtraDumpArgs        []string `json:"extra_dump_args,omitempty"`
}

MySQLOptions are the MySQL/MariaDB-specific job settings

type MySQLState

type MySQLState struct {
	BinlogFile string `json:"binlog_file"`
	BinlogPos  int64  `json:"binlog_pos"`
	GTID       string `json:"gtid,omitempty"`
	ServerID   int64  `json:"server_id,omitempty"`
	ServerUUID string `json:"server_uuid,omitempty"`
}

type PostgresBasebackup

type PostgresBasebackup struct {
	Dir          string   `json:"dir"`
	Files        []string `json:"files"`
	Label        string   `json:"label"`
	Timeline     int64    `json:"timeline,omitempty"`
	WALStartLSN  string   `json:"wal_start_lsn,omitempty"`
	WALEndLSN    string   `json:"wal_end_lsn,omitempty"`
	WALStartFile string   `json:"wal_start_file,omitempty"`
	WALEndFile   string   `json:"wal_end_file,omitempty"`
	WALSource    string   `json:"wal_source,omitempty"`
	Compression  string   `json:"compression"`
}

type PostgresGlobals

type PostgresGlobals struct {
	File                  string `json:"file"`
	RolePasswordsIncluded bool   `json:"role_passwords_included"`
}

type PostgresManifest

type PostgresManifest struct {
	Mode       string              `json:"mode"`
	Globals    *PostgresGlobals    `json:"globals,omitempty"`
	Basebackup *PostgresBasebackup `json:"basebackup,omitempty"`
	// WAL position of the server when the logical dumps were taken
	WALLSN string `json:"wal_lsn,omitempty"`
}

type PostgresOptions

type PostgresOptions struct {
	Mode string `json:"mode,omitempty"`
	// Format is "custom" (default, pg_restore) or "plain" (psql)
	Format string `json:"format,omitempty"`
	// Globals dumps roles and tablespaces with pg_dumpall --globals-only (default true)
	Globals *bool `json:"globals,omitempty"`
	// RolePasswords: "auto" (include when superuser), "include", "omit"
	RolePasswords string   `json:"role_passwords,omitempty"`
	ExtraDumpArgs []string `json:"extra_dump_args,omitempty"`
}

PostgresOptions are the PostgreSQL-specific job settings

type PostgresState

type PostgresState struct {
	WALEndLSN string `json:"wal_end_lsn,omitempty"`
	Timeline  int64  `json:"timeline,omitempty"`
}

type Restore

type Restore struct {
	Summary  string   `json:"summary"`
	Commands []string `json:"commands"`
}

Restore carries a human-readable summary and ready-to-run commands

type Spec

type Spec struct {
	Engine string `json:"engine"`
	// Host is a hostname/IP, or a path (leading "/") for a local unix socket
	Host string `json:"host"`
	Port int    `json:"port,omitempty"`
	User string `json:"user,omitempty"`
	// Password is stored server-side; blank means the agent resolves it from Profile
	Password string `json:"password,omitempty"`
	// HasPassword is set on API responses instead of the password itself
	HasPassword bool `json:"has_password,omitempty"`
	// ClearPassword (request only) removes a stored password on update
	ClearPassword bool `json:"clear_password,omitempty"`
	// Profile names the agent-side credential profile (default: the engine name)
	Profile          string   `json:"profile,omitempty"`
	TLS              TLSSpec  `json:"tls"`
	Databases        []string `json:"databases,omitempty"`
	ExcludeDatabases []string `json:"exclude_databases,omitempty"`
	Compression      string   `json:"compression,omitempty"`
	// BackupID overrides the PBS backup group id (default db-<engine>-<host>)
	BackupID string `json:"backup_id,omitempty"`

	Postgres *PostgresOptions `json:"postgres,omitempty"`
	MySQL    *MySQLOptions    `json:"mysql,omitempty"`
	MongoDB  *MongoOptions    `json:"mongodb,omitempty"`
}

Spec is the database job configuration stored server-side (backup_jobs.db_config) and handed to the agent (with the password redacted) for every run.

func (*Spec) Clone

func (s *Spec) Clone() *Spec

Clone returns a deep copy of the spec

func (*Spec) EffectivePort

func (s *Spec) EffectivePort() int

EffectivePort returns the configured port or the engine default

func (*Spec) IsSocket

func (s *Spec) IsSocket() bool

IsSocket reports whether Host is a local unix socket path

func (*Spec) Redacted

func (s *Spec) Redacted() *Spec

Redacted returns a copy without the password (HasPassword tells whether one is stored)

func (*Spec) Summary

func (s *Spec) Summary() string

Summary is a short human-readable description (UI tables, logs)

func (*Spec) Validate

func (s *Spec) Validate() error

Validate normalises the engine name, applies defaults where they are unambiguous and rejects inconsistent settings. It is used by the server (job create/update) and by the agent before a run.

type TLSSpec

type TLSSpec struct {
	Mode     string `json:"mode,omitempty" yaml:"mode,omitempty"`
	CAFile   string `json:"ca_file,omitempty" yaml:"ca_file,omitempty"`
	CertFile string `json:"cert_file,omitempty" yaml:"cert_file,omitempty"`
	KeyFile  string `json:"key_file,omitempty" yaml:"key_file,omitempty"`
}

TLSSpec describes how the agent connects to the database server. File paths refer to the agent host.

type Tool

type Tool struct {
	Path    string `json:"path"`
	Version string `json:"version"`
}

Tool is a detected client binary

type ToolConfig

type ToolConfig struct {
	PostgresBinDir string `yaml:"postgres_bindir,omitempty" json:"postgres_bindir,omitempty"`
	MySQLBinDir    string `yaml:"mysql_bindir,omitempty" json:"mysql_bindir,omitempty"`
	MongoBinDir    string `yaml:"mongo_bindir,omitempty" json:"mongo_bindir,omitempty"`
}

ToolConfig pins the client tool directories (default: PATH and well-known locations)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL