mysql

package
v1.63.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 13, 2026 License: Apache-2.0 Imports: 31 Imported by: 0

Documentation

Overview

Package mysql backs up MySQL servers into goback sets: logical dumps through mysqldump into one set, and the binary logs, received as a replica, into another for point-in-time recovery.

Index

Constants

View Source
const (
	// MetaFlavor is always mysql for now.
	MetaFlavor  = "mysql.flavor"
	MetaVersion = "mysql.version"
	// MetaServerID and MetaServerUUID name the server; MySQL 5.5 has no
	// server_uuid.
	MetaServerID   = "mysql.server_id"
	MetaServerUUID = "mysql.server_uuid"
	// MetaBinlogFile and MetaBinlogPos are the binary log position a dump
	// is consistent with.
	MetaBinlogFile = "mysql.binlog_file"
	MetaBinlogPos  = "mysql.binlog_pos"
	// MetaLock is snapshot or all-tables: how the dump was kept consistent.
	MetaLock = "mysql.lock"
	// MetaGTIDMode is ON for a dump of a server with GTIDs on.
	MetaGTIDMode = "mysql.gtid_mode"
	// MetaChain numbers the binary log chain a dump or binlog commit
	// belongs to, in decimal; the numbering starts over after RESET MASTER.
	MetaChain = "mysql.chain"
	// MetaFirstBinlog and MetaLastBinlog bound the files a binlog commit
	// holds of its chain.
	MetaFirstBinlog  = "mysql.first_binlog"
	MetaLastBinlog   = "mysql.last_binlog"
	MetaBinlogFormat = "mysql.binlog_format"
	// MetaEndBinlog and MetaEndPos are where the binlogs a binlog commit
	// holds of its chain end, its partial binlog included.
	MetaEndBinlog = "mysql.end_binlog"
	MetaEndPos    = "mysql.end_pos"
	// MetaCapture is how a binlog commit's binlogs were collected:
	// CaptureReplication, or files for binlogs copied from the server's
	// binlog directory, which restore reads alike.
	MetaCapture = "mysql.capture"
	// MetaDumpSet and MetaWindow are, on a binlog commit, the set its dumps
	// go to and how far back point-in-time recovery reaches, as a Go
	// duration.
	MetaDumpSet = "mysql.dump_set"
	MetaWindow  = "mysql.window"
)

The commit metadata the MySQL sets carry.

View Source
const (
	// DefaultCommitEvery is how often the receiver commits what is new.
	DefaultCommitEvery = time.Minute
	// DefaultCommitBytes is how much new binlog makes it commit early.
	DefaultCommitBytes = 256 << 20
	// MinCommitEvery is the shortest interval it takes, and the least time
	// between two commits.
	MinCommitEvery = 10 * time.Second
)

The receiver's commit cadence.

View Source
const CaptureReplication = "replication"

CaptureReplication is the MetaCapture of binlogs received as a replica.

View Source
const DumpFile = "dump.sql"

DumpFile is the name of the one file a dump commit holds.

Variables

View Source
var (
	// ErrForeignServer is returned for a dump of another server than the
	// dump set's.
	ErrForeignServer = errors.New("dump of another server")
	// ErrForeignBinlog is returned for binlogs of another server than the
	// binlog set's.
	ErrForeignBinlog = errors.New("binlog of another server")
	// ErrGap is returned when binlogs a dump needs are missing.
	ErrGap = errors.New("binlogs are missing")
	// ErrBinlogChanged is returned when the server's copy of a binlog no
	// longer matches what was received of it.
	ErrBinlogChanged = errors.New("a received binlog changed on the server")
	// ErrNoBinlog is returned when the server's binary log is off.
	ErrNoBinlog = errors.New("the server's binary log is off")
	// ErrMariaDB is returned for a MariaDB server, which is not supported yet.
	ErrMariaDB = errors.New("MariaDB is not supported yet")
)
View Source
var ErrNoDump = errors.New("no dump to restore from")

ErrNoDump is returned when no dump can be restored to the moment asked for.

View Source
var ErrServerIDInUse = errors.New("another replica uses the receiver's server id")

ErrServerIDInUse is returned when another replica connects with the receiver's server id, so the server ends one dump after the other.

Functions

func Check

func Check(ctx context.Context, defaultsExtraFile string, serverID uint32) []string

Check connects as the option files say, defaultsExtraFile first, and lists what the server lacks for receiving its binlogs as replica serverID; nil when it lacks nothing.

func CheckDump

func CheckDump(ctx context.Context, defaultsExtraFile string, position bool) []string

CheckDump connects as the option files say, defaultsExtraFile first, and lists what the server lacks for dumps, which record their binlog position when position is set and log_bin is on; nil when it lacks nothing.

func DumpArgs

func DumpArgs(version [3]int, lockAll, position, gtid bool) []string

DumpArgs are the arguments mysqldump runs with: version is mysqldump's own, lockAll replaces the snapshot with a global read lock for the whole dump, and position records the binary log position.

func ReplayArgs

func ReplayArgs(meta map[string]string, at time.Time, files []string) []string

ReplayArgs are mysqlbinlog's arguments for replaying files after the dump whose metadata is meta, up to at when it is set.

func ReplicaServerID

func ReplicaServerID(set string) uint32

ReplicaServerID is the server id a binlog set's receiver connects with unless told otherwise: a hash of the set's name in 2^31 to 2^32-1, above the small ids servers are usually given.

func Window

func Window(dumps []*proto.Commit, window time.Duration, now time.Time) (map[int]string, bool)

Window decides which binlogs a binlog set keeps from its dumps, newest first: in each chain, those from the start file of the chain's oldest dump received within window on. The newest dump counts as received within window. Chains no such dump belongs to are let go. ok is false when no dump records a position, and then every binlog is kept.

Types

type Binlog

type Binlog struct {
	Name string
	Size int64
}

Binlog is one binary log the server lists.

func BinaryLogs

func BinaryLogs(ctx context.Context, c Querier) ([]Binlog, error)

BinaryLogs lists the server's binary logs, oldest first; the last one is the one the server writes to.

type Client

type Client interface {
	Querier
	// DumpVersion is the output of mysqldump --version.
	DumpVersion(ctx context.Context) (string, error)
	// Dump starts mysqldump with args. Close on the returned reader waits
	// for mysqldump, stopping it first unless its output was read to the
	// end, and reports how it ended.
	Dump(ctx context.Context, args []string) (io.ReadCloser, error)
	// Load runs the SQL r holds through mysql.
	Load(ctx context.Context, r io.Reader) error
	// BinlogVersion is the output of mysqlbinlog --version; it fails when
	// there is no mysqlbinlog.
	BinlogVersion(ctx context.Context) (string, error)
	// Replay runs mysqlbinlog with args, under TZ=UTC, into mysql.
	Replay(ctx context.Context, args []string) error
}

Client runs the MySQL client programs against one server. Exec runs the installed ones.

type Dump

type Dump struct {
	// Walker stores the commit; its Set is the dump set. Run sets its
	// Stream and Metadata.
	Walker *backup.Walker
	Client Client
	// NoPosition leaves the binary log position out even when log_bin is
	// on, which spares the dump the global read lock and the RELOAD and
	// REPLICATION CLIENT privileges.
	NoPosition bool
	// BinlogSet, when set, is the set the server's binlogs go to; a dump
	// joins the newest chain it has seen, which catches a reset the
	// numbering since the previous dump does not show.
	BinlogSet string
	// Logger is where the dump reports; nil means slog.Default.
	Logger *slog.Logger
}

Dump commits one mysqldump of the whole server to the dump set.

func (*Dump) Run

func (d *Dump) Run(ctx context.Context) (*backup.WalkResult, error)

Run dumps the server into the set's next commit. A dump of another server than the set's fails with ErrForeignServer, and a dump mysqldump did not finish leaves the set without a commit.

type Exec

type Exec struct {
	Mysql       string
	Mysqldump   string
	Mysqlbinlog string
	// DefaultsFile, when set, goes to every program as
	// --defaults-extra-file.
	DefaultsFile string
	// Stderr receives the programs' errors; nil means os.Stderr.
	Stderr io.Writer
}

Exec is a Client that runs the installed client programs. They connect as MySQL's option files say: ~/.my.cnf, or DefaultsFile.

func (*Exec) BinlogVersion

func (e *Exec) BinlogVersion(ctx context.Context) (string, error)

BinlogVersion implements Client.

func (*Exec) Dump

func (e *Exec) Dump(ctx context.Context, args []string) (io.ReadCloser, error)

Dump implements Client.

func (*Exec) DumpVersion

func (e *Exec) DumpVersion(ctx context.Context) (string, error)

DumpVersion implements Client.

func (*Exec) Load

func (e *Exec) Load(ctx context.Context, r io.Reader) error

Load implements Client.

func (*Exec) LoadCommand

func (e *Exec) LoadCommand(file string) string

LoadCommand is the shell command that loads the dump in file.

func (*Exec) Query

func (e *Exec) Query(ctx context.Context, sql string) ([][]string, error)

Query implements Client.

func (*Exec) Replay

func (e *Exec) Replay(ctx context.Context, args []string) error

Replay implements Client. Every file goes through one mysqlbinlog and one mysql, so a temporary table lives across files.

func (*Exec) ReplayCommand

func (e *Exec) ReplayCommand(args []string) string

ReplayCommand is the shell command that replays binlogs as Replay does.

type Identity

type Identity struct {
	ServerID   uint32
	ServerUUID string
	Base       string
}

Identity names the server a commit belongs to: by server_uuid where both sides have one, else by server_id and the binary log's base name.

func (Identity) Same

func (i Identity) Same(o Identity) bool

Same reports whether i and o name the same server.

func (Identity) String

func (i Identity) String() string

type OpenBinlog

type OpenBinlog struct {
	Name string
	Path string
	// Size is how much of it holds whole events.
	Size int64
	// ModTime is its last event's time.
	ModTime time.Time
	// Created is its format description event's time.
	Created time.Time
}

OpenBinlog is a binlog a receiver is still writing.

type Options

type Options struct {
	Host     string
	Port     int
	Socket   string
	User     string
	Password string
	// Protocol is tcp or socket, as --protocol takes it; empty picks by
	// Host.
	Protocol string
	// SSLMode, SSLCA, SSLCert and SSLKey are the --ssl-* options.
	SSLMode string
	SSLCA   string
	SSLCert string
	SSLKey  string
}

Options are the connection settings MySQL's option files give the client group.

func ReadOptions

func ReadOptions(extra string) (*Options, error)

ReadOptions reads the client group of extra, when set, and then of ~/.my.cnf, a later value winning, as MySQL's client programs do with --defaults-extra-file. A missing ~/.my.cnf is no error; a missing extra is.

type Querier

type Querier interface {
	// Query runs sql and returns its rows as mysql --batch
	// --skip-column-names prints them, NULL as "NULL".
	Query(ctx context.Context, sql string) ([][]string, error)
}

Querier runs SQL against a server.

type Receiver

type Receiver struct {
	// DefaultsExtraFile is the option file read before ~/.my.cnf for the
	// connection and its password.
	DefaultsExtraFile string
	// ServerID is the replica server id the receiver connects with;
	// ReplicaServerID of the set when zero.
	ServerID uint32
	// Spool is where binlogs wait for their commit; one receiver per spool.
	Spool Spool
	// Walker returns the walker of one commit: its Set is the binlog set
	// and its Root the spool's directory.
	Walker func() *backup.Walker
	// DumpSet is the set the server's dumps go to.
	DumpSet string
	// Window is how far back point-in-time recovery reaches.
	Window time.Duration
	// CommitEvery is how often new binlog is committed; DefaultCommitEvery
	// when zero, and at least MinCommitEvery.
	CommitEvery time.Duration
	// CommitBytes of new binlog commit early; DefaultCommitBytes when zero.
	CommitBytes int64
	// Logger is where the receiver reports; nil means slog.Default.
	Logger *slog.Logger
	// Committed, when set, hears of each commit once it is stored.
	Committed func(*backup.WalkResult)
	// Report, when set, hears how the receiver stands every ten seconds.
	Report func(ReceiverStatus)
	// contains filtered or unexported fields
}

Receiver receives a server's binary log as a replica does, through the binlog dump protocol, into a spool, and commits it to the binlog set on a timer: whole binlogs as the server moves on from them, and the binlog being received up to its last event as <binlog>.partial. It sends the server nothing back: MySQL keeps its binlogs by its own expiry.

func (*Receiver) Run

func (r *Receiver) Run(ctx context.Context, stop <-chan struct{}) error

Run receives and commits until stop is closed, then commits what it received and returns nil. It reconnects after the connection fails. It returns early with an error that needs a person (ErrGap, ErrNoBinlog, ErrForeignBinlog, ErrBinlogChanged, ErrServerIDInUse), or with one a restart may clear. When ctx ends, it stops without the last commit.

type ReceiverStatus

type ReceiverStatus struct {
	Chain     int  `json:"chain"`
	Connected bool `json:"connected"`
	// Received is the binlog and position the spool's binlogs end at.
	Received string `json:"received,omitempty"`
	// Committed is where the binlogs the newest stored commit holds end.
	Committed string `json:"committed,omitempty"`
	// LagBytes is how much binlog the server holds that no commit holds,
	// as of the last minute; LagSeconds how long the oldest received byte
	// no commit holds has waited.
	LagBytes   int64   `json:"lag_bytes"`
	LagSeconds float64 `json:"lag_seconds"`
	SpoolBytes int64   `json:"spool_bytes"`
	ServerID   uint32  `json:"server_id"`
	// Healthy is false while a received byte has waited for a commit, or
	// the receiver for a connection, longer than three commit intervals.
	Healthy bool `json:"-"`
}

ReceiverStatus is how a receiver stands.

type Restore

type Restore struct {
	Objects backup.ObjectStore
	Key     *storekey.Key
	// Latest finds a set's newest commit; it returns backup.ErrNotFound for
	// a set without one.
	Latest  func(ctx context.Context, set string) (*proto.Ref, error)
	DumpSet string
	// BinlogSet, when empty, restores the dump alone.
	BinlogSet string
	// At, when set, is the moment the restore stops at; zero replays to the
	// end of the binlog set.
	At     time.Time
	Client Client
	// WorkDir is where the binlogs are fetched to before the replay.
	WorkDir string
	// To, when set, only writes the dump and the binlogs into this empty
	// directory, for the caller to apply.
	To string
	// FS is the tree To names a directory of; nil is the local disk.
	FS backup.WriteFS
	// Force loads into a server that holds databases already.
	Force bool
}

Restore loads a dump into an empty server and, given a binlog set, replays the binlogs after it.

func (*Restore) Run

func (r *Restore) Run(ctx context.Context) (*RestoreResult, error)

Run restores into the server, or into To.

type RestoreResult

type RestoreResult struct {
	Dump    *proto.Commit
	DumpRef *proto.Ref
	// Binlog is the binlog commit the binlogs come from; nil without a
	// binlog set or when it has no commits.
	Binlog       *proto.Ref
	BinlogFormat string
	// Binlogs names the binlogs replayed, in order.
	Binlogs []string
	// ReplayArgs are mysqlbinlog's arguments; empty when nothing is
	// replayed. With To, they name the binlogs as FS names them.
	ReplayArgs []string
	// DumpPath is where a restore with To wrote the dump, as FS names it.
	DumpPath string
}

RestoreResult says what a restore chose and did.

type Server

type Server struct {
	Version        string
	Major, Minor   int
	Patch          int
	LogBin         bool
	ServerID       uint32
	ServerUUID     string
	BinlogBasename string
	BinlogFormat   string
	GTIDMode       bool
}

Server is what goback reads about a server before it backs it up.

func ReadServer

func ReadServer(ctx context.Context, c Querier) (*Server, error)

ReadServer reads the server's version and binary log settings.

type Spool

type Spool struct {
	Dir string
}

Spool is the directory a receiver keeps binlogs in until a commit holds them: whole binlogs under their names, and the one being received in .receiving. A binlog appears under its name only once it is whole.

func (Spool) Files

func (s Spool) Files() ([]string, error)

Files lists the whole binlogs in the spool, oldest first.

func (Spool) Remove

func (s Spool) Remove(names ...string) error

Remove deletes binlogs a commit holds.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL