agent

package
v1.64.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 13, 2026 License: Apache-2.0 Imports: 34 Imported by: 0

Documentation

Overview

Package agent runs a machine's backups on their schedules in one long-running process: it reads a config file listing the backups, derives when each last ran from the store, and runs each kind's job functions against one shared, open store.

Index

Constants

View Source
const (
	AgentStarting = "starting"
	AgentRunning  = "running"
	AgentStopping = "stopping"
	AgentStopped  = "stopped"
)

The states of an agent.

View Source
const (
	JobPlanning = "planning"
	// JobBlocked waits for the first complete commit of the set it runs
	// after.
	JobBlocked = "blocked"
	// JobWaiting is due and waits for a free slot.
	JobWaiting = "waiting"
	JobIdle    = "idle"
	JobRunning = "running"
)

The states of a task.

View Source
const (
	ResultOK        = "ok"
	ResultDirty     = "dirty"
	ResultFailed    = "failed"
	ResultCancelled = "cancelled"
)

The results of a run.

View Source
const StatusFile = "status.json"

StatusFile is the name of the status file in the state dir.

Variables

View Source
var ErrLocked = errors.New("another goback agent uses this state dir")

ErrLocked is returned by Lock when another agent holds the state dir.

Functions

func JitterOffset

func JitterOffset(agent, task string, window time.Duration) time.Duration

JitterOffset is the offset in [0, window) every slot of the named task of agent moves by. It is the same on every start.

func LastRun

func LastRun(ctx context.Context, index backup.Index, set string, now time.Time) (time.Time, *proto.Commit, error)

LastRun is when the set's newest complete commit started, by its scan start, with the commit; zero and nil when the set has none.

func Lock

func Lock(dir string) (func(), error)

Lock creates dir and takes the exclusive lock on its lock file, so a second agent on the same state dir refuses to start. The returned func releases it.

Types

type Agent

type Agent struct {
	// ID is the agent id runs commit as; it also picks each task's jitter.
	ID string
	// Settings and Caches name the store and the caches every run shares,
	// which the agent opens when Open is nil; ID, when set, is the agent id.
	Settings goback.Settings
	Caches   goback.Caches
	// Open opens the store, the index, the key and the caches that every
	// run shares. The agent retries it with backoff until it works. When
	// Open is nil, the agent opens Settings, and a nil Close and AfterRun
	// close it and run job.AfterRun.
	Open func(ctx context.Context) (*job.Env, error)
	// Close closes what Open opened, once every run has ended.
	Close func(env *job.Env)
	// AfterRun runs after each run, never two at once: compaction and cache
	// sweeps.
	AfterRun func(ctx context.Context, env *job.Env)
	// Concurrency is the most tasks running at once, log tasks aside.
	Concurrency int
	// Grace is how long running tasks get once Run's ctx ends, before they
	// are cancelled.
	Grace time.Duration
	// StateDir is where the status file goes; empty writes none.
	StateDir string
	Logger   *slog.Logger
	// Now is the wall clock; nil is time.Now.
	Now func() time.Time
	// contains filtered or unexported fields
}

Agent runs tasks on their plans against one store it opens once. It keeps no schedule of its own: each task's last run is the scan start of its set's newest complete commit.

func (*Agent) Abort

func (a *Agent) Abort()

Abort cancels running tasks at once instead of waiting for the grace, as a second signal does.

func (*Agent) Reload

func (a *Agent) Reload(tasks []*Task)

Reload replaces the tasks Run schedules. A task whose Spec is unchanged keeps its timing; a new or changed one is planned from the store; a run of a removed or changed one already going finishes as it was.

func (*Agent) Run

func (a *Agent) Run(ctx context.Context, tasks []*Task) error

Run schedules the tasks until ctx ends, then starts nothing new, gives running tasks the grace, cancels what is left, closes the store and returns. Failed runs never end it.

type Backup

type Backup struct {
	// Name is the set a path backup writes, and the group a database
	// backup's sets go under; slashes nest it in groups, as in srv1/db.
	Name string `yaml:"name"`
	// Every and Cron say when full backups run; the kind's default when
	// both are empty.
	Every time.Duration `yaml:"every"`
	Cron  string        `yaml:"cron"`
	// Jitter moves every full backup of this agent's backup by the same
	// offset inside [0, Jitter).
	Jitter time.Duration `yaml:"jitter"`
	// Timeout cancels a full backup run after this long; zero never does.
	Timeout time.Duration `yaml:"timeout"`

	Path     *PathBackup     `yaml:"path"`
	Postgres *PostgresBackup `yaml:"postgres"`
	MySQL    *MySQLBackup    `yaml:"mysql"`
}

Backup is one thing to protect: exactly one of Path, Postgres and MySQL is set.

func (*Backup) CheckDatabase

func (b *Backup) CheckDatabase(ctx context.Context) []string

CheckDatabase connects to the backup's database and lists what it lacks for the backup as configured; nil for a backup of no database.

type Checked

type Checked struct {
	Jobs      []Planned       `json:"jobs"`
	Databases []DatabaseCheck `json:"databases"`
}

Checked is what Check found.

type Config

type Config struct {
	Store    Store    `yaml:"store"`
	Schedule Schedule `yaml:"schedule"`
	Backups  []Backup `yaml:"backups"`
}

Config is a goback agent config file.

func Decode

func Decode(r io.Reader) (*Config, error)

Decode reads a config strictly, refusing unknown keys, fills in the defaults and validates it, reporting every problem it finds.

func Load

func Load(path string) (*Config, error)

Load reads and validates the config file at path.

func (*Config) Check

func (c *Config) Check(ctx context.Context, idx backup.Index, agent string, now time.Time) (Checked, error)

Check reads when each of the config's tasks last ran from idx and works out when it is next due for agent, and connects to each backup's database to list what it lacks. The error joins every task it could not read and every problem found; what was found is returned either way.

func (*Config) RestartSections

func (c *Config) RestartSections(old *Config) []string

RestartSections names the sections of c that differ from old and that a running agent keeps until it restarts.

func (*Config) StateDir

func (c *Config) StateDir(cacheDir string) (string, error)

StateDir is where the status file and the lock go: the config's state-dir, else <cacheDir>/schedule.

func (*Config) Tasks

func (c *Config) Tasks() []*Task

Tasks lists what the config's backups run, in config order.

type Control

type Control struct {
	// contains filtered or unexported fields
}

Control is what the agent hands a ServeFunc.

func (*Control) Committed

func (c *Control) Committed(result *backup.WalkResult)

Committed reports a commit the task made, for the status file and the metrics.

func (*Control) Report

func (c *Control) Report(details any, healthy bool)

Report puts details under the task in the status file. healthy says the task works as it should now; the task's staleness is judged by the last healthy report.

func (*Control) StateDir

func (c *Control) StateDir() string

StateDir is the agent's state dir, where a task may keep local state under a name of its own; empty when the agent has none.

func (*Control) Stop

func (c *Control) Stop() <-chan struct{}

Stop is closed when the agent wants the task to make its last commit and return.

type DatabaseCheck

type DatabaseCheck struct {
	Backup   string   `json:"backup"`
	Problems []string `json:"problems"`
}

DatabaseCheck is what a backup's database lacks for the backup.

type JobStatus

type JobStatus struct {
	Name   string `json:"name"`
	Backup string `json:"backup"`
	Set    string `json:"set"`
	Kind   string `json:"kind"`
	// State is planning, blocked, waiting, idle or running.
	State       string    `json:"state"`
	LastStart   time.Time `json:"last_start,omitzero"`
	LastResult  string    `json:"last_result,omitempty"`
	LastError   string    `json:"last_error,omitempty"`
	LastSuccess time.Time `json:"last_success,omitzero"`
	LastCommit  string    `json:"last_commit,omitempty"`
	Failures    int       `json:"failures"`
	NextDue     time.Time `json:"next_due,omitzero"`
	// StaleAfter is how long without a success makes the job unhealthy.
	StaleAfter int64 `json:"stale_after_seconds,omitempty"`
	// Details are what a long-running task last reported about itself.
	Details any `json:"details,omitempty"`
}

JobStatus is one task in the status file.

type MySQLBackup

type MySQLBackup struct {
	DefaultsExtraFile string `yaml:"defaults-extra-file"`
	Mysql             string `yaml:"mysql"`
	Mysqldump         string `yaml:"mysqldump"`
	// Spool is where binlogs wait for their commit; <state-dir>/spool/<set>
	// when empty.
	Spool       string        `yaml:"spool"`
	ServerID    uint32        `yaml:"server-id"`
	CommitEvery time.Duration `yaml:"commit-every"`
	CommitBytes string        `yaml:"commit-bytes"`
	Window      time.Duration `yaml:"window"`
	PITR        bool          `yaml:"pitr"`
	// NoBinlogPosition, with pitr off, has dumps record no binlog position,
	// which spares them the global read lock.
	NoBinlogPosition bool `yaml:"no-binlog-position"`
}

MySQLBackup backs up a MySQL server: dumps and, with PITR, its binlogs received as a replica, with goback mysql receive's settings.

func (*MySQLBackup) UnmarshalYAML

func (m *MySQLBackup) UnmarshalYAML(unmarshal func(any) error) error

UnmarshalYAML fills in the mysql commands' defaults before decoding.

type PathBackup

type PathBackup struct {
	Dir                string            `yaml:"dir"`
	Include            []string          `yaml:"include"`
	Exclude            []string          `yaml:"exclude"`
	Workers            int               `yaml:"workers"`
	ForceHash          int               `yaml:"force-hash"`
	CheckpointInterval time.Duration     `yaml:"checkpoint-interval"`
	ScanWorkers        int               `yaml:"scan-workers"`
	ReadRetries        int               `yaml:"read-retries"`
	LostRetries        int               `yaml:"lost-retries"`
	ProgressInterval   time.Duration     `yaml:"progress-interval"`
	Meta               map[string]string `yaml:"meta"`
	PreHook            string            `yaml:"pre-hook"`
	PostHook           string            `yaml:"post-hook"`
	HookTimeout        time.Duration     `yaml:"hook-timeout"`
	HookOptional       bool              `yaml:"hook-optional"`
}

PathBackup backs up a directory, with commit new's settings.

func (*PathBackup) UnmarshalYAML

func (p *PathBackup) UnmarshalYAML(unmarshal func(any) error) error

UnmarshalYAML fills in commit new's defaults before decoding. It takes the decoder's own unmarshal, which keeps unknown keys an error.

type Plan

type Plan struct {
	// contains filtered or unexported fields
}

Plan is when a task runs: an interval from the last run's start, or a cron expression read in a time zone.

func Cron

func Cron(expr string, loc *time.Location) (*Plan, error)

Cron runs a task at the times a five-field cron expression, or @hourly, @daily, @weekly or @monthly, names in loc. A time a DST change skips runs once in the hour after the change; a repeated one runs once.

func Every

func Every(d time.Duration) *Plan

Every runs a task d after its last run started.

func (*Plan) Due

func (p *Plan) Due(offset time.Duration, last, now time.Time) time.Time

Due is when a task with this plan next runs, its slots moved by offset, given when its last run started and the time now. A task that never ran is due now, as is one that missed a slot, however many; a last start ahead of now counts as now.

func (*Plan) String

func (p *Plan) String() string

String is the plan as the config says it.

type Planned

type Planned struct {
	Name   string `json:"name"`
	Backup string `json:"backup"`
	Set    string `json:"set"`
	Kind   string `json:"kind"`
	// Plan is the task's schedule, "always" for one that runs all the time.
	Plan    string    `json:"plan"`
	LastRun time.Time `json:"last_run,omitzero"`
	NextDue time.Time `json:"next_due,omitzero"`
}

Planned is one task as Check finds it.

type PostgresBackup

type PostgresBackup struct {
	DBName       string `yaml:"dbname"`
	PGBasebackup string `yaml:"pg-basebackup"`
	// Spool is where WAL waits for its commit; <state-dir>/spool/<set>
	// when empty.
	Spool             string        `yaml:"spool"`
	Slot              string        `yaml:"slot"`
	CommitEvery       time.Duration `yaml:"commit-every"`
	CommitBytes       string        `yaml:"commit-bytes"`
	AllowUncappedSlot bool          `yaml:"allow-uncapped-slot"`
	Window            time.Duration `yaml:"window"`
	PITR              bool          `yaml:"pitr"`
}

PostgresBackup backs up a Postgres cluster: base backups and, with PITR, its WAL streamed through a replication slot, with goback postgres receive's settings.

func (*PostgresBackup) UnmarshalYAML

func (p *PostgresBackup) UnmarshalYAML(unmarshal func(any) error) error

UnmarshalYAML fills in the postgres commands' defaults before decoding.

type RunFunc

type RunFunc func(ctx context.Context, env *job.Env) (*backup.WalkResult, error)

RunFunc runs a task once. A nil result with a nil error means there was nothing to commit.

type Schedule

type Schedule struct {
	// Concurrency is the most full backups running at once; log capture
	// doesn't count.
	Concurrency int `yaml:"concurrency"`
	// ShutdownGrace is how long running backups get after SIGTERM.
	ShutdownGrace time.Duration `yaml:"shutdown-grace"`
	// StateDir holds the status file and the lock; <cache-dir>/schedule
	// when empty.
	StateDir string `yaml:"state-dir"`
}

Schedule says how the agent runs its backups.

type ServeFunc

type ServeFunc func(ctx context.Context, env *job.Env, c *Control) error

ServeFunc runs a task that runs all the time, committing on its own timer, until ctx ends or c.Stop is closed. After Stop it makes its final commit and returns; ctx ends when the shutdown grace runs out.

type Status

type Status struct {
	PID       int         `json:"pid"`
	State     string      `json:"state"`
	StartedAt time.Time   `json:"started_at"`
	UpdatedAt time.Time   `json:"updated_at"`
	Jobs      []JobStatus `json:"jobs"`
}

Status is what the status file reports: the agent and each of its tasks. The agent writes it and never reads it.

func ReadStatus

func ReadStatus(dir string) (*Status, error)

ReadStatus reads the status file in dir.

func (*Status) Problems

func (s *Status) Problems(now time.Time) []string

Problems lists what makes the agent unhealthy at now: having stopped, a heartbeat older than three minutes, and every job whose last success is older than its bound. A job that never succeeded is stale once the agent has been up for its bound; a blocked one is not judged.

type Store

type Store struct {
	Storage           string `yaml:"storage"`
	Index             string `yaml:"index"`
	StoreKey          string `yaml:"store-key"`
	AtRestKey         string `yaml:"at-rest-key"`
	PassphraseFile    string `yaml:"passphrase-file"`
	AgentID           string `yaml:"agent-id"`
	CacheDir          string `yaml:"cache-dir"`
	MetadataCacheSize string `yaml:"metadata-cache-size"`
	BlobCacheSize     string `yaml:"blob-cache-size"`
	BlobCacheOnBackup *bool  `yaml:"blob-cache-on-backup"`
}

Store names the store and how to open it, under the global flags' names. A flag or its environment variable overrides a value set here.

func (Store) Caches

func (s Store) Caches() goback.Caches

Caches is every cache a run uses; the blob cache unless BlobCacheOnBackup turns it off.

func (Store) Settings

func (s Store) Settings() (goback.Settings, error)

Settings is the store section as goback.Settings, with the passphrase read from PassphraseFile. A cache size of 0 means no bound.

type Task

type Task struct {
	// Name is unique among an agent's tasks; it names the task in logs,
	// the status file and metrics.
	Name   string
	Set    string
	Kind   string
	Backup string
	Plan   *Plan
	// Jitter is the window the task's slots move by a fixed offset in.
	Jitter time.Duration
	// Timeout cancels a run after this long; zero never does.
	Timeout time.Duration
	// StaleAfter is how long a Serve task may go without reporting itself
	// healthy before it counts as unhealthy; zero never judges it.
	StaleAfter time.Duration
	// Log marks log capture, which runs outside the concurrency limit.
	Log bool
	// After names a set that must hold a complete commit before the task
	// starts.
	After string
	// Spec is what the task was built from; a reload replaces a task whose
	// Spec changed.
	Spec  string
	Run   RunFunc
	Serve ServeFunc
}

Task is one thing a backup does on its own timing: its full backups, or its log capture. Exactly one of Run, with a Plan, and Serve is set.

func (*Task) NextDue

func (t *Task) NextDue(agent string, last, now time.Time) time.Time

NextDue is when the task runs next on agent, given when its last run started (zero for never) and the time now.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL