backup

package
v0.8.22 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 17, 2026 License: MIT Imports: 30 Imported by: 0

Documentation

Overview

Package backup copies volume disks to S3-compatible object storage and back. A volume's backups live under one path in a bucket:

<path>/jokku-backup.json      what the backups are: format, volume, key ID
<path>/backups/<time>.backup  one per backup: the disk's size and which block goes where
<path>/blocks/ab/abcd...      the disk's 1 MiB blocks, named by their contents

A block is uploaded once and shared by every backup that holds it, so each backup is complete on its own but only adds the blocks that changed. Holes and all-zero blocks are never stored. Blocks and backup files are compressed, and encrypted when there is a key.

Index

Constants

View Source
const (
	StateDir     = "state"
	ArtifactsDir = "artifacts"
	// StateVolume and ArtifactsVolume stand for the volume ID in their
	// paths' jokku-backup.json.
	StateVolume     = "cluster-state"
	ArtifactsVolume = "cluster-artifacts"
)

A cluster backup holds the control node's state, under <path>/state, and the root filesystems of recent releases, under <path>/artifacts (one backup per artifact, named by its file, sharing blocks).

View Source
const (

	// NameFormat names backups by when they were made, in UTC; names sort
	// in time order.
	NameFormat = "2006-01-02T15-04-05Z"
)

Variables

View Source
var ErrNotFound = errors.New("not found")

ErrNotFound is a key the store doesn't have.

View Source
var Open = OpenS3

Open connects to a destination's bucket. Tests replace it.

View Source
var StateFiles = []string{"jokku.db", "tls/control.key", "tls/control.crt", "wireguard.key", "control-token", "agent-token"}

StateFiles are what the state backup holds, relative to the data dir: a snapshot of the database, and the control node's identity (its TLS key, which nodes pin, its WireGuard key, and its tokens).

Functions

func CheckPath

func CheckPath(p string) error

CheckPath refuses a backup path that is empty or climbs out of the bucket's folders.

func Delete

func Delete(ctx context.Context, st Store, p, name string) error

Delete deletes one backup from a path. The blocks only it used stay until Collect.

func List

func List(ctx context.Context, st Store, p string) ([]string, error)

List returns the names of the backups in a path, oldest first.

func PackState

func PackState(dataDir, db string, w io.Writer) error

PackState writes the state as a tar: db (a snapshot of the database) as jokku.db, and the other StateFiles from dataDir that exist. Uncompressed: it is backed up block by block, and blocks are compressed then.

func ParseEndpoint

func ParseEndpoint(endpoint string) (*url.URL, error)

ParseEndpoint checks an endpoint is a bare https:// (or http://) host, adding https:// when no scheme is given.

func Prune

func Prune(ctx context.Context, st Store, p string, r Retention, now time.Time) ([]string, int, error)

Prune deletes the backups in a path that r doesn't keep, returning the names deleted and how many are left. The blocks only they used stay until Collect.

func UnpackState

func UnpackState(r io.Reader, dir string) ([]string, error)

UnpackState extracts a state tar into dir (files 0600, directories 0700), returning what it held. Only StateFiles are extracted.

Types

type Dir

type Dir string

Dir is a store in a local directory, for tests.

func (Dir) Check

func (d Dir) Check(context.Context) error

func (Dir) Delete

func (d Dir) Delete(_ context.Context, key string) error

func (Dir) Get

func (d Dir) Get(_ context.Context, key string) ([]byte, error)

func (Dir) List

func (d Dir) List(_ context.Context, prefix string) ([]Object, error)

func (Dir) Put

func (d Dir) Put(_ context.Context, key string, data []byte) error

type Info

type Info struct {
	Format    int    `json:"format"`
	BlockSize int    `json:"block_size"`
	Encrypted bool   `json:"encrypted"`
	KeyID     string `json:"key_id,omitempty"`
	// Volume is the ID of the volume whose backups these are, so two
	// volumes (on two clusters, say) never mix their backups in one path.
	Volume string `json:"volume"`
	App    string `json:"app,omitempty"`
	Name   string `json:"name,omitempty"`
}

Info is jokku-backup.json, which says what a path holds.

func ReadInfo

func ReadInfo(ctx context.Context, st Store, p string) (*Info, error)

ReadInfo reads what a path holds; nil if it holds nothing yet.

type Key

type Key struct {
	// contains filtered or unexported fields
}

Key encrypts a cluster's backups. It is shown to the user once and is the only way to read them: Jokku keeps a copy, but losing the cluster loses that.

func NewKey

func NewKey() *Key

func ParseKey

func ParseKey(s string) (*Key, error)

ParseKey reads a key as String writes it.

func (*Key) ID

func (k *Key) ID() string

ID names the key without giving it away, so a bucket can say which key its backups need.

func (*Key) String

func (k *Key) String() string

type Manifest

type Manifest struct {
	Version   int       `json:"version"`
	Name      string    `json:"name"`
	Volume    string    `json:"volume"`
	Created   time.Time `json:"created"`
	Size      int64     `json:"size"`
	BlockSize int       `json:"block_size"`
	// Index[i] is where the block Names[i] goes, in blocks.
	Index []int64  `json:"index"`
	Names []string `json:"names"`
}

Manifest is one backup: the disk's size and its blocks.

func ReadManifest

func ReadManifest(ctx context.Context, st Store, key *Key, p, name string) (*Manifest, error)

ReadManifest reads one backup's manifest.

func Restore

func Restore(ctx context.Context, st Store, key *Key, p, name, dst string, progress func(done, total int64)) (*Manifest, error)

Restore writes backup name of a path to dst, a new disk image, checking every block against its name. progress is told the bytes written so far and the total.

type Object

type Object struct {
	Key  string
	Size int64
}

Object is a listed object.

type Options

type Options struct {
	Store Store
	Key   *Key // nil: not encrypted
	Path  string

	Volume, App, VolumeName string // recorded in Info
	Disk                    string // the disk image
	Name                    string // the backup's name, from NameFormat

	// Freeze, when something may be writing to the disk, stops writes
	// until thaw is called, so the backup is the disk at one moment.
	Freeze func(ctx context.Context) (thaw func() error, err error)
	// Staging is a directory for blocks that change while the backup runs.
	Staging string
	Log     func(string)
}

Options say what to back up, and where.

type Result

type Result struct {
	Name      string `json:"name"`
	Size      int64  `json:"size"`       // the disk's size
	Blocks    int    `json:"blocks"`     // blocks holding data
	NewBlocks int    `json:"new_blocks"` // of them, not already in the bucket
	NewBytes  int64  `json:"new_bytes"`  // what was uploaded, after compression
}

Result says what a backup stored.

func Run

func Run(ctx context.Context, o Options) (*Result, error)

Run backs up a disk: blocks the bucket doesn't have yet are uploaded while the app runs, then, with writes frozen, only the blocks that changed meanwhile are copied aside, to be uploaded once writes resume.

type Retention

type Retention struct {
	Recent time.Duration
	Daily  int
	Also   []string // backups to keep regardless, such as one about to be restored
}

Retention says which backups to keep: every one from the last Recent, the last one of each of the last Daily days (UTC, today included), and always the newest.

func (Retention) Keep

func (r Retention) Keep(names []string, now time.Time) map[string]bool

Keep picks the backups (names, from NameFormat) that r keeps at now. Names that aren't times are kept: they aren't Jokku's to delete.

type Store

type Store interface {
	// Check reports whether the bucket can be reached with these
	// credentials.
	Check(ctx context.Context) error
	Put(ctx context.Context, key string, data []byte) error
	// Get returns ErrNotFound for a key that isn't there.
	Get(ctx context.Context, key string) ([]byte, error)
	// List returns the objects under prefix, sorted by key.
	List(ctx context.Context, prefix string) ([]Object, error)
	Delete(ctx context.Context, key string) error
}

Store is where backups are kept: a bucket, as objects named by key.

func OpenS3

func OpenS3(d types.BackupDestination) (Store, error)

OpenS3 connects to an S3-compatible bucket: AWS, Tigris, R2, B2, MinIO.

type Usage

type Usage struct {
	Backups int
	Blocks  int
	Bytes   int64 // blocks and backup files
}

Usage is what a path holds.

func Collect

func Collect(ctx context.Context, st Store, key *Key, p string) (Usage, int, int64, error)

Collect deletes the blocks no backup in a path uses, and says what is left. It must not run while a backup of the path is: that backup's new blocks aren't in any backup file yet.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL