backup

package
v3.0.0-alpha.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 5, 2026 License: BSD-2-Clause Imports: 31 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// PromotedTopologyChanged: the replicaset set is not the one backed up.
	PromotedTopologyChanged = "topology_changed"
	// PromotedMasterChanged: a replicaset is backed up on another instance now.
	PromotedMasterChanged = "master_changed"
)

Reasons a full backup was not the plan.

View Source 
const BackupIDLayout = "20060102T150405Z"

BackupIDLayout is the time layout of the ids NewBackupID produces: a zero-padded UTC timestamp, which sorts as text in the order it was taken.

View Source 
const SchemaVersion = 1

SchemaVersion is the current cluster manifest JSON schema version.

Variables

View Source 
var (
	ErrNoBackups = errors.New(
		"no backups found in storage; use --target=full for the first backup",
	)

	ErrShardDegraded = errors.New(
		"latest backup is degraded; use --target=full to start a new chain",
	)

	ErrTopologyChanged = errors.New(
		"topology changed since the last backup; use --target=full to start a new chain",
	)

	ErrMasterChanged = errors.New(
		"master changed since the last backup; use --target=full to start a new chain",
	)

	// ErrReplicasetUnreachable means a shard nobody could reach says nothing about where
	// the master is, so
	// this one deliberately does not recommend --target=full: answering a
	// restart with a new full chain is not reversible.
	ErrReplicasetUnreachable = errors.New(
		"replicaset is not reachable; retry once the instance is back",
	)

	ErrNoMaster = errors.New("no RW master found in replicaset")
)
View Source 
var ErrAlreadyInProgress = errors.New(
	"backup already in progress, run finalize first",
)

ErrAlreadyInProgress signals that box.backup is already open on the instance.

View Source 
var ErrInvalidBackupID = errors.New("invalid backup id")

ErrInvalidBackupID is wrapped by every backup id rejection, so a caller can tell a malformed identifier from a runtime failure.

Functions

func CheckChainHead 

func CheckChainHead(head *ClusterManifest, plan *BackupPlan, backupID BackupID) error

CheckChainHead compares the storage against the plan the backup was taken from. head is the newest manifest in the storage, nil when there is none.

Two things can have happened between planning and uploading, and both are silent: another upload landed -- so the increment about to be stored does not continue the backup it was planned against -- or the id of this backup does not sort above the stored one, and the manifest would land in the middle of a chain that is read by id. Neither is recoverable afterwards, so both are refused before anything is written.

Ids are compared as strings, which is how every reader of a storage orders them -- LatestManifest takes the greatest key, the chain sorts by backup id, gc walks that order. An id scheme that does not sort in the order the backups are taken (a bare counter: backup-2, backup-10) is therefore refused here rather than quietly producing a storage whose newest backup is not the last one taken.

func CloseIfOpen 

func CloseIfOpen(conn connector.Connector) error

CloseIfOpen calls box.backup.stop() if a backup is currently open, and does nothing if it is not.

func NewBackupID 

func NewBackupID() string

NewBackupID returns an id for a backup taken now: the current UTC second in BackupIDLayout.

It returns only once that second is over, which is what keeps ids unique without widening the layout: the next call on this host reads a later second, so two ids taken one after the other never collide and sort in the order they were taken. The wait is under a second.

Uniqueness holds for sequential calls on one host only. Calls running concurrently within one second, calls on different hosts, and a wall clock stepped back all can return the same id: upload refuses such a duplicate, but start and finalize take the id as a directory name and do not.

A finer layout cannot replace the wait: "20260918T143000.123Z" sorts below "20260918T143000Z", out of order with ids already stored at second precision.

func OpenStorage 

func OpenStorage(cfg *StorageConfig) (storage.Storage, error)

OpenStorage creates a backup storage backend from a StorageConfig.

func PlanIsAuthentic 

func PlanIsAuthentic(plan *BackupPlan) (bool, error)

PlanIsAuthentic reports whether the plan still carries the checksum tt wrote for exactly this content.

A plan without one was not written by `tt backup plan`; a plan whose checksum does not match was edited after it. Both mean the same thing for the checks upload makes against the plan: what it says is the operator's word, not something tt derived from the cluster, so a disagreement is reported and not refused. Anyone can recompute the checksum, so this stops nobody -- it only keeps tt from claiming a guarantee it does not have.

func SplitPaths 

func SplitPaths(commaSeparated string) []string

SplitPaths splits a comma-separated list of paths and trims whitespace.

func Start 

func Start(conn connector.Connector, opts BackupStartOpts) (string, error)

Start opens box.backup on the instance, packs the WAL files and a per-shard fragment into a .tar.zst archive in opts.Dir (or under <os.TempDir()>/tt-backup/<backup-id>/ when it is empty), and leaves box.backup open. The archive path is returned; the caller is expected to print it to stdout. A run that produces no archive closes box.backup again.

func Stop 

func Stop(conn connector.Connector, backupID, dir string) error

Stop closes box.backup on the instance and removes only this replicaset's local archive and manifest fragment from dir, or from <os.TempDir()>/tt-backup/<backup-id> when dir is empty. Stop is idempotent: if the backup is already closed, stop() is skipped and stale local artifacts are still removed. A backupID that is not a safe path component (see ValidateBackupID) is rejected before the instance is touched.

The directory itself is removed only in the default layout, where tt owns it; a directory named by the caller belongs to the caller and stays.

func Upload 

func Upload(
	ctx context.Context,
	store storage.Storage,
	backupID BackupID,
	manifestData []byte,
	archives []ArchiveToUpload,
) error

Upload uploads archives and the manifest to storage. If the upload fails, all previously uploaded archives are deleted.

func ValidateBackupID 

func ValidateBackupID(replicaID string) error

ValidateBackupID checks that id is usable in the three roles it carries: a single path component under the local backup root (start, finalize), a storage key component (upload), and the sort key that decides which backup is the latest (last, verify, gc). Ids come from job names, k8s annotations and date templates, so one that addresses a path other than its own has to be refused before the instance or the storage is touched.

The rules are a denylist on purpose: orchestrators emit timestamps (20260326T120000Z), dashed ids (2026-01-01-full) and non-ASCII names, and all of those stay valid.

What this cannot check is the one property the storage layout rests on: ids have to sort, as text, in the order the backups are taken. A single id says nothing about that, so the storage is where it is enforced -- `upload` refuses an id that does not sort above the newest stored backup, which is what a bare counter (backup-2 then backup-10) produces. A zero-padded UTC timestamp satisfies it by construction.

func ValidateFragmentsAgainstPlan 

func ValidateFragmentsAgainstPlan(
	fragments []*Fragment,
	plan *BackupPlan,
	authentic bool,
) ([]string, error)

ValidateFragmentsAgainstPlan checks that the fragments are the ones the plan asked for: each was taken on the instance the plan named as that replicaset's master, and each is of the backup type the plan asked for.

The master check is what catches a failover between `plan` and `start`: the backup is then taken on another instance, and an increment of another instance's journal does not continue the chain it is being appended to. tt notices a master change when it plans, but the window after that is exactly where nothing was looking.

A replicaset the plan expects and no fragment covers is deliberately not checked here. It is not a disagreement about what was backed up -- it is a shard that produced nothing, which a backup records rather than dies of: the aggregation turns it into a failed shard with a shard_unreachable warning, and the shards that did produce data are still stored.

The comparisons are only as good as the plan: what tt derived from the cluster is authoritative, what an operator wrote by hand is their word. So a disagreement with a plan tt produced is refused, and a disagreement with a hand-written or edited one is returned as a note for the caller to report. Which of the two it is comes from the plan's own checksum (PlanIsAuthentic), and is the caller's answer to pass, since only it knows how the plan arrived.

Returns the notes about checks that were made but not enforced, and about checks the plan states too little to make at all.

func VerifyArchives 

func VerifyArchives(archives []ArchiveToUpload, fragments []*Fragment) ([]string, error)

VerifyArchives recomputes the sha256 of every archive about to be uploaded and checks it against the fragment describing the same replicaset. The fragment's checksum was computed on the node, before the archive crossed the network to this host: a copy that went wrong in between is otherwise stored and published as healthy, and found out at restore time.

A fragment that carries no checksum gets the computed one, so the manifest describes the archive either way. Those replicaset UUIDs are returned rather than silently accepted: nothing was verified for them, and the caller says so.

An archive no fragment describes is refused rather than skipped. Today the caller cannot produce one -- the counts have to match, neither side may hold a duplicate replicaset, and every fragment needs an archive -- but that is an invariant of three other checks, and a function that verifies archives must not report success on one it never looked at.

The fragment ends up holding the checksum computed here, in the canonical lower-case form, so the manifest records what this host actually read.

Types

type AggregateInput 

type AggregateInput struct {
	BackupID         BackupID
	BaseFullBackupID BackupID
	PreviousBackupID BackupID
	CreationTime     time.Time
	Topology         Topology
	Shards           []*ShardInput
	// Warnings are the ones the caller derived rather than the shards did --
	// today only promoted_to_full, which needs the previous manifest to see.
	Warnings []Warning
}

AggregateInput contains all external data needed to build a manifest.

func NewAggregateInput 

func NewAggregateInput(
	backupID BackupID,
	previousBackupID BackupID,
	baseFullBackupID BackupID,
	creationTime time.Time,
	topology Topology,
	shards []*ShardInput,
) AggregateInput

NewAggregateInput collects backup metadata, topology and shard inputs.

type ArchiveToUpload 

type ArchiveToUpload struct {
	LocalPath      string
	StorageKey     string
	Size           int64
	ReplicasetUUID string
}

ArchiveToUpload pairs a local archive file with its storage destination.

func PrepareArchives 

func PrepareArchives(
	paths []string,
	backupID BackupID,
) ([]ArchiveToUpload, map[string]*ArtifactLocation, error)

PrepareArchives stats each local archive, extracts the replicaset UUID from its filename, and computes the storage key. Returns a slice ready for Upload and a map of replicaset_uuid → ArtifactLocation for manifest building.

Keys are relative to the storage root, which is where the manifest stored beside them says they are and what every reader resolves them against. The <cluster_name>/<environment>/ segment of the layout belongs to the storage the objects are written into (StorageConfig.Scope), not to the keys: a manifest carrying its own prefix sends readers looking for <prefix>/<prefix>/data/…, where restore cannot download the archive at all, verify reports the live one missing and the stored one dangling, and gc counts the stored one as an orphan.

type Artifact 

type Artifact struct {
	Path           string          `json:"path"`
	SizeBytes      int64           `json:"size_bytes"`
	ChecksumSHA256 string          `json:"checksum_sha256"`
	Compression    string          `json:"compression"`
	Files          []string        `json:"files"`
	RecoveryPoints []RecoveryPoint `json:"recovery_points"`
	Type           BackupType      `json:"type"`
}

Artifact describes the stored archive produced for a shard.

type ArtifactLocation 

type ArtifactLocation struct {
	Path      string
	SizeBytes int64
}

ArtifactLocation identifies an already uploaded shard archive.

type BackupID 

type BackupID string

BackupID is an opaque identifier of a backup in a chain.

type BackupInfo 

type BackupInfo struct {
	Files          []string          `json:"files"`
	Type           BackupType        `json:"type"`
	Vclock         Vclock            `json:"vclock"`
	PrevVclock     Vclock            `json:"prev_vclock"`
	RecoveryPoints *[]*RecoveryPoint `json:"recovery_points"`
}

BackupInfo is the decoded box.backup.info() result: files, vclocks, type and recovery points of an open backup.

Tarantool 3.8.0 returns:

  • vclock: the end vclock (max vclock to restore to);
  • prev_vclock: the begin vclock (only for incremental);
  • checkpoint_vclock: the begin vclock for full (cleared from output);
  • files: full paths to snap/xlog files;
  • recovery_points: {timestamp, replica_id, lsn, label}.

func GetInfo 

func GetInfo(conn connector.Connector) (*BackupInfo, error)

GetInfo runs box.backup.info() and decodes the result. It returns nil if no backup is open.

type BackupPlan 

type BackupPlan struct {
	FormatVersion PlanFormat `json:"format_version"`
	Type          BackupType `json:"mode"`
	// ClusterName and Environment name the storage subtree the backup belongs
	// to. They are decided when the backup is planned and travel with the plan,
	// so upload does not have to be told again what it is uploading.
	ClusterName string                    `json:"cluster_name,omitempty"`
	Environment string                    `json:"environment,omitempty"`
	Replicasets map[string]ReplicasetPlan `json:"replicasets,omitempty"`
	// The chain head the next backup builds on, absent whenever there is none
	// to name -- which today is every full plan, since planFull does not read
	// the storage at all. Both are null rather than missing keys: a consumer
	// reading plan["previous_backup_id"] gets None, not a KeyError.
	PreviousBackupID OptionalBackupID `json:"previous_backup_id"`
	BaseFullBackupID OptionalBackupID `json:"base_full_backup_id"`
	// ChecksumSHA256 is what `tt backup plan` computed over the rest of this
	// document. It is not a signature and guards nothing: it only tells upload
	// whether the plan in front of it is the one tt produced, so that a plan
	// written or edited by hand is treated as the operator's word rather than
	// as tt's -- see PlanIsAuthentic.
	ChecksumSHA256 string `json:"checksum_sha256,omitempty"`
}

BackupPlan is the output of the plan command.

func Plan 

func Plan(
	target BackupType,
	latest *ClusterManifest,
	live *LiveTopology,
	scope PlanScope,
) (*BackupPlan, error)

Plan builds a backup plan from the latest manifest and the live topology.

func ReadPlan 

func ReadPlan(filePath string) (*BackupPlan, error)

ReadPlan reads and decodes a tt backup plan JSON file.

func (*BackupPlan) Sign 

func (plan *BackupPlan) Sign() error

Sign records the checksum of the plan's own content, which is how upload later tells a plan tt produced from one written by hand.

type BackupStartOpts 

type BackupStartOpts struct {
	// BackupID identifies the backup; used in the archive path.
	BackupID string
	// FromVclock selects an incremental backup; nil means a full backup.
	FromVclock Vclock
	// TTL is the backup lease duration. Zero falls back to the default (1h).
	TTL time.Duration
	// InstName is a fallback instance name (from <APP:INSTANCE>).
	InstName string
	// Dir is the directory the archive and the manifest fragment are written
	// to. Empty means <os.TempDir()>/tt-backup/<backup-id>.
	Dir string
	// Snapshot takes a new checkpoint with box.snapshot() before a full backup
	// opens. Without it the full ships the newest checkpoint as it is. Full
	// backups only.
	Snapshot bool
}

BackupStartOpts are the parameters of tt backup start.

type BackupType 

type BackupType string

BackupType is the backup mode: full or incremental. 

const (
	// BackupTypeFull marks a complete backup chain starting point.
	BackupTypeFull BackupType = "full"
	// BackupTypeIncremental marks a backup based on a previous one.
	BackupTypeIncremental BackupType = "incremental"
)

type ClusterManifest 

type ClusterManifest struct {
	SchemaVersion    int              `json:"schema_version"`
	BackupID         BackupID         `json:"backup_id"`
	PreviousBackupID OptionalBackupID `json:"previous_backup_id"`
	BaseFullBackupID BackupID         `json:"base_full_backup_id"`
	Status           Status           `json:"status"`
	CreationTime     time.Time        `json:"creation_time"`
	Shards           map[string]Shard `json:"shards"` // Key is replicaset.
	Topology         Topology         `json:"topology"`
	Warnings         []Warning        `json:"warnings"` // Empty is [].
}

ClusterManifest is the complete cluster-level backup manifest.

func Aggregate 

func Aggregate(in AggregateInput) (*ClusterManifest, error)

Aggregate builds and validates a cluster manifest from shard fragments.

func LatestManifest 

func LatestManifest(ctx context.Context, store storage.Storage) (*ClusterManifest, error)

LatestManifest reads the manifest with the lexicographically greatest manifest key. Backup IDs are sortable, so there is no need to load and build every backup chain. Anything under manifests/ that is not a manifest key (an operator note, an S3 console folder placeholder) is skipped rather than decoded, and the manifest that is returned has to be usable: callers chain the next backup onto the backup id they read here.

func (ClusterManifest) Validate 

func (manifest ClusterManifest) Validate() error

Validate checks manifest structure without chain or storage verification.

type ClusterMember 

type ClusterMember struct {
	ID   uint32 `json:"id"`
	UUID string `json:"uuid"`
	Name string `json:"name"`
}

ClusterMember is one box.space._cluster row: a replicaset member as the backed-up instance saw it. Name is empty on rows that carry none, which is what members written by old Tarantool replicas look like.

func GetClusterMembers 

func GetClusterMembers(conn connector.Connector) ([]ClusterMember, error)

GetClusterMembers fetches the _cluster membership from the instance.

type Fragment 

type Fragment struct {
	ReplicasetUUID string `json:"replicaset_uuid"`
	InstanceUUID   string `json:"instance_uuid"`
	InstanceName   string `json:"instance_name"`
	// InstanceID is the backed-up instance's own box.space._cluster id, from
	// the same membership read the members came from. Zero (omitted) when
	// that read named no row for the instance.
	InstanceID uint32 `json:"instance_id,omitempty"`
	Hostname   string `json:"hostname"`
	// Members are the other members of the replicaset as the backed-up
	// snapshot records them in _cluster, ordered by name. Absent when
	// the replicaset has no other members.
	Members        []ClusterMember   `json:"members,omitempty"`
	Type           BackupType        `json:"type"`
	VclockBegin    Vclock            `json:"vclock_begin"`
	VclockEnd      Vclock            `json:"vclock_end"`
	Files          []string          `json:"files"`
	ChecksumSHA256 string            `json:"checksum_sha256"`
	RecoveryPoints *[]*RecoveryPoint `json:"recovery_points,omitempty"`
}

Fragment is a per-replicaset backup description stored in an instance archive.

func DecodeFragment 

func DecodeFragment(data []byte) (*Fragment, error)

DecodeFragment decodes and validates one instance_backup.json payload.

func ReadFragments 

func ReadFragments(paths []string) ([]*Fragment, error)

ReadFragments reads and decodes per-shard instance_backup.json files.

func (Fragment) Validate 

func (fragment Fragment) Validate() error

Validate checks that a fragment has required structural fields.

type InstanceInfo 

type InstanceInfo struct {
	ReplicasetUUID string
	InstanceUUID   string
	InstanceName   string
	Hostname       string
	WalDir         string
	MemtxDir       string
	VinylDir       string
}

InstanceInfo holds instance-identifying fields and data directories fetched from the instance via box.info/box.cfg.

func GetInstanceInfo 

func GetInstanceInfo(conn connector.Connector) (*InstanceInfo, error)

GetInstanceInfo fetches instance-identifying fields, the hostname and data directories from the instance (box.info/box.cfg).

type LiveInstance 

type LiveInstance struct {
	InstanceUUID string `json:"instance_uuid"`
	InstanceName string `json:"instance_name"`
	Hostname     string `json:"hostname"`
	Mode         string `json:"mode"`
	Status       string `json:"status"`
}

LiveInstance is one instance in the live cluster topology.

type LiveTopology 

type LiveTopology struct {
	Replicasets map[string][]LiveInstance `json:"replicasets"`
}

LiveTopology is the current cluster topology discovered from live nodes.

type OptionalBackupID 

type OptionalBackupID BackupID

OptionalBackupID is a backup id that a document may legitimately not have: the first full backup of a chain has no previous backup, and a plan for one has no base full backup either. The manifest schema types those as "string | null", so an empty one is written and read as null rather than as "": a consumer branching on `=== null` would otherwise decide that a backup it holds has a predecessor named "".

It stays a string type so that the internal "" means "chain start" reading, which every walker of a chain already does, keeps working unchanged.

func (OptionalBackupID) MarshalJSON 

func (id OptionalBackupID) MarshalJSON() ([]byte, error)

MarshalJSON writes an absent id as null.

func (*OptionalBackupID) UnmarshalJSON 

func (id *OptionalBackupID) UnmarshalJSON(data []byte) error

UnmarshalJSON reads null as an absent id, and so accepts the "" that manifests written before this field was nullable carry. A key that is missing entirely never reaches here at all -- the zero value is already what it means.

type PlanFormat 

type PlanFormat int

PlanFormat is the plan's format_version. It decodes from any JSON number with an integral value, because JSON has no integer type and an orchestrator building the plan through jq or JavaScript emits 1.0 for what it means as 1. Anything else -- a string, a fraction -- is reported as what it is rather than as a Go decoding failure. 

const PlanFormatVersion PlanFormat = 1

PlanFormatVersion is the current backup plan JSON format version. The plan travels from the manager host that produced it to the one that uploads, possibly across a tt upgrade, so upload refuses a version it does not know rather than reading the fields it knows and ignoring the rest.

func (*PlanFormat) UnmarshalJSON 

func (v *PlanFormat) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes a plan format version.

type PlanScope 

type PlanScope struct {
	ClusterName string
	Environment string
}

PlanScope is the storage subtree the planned backup belongs to. It is an argument of its own because it is the one part of a plan that does not come from the cluster: the operator names it, once, when planning.

type RecoveryPoint 

type RecoveryPoint struct {
	Label     string  `json:"label"`
	ReplicaID uint32  `json:"replica_id"`
	LSN       uint64  `json:"lsn"`
	Timestamp float64 `json:"timestamp"` // unix-time (seconds, float).
}

RecoveryPoint describes one engine recovery point returned by Tarantool. box.backup.info() returns {timestamp, replica_id, lsn, label}.

type ReplicasetPlan 

type ReplicasetPlan struct {
	MasterInstanceUUID string `json:"master_instance_uuid"`
	MasterInstanceName string `json:"master_instance_name"`
	FromVclock         Vclock `json:"from_vclock,omitempty"`
}

ReplicasetPlan describes the backup plan for one replicaset.

type Shard 

type Shard struct {
	Instance *ShardInstance `json:"instance,omitempty"`
	Error    string         `json:"error,omitempty"`
}

Shard is one replicaset result: either a backed-up instance or an error.

type ShardInput 

type ShardInput struct {
	ReplicasetUUID string
	Fragment       *Fragment
	Location       *ArtifactLocation
	Err            error
}

ShardInput describes one expected replicaset backup result.

type ShardInstance 

type ShardInstance struct {
	InstanceUUID string   `json:"instance_uuid"`
	InstanceName string   `json:"instance_name"`
	Hostname     string   `json:"hostname"`
	VclockBegin  Vclock   `json:"vclock_begin"`
	VclockEnd    Vclock   `json:"vclock_end"`
	Artifact     Artifact `json:"artifact"`
}

ShardInstance contains successful backup metadata for one instance.

type StartOpts 

type StartOpts struct {
	// FromVclock selects an incremental backup; nil means a full backup.
	FromVclock Vclock
	// TTL is the backup lease duration. Zero falls back to defaultTTL (1h).
	TTL time.Duration
}

StartOpts are the parameters of box.backup.start().

type Status 

type Status string

Status is the aggregate health of the cluster backup. 

const (
	// StatusOK means all expected shards were backed up without warnings.
	StatusOK Status = "OK"
	// StatusDegraded means some data exists, but the backup has warnings or shard errors.
	StatusDegraded Status = "degraded"
	// StatusFailed means no shard was successfully backed up.
	StatusFailed Status = "failed"
)

type StorageConfig 

type StorageConfig struct {
	Type StorageType
	// File storage.
	Path string
	// S3 storage.
	Endpoint        string
	Bucket          string
	Region          string
	AccessKeyID     string
	SecretAccessKey string
	SessionToken    string
	UseSSL          bool
	// CACert is a PEM file holding the certificate authority of an endpoint the
	// system roots do not cover; SkipVerify drops certificate checks entirely.
	CACert     string
	SkipVerify bool
	// Common.
	Prefix string
}

StorageConfig is a parsed backup storage configuration.

func ParseStorageFile 

func ParseStorageFile(path string) (*StorageConfig, error)

ParseStorageFile reads the full form of a storage configuration: a YAML file whose "type" field names the backend and decides what the other fields mean.

func ParseStorageURI 

func ParseStorageURI(rawURI string) (*StorageConfig, error)

ParseStorageURI parses a --backup-storage value into a StorageConfig.

The value is either a URI naming the backend by its scheme, or "@" followed by the path of a YAML config file (see ParseStorageFile). 

file://<abs_path>[?Prefix=<subdir>]
    Local filesystem storage. The path must be absolute. Query parameters:
      Prefix           - subdirectory within the path (optional).

s3://<bucket>[/<prefix>]
    S3 storage configured by the standard AWS_* variables: credentials and
    TLS are not written in the URI, which is what a command line, a process
    listing and a cron log all show.

s3+https://<endpoint>/<bucket>[/<prefix>]?Region=...&AccessKeyID=...&SecretAccessKey=...
s3+http://<endpoint>/<bucket>[/<prefix>]?Region=...&AccessKeyID=...&SecretAccessKey=...
    S3-compatible storage with the endpoint and the credentials in the URI.
    Use s3+https for TLS, s3+http for plain TCP. The first path segment
    after the host is the bucket name, the rest is the optional key prefix.
    Query parameters:
      Region           - AWS region (optional).
      AccessKeyID      - access key ID (required).
      SecretAccessKey  - secret access key (required).

func (*StorageConfig) Scope 

func (cfg *StorageConfig) Scope(clusterName, environment string) error

Scope narrows the config to one cluster in one environment, the <storage_root>/<cluster_name>/<environment>/ layout of the RFC. It appends the two segments to whatever prefix the URI already carried, so a storage pointed at a subtree and a storage scoped by the flags compose.

Every command that touches a storage takes the same two flags, because a backup written into the subtree has to be readable from it: a verify or a gc aimed at the root of a storage laid out this way sees nothing at all, and reports a healthy storage or a clean sweep for it.

An environment without a cluster name is refused rather than dropped: the pair names one directory level each, and honouring only the second one would silently read or write somewhere else entirely. So is a value that is not one path component, or is nothing but whitespace -- a variable the caller meant to expand and did not would otherwise name a directory of blanks that only a run with the same unexpanded variable ever finds again.

The two are named as fields rather than as flags, because the values reach here from a flag on one command and out of a plan file on another; the caller says which by wrapping the error.

type StorageType 

type StorageType string

StorageType is the type of a backup storage backend. 

const (
	StorageTypeFile StorageType = "file"
	StorageTypeS3   StorageType = "s3"
)

type Topology 

type Topology struct {
	Replicasets map[string][]TopologyInstance `json:"replicasets"`
}

Topology lists expected instances grouped by replicaset UUID.

func BuildTopologyFromFragments 

func BuildTopologyFromFragments(fragments []*Fragment) Topology

BuildTopologyFromFragments builds a Topology from the collected fragments. Each replicaset lists the master instance that produced the backup first, then the members its fragment recorded, ordered by name.

type TopologyInstance 

type TopologyInstance struct {
	// InstanceID is the instance's box.space._cluster id as the backed-up
	// snapshot records it: the backed-up instance's own on the master entry
	// of a replicaset, a member's on every other entry. Zero when the
	// fragment the entry came from carried none for it.
	InstanceID   uint32 `json:"instance_id,omitempty"`
	InstanceUUID string `json:"instance_uuid"`
	InstanceName string `json:"instance_name"`
	Hostname     string `json:"hostname"`
}

TopologyInstance is one expected instance in cluster topology.

type Vclock 

type Vclock map[uint32]uint64

Vclock maps replica IDs to their LSNs, including replica 0.

type Warning 

type Warning struct {
	Code    WarningCode    `json:"code"`
	Message string         `json:"message"`
	Details map[string]any `json:"details"`
}

Warning is a typed, serializable non-fatal backup issue.

func NewPromotedToFullWarning 

func NewPromotedToFullWarning(reason, message string) Warning

NewPromotedToFullWarning records why a full backup landed on top of an existing chain: an operator reading the storage months later cannot otherwise tell a scheduled full backup from one the cluster forced.

func NewRecoveryPointsUnavailableWarning 

func NewRecoveryPointsUnavailableWarning(replicasetUUID, errMsg string) Warning

NewRecoveryPointsUnavailableWarning reports unavailable recovery points.

func NewShardPartialWarning 

func NewShardPartialWarning(replicasetUUID, instanceUUID, reason string) Warning

NewShardPartialWarning reports a partial shard backup.

func NewShardUnreachableWarning 

func NewShardUnreachableWarning(replicasetUUID string) Warning

NewShardUnreachableWarning reports a missing shard fragment.

func NewStoragePartialUploadWarning 

func NewStoragePartialUploadWarning(keys []string) Warning

NewStoragePartialUploadWarning reports storage keys that failed to upload.

func PromotionWarning 

func PromotionWarning(head *ClusterManifest, plan *BackupPlan) *Warning

PromotionWarning reports why a full backup is landing on top of an existing chain, or nil when it is not landing on one or nothing changed.

The reason is derived rather than declared: an orchestrator asking for a full backup has no way to tell tt why, and the two manifests already say it. A scheduled full backup of an unchanged cluster produces no warning.

type WarningCode 

type WarningCode string

WarningCode identifies a non-fatal manifest issue. 

const (

	// WarnShardPartial marks a shard backup that completed only partially.
	WarnShardPartial WarningCode = "shard_partial"
	// WarnShardUnreachable marks a shard that did not produce a fragment.
	WarnShardUnreachable WarningCode = "shard_unreachable"
	// WarnRecoveryPointsUnavailable marks a missing recovery_points field.
	WarnRecoveryPointsUnavailable WarningCode = "recovery_points_unavailable"
	// WarnStoragePartialUpload marks archives that were not fully uploaded.
	WarnStoragePartialUpload WarningCode = "storage_partial_upload"
	// WarnPromotedToFull marks a full backup taken because an incremental one
	// had become impossible: the replicaset set or a master changed since the
	// backup this chain would have continued.
	WarnPromotedToFull WarningCode = "promoted_to_full"
)

func (WarningCode) Informational 

func (code WarningCode) Informational() bool

Informational reports whether a warning describes the backup rather than something missing from it. Only these leave the status alone: a full backup forced by a master change is as complete as a planned one.

Source Files

View all Source files

Directories

Path Synopsis
Package chain builds recovery chains from backup manifests.
 Package chain builds recovery chains from backup manifests.
Package gc deletes backup chains by retention rules and cleans up dangling archives.
 Package gc deletes backup chains by retention rules and cleans up dangling archives.
Package verify health-checks a backup storage: manifest chain integrity, archive presence and checksums, dangling archives.
 Package verify health-checks a backup storage: manifest chain integrity, archive presence and checksums, dangling archives.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.