dumps

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 26, 2026 License: MIT Imports: 11 Imported by: 0

Documentation

Overview

Package dumps implements the catalog of saved dump files. Dumps live on a Storage backend (filesystem in Phase B; cloud backends in Phase G); each entry has a sidecar <id>.meta.json carrying schema, checksum, etc.

Index

Constants

View Source 
const (
	EnvelopeMagic = "SIPH"
	EnvelopeSize  = 4096
)

Variables

View Source 
var ErrInvalidEnvelope = errors.New("invalid siphon envelope")

Functions

func WriteEnvelope 

func WriteEnvelope(w io.Writer, e *Envelope) (int, error)

WriteEnvelope writes a padded 4 KB header to w. Returns the number of bytes written (always 4096 on success).

Types

type Catalog 

type Catalog struct {
	// contains filtered or unexported fields
}

Catalog is a dump store addressed by dump ID. It holds a storage.Store substrate (local directory or object store) and maps each ID to two keys: "<id>.dump" (the dump body, envelope-prefixed) and "<id>.meta.json" (the sidecar metadata). Callers never see storage paths — only IDs.

func New 

func New(store storage.Store) *Catalog

New returns a Catalog over the given storage backend.

func NewCatalog 

func NewCatalog(root string) (*Catalog, error)

NewCatalog returns a Catalog backed by a local directory, creating it if missing. Retained for callers (and tests) that want the default local backend without constructing a storage.Store themselves.

func (*Catalog) Delete 

func (c *Catalog) Delete(ctx context.Context, id string) error

Delete removes both the dump body and its sidecar. Delete is idempotent in the store, so a missing object is not an error.

func (*Catalog) List 

func (c *Catalog) List(ctx context.Context) ([]Meta, error)

List returns metadata for every dump in the catalog, sorted newest first. It enumerates meta keys, reading each; a corrupt entry is skipped rather than failing the whole listing.

func (*Catalog) OpenDump 

func (c *Catalog) OpenDump(ctx context.Context, id string) (io.ReadCloser, error)

OpenDump opens the dump body for id as a one-shot forward stream. The caller must Close it. A missing dump maps to a CodeUser error.

func (*Catalog) PruneDryRun 

func (c *Catalog) PruneDryRun(ctx context.Context, p RetentionPolicy) (PruneReport, error)

PruneDryRun returns which dumps would be deleted under policy p without deleting anything. It groups the catalog into chains, runs the chain-aware retention engine, and flattens the plan back to dumps. Chain-aware: a base is only in Would when its whole chain is pruned, so an incremental is never orphaned.

func (*Catalog) PutDump 

func (c *Catalog) PutDump(ctx context.Context, id string, r io.Reader) error

PutDump streams r (the envelope-prefixed dump body) to the store under the dump key for id. The store guarantees the object is published atomically, so a failed or cancelled write never leaves a partial dump addressable by id.

func (*Catalog) ReadMeta 

func (c *Catalog) ReadMeta(ctx context.Context, id string) (*Meta, error)

ReadMeta loads and decodes the sidecar metadata for id.

func (*Catalog) ResolveChain 

func (c *Catalog) ResolveChain(ctx context.Context, targetID string) ([]Meta, error)

ResolveChain returns the ordered list of Metas that, applied in order, reconstruct targetID. Element 0 is the base; the last element is the target.

It walks ParentID backwards until it reaches a base (BaseID == ID, or a legacy empty BaseID). Cycles and broken chains (a missing parent) are reported as errors rather than looping forever or silently truncating.

func (*Catalog) WriteMeta 

func (c *Catalog) WriteMeta(ctx context.Context, m *Meta) error

WriteMeta serializes m and writes it under the meta key. Write meta LAST when publishing a dump: the catalog enumerates by meta, so a dump body without its meta is an invisible (prunable) orphan, whereas a meta without its body would be a dangling catalog entry.

type Chain 

type Chain struct {
	Root    string // root dump ID (the base), the chain's stable key
	Members []Meta // base first, then incrementals in apply order
}

Chain is a restorable unit: a base dump plus its ordered incrementals. The chain is the unit of retention — it is kept or pruned as a whole, so an incremental can never be orphaned from its base.

func GroupChains 

func GroupChains(dumps []Meta) []Chain

GroupChains folds a flat dump list into chains keyed by root BaseID. A full backup (BaseID == ID, or legacy empty BaseID) is a singleton chain; its incrementals attach to it via their BaseID. A dump whose root is missing from the set anchors its own chain rather than being dropped, so a partially present catalog never loses entries silently.

Within each chain, members are ordered by ParentID/BaseID TOPOLOGY (base first, then each child after its parent), with Created only as a tie-breaker. This is the contract the leaf-inward delete path relies on: deleting members last-to-first must remove every descendant before its ancestor. Ordering by Created alone would break that on tied or skewed timestamps — a child could sort ahead of its parent and the base could be deleted while a leaf survives.

type Envelope 

type Envelope struct {
	Siphon        string       `json:"siphon"`
	Type          EnvelopeType `json:"type"`
	Driver        string       `json:"driver"`
	EngineVersion string       `json:"engine_version,omitempty"`
	BaseID        string       `json:"base_id,omitempty"`
	ParentID      string       `json:"parent_id,omitempty"`
	WALStart      string       `json:"wal_start,omitempty"`
	WALEnd        string       `json:"wal_end,omitempty"`
	BinlogFile    string       `json:"binlog_file,omitempty"`
	BinlogStart   uint64       `json:"binlog_start,omitempty"`
	BinlogEnd     uint64       `json:"binlog_end,omitempty"`
	Checksum      string       `json:"checksum,omitempty"`
	Tables        []TableEntry `json:"tables,omitempty"`
	Created       time.Time    `json:"created"`
}

Envelope is the 4 KB JSON header prepended to every dump. The native dump bytes follow immediately after.

func ReadEnvelope 

func ReadEnvelope(r io.Reader) (*Envelope, io.Reader, error)

ReadEnvelope reads and validates the 4 KB header from r. Returns the parsed Envelope and a reader positioned at the start of the native dump bytes.

type EnvelopeType 

type EnvelopeType string
const (
	EnvelopeBase        EnvelopeType = "base"
	EnvelopeIncremental EnvelopeType = "incremental"
)

type GFSPolicy 

type GFSPolicy struct {
	Daily   int
	Weekly  int
	Monthly int
}

GFSPolicy is a grandfather-father-son retention rule: keep the newest chain in each of the most-recent Daily calendar days, Weekly ISO weeks, and Monthly calendar months. A zero field disables that tier; an all-zero GFSPolicy is off.

type Meta 

type Meta struct {
	ID            string            `json:"id"`
	Profile       string            `json:"profile"`
	Driver        string            `json:"driver"`
	EngineVersion string            `json:"engine_version"`
	DumpFormat    string            `json:"dump_format"`
	SizeBytes     int64             `json:"size_bytes"`
	Checksum      string            `json:"checksum"`
	Created       time.Time         `json:"created"`
	Tables        []TableEntry      `json:"tables"`
	Annotations   map[string]string `json:"annotations,omitempty"`

	// Incremental chain (populated in Phase F; nil for full backups in B).
	BaseID   string `json:"base_id,omitempty"`
	ParentID string `json:"parent_id,omitempty"`
}

Meta is the sidecar JSON written next to each dump file.

type PruneReport 

type PruneReport struct {
	Would []Meta
	Kept  []Meta
}

PruneReport contains the outcome of a dry-run prune operation, flattened to individual dumps (chain members of pruned chains are listed in Would, members of kept chains in Kept).

type RetentionPlan 

type RetentionPlan struct {
	Keep  []Chain
	Prune []Chain
}

RetentionPlan is the engine's decision: which chains to keep and which to prune, with no side effects.

func Plan 

func Plan(chains []Chain, p RetentionPolicy, now time.Time) RetentionPlan

Plan decides which chains to keep vs prune under p, as of now. It is pure: no I/O, no clock — now is injected — so every rule and edge case is unit-testable with synthetic fixtures. An empty policy keeps everything.

type RetentionPolicy 

type RetentionPolicy struct {
	KeepLast int           // keep the N newest chains (0 = rule off)
	MaxAge   time.Duration // keep chains younger than this (0 = rule off)
	GFS      GFSPolicy     // keep-by-calendar-bucket (all-zero = off)
}

RetentionPolicy decides which dump chains to keep. A chain is kept if it satisfies ANY active rule (union semantics): adding a rule can only ever protect more chains, never fewer, so a misconfiguration cannot silently delete data a rule meant to keep. An all-zero policy keeps everything (prune is a no-op) — the dangerous "delete everything" direction requires explicit configuration, never silence.

func (RetentionPolicy) IsEmpty 

func (p RetentionPolicy) IsEmpty() bool

IsEmpty reports whether no rule is active, i.e. the policy keeps everything.

type TableEntry 

type TableEntry struct {
	Name      string `json:"name"`
	Rows      int64  `json:"rows"`
	SizeBytes int64  `json:"size_bytes"`
}

TableEntry holds per-table statistics recorded at dump time.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.