Documentation
¶
Overview ¶
Package backup is the archive format of a sluis backup and its encryption. It holds no exporter and no restore logic: those write and read records through Writer and Reader.
Layout ¶
A backup is a set of objects in an S3-compatible bucket, written through a port.Blob, under
backup/<installation>/<id>/
manifest.json
state/<module>/<n>
secrets/<module>/<n>
blobs/<module>/<n>
The three sections hold the module's State records, its secrets and its blobs. Each is a run of chunks numbered from 0. A chunk is a sealed run of records; the records are opaque bytes to this package (the exporters define them). The manifest is written last, so a backup without a manifest is not a backup.
Sealing ¶
One data key is generated per backup with the configured key (keys.Archive; a KMS alias or an OpenBao transit key) under its configured encryption context, and stored wrapped in the manifest. Two keys are derived from it with HKDF-SHA-256: one per chunk, bound to the chunk's object name, and one for the manifest's MAC. A chunk is sealed with AES-256-GCM; its additional data names the installation, the backup id, the section, the module and the chunk number, so a chunk moved to another place, another backup or another position does not open. The manifest lists every chunk's SHA-256 (of the stored bytes) and size and is authenticated by an HMAC-SHA-256, so a changed, dropped, added or reordered chunk is found before any record is used.
Nothing in the manifest is secret: it holds the module names, counts, sizes and hashes, the wrapped data key and the creator. Every record, whether a State item, a secret value or a blob, is inside a sealed chunk; there is no plaintext copy of any of them in the archive. Without the right to decrypt with the key the data key is not recovered and Open fails closed.
Versions ¶
The manifest carries a format version. A reader opens the current version and the one before it (Readable); anything else is refused with ErrFormat. The golden archives under testdata keep that promise: a released format's archive stays in the tree and is opened by every later release.
Index ¶
- Constants
- Variables
- func ChunkName(installation, id string, s Section, m port.Module, n int) string
- func ManifestName(installation, id string) string
- func Prefix(installation, id string) string
- func Readable(v int) bool
- func ValidName(s string) bool
- type Chunk
- type Key
- type Layout
- type Manifest
- type ModuleEntry
- type Params
- type Part
- type Progress
- type Reader
- type Section
- type Writer
Constants ¶
const ChunkBytes = 8 << 20
ChunkBytes is the plaintext size a chunk is cut at: a chunk is flushed when the next record would take it past this, so a chunk is smaller unless one record is larger.
const Version = 1
Version is the format version this release writes.
Variables ¶
var ( // ErrFormat is an archive this release cannot read: not a manifest, an // unknown or too old version, or a field out of range. ErrFormat = errors.New("backup: unreadable archive") // ErrIntegrity is an archive that was changed, truncated or reordered, or // whose chunk does not open: the manifest's MAC, a chunk's hash, its // authentication, or a count did not match. ErrIntegrity = errors.New("backup: archive failed verification") // ErrKey is a data key that could not be recovered: the key refused to // decrypt, or the context differs. Nothing was read. ErrKey = errors.New("backup: cannot recover the data key") )
Errors a caller can match with errors.Is.
Functions ¶
func ManifestName ¶
ManifestName is the object name of a backup's manifest.
Types ¶
type Chunk ¶
type Chunk struct {
N int `json:"n"`
Records int `json:"records"`
Size int64 `json:"size"`
SHA256 string `json:"sha256"`
}
Chunk is one sealed object.
type Key ¶
type Key interface {
GenerateDataKey(ctx context.Context) (keys.DataKey, error)
UnwrapDataKey(ctx context.Context, wrapped []byte, opts ...keys.DecryptOption) ([]byte, error)
}
Key is what the archive needs from the configured key: a fresh wrapped data key to write with, and the unwrapping of one to read with. *keys.Key is one; it carries the encryption context, which this package neither adds to nor changes.
type Layout ¶
type Layout string
Layout names the storage layout of the installation that was backed up.
type Manifest ¶
type Manifest struct {
// Format is the archive format version (see [Version]).
Format int `json:"format"`
// Installation and ID name the backup; they are also in its path.
Installation string `json:"installation"`
ID string `json:"id"`
// Layout is the storage layout of the source installation.
Layout Layout `json:"layout"`
// Created is the start of the backup, UTC, RFC 3339.
Created string `json:"created"`
// Creator names what wrote it, for example "sluis-backup 1.75.0".
Creator string `json:"creator"`
// ChunkBytes is the plaintext size a chunk is cut at.
ChunkBytes int `json:"chunkBytes"`
// Modules lists the modules in the backup in [port.Modules] order, each
// once.
Modules []ModuleEntry `json:"modules"`
}
Manifest describes one backup. It is the authenticated body of manifest.json.
func Peek ¶
Peek reads a manifest without verifying it: nothing authenticates the result, so it is for listing and for showing a person what is there, never for deciding what to restore. Open verifies.
type ModuleEntry ¶
type ModuleEntry struct {
Module port.Module `json:"module"`
State Part `json:"state"`
Secrets Part `json:"secrets"`
Blobs Part `json:"blobs"`
}
ModuleEntry is one module's three sections.
type Params ¶
type Params struct {
Installation string
ID string
Layout Layout
// Creator names what writes it, for example "sluis-backup 1.75.0".
Creator string
// Now is the clock; time.Now when nil.
Now func() time.Time
// ChunkBytes overrides [ChunkBytes], for tests.
ChunkBytes int
}
Params describe the backup a Writer makes.
type Part ¶
type Part struct {
// Records is the number of records in all chunks.
Records int64 `json:"records"`
// Chunks lists the chunks in order; their numbers run from 0.
Chunks []Chunk `json:"chunks,omitempty"`
}
Part is one section of one module.
type Progress ¶
type Progress struct {
Wrapped []byte `json:"wrapped"`
Created string `json:"created"`
Modules []ModuleEntry `json:"modules"`
}
Progress is what a run must keep to continue after a stop: the wrapped data key and the chunks written so far. It holds no secret (the data key is wrapped) and no record.
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Reader reads one verified backup.
func Open ¶
func Open(ctx context.Context, store port.Blob, key Key, installation, id string, opts ...keys.DecryptOption) (*Reader, error)
Open reads a backup's manifest, recovers the data key with the key (an option may name the context the key was written under, see keys.WithContext) and verifies the manifest's MAC. No chunk is read; a key that may not decrypt fails here with ErrKey.
func (*Reader) Records ¶
func (r *Reader) Records(ctx context.Context, s Section, m port.Module, fn func(record []byte) error) error
Records calls fn with each record of a section of a module, in the order they were added. Each chunk is checked against the manifest (size, SHA-256, authentication, record count) before its records are given out. The slice passed to fn is valid only during the call. A module or section the backup does not hold has no records.
type Section ¶
type Section string
Section is one of the three kinds of content a module contributes.
The sections, in archive order.
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Writer writes one backup. It holds at most one chunk of records in memory. It is not safe for concurrent use.
func NewWriter ¶
NewWriter starts a backup: it asks the key for a data key. Nothing is written to the store until the first chunk is flushed.
func Resume ¶
func Resume(ctx context.Context, store port.Blob, key Key, p Params, prog Progress) (*Writer, error)
Resume continues a backup from the Progress a stopped Writer returned from Flush: it unwraps the same data key and takes the chunks as written. The caller must not add records twice; it resumes from the first record not in a chunk listed by the progress.
func (*Writer) Abort ¶
func (w *Writer) Abort()
Abort wipes the data key without writing a manifest. The chunks already written stay; without a manifest they are not a backup.
func (*Writer) Add ¶
Add appends one record to a section of a module. A change of section or module flushes the chunk in progress first.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package app assembles the backup module as a process of its own (docs/decisions/0071): the storage it reads an installation from, the archive bucket and key it writes to, the audit trail, and the job that runs under the backup lease (internal/backup/job).
|
Package app assembles the backup module as a process of its own (docs/decisions/0071): the storage it reads an installation from, the archive bucket and key it writes to, the audit trail, and the job that runs under the backup lease (internal/backup/job). |
|
Package export reads an installation through the ports and writes it into a backup archive (backup.Writer).
|
Package export reads an installation through the ports and writes it into a backup archive (backup.Writer). |
|
Package job runs the backup module's work: one backup under the backup lease, its status and retention records, the audit records, the listing of the archive and the prune.
|
Package job runs the backup module's work: one backup under the backup lease, its status and retention records, the audit records, the listing of the archive and the prune. |
|
Package restore writes a backup (backup.Reader) into an installation of layout v5, and shows beforehand what that would do.
|
Package restore writes a backup (backup.Reader) into an installation of layout v5, and shows beforehand what that would do. |
|
Package restorejob is the restore function's work: one restore at a time, asynchronous, with every module under maintenance while it writes (docs/decisions/0071 and 0072, point 9).
|
Package restorejob is the restore function's work: one restore at a time, asynchronous, with every module under maintenance while it writes (docs/decisions/0071 and 0072, point 9). |
|
Package rpc is the backup module as another module sees it (docs/decisions/0071): the status of the backups, the list of what the archive holds, and a backup now, as methods of the `backup` module over internal/modcall.
|
Package rpc is the backup module as another module sees it (docs/decisions/0071): the status of the backups, the list of what the archive holds, and a backup now, as methods of the `backup` module over internal/modcall. |