Documentation
¶
Overview ¶
Package export reads an installation through the ports and writes it into a backup archive (backup.Writer). It is the producer half of the archive: the restore engine decodes what it writes with Decode.
What is exported ¶
For each module of layout v5:
- state: every State record the module owns, with the lifetime it has left, and, for oidc, every Index set. The leases, notifications, gates, caches and dedupe records of the controllers, and the maintenance record, are not exported: they are rebuilt, or (the maintenance record) managed by the restore itself.
- secrets: every secret under internal/<module> and, where the module has an external namespace, external/<module>, as the stored document (its latest version).
- blobs: the objects under the module's blob prefixes (BlobPrefixes); an object over BlobPart bytes is split into parts, each its own record.
Records ¶
A record is one JSON object and the archive's chunks hold it as opaque bytes. Every record carries `v` (the record version, RecordVersion) and `t` (its type), so it is read without the section that held it:
state {"v":1,"t":"state","key":..., "value":<base64>, "expires":<RFC 3339>}
index {"v":1,"t":"index","key":..., "members":[...], "expires":<RFC 3339>}
secret {"v":1,"t":"secret","ns":"internal|external","name":..., "doc":<the stored JSON object>}
blob {"v":1,"t":"blob","name":..., "size":N, "sha256":..., "part":i, "parts":n, "data":<base64>}
A State key is the logical key, which port.Locate5 addresses; `expires` is absolute and absent for a permanent record, so a restore later computes what is left. The state section holds state and index records, the secrets section secret records and the blobs section blob records. The values of secrets and records appear only in the sealed chunks.
Resuming ¶
A run is a fixed list of units (one State family, one set family, one secret namespace or one blob prefix of one module). After each unit the run flushes the writer and hands a Checkpoint to Options.Checkpoint. A run that was stopped starts again from the last checkpoint with backup.Resume and Options.Resume: the unit that was in progress is read again from its start, and the chunks it had already written are written over, so the order in which an adapter lists records does not matter.
Index ¶
Constants ¶
const ( TypeState = "state" TypeIndex = "index" TypeSecret = "secret" TypeBlob = "blob" )
Record types.
const ( Internal = "internal" External = "external" )
The secret namespaces.
const BlobPart = 8 << 20
BlobPart is the size of the pieces a blob is split into.
const RecordVersion = 1
RecordVersion is the version of the record encoding this release writes.
Variables ¶
var BlobPrefixes = map[port.Module][]string{ port.ModuleGitHub: {blobprefix.GitHubReports}, port.ModuleSlack: {blobprefix.SlackReports}, }
BlobPrefixes are the blob prefixes each module owns and a backup keeps: what each controller last reported. The hub's directory snapshots (`google/`) are a cache that the module rewrites on its next refresh and are not kept.
var ErrRecord = errors.New("export: unreadable record")
ErrRecord is a record that cannot be decoded: not JSON, an unknown version or type, a type that does not belong in the section, or a field out of range.
Functions ¶
Types ¶
type Assembler ¶
type Assembler struct {
// contains filtered or unexported fields
}
Assembler puts the parts of blobs back together. Parts of one object arrive in order, one object after another, as they were written; memory is one object.
type Blob ¶
type Blob struct {
V int `json:"v"`
T string `json:"t"`
Name string `json:"name"`
// Size and SHA256 are the whole object's.
Size int64 `json:"size"`
SHA256 string `json:"sha256"`
// Part counts from 0 of Parts.
Part int `json:"part"`
Parts int `json:"parts"`
Data []byte `json:"data"`
}
Blob is one part of one object.
type Checkpoint ¶
type Checkpoint struct {
// Units is the length of the run's list of units; a resume refuses another.
Units int `json:"units"`
// Next is the number of units done.
Next int `json:"next"`
Progress backup.Progress `json:"progress"`
}
Checkpoint is what a run keeps to continue after a stop: the number of units done and the writer's backup.Progress. It holds no secret and no record.
type Index ¶
type Index struct {
V int `json:"v"`
T string `json:"t"`
Key string `json:"key"`
Members []string `json:"members"`
Expires string `json:"expires,omitempty"`
}
Index is one Index set.
type Options ¶
type Options struct {
// Modules limits the run; the default is every module of [port.Modules].
// A resume names the same ones.
Modules []port.Module
// Now is the clock the remaining lifetimes are made absolute by.
Now func() time.Time
// Resume continues from the last [Checkpoint] seen. The writer must be
// one made with [backup.Resume] from the checkpoint's Progress.
Resume *Checkpoint
// Checkpoint is called after each unit with what a resume needs. An error
// stops the run.
Checkpoint func(context.Context, Checkpoint) error
// Log receives one line per unit, with counts only. Nil is silent.
Log *slog.Logger
}
Options are the optional parts of a run.
type Secret ¶
type Secret struct {
V int `json:"v"`
T string `json:"t"`
// NS is [Internal] or [External]; Name is the path below
// <ns>/<module>.
NS string `json:"ns"`
Name string `json:"name"`
// Doc is the stored JSON object, the latest version.
Doc json.RawMessage `json:"doc"`
}
Secret is one secret as the store holds it.
type Source ¶
type Source struct {
// State and Index read every module's records and sets; the Router of the
// DynamoDB adapter is both.
State port.StateExporter
Index port.IndexExporter
// Secrets are the installation's secrets on layout v5.
Secrets *secretstore.StoresV5
// Blob holds the modules' blobs.
Blob port.Blob
}
Source is what an export reads. All four are required.
Source Files