Documentation
¶
Overview ¶
Package migrate copies the service's State from any adapter to any other through the ports and the domain stores (ADR 0031), and is the whole of `sluis migrate`.
It copies through the BUSINESS interfaces, never raw bytes: a domain record and its credential are read from the source's own store (the ConfigMaps and Secrets of internal/kube, or the State records of internal/portstore) and written to the destination's, so a credential lands in the destination's Secrets, and a record lands in the layout its adapter keeps. The issuer's logins in progress (sessions, refresh tokens, the keyring's schedule) are not a domain store; they are State records and Index sets under `issuer:`, which are copied with the lifetime each has left, so nobody signs in again.
A run has three phases, and the first writes nothing:
- Plan: read both sides, and say for every item whether it is new on the destination, already there with the same value, or there with another one. Without Overwrite any difference is a conflict and the run stops here, naming the keys, before a single write.
- Copy what is new (and, with Overwrite, what differs), item by item.
- Verify: read the source and the destination again, fresh, and compare every source item with the destination's, secrets read from each side's Secrets.
A re-run after a failure plans again and so copies exactly what is missing: the copy is idempotent, and an item the destination already holds equal is never rewritten.
What is not copied: leases (transient, and owned by whoever is running), the hub's snapshots (a cache the hub rewrites on its first refresh), and the controllers' in-flight hand-offs and caches (`share.`, `cache.`, `dedupe.`), which a tick rebuilds.
Index ¶
- Constants
- Variables
- func BlobID(c store.Config) string
- func V3ConfigName(v2 string) string
- type BlobMode
- type ConcernSummary
- type ConfigStore
- type Domains
- type LayoutItem
- type Options
- type Problem
- type Report
- type SSMAPI
- type SSMConfig
- type SSMConfigAPI
- type SSMLayoutOptions
- type SSMLayoutReport
- type SecretsLayoutOptions
- type SecretsLayoutReport
- type Side
- type Step
Constants ¶
const ( DomainDirectory = "directory" DomainGitHub = "github" DomainSlack = "slack" DomainConsole = "console" DomainIssuer = "issuer" DomainBlobs = "blobs" )
The domains a run can skip by name.
Variables ¶
var ( // ErrLayoutConflict is a v4 address that already holds something other // than what the v3 item says. Nothing is written. ErrLayoutConflict = errors.New("migrate: a v4 address already holds a different value") // ErrLayoutMismatch is a write that did not read back equal. ErrLayoutMismatch = errors.New("migrate: a v4 address does not read back as the v3 item") // ErrNotV4 is a request to delete v3 while a consumer could still read it. ErrNotV4 = errors.New("migrate: v3 is deleted only once the installation runs on layout v4: " + "set secrets.layout to v4, roll every consumer, and run this again") // ErrV4Incomplete is a v3 item with nothing at its v4 address. ErrV4Incomplete = errors.New("migrate: v3 holds items that v4 does not: run the move again before deleting v3") )
Errors a layout run ends with. The report says the rest.
var ( // ErrWritersRunning is a run that was not told the source is quiet. ErrWritersRunning = errors.New("migrate: refusing to copy while the source can still change: " + "stop every writer first (scale the issuer, the console and both controllers to 0) and pass --i-have-stopped-writers; " + "--dry-run reads and writes nothing and needs neither") // ErrConflict is a destination that already holds a different value. ErrConflict = errors.New("migrate: the destination already holds a different value") // ErrRefused is a source item the destination would refuse: nothing is // written until it is dealt with, and --overwrite does not change that. ErrRefused = errors.New("migrate: the destination would refuse source items") // ErrMismatch is a copy that did not verify. ErrMismatch = errors.New("migrate: the destination does not match the source after the copy") // ErrUnreadable is a source item that cannot be presented through its store. ErrUnreadable = errors.New("migrate: the source holds items its store cannot read") )
Errors a run ends with. The report says the rest.
var AllDomains = []string{DomainDirectory, DomainGitHub, DomainSlack, DomainConsole, DomainIssuer, DomainBlobs}
AllDomains is every name a run accepts to skip.
Functions ¶
func BlobID ¶
BlobID says where a configuration keeps its blobs, so that two sides that keep them in one place are known to: the same S3 bucket and prefix, or, with no `ports.blob`, the same release's objects. The memory adapter keeps them nowhere another process reads, so it is never "the same".
func V3ConfigName ¶ added in v1.62.0
V3ConfigName is the v3 name of a configuration secret v2 kept at `<root>/private/config/<v2>`: the Google OAuth client becomes the provider `default`, and a client's secret gains its `/secret`. Every other name is the same in both.
Types ¶
type BlobMode ¶
type BlobMode string
BlobMode says whether the controllers' reports are copied.
const ( // BlobsAuto copies them only when the two sides keep them in different // places. When they share one the next pass rewrites them anyway. BlobsAuto BlobMode = "auto" // BlobsCopy always copies them. BlobsCopy BlobMode = "copy" // BlobsSkip never does. BlobsSkip BlobMode = "skip" )
The blob modes.
type ConcernSummary ¶ added in v1.58.0
type ConcernSummary struct {
Concern string `json:"concern"`
Items int `json:"items"`
Bytes int `json:"bytes"`
Refused int `json:"refused,omitempty"`
}
ConcernSummary is what a run moves for one concern of the installation: state records, secrets, blobs. Items and Bytes are what the source holds and the run would write; Refused is what the destination would not take.
type ConfigStore ¶ added in v1.74.0
type ConfigStore interface {
// Get returns the text of name; ok is false when it is absent.
Get(ctx context.Context, name string) (value string, ok bool, err error)
// Create writes name only if it is absent.
Create(ctx context.Context, name, value string) error
}
ConfigStore is `<root>/internal/config/`, the parameters an operator seeds. The secrets source reads them as plain text, so they are not documents and do not go through a state.Store.
type Domains ¶
type Domains struct {
Workspaces hub.Store
Credentials hub.CredentialStore
Orgs orgStore
Links linkStore
Runner runnerStore
Catalogue catalogueStore
SlackApps slackCatalogueStore
Slack slackStore
Channels channelStore
SessionKey sessionKeys
}
Domains is every domain store of one side.
func OpenDomains ¶
OpenDomains builds the domain stores of one side by the switch the service itself makes (internal/app): any adapter but `legacy` keeps them on the State port with every credential in Secrets, and `legacy` keeps today's ConfigMaps and Secrets. A source is opened as it is; a destination (create) also makes the objects a legacy store needs to exist, as the service does at its start.
func OpenDomainsExporting ¶ added in v1.74.0
func OpenDomainsExporting(ctx context.Context, st *store.Stores, create bool, exported func(id string) bool) (*Domains, error)
OpenDomainsExporting is OpenDomains for a side that may be on layout v4: an installed runner App, and a catalogue App for which exported says true, is the document external/github/<app> there and not an internal credential, and a Slack App's bot token is external/slack/<app> (ADR 0041). exported may be nil (no catalogue App is exported). On layout v3 it changes nothing.
type LayoutItem ¶ added in v1.74.0
type LayoutItem struct {
// Source is the v3 Secrets path, or `state:<record>` for an App whose ids
// are in State.
Source string `json:"source"`
// Address is where it goes in v4: `internal/...`, `external/<kind>/<id>`
// or `internal/config/...`.
Address string `json:"address"`
// Action is new, resume (a rotation half written), unchanged, conflict,
// or, for the deletion, delete or missing.
Action string `json:"action"`
// V3Version and V4Revision are the versions a conflict is between.
V3Version string `json:"v3Version,omitempty"`
V4Revision string `json:"v4Revision,omitempty"`
Note string `json:"note,omitempty"`
}
LayoutItem is one v3 item and what the run found or did.
type Options ¶
type Options struct {
// DryRun plans and reports and writes nothing, the destination's objects
// included.
DryRun bool
// Overwrite replaces what the destination holds with the source's. Without
// it a destination value that differs fails the run, naming the key.
Overwrite bool
// WritersStopped is the operator's statement that nothing writes to the
// source. A copy is a snapshot: a write during it is a copy of neither
// state. A run that writes refuses without it.
WritersStopped bool
// Sessions also copies the issuer's logins in progress (sessions, refresh
// tokens, the codes in flight, the Index sets). An installation's move does
// not: people sign in again, and only the key ring's schedule is copied, so
// that a token issued before the move keeps verifying.
Sessions bool
// Skip names domains left out (see the Domain constants).
Skip []string
// Blobs says whether the controllers' reports are copied.
Blobs BlobMode
// ReportBlob, when set, is a Blob name the report is also written to on
// the destination (not by a dry run).
ReportBlob string
// ExportedGitHubApp says which catalogue GitHub Apps of the destination have
// `export: true` (layout v4). Nil exports none; a runner App is always
// exported.
ExportedGitHubApp func(id string) bool
// Log receives progress. Nil is silent.
Log *slog.Logger
// Now is the clock the copied lifetimes are measured against; nil is
// time.Now. For tests, which inject the delay between reading and writing.
Now func() time.Time
}
Options is how a run behaves.
type Problem ¶
type Problem struct {
Domain string `json:"domain"`
Kind string `json:"kind"`
Key string `json:"key"`
Reason string `json:"reason"`
}
Problem names one item that is not as it should be. Key is the item's key within its kind; for the issuer's state, whose keys carry bearer values (a code, a session), it is a short hash of the key so that a report is safe to keep.
type Report ¶
type Report struct {
From string `json:"from"`
To string `json:"to"`
FromAdapter string `json:"fromAdapter"`
ToAdapter string `json:"toAdapter"`
DryRun bool `json:"dryRun"`
Overwrite bool `json:"overwrite"`
StartedAt time.Time `json:"startedAt"`
FinishedAt time.Time `json:"finishedAt"`
Steps []Step `json:"steps"`
Totals Step `json:"totals"`
Conflicts []Problem `json:"conflicts,omitempty"`
Unreadable []Problem `json:"unreadable,omitempty"`
Mismatches []Problem `json:"mismatches,omitempty"`
// Refused are the items the destination would refuse, and why.
Refused []Problem `json:"refused,omitempty"`
// Concerns is the summary per concern: state, secrets, blobs.
Concerns []ConcernSummary `json:"concerns,omitempty"`
// Notes say what was left out and why (a domain skipped, a side with
// nothing to read).
Notes []string `json:"notes,omitempty"`
// Error is how the run ended, if it did not end well.
Error string `json:"error,omitempty"`
// OK is true when the plan had no conflict, nothing unreadable, and, unless
// this was a dry run, every item verified.
OK bool `json:"ok"`
}
Report is what a run says it did: counts and the keys of what went wrong, never a value.
type SSMAPI ¶ added in v1.62.0
type SSMAPI interface {
GetParametersByPath(ctx context.Context, in *awsssm.GetParametersByPathInput, opts ...func(*awsssm.Options)) (*awsssm.GetParametersByPathOutput, error)
PutParameter(ctx context.Context, in *awsssm.PutParameterInput, opts ...func(*awsssm.Options)) (*awsssm.PutParameterOutput, error)
DescribeParameters(ctx context.Context, in *awsssm.DescribeParametersInput, opts ...func(*awsssm.Options)) (*awsssm.DescribeParametersOutput, error)
}
SSMAPI is the part of the SSM client the layout migration calls.
type SSMConfig ¶ added in v1.74.0
type SSMConfig struct {
API SSMConfigAPI
Root string
// KMSKeyID encrypts what is written; empty is the AWS-managed key.
KMSKeyID string
}
SSMConfig is the ConfigStore over `<root>/internal/config/` in Parameter Store: plain-text SecureStrings, which the secrets source reads as they are.
func (SSMConfig) Create ¶ added in v1.74.0
Create implements ConfigStore: SSM refuses a parameter that exists.
type SSMConfigAPI ¶ added in v1.74.0
type SSMConfigAPI interface {
GetParameter(ctx context.Context, in *awsssm.GetParameterInput, opts ...func(*awsssm.Options)) (*awsssm.GetParameterOutput, error)
PutParameter(ctx context.Context, in *awsssm.PutParameterInput, opts ...func(*awsssm.Options)) (*awsssm.PutParameterOutput, error)
}
SSMConfigAPI is the part of the SSM client SSMConfig calls.
type SSMLayoutOptions ¶ added in v1.62.0
type SSMLayoutOptions struct {
// From is the v2 root (`/sluis`), To the v3 one (`/sluis/<instance>`).
From, To string
// KMSKeyID encrypts what is written. Empty keeps each parameter's own key:
// a customer key is never silently replaced by the AWS-managed one.
KMSKeyID string
// DryRun plans and writes nothing; Overwrite replaces a parameter that
// is already there with a different value.
DryRun, Overwrite bool
}
SSMLayoutOptions is one run of the configuration secrets' move to layout v3.
type SSMLayoutReport ¶ added in v1.62.0
type SSMLayoutReport struct {
// Account and Region are where it ran, as STS and the SDK say: what an
// operator checks before trusting a dry run.
Account string `json:"account,omitempty"`
Region string `json:"region,omitempty"`
From string `json:"from"`
To string `json:"to"`
Copied []string `json:"copied"`
Unchanged []string `json:"unchanged"`
Conflicts []string `json:"conflicts,omitempty"`
DryRun bool `json:"dryRun"`
}
SSMLayoutReport is what a run did, by v3 name. It holds names, never a value.
func MoveSSMLayout ¶ added in v1.62.0
func MoveSSMLayout(ctx context.Context, api SSMAPI, o SSMLayoutOptions) (SSMLayoutReport, error)
MoveSSMLayout copies every configuration secret under the v2 root to its v3 name under the new root. It writes only where the destination is absent or, with Overwrite, different; the source is left as it is, to be deleted by the operator once the installation runs on v3.
type SecretsLayoutOptions ¶ added in v1.74.0
type SecretsLayoutOptions struct {
// V3 is the layout-v3 Secrets port of the installation (its `private/`
// tree, not the layout-aware one), State the port its App records are in.
V3 port.Secrets
State port.State
// Dest are the v4 stores, Config the plain-text parameters.
Dest *secretstore.Stores
Config ConfigStore
// ExportedGitHubApp says which catalogue GitHub Apps have `export: true`
// (a runner App is always exported). Nil exports none.
ExportedGitHubApp func(id string) bool
// DryRun plans and writes nothing.
DryRun bool
// Layout is the installation's `secrets.layout` (DeleteV3 only).
Layout string
// Now is the clock a rotated client's previous secret is judged by.
Now func() time.Time
}
SecretsLayoutOptions is one run of the move to layout v4.
type SecretsLayoutReport ¶ added in v1.74.0
type SecretsLayoutReport struct {
To string `json:"to,omitempty"`
DryRun bool `json:"dryRun"`
Items []LayoutItem `json:"items"`
New int `json:"new"`
Unchanged int `json:"unchanged"`
Conflicts int `json:"conflicts,omitempty"`
Written int `json:"written"`
Verified int `json:"verified"`
Deleted int `json:"deleted,omitempty"`
// Skipped are v3 paths not migrated on purpose (the retired exports
// copies); DeleteV3 removes them.
Skipped []string `json:"skipped,omitempty"`
OK bool `json:"ok"`
Error string `json:"error,omitempty"`
}
SecretsLayoutReport is what a run says it did: sources, addresses and versions, never a value.
func DeleteV3 ¶ added in v1.74.0
func DeleteV3(ctx context.Context, o SecretsLayoutOptions) (*SecretsLayoutReport, error)
DeleteV3 removes the v3 items once v4 holds them. It refuses unless the installation is on layout v4 and every v3 item has its v4 address.
func MoveSecretsLayout ¶ added in v1.74.0
func MoveSecretsLayout(ctx context.Context, o SecretsLayoutOptions) (*SecretsLayoutReport, error)
MoveSecretsLayout backfills every v3 item into its v4 address.
func (*SecretsLayoutReport) JSON ¶ added in v1.74.0
func (r *SecretsLayoutReport) JSON() []byte
JSON is the report as the command prints it.
type Side ¶
type Side struct {
// Name labels it in the report: the configuration file it came from.
Name string
// Stores is the opened ports.
Stores *store.Stores
// BlobID identifies where this side keeps blobs, so that two sides that
// share a place are known to; empty is "nowhere shared".
BlobID string
// SecretsLayout is the `secrets.layout` of an ssm secrets source (v3, the
// default, transition or v4); empty when the secrets are not in ssm.
SecretsLayout string
}
Side is one end of a run: the ports and stores of a configuration, opened.
type Step ¶
type Step struct {
Domain string `json:"domain"`
Kind string `json:"kind"`
// Source is how many readable items the source holds.
Source int `json:"source"`
// Unreadable is how many it holds that its store cannot present: they are
// listed and not copied.
Unreadable int `json:"unreadable,omitempty"`
// New is how many the destination lacks, Present how many it already holds
// with the same value, and Conflicts how many it holds with another.
New int `json:"new"`
Present int `json:"present"`
Conflicts int `json:"conflicts,omitempty"`
// Copied is how many were written: New, and with Overwrite the Conflicts.
Copied int `json:"copied"`
// Verified is how many matched on the second read; Mismatched how many did
// not.
Verified int `json:"verified"`
Mismatched int `json:"mismatched,omitempty"`
// Refused is how many the destination would refuse (an item over a size
// limit, an invalid key): they are listed under refused and stop the run
// before anything is written.
Refused int `json:"refused,omitempty"`
// Bytes is the size of the readable source items (a record with its
// credentials, a blob, a State value); Secrets and SecretBytes are the
// credentials among them, which go to the Secrets port.
Bytes int `json:"bytes"`
Secrets int `json:"secrets,omitempty"`
SecretBytes int `json:"secretBytes,omitempty"`
}
Step is the outcome of one collection.