Documentation
¶
Overview ¶
Package migrate moves a live node from one isolation mode to another. The orchestration is written against isolation.Backend so it is independent of any particular mode, and every host or cluster side effect it needs is a hook supplied by the caller.
Index ¶
Constants ¶
const JournalVersion = 1
JournalVersion is the schema version of a Journal.
Variables ¶
var ErrInterrupted = errors.New("isolation migration was interrupted")
ErrInterrupted is returned, joined with the outcome of the rollback, by ResumeInPlace when it undid a migration that a restart interrupted.
var ErrNoJournal = errors.New("no migration journal")
ErrNoJournal reports that no migration is recorded.
var ErrRollbackFailed = errors.New("rollback to the source failed")
ErrRollbackFailed is returned, joined with the original failure, when a failed migration could not restore the source. The node needs attention.
Functions ¶
This section is empty.
Types ¶
type InPlaceSpec ¶
type InPlaceSpec struct {
Source isolation.Backend
// SourceRoot names the slot the source runs in.
SourceRoot isolation.SlotRef
Target isolation.Backend
// TargetSpec names the alternate slot and the node to prepare.
TargetSpec isolation.NodeSpec
// Preflight checks the target backend's requirements before anything is
// mutated.
Preflight func(ctx context.Context, target isolation.Backend) error
// Download fetches the artifacts the target needs. Optional.
Download func(ctx context.Context) error
// SnapshotConfig saves the host configuration files both stacks write, before
// the first step that changes the host. Optional; when set, RestoreConfig
// should be too. A failure ends the migration with nothing changed.
SnapshotConfig func(ctx context.Context) error
// RestoreConfig puts the saved files back during a rollback, after the target
// is gone and before the source starts. It reports nil when there is no
// snapshot to restore. The snapshot is kept when it fails.
RestoreConfig func(ctx context.Context) error
// DiscardConfig removes the snapshot once the migration is over, after a
// success or a completed rollback.
DiscardConfig func(ctx context.Context) error
// CleanupNetwork removes host networking the source left behind, as a
// repave does.
CleanupNetwork func(ctx context.Context) error
// PersistTarget records the new mode, slot and owning backend.
PersistTarget func(ctx context.Context, target isolation.SlotRef) error
// RestoreSource puts the source's applied config back after PersistTarget
// when the migration rolls back. Required whenever PersistTarget is set.
RestoreSource func(ctx context.Context) error
// WaitReady blocks until the target node stack is up on the host or a
// bounded timeout passes. Cordoning, draining and uncordoning the Node
// belong to the operator, not to the migration.
WaitReady func(ctx context.Context) error
// RemoveSourceConfig removes the source's applied config after success.
RemoveSourceConfig func(ctx context.Context) error
// PrepareHost applies the host preparation a bootstrap in the target mode
// would have, once the target is up: the mode's boot-time firewall flush
// unit and package-manager guards. Optional. Like the source cleanup, a
// failure does not undo the migration; it leaves the migration short of
// StepComplete, so a resume runs it again.
PrepareHost func(ctx context.Context) error
// Record, when set, is called before each step that changes the host, with
// the step about to run (and with StepReady and StepComplete after the
// matching work). It is a write-ahead log: when Record returns an error the
// step does not run and the migration is rolled back like any failure.
// Recording is what lets ResumeInPlace decide what to do after a crash.
Record func(ctx context.Context, step Step) error
// TargetRoot names the target slot. Required by ResumeInPlace, which has no
// Prepare result to take it from; RunInPlace uses it only to clean up after
// a failed Prepare.
TargetRoot isolation.SlotRef
}
InPlaceSpec describes an in-place migration. Hooks may be nil only where noted; a nil required hook is a programming error reported by Run.
type Journal ¶
type Journal struct {
Version int `json:"version"`
Strategy Strategy `json:"strategy"`
From isolation.Mode `json:"from"`
To isolation.Mode `json:"to"`
// SourceSlot and TargetSlot are the blue/green slots of an in-place
// migration. Reprovision leaves them empty.
SourceSlot string `json:"sourceSlot,omitempty"`
TargetSlot string `json:"targetSlot,omitempty"`
Step Step `json:"step"`
StartedAt time.Time `json:"startedAt"`
UpdatedAt time.Time `json:"updatedAt"`
// Attempts counts the runs of a reprovision handoff that started, so the
// handoff can give up after a bounded number of them.
Attempts int `json:"attempts,omitempty"`
// Message is the last failure the handoff command saw, for operators.
Message string `json:"message,omitempty"`
}
Journal is the small state file of a migration in progress. It is written atomically before each step (see InPlaceSpec.Record), so a daemon that restarts mid-migration can tell what may have happened and resume or roll back. It holds no credentials.
type JournalStore ¶
type JournalStore struct {
Path string
// Now overrides the clock. Nil means time.Now.
Now func() time.Time
}
JournalStore reads and writes a Journal at one path.
func (JournalStore) Load ¶
func (s JournalStore) Load() (Journal, error)
Load reads the journal. A missing file is ErrNoJournal. A file that does not parse is an error and not "no journal": the daemon must not carry on as if no migration were under way when one may be.
func (JournalStore) Remove ¶
func (s JournalStore) Remove() error
Remove deletes the journal and syncs its directory, so a removed journal does not come back after a crash. A missing file is not an error.
func (JournalStore) Save ¶
func (s JournalStore) Save(j Journal) error
Save writes j atomically and durably, so a crash leaves the previous journal or the new one and never a partial file.
type Result ¶
type Result struct {
// Target is the slot the node now runs in. Nil unless the migration
// succeeded.
Target isolation.SlotRef
// RolledBack is true when a failure was followed by a restart of the source.
RolledBack bool
}
Result reports how a migration ended.
func ResumeInPlace ¶
ResumeInPlace decides what to do with an in-place migration a restart interrupted, from the last step Record saw. Before the source was stopped it only removes the partial target. Once the source may be stopped it rolls back to the source and returns ErrInterrupted, since repeating a step that crashed the process could loop. When the target was already running and recorded it waits for the target as the original run would, then finishes or rolls back.
func RunInPlace ¶
func RunInPlace(ctx context.Context, s InPlaceSpec) (Result, error)
RunInPlace performs the in-place steps from the design. Nothing is mutated when preflight or download fails. Once the target exists, any failure destroys it and, if the source was stopped, starts the source again. The source root is retained until WaitReady succeeds.
type Step ¶
type Step string
Step is how far an in-place migration got. Each value is written before the step it names starts, so after a crash it says what may have happened, not what finished.
StepRolledBack is terminal: the migration failed and the node runs in its original mode again. StepFailed is terminal too: the migration failed and the node could not be put back, so it needs attention. In both the journal's Message says why, until whoever reports the outcome removes it.
const ( // StepPreparing precedes Prepare: a target slot may be partially built. StepPreparing Step = "preparing" // StepStoppingSource precedes the source stop: the source may be stopped. StepStoppingSource Step = "stopping-source" // StepStartingTarget precedes the target start: the target may be running. StepStartingTarget Step = "starting-target" // StepPersisting precedes PersistTarget: the target's applied config may // exist. StepPersisting Step = "persisting" // StepAwaitingReady follows PersistTarget: the target runs and is recorded. StepAwaitingReady Step = "awaiting-ready" // StepReady follows a successful WaitReady: only source cleanup is left. StepReady Step = "ready" // StepComplete follows source cleanup. StepComplete Step = "complete" )
const StepBootstrapped Step = "bootstrapped"
StepBootstrapped is the last step of a reprovision migration: the node runs in the target mode and only the report in the Machine status is left.
const StepBootstrapping Step = "bootstrapping"
StepBootstrapping follows the reset of a reprovision migration: the old node is gone and bootstrap in the target mode is running or retrying.
const StepReprovisioning Step = "reprovisioning"
StepReprovisioning is the step of a reprovision mode change whose reset and bootstrap are carried out by the handoff command; the daemon that started the migration is gone by then.
type Strategy ¶
type Strategy string
Strategy is how a migration is carried out.