nodemove

package
v1.3.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 16, 2026 License: Apache-2.0 Imports: 9 Imported by: 0

Documentation

Overview

Package nodemove runs persisted, resumable moves of apps and databases between nodes, and node drains built on those moves. Every phase is idempotent and recorded before the next starts, so a restart resumes.

Index

Constants

View Source
const (
	PhasePreflight   = "preflight"
	PhaseStopSource  = "stop_source"
	PhaseCopyVolumes = "copy_volumes"
	PhasePlace       = "place"
	PhaseVerify      = "verify"
	PhaseStopOld     = "stop_old"
	PhaseFinalize    = "finalize"
	PhaseRollback    = "rollback"
	PhaseDone        = "done"
)

Phases of a move, in order. Stateless moves skip stop_source and copy_volumes, stateful ones skip stop_old (the source is already stopped).

View Source
const (
	OutcomeOK      = "ok"
	OutcomeRetry   = "retry"
	OutcomeFailed  = "failed"
	OutcomeWaiting = "waiting"
)

Step outcomes recorded in a move's history.

Variables

This section is empty.

Functions

func IsPermanent

func IsPermanent(err error) bool

IsPermanent reports whether err was marked with Permanent.

func Permanent

func Permanent(err error) error

Permanent marks err as not worth retrying.

Types

type BlockedError

type BlockedError struct{ Reason string }

BlockedError means a resource cannot be moved by a drain at all.

func (*BlockedError) Error

func (e *BlockedError) Error() string

type Config

type Config struct {
	VerifyTimeout time.Duration
	MaxAttempts   int
	Interval      time.Duration
}

Config tunes the runner; ConfigFromEnv reads it from APP_NODE_MOVE_*.

func ConfigFromEnv

func ConfigFromEnv(lookup func(string) (string, bool)) Config

ConfigFromEnv reads APP_NODE_MOVE_VERIFY_TIMEOUT, APP_NODE_MOVE_MAX_ATTEMPTS and APP_NODE_MOVE_INTERVAL, falling back to defaults on unset or invalid values.

type DrainPlanner

type DrainPlanner interface {
	Cordon(ctx context.Context, nodeID string) error
	// Remaining lists what still runs on nodeID, databases before apps so
	// an app can follow its database.
	Remaining(ctx context.Context, nodeID string) ([]Resource, error)
	// PickTarget chooses where res goes, honoring explicit when set. A
	// *BlockedError skips res and records why.
	PickTarget(ctx context.Context, res Resource, fromNodeID, explicit string) (string, error)
	// Blocked lists resources on nodeID a drain can never move (e.g. models).
	Blocked(ctx context.Context, nodeID string) ([]store.NodeDrainBlocked, error)
}

DrainPlanner supplies what a drain needs from the platform.

type Executor

type Executor interface {
	// Preflight refuses a move that cannot work (wrap with Permanent) and
	// prepares dependencies such as a database's mesh endpoint.
	Preflight(ctx context.Context, m store.NodeMove) error
	// Stop suspends the resource and removes its containers from nodeID.
	Stop(ctx context.Context, kind, name, nodeID string) error
	CopyVolume(ctx context.Context, fromNodeID, toNodeID, volume string) error
	Place(ctx context.Context, kind, name, nodeID string) error
	Resume(ctx context.Context, kind, name string) error
	// Ready reports whether the resource runs and passes readiness on
	// nodeID, judged only on observations made after since.
	Ready(ctx context.Context, kind, name, nodeID string, since time.Time) (bool, string, error)
	// RemoveFrom removes the resource's containers from nodeID only.
	RemoveFrom(ctx context.Context, kind, name, nodeID string) error
	// Finalize runs after a verified move, e.g. restarting a database's dependents.
	Finalize(ctx context.Context, m store.NodeMove) error
}

Executor performs the side effects of each phase against the platform. Every method must be idempotent: a crash between an action and the phase save replays the action.

type Option

type Option func(*Runner)

Option configures a Runner.

func WithClock

func WithClock(now func() time.Time) Option

WithClock overrides time.Now, for tests.

func WithDrainPlanner

func WithDrainPlanner(p DrainPlanner) Option

WithDrainPlanner enables drains.

func WithIDs

func WithIDs(f func(prefix string) (string, error)) Option

WithIDs overrides ID generation, for tests.

type Request

type Request struct {
	Kind       string
	Resource   string
	FromNodeID string
	ToNodeID   string
	Volumes    []string
	DrainID    string
}

Request starts one move.

type Resource

type Resource struct {
	Kind    string
	Name    string
	Volumes []string
}

Resource is one app or database a drain must move off a node.

type Runner

type Runner struct {
	// contains filtered or unexported fields
}

Runner advances moves and drains.

func NewRunner

func NewRunner(s Store, exec Executor, cfg Config, logger *slog.Logger, opts ...Option) *Runner

NewRunner builds a Runner.

func (*Runner) Advance

func (r *Runner) Advance(ctx context.Context, id string) (store.NodeMove, error)

Advance runs phases of one move until it waits, finishes, or a phase needs a retry. It returns the persisted move.

func (*Runner) AdvanceDrain

func (r *Runner) AdvanceDrain(ctx context.Context, id string) error

AdvanceDrain moves the next resource off the drained node, one at a time, each through a full verified move. It stops at the first failed move and leaves the node cordoned.

func (*Runner) Kick

func (r *Runner) Kick()

Kick wakes Run without waiting for the next interval.

func (*Runner) Run

func (r *Runner) Run(ctx context.Context)

Run ticks until ctx is done. Running moves found at startup resume.

func (*Runner) Start

func (r *Runner) Start(ctx context.Context, req Request) (store.NodeMove, error)

Start records a new move and wakes the runner. It fails with store.ErrNodeMoveInProgress when the resource is already moving.

func (*Runner) StartDrain

func (r *Runner) StartDrain(ctx context.Context, nodeID, explicitTarget string) (store.NodeDrain, error)

StartDrain cordons nodeID and records a running drain. It fails with store.ErrNodeMoveInProgress when the node is already draining.

func (*Runner) Tick

func (r *Runner) Tick(ctx context.Context)

Tick advances every running move, then every running drain.

type Store

type Store interface {
	CreateNodeMove(ctx context.Context, m store.NodeMove) error
	SaveNodeMove(ctx context.Context, m store.NodeMove) error
	GetNodeMove(ctx context.Context, id string) (store.NodeMove, error)
	ListNodeMoves(ctx context.Context, f store.NodeMoveFilter) ([]store.NodeMove, error)
	CreateNodeDrain(ctx context.Context, d store.NodeDrain) error
	SaveNodeDrain(ctx context.Context, d store.NodeDrain) error
	GetNodeDrain(ctx context.Context, id string) (store.NodeDrain, error)
	ListNodeDrains(ctx context.Context, nodeID string, limit int) ([]store.NodeDrain, error)
}

Store is the persistence the runner needs; *store.DB satisfies it.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL