selfupgrade

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 12, 2026 License: Apache-2.0 Imports: 22 Imported by: 0

Documentation

Overview

Package selfupgrade replaces the control plane binary with a verified newer release and rolls binary and data back if it does not become healthy. It runs from a host command, never inside the process being replaced.

Index

Constants

View Source
const (
	StepGuard          = "guard"
	StepDownload       = "download"
	StepChecksum       = "verify_checksum"
	StepSignature      = "verify_signature"
	StepProbe          = "probe_binary"
	StepBackup         = "backup"
	StepMigrationCheck = "migration_check"
	StepStop           = "stop"
	StepSwap           = "swap"
	StepStart          = "start"
	StepHealth         = "health"
	StepRollback       = "rollback"
)

Step names, in the order Apply runs them.

View Source
const (
	StatusOK      = "ok"
	StatusFailed  = "failed"
	StatusSkipped = "skipped"
)

Step statuses.

View Source
const (
	OutcomeRunning    = "running"
	OutcomeSucceeded  = "succeeded"
	OutcomeRolledBack = "rolled_back"
	OutcomeRefused    = "refused"
	OutcomeFailed     = "failed"
)

Attempt outcomes. Running is only ever seen in a journal.

View Source
const (
	SourceMetadata = "metadata"
	SourceHeading  = "heading"
)

Breaking change sources.

View Source
const (
	VerifyAuto    = "auto"
	VerifyRequire = "require"
	VerifyOff     = "off"
)

Signature modes, the same vocabulary as the installer's APP_INSTALL_VERIFY.

View Source
const DefaultHealthTimeout = 90 * time.Second

DefaultHealthTimeout bounds how long a new release may take to become healthy before the previous binary and data are put back.

View Source
const DetailOffWhilePinned = "signature check disabled by the operator although this host is pinned to signed releases"

DetailOffWhilePinned is the timeline detail for an operator override.

View Source
const ExitDowngradeRefused = 78

ExitDowngradeRefused is the process exit status when a binary refuses to start on a database written by a newer release. systemd can be told not to restart on it (RestartPreventExitStatus).

View Source
const JournalDir = "self-upgrade"

JournalDir is the directory under the data dir that holds attempt files.

View Source
const MetaMarker = "levelrail-upgrade:"

MetaMarker opens the machine-readable upgrade metadata comment a release body may carry. The JSON inside lists breaking changes explicitly.

View Source
const PinEscapeHint = "" /* 136-byte string literal not displayed */

PinEscapeHint names the documented, explicit ways past a pinned refusal.

View Source
const PinFileName = "release-signature.pin"

PinFileName is the marker, under the data dir, that pins this host to signed releases. install.sh writes the same key=value format.

Variables

View Source
var (
	ErrChecksumMismatch  = errors.New("downloaded binary does not match its published checksum")
	ErrChecksumMissing   = errors.New("release publishes no checksum for this binary")
	ErrSignatureInvalid  = errors.New("release signature verification failed")
	ErrSignatureRequired = errors.New("signature verification is required but unavailable")
	ErrSignaturePinned   = errors.New("this host only accepts signed releases")
	ErrBreakingNotAcked  = errors.New("breaking changes need acknowledgement")
	ErrDowngrade         = errors.New("target release is older than the database schema")
	ErrVersionMismatch   = errors.New("downloaded binary reports a different version than requested")
	ErrMigrationCheck    = errors.New("migration dry-run failed")
	ErrNotHealthy        = errors.New("new release did not become healthy")
	// ErrDryRunUnsupported is returned by a migration check when the target
	// binary has no dry-run subcommand, which is not a failure.
	ErrDryRunUnsupported = errors.New("target binary has no migration dry-run")
)

Refusals that change nothing on the host.

Functions

func AckError

func AckError(missing []Breaking) error

AckError names the unacknowledged breaking changes in a refusal message.

func DowngradeMessage

func DowngradeMessage(in GuardInput) string

DowngradeMessage explains the refusal and gives the exact recovery commands. Migrations only go forward, so the database cannot be opened by an older binary, and the message says which release to run instead.

func ImportJournal

func ImportJournal(ctx context.Context, j *Journal, db AttemptStore) (int, error)

ImportJournal copies every journaled attempt into the database and removes the files of finished ones. Running attempts stay on disk so a recovery can still read their paths. It returns how many rows were written.

func ParseChecksums

func ParseChecksums(r io.Reader) (map[string]string, error)

ParseChecksums reads sha256sum output into name to hex digest.

func PinPath added in v0.3.0

func PinPath(dataDir string) string

PinPath is the pin file's location under dataDir.

func RemovePin added in v0.3.0

func RemovePin(dataDir string) (bool, error)

RemovePin clears the pin and reports whether one existed.

func ToStore

func ToStore(a Attempt) store.SelfUpgradeAttempt

ToStore converts an attempt to its database row.

func ValidTag

func ValidTag(tag string) bool

ValidTag accepts release tags such as v1.2.3 or v1.2.3-beta.4 and nothing that could escape a URL path or a file name.

func VerifyChecksum

func VerifyChecksum(path, checksumsPath, asset string) error

VerifyChecksum checks the file at path against asset's entry in the checksums file.

func WritePin added in v0.3.0

func WritePin(dataDir string, p Pin) error

WritePin atomically records p. World-readable so a non-root control plane can report it; only root can write the data dir.

Types

type Attempt

type Attempt struct {
	ID          string       `json:"id"`
	FromVersion string       `json:"from_version"`
	ToVersion   string       `json:"to_version"`
	FromSchema  int          `json:"from_schema"`
	ToSchema    int          `json:"to_schema"`
	Initiator   string       `json:"initiator"`
	Outcome     string       `json:"outcome"`
	FailedStep  string       `json:"failed_step,omitempty"`
	Error       string       `json:"error,omitempty"`
	BackupName  string       `json:"backup_name,omitempty"`
	Acked       []string     `json:"acked,omitempty"`
	Steps       []StepRecord `json:"steps"`
	StartedAt   time.Time    `json:"started_at"`
	FinishedAt  time.Time    `json:"finished_at,omitempty"`

	PrevBinary    string `json:"prev_binary,omitempty"`
	SnapshotPath  string `json:"snapshot_path,omitempty"`
	SwappedBinary bool   `json:"swapped_binary,omitempty"`
}

Attempt is one upgrade run. The unexported-to-the-database fields at the end (paths) only live in the journal so a restart in the middle can recover.

func Apply

func Apply(ctx context.Context, d Deps, req Request) (Attempt, error)

Apply runs the upgrade: guard, download, verify, back up, migration dry run, swap, restart, health check, and rollback on failure. Nothing on the host changes before the stop step. err is nil only on success.

func Recover

func Recover(ctx context.Context, d Deps, a Attempt, healthTimeout time.Duration) (Attempt, error)

Recover finishes an attempt whose host command died mid-way: closes one that never stopped the service, rolls back one that did, confirms a healthy one.

func (Attempt) AckedJSON

func (a Attempt) AckedJSON() string

AckedJSON encodes the acknowledged breaking-change ids for the database.

func (Attempt) StepsJSON

func (a Attempt) StepsJSON() string

StepsJSON encodes the timeline for the database column.

type AttemptStore

type AttemptStore interface {
	UpsertSelfUpgradeAttempt(ctx context.Context, a store.SelfUpgradeAttempt) error
}

AttemptStore is the persistence ImportJournal needs; *store.DB satisfies it.

type BackupInfo

type BackupInfo struct {
	Name         string
	SnapshotPath string
}

BackupInfo names the pre-upgrade restore point.

type Breaking

type Breaking struct {
	ID          string `json:"id"`
	Version     string `json:"version"`
	Summary     string `json:"summary"`
	RequiresAck bool   `json:"requires_ack"`
	Source      string `json:"source"`
}

Breaking is one breaking change between two versions. RequiresAck means the upgrade refuses to start until the operator acknowledges it by id.

func BreakingBetween

func BreakingBetween(releases []ReleaseNotes, current, target string) []Breaking

BreakingBetween collects the breaking changes of every release newer than current and no newer than target, oldest first. Versions that do not order are skipped: an unknown tag cannot be said to sit between the two.

func MissingAcks

func MissingAcks(breaking []Breaking, acked []string) []Breaking

MissingAcks returns the breaking changes that require acknowledgement and were not acknowledged.

func ParseBreaking

func ParseBreaking(tag, body string) []Breaking

ParseBreaking extracts the breaking changes one release body declares. The explicit metadata comment wins; without it, bullets under a "Breaking changes" heading count and always require acknowledgement.

type Deps

type Deps struct {
	BinaryPath string
	WorkDir    string
	Asset      string
	Journal    *Journal
	Log        *slog.Logger
	Now        func() time.Time

	Fetch       func(ctx context.Context, tag, asset, dir string) (Fetched, error)
	VerifySig   func(ctx context.Context, checksumsPath, bundlePath string) (string, error)
	Probe       func(ctx context.Context, binary string) (ProbeInfo, error)
	DBSchema    func(ctx context.Context) (int, error)
	Backup      func(ctx context.Context) (BackupInfo, error)
	MigrateTest func(ctx context.Context, binary, snapshotPath string) (int, error)
	Stop        func(ctx context.Context) error
	Start       func(ctx context.Context) error
	WaitHealthy func(ctx context.Context, timeout time.Duration) error
	// RestoreDB puts the snapshot in place of the live database and returns
	// where the replaced database was kept.
	RestoreDB   func(ctx context.Context, snapshotPath string) (string, error)
	WriteMarker func(ctx context.Context, a Attempt) error
	ClearMarker func()
	Audit       func(ctx context.Context, detail string) error
}

Deps are the host operations Apply drives. Tests fake them; the host command wires real ones (HTTPS, cosign, systemd, sqlite, filesystem).

type Fetched

type Fetched struct {
	BinaryPath    string
	ChecksumsPath string
	BundlePath    string
}

Fetched is a downloaded release: the binary, its checksums file and the signature bundle (empty when the release publishes none).

type Fetcher

type Fetcher struct {
	// BaseURL is the release download prefix with the tag as its last path
	// segment appended by Fetch, for example https://host/org/repo/releases/download.
	BaseURL string
	Client  *http.Client
	// AllowHTTP permits a plain-http BaseURL, for tests against a local server.
	AllowHTTP bool
	MaxBytes  int64
}

Fetcher downloads release assets over HTTPS.

func (Fetcher) Fetch

func (f Fetcher) Fetch(ctx context.Context, tag, asset, dir string) (Fetched, error)

Fetch downloads asset, checksums.txt and the signature bundle of tag into dir. The checksums file is required, the bundle is optional.

type GuardInput

type GuardInput struct {
	BinaryVersion string
	BinarySchema  int
	DBSchema      int
	// DBVersion is the release that last ran on the database, "" if unknown.
	DBVersion string
	// Program is the executable's name, used in the recovery commands.
	Program string
	// InstallerCommand is the verified installer invocation prefix, for
	// example "curl -fsSL <url> | sudo".
	InstallerCommand string
}

GuardInput describes a binary that found a database newer than itself.

type Journal

type Journal struct {
	Dir string
}

Journal persists attempts as one JSON file each. It is the source of truth while an attempt runs, because the database may be swapped underneath it.

func NewJournal

func NewJournal(dataDir string) *Journal

NewJournal returns the journal under dataDir.

func (*Journal) List

func (j *Journal) List() ([]Attempt, error)

List returns every readable attempt, oldest first. Unreadable files are skipped: a half-written journal must not block an upgrade.

func (*Journal) Remove

func (j *Journal) Remove(id string)

Remove deletes an attempt's file.

func (*Journal) Running

func (j *Journal) Running() ([]Attempt, error)

Running returns attempts that never reached a final outcome.

func (*Journal) Save

func (j *Journal) Save(a Attempt) error

Save writes a atomically with owner-only permissions.

type Pin added in v0.3.0

type Pin struct {
	Version    string
	VerifiedAt time.Time
}

Pin records the first release whose signature this host verified.

func ReadPin added in v0.3.0

func ReadPin(dataDir string) (Pin, bool, error)

ReadPin reports whether dataDir holds a pin. A present but unreadable or empty file still pins: failing open would let a truncated file unpin.

type ProbeInfo

type ProbeInfo struct {
	Version string
	Schema  int
}

ProbeInfo is what a downloaded binary reports about itself.

type ReleaseNotes

type ReleaseNotes struct {
	Tag  string
	Body string
}

ReleaseNotes is one release's tag and notes body.

type Request

type Request struct {
	ID             string
	CurrentVersion string
	TargetVersion  string
	Initiator      string
	Breaking       []Breaking
	Acked          []string
	HealthTimeout  time.Duration
}

Request is one upgrade the operator asked for.

type Signer

type Signer struct {
	Mode           string
	CosignPath     string
	IdentityRegexp string
	Issuer         string
	// Pinned is set once this host has verified a release signature: from
	// then on a missing bundle or cosign is refused unless Mode is off.
	Pinned bool
	// OnVerified runs after a successful verification, to write the pin.
	OnVerified func()
	// Run executes the verifier; nil uses os/exec. Tests inject a fake.
	Run func(ctx context.Context, name string, args ...string) ([]byte, error)
}

Signer verifies a cosign keyless bundle over a checksums file.

func (Signer) Verify

func (s Signer) Verify(ctx context.Context, checksumsPath, bundlePath string) (string, error)

Verify returns a human detail for the timeline. A failed verification is always an error. A missing cosign or bundle is an error in require mode and once the host is pinned.

type StepRecord

type StepRecord struct {
	Name       string    `json:"name"`
	Status     string    `json:"status"`
	Detail     string    `json:"detail,omitempty"`
	At         time.Time `json:"at"`
	DurationMS int64     `json:"duration_ms"`
}

StepRecord is one step's result in an attempt's timeline.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL