restartpolicy

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 8, 2026 License: MIT Imports: 3 Imported by: 0

README

Restart Policy

Restart policy lets Hypeman keep an instance running after the guest program exits, or after a configured health check marks a running workload unhealthy.

This supervises the whole instance, not an individual in-guest process. If the image runs systemd or multiple processes, a restart boots the instance again using the same persisted instance configuration.

Policies

never is the default. Hypeman records exit information, but never restarts the instance.

on_failure restarts when the last run failed.

Failure means:

  • exit code is nonzero
  • the guest was killed by signal or OOM
  • the instance stopped unexpectedly and no clean exit code is available
  • a configured health check reached unhealthy

Exit code 0 does not restart under on_failure.

always restarts after any guest exit, including exit code 0.

Manual Stops

Manual stop suppresses restart policy.

When an instance is stopped through the API, Hypeman records an internal suppression marker before shutdown begins. The public instance state remains the single lifecycle state; the suppression marker is exposed only as restart_status.blocked_reason=manual_stop.

Calling start clears manual suppression and retry status.

Deleting an instance removes it entirely and no restart is attempted.

Unexpected guest exit does not set manual suppression. If the instance has a restart policy, the controller may restart it.

Attempts And Backoff

Each automatic restart waits for backoff before another restart attempt is allowed.

max_attempts limits consecutive automatic restart attempts. 0 means unlimited attempts.

If max_attempts is exceeded, restart policy is blocked for that failure window and restart_status.blocked_reason is set to max_attempts_exceeded.

If an instance runs for stable_after, the consecutive attempt count resets. This prevents old transient failures from permanently consuming the retry budget.

Manual start clears blocked restart status and starts a new failure window.

Updating the restart policy clears retry status unless the instance was manually stopped. Changing the policy on a manually stopped instance does not start it; the user must call start.

Health Checks

Health checks are a separate feature. They report workload health without changing instance lifecycle state.

When health status reaches unhealthy, restart policy treats that as a failure signal for on_failure and always. never still reports health without restarting.

Health-triggered restart uses the same attempt counter, backoff, max-attempt limit, manual-stop suppression, and stable-window reset as exit-triggered restart.

Lifecycle Behavior

Exit-triggered restart only starts instances from Stopped.

Health-triggered restart acts on Running instances by stopping the instance and starting it again. The health check itself does not keep the lifecycle state in Initializing or otherwise hide the distinction between boot readiness and workload health.

It does not restore Standby instances, wake templates, or act on Unknown state.

Automatic restart uses the normal instance start path. Resource validation, network allocation, config disk generation, volumes, GPU attachment, egress policy, and command/env metadata behave the same as a manual start.

The restart keeps the same image, overlay disk, volumes, env, tags, entrypoint, and cmd stored on the instance.

Status

Instance responses include the configured restart_policy and current restart_status.

restart_status.next_attempt_at is set while waiting for backoff.

restart_status.attempts counts consecutive automatic restart attempts in the current failure window.

restart_status.blocked_reason explains why no more retries will happen despite a restart policy being configured.

restart_status.last_reason=health_check_failed means the current retry window was entered because health checks marked the workload unhealthy rather than because the guest process exited on its own.

Non-goals

Restart policy is not a health check.

It does not supervise individual processes inside the guest.

It does not replace systemd for images that want in-guest service supervision.

Documentation

Index

Constants

View Source
const (
	DefaultBackoff     = 5 * time.Second
	DefaultStableAfter = 10 * time.Minute
)

Variables

This section is empty.

Functions

func Backoff

func Backoff(policy *Policy) time.Duration

func EqualStatus

func EqualStatus(a, b Status) bool

func ShouldRestart

func ShouldRestart(policy *Policy, exitCode *int) bool

func StableAfter

func StableAfter(policy *Policy) time.Duration

Types

type BlockedReason

type BlockedReason string
const (
	BlockedReasonManualStop          BlockedReason = "manual_stop"
	BlockedReasonMaxAttemptsExceeded BlockedReason = "max_attempts_exceeded"
)

type Policy

type Policy struct {
	Policy      PolicyMode `json:"policy"`
	Backoff     string     `json:"backoff,omitempty"`
	MaxAttempts int        `json:"max_attempts,omitempty"`
	StableAfter string     `json:"stable_after,omitempty"`
}

func NormalizePolicy

func NormalizePolicy(policy *Policy) (*Policy, error)

type PolicyMode

type PolicyMode string
const (
	PolicyNever     PolicyMode = "never"
	PolicyAlways    PolicyMode = "always"
	PolicyOnFailure PolicyMode = "on_failure"
)

type RestartReason

type RestartReason string
const (
	RestartReasonHealthCheckFailed RestartReason = "health_check_failed"
)

type Status

type Status struct {
	Attempts      int           `json:"attempts,omitempty"`
	LastAttemptAt *time.Time    `json:"last_attempt_at,omitempty"`
	NextAttemptAt *time.Time    `json:"next_attempt_at,omitempty"`
	BlockedReason BlockedReason `json:"blocked_reason,omitempty"`
	LastReason    RestartReason `json:"last_reason,omitempty"`
}

func AfterFailedAttempt

func AfterFailedAttempt(policy *Policy, status Status, now time.Time) Status

func PrepareAttempt

func PrepareAttempt(policy *Policy, status Status, now time.Time) (Status, bool)

func (Status) IsZero

func (s Status) IsZero() bool

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL