instances

package
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 12, 2026 License: MIT Imports: 64 Imported by: 0

README

Instance Manager

Manages VM instance lifecycle across multiple hypervisors (Cloud Hypervisor, QEMU on Linux; vz on macOS).

Design Decisions

Why State Machine? (state.go)

What: Single-hop state transitions matching hypervisor states

Why:

  • Validates transitions before execution (prevents invalid operations)
  • Manager orchestrates multi-hop flows (e.g., Running → Paused → Standby)
  • Clear separation: state machine = rules, manager = orchestration

States:

  • Stopped - No VMM, no snapshot
  • Created - VMM created but not booted (CH native)
  • Initializing - VM is running while guest init is still in progress
  • Running - Guest program start boundary reached and guest-agent readiness observed (unless skip_guest_agent=true)
  • Paused - VM paused (CH native)
  • Shutdown - VM shutdown, VMM exists (CH native)
  • Standby - No VMM, snapshot exists (can restore)
Why Config Disk? (configdisk.go)

What: Read-only erofs disk with instance configuration

Why:

  • Zero modifications to OCI images (images used as-is)
  • Config injected at boot time (not baked into image)
  • Efficient (compressed erofs, ~few KB)
  • Contains: entrypoint, cmd, env vars, workdir

Filesystem Layout (storage.go)

/var/lib/hypeman/
  guests/
    {instance-id}/              # ULID-based ID
      metadata.json             # State, versions, timestamps
      overlay.raw               # 50GB sparse writable overlay
      config.erofs              # Compressed config disk
      ch.sock                   # Hypervisor API socket (abbreviated for SUN_LEN limit)
      logs/
        app.log                 # Guest application log (serial console output)
        vmm.log                 # Hypervisor log (stdout+stderr)
        hypeman.log             # Hypeman operations log
      snapshots/
        snapshot-latest/        # Snapshot directory
          config.json           # VM configuration
          memory-ranges         # Memory state

metadata.json also carries controller-owned auto-standby runtime timestamps when that feature is enabled, so idle countdown state can survive Hypeman restarts.

Benefits:

  • Content-addressable IDs (ULID = time-ordered)
  • Self-contained: all instance data in one directory
  • Easy cleanup: delete directory = full cleanup
  • Sparse overlays: only store diffs from base image

Multi-Hop Orchestrations (manager.go)

Manager orchestrates multiple single-hop state transitions:

CreateInstance:

Stopped → Created → Initializing → Running
1. Start VMM process
2. Create VM config
3. Boot VM
4. Wait for guest-agent readiness gate (event-driven, exec mode, unless skipped)
5. Guest program start marker observed
6. Kernel headers setup continues asynchronously (does not gate `Running`)
7. Expand memory (if hotplug configured)

StandbyInstance:

Running → Paused → Standby
1. Reduce memory (virtio-mem hotplug)
2. Pause VM
3. Create snapshot
4. Stop VMM

RestoreInstance:

Standby → Paused → Running
1. Start VMM
2. Restore from snapshot
3. Resume VM

DeleteInstance:

Any State → Stopped
1. Stop VMM (if running)
2. Delete all instance data

Snapshot Optimization (standby.go, restore.go)

Reduce snapshot size:

  • Memory hotplug: Reduce to base size before snapshot (virtio-mem)
  • Sparse overlays: Only store diffs from base image

Fast restore:

  • Don't prefault pages (lazy loading)
  • Parallel with TAP device setup

Scheduled Snapshot Behavior

  • Schedules are configured per instance and persisted in the server data store (outside snapshot payloads).
  • A background scheduler evaluates due schedules every minute.
  • Each due run chooses snapshot behavior from current source state:
    • Running/Standby sources use Standby snapshots.
    • Stopped sources use Stopped snapshots.
  • Standby runs from Running sources perform a brief pause/resume cycle during capture.
  • The minimum interval is 1m, but larger intervals are recommended for heavier or latency-sensitive workloads because running captures pause/resume the guest.
  • Scheduled snapshot name_prefix is optional and capped at 47 chars so generated names stay within the 63-char snapshot name limit.
  • New schedules establish cadence at now + interval + deterministic jitter derived from the instance ID.
  • Updating only retention, metadata, or name_prefix preserves next_run_at; changing interval establishes a new cadence.
  • Schedule runs advance to the next future interval (no backfill flood after downtime).
  • Each schedule stores operational status:
    • next_run_at
    • last_run_at
    • last_snapshot_id
    • last_error
  • Retention cleanup runs after successful scheduled snapshot creation and only affects scheduled snapshots for that instance.
  • If an instance is deleted, its schedule is retained so retention can continue cleaning existing scheduled snapshots.
  • Once the deleted instance has no scheduled snapshots left, the scheduler removes that schedule automatically.

WaitForState (wait.go, lifecycle_events.go)

Waits for an instance to reach a target state using the shared lifecycle event bus with a polling fallback. State-changing manager methods publish lifecycle events after successful mutations, and WaitForState filters them by instance_id. A 5s polling fallback guards against missed or dropped events. Returns early on terminal (Stopped, Standby, Shutdown) or error (Unknown) states. Used by GET /instances/{id}/wait.

Reference Handling

Instances use OCI image references directly:

req := CreateInstanceRequest{
    Image: "docker.io/library/alpine:latest",  // OCI reference
}
// Validates image exists and is ready via image manager

Testing

Tests focus on testable components:

# State machine (pure logic, no VM needed)
TestStateTransitions - validates all transition rules

# Storage operations (filesystem only, no VM needed)
TestStorageOperations - metadata persistence, directory cleanup

# Full integration (requires kernel/initrd)
# Skipped by default, needs system files from system manager

Dependencies

  • lib/images - Image manager for OCI image validation
  • lib/system - System manager for kernel/initrd files
  • lib/hypervisor - Hypervisor abstraction for VM operations
  • System tools: mkfs.erofs, cpio, gzip (Linux); mkfs.ext4 (macOS)

Documentation

Index

Constants

View Source
const (
	// SnapshotKindStandby captures snapshot-based standby state (memory/device/disk).
	SnapshotKindStandby = snapshot.SnapshotKindStandby
	// SnapshotKindStopped captures stopped-state disk+metadata only.
	SnapshotKindStopped = snapshot.SnapshotKindStopped
)
View Source
const (
	WaitForStateDefaultTimeout = 60 * time.Second
	WaitForStateMaxTimeout     = 5 * time.Minute
	WaitForStatePollInterval   = 5 * time.Second
)
View Source
const DefaultStopTimeout = 5

DefaultStopTimeout is the default grace period for graceful shutdown (seconds).

View Source
const (
	// MaxVolumesPerInstance is the maximum number of volumes that can be attached
	// to a single instance. This limit exists because volume devices are named
	// /dev/vdd, /dev/vde, ... /dev/vdz (letters d-z = 23 devices).
	// Devices a-c are reserved for rootfs, overlay, and config disk.
	MaxVolumesPerInstance = 23
)

Variables

View Source
var (
	// ErrNotFound is returned when an instance is not found
	ErrNotFound = errors.New("instance not found")

	// ErrInvalidState is returned when a state transition is not valid
	ErrInvalidState = errors.New("invalid state transition")

	// ErrInvalidRequest is returned when request validation fails
	ErrInvalidRequest = errors.New("invalid request")

	// ErrAlreadyExists is returned when creating an instance that already exists
	ErrAlreadyExists = errors.New("instance already exists")

	// ErrImageNotReady is returned when the image is not ready for use
	ErrImageNotReady = errors.New("image not ready")

	// ErrAmbiguousName is returned when multiple instances have the same name
	ErrAmbiguousName = errors.New("multiple instances with the same name")

	// ErrInsufficientResources is returned when resources (CPU, memory, network, GPU) are not available
	ErrInsufficientResources = errors.New("insufficient resources")

	// ErrNotSupported is returned when an operation is not supported for the instance hypervisor
	ErrNotSupported = errors.New("operation not supported")

	// ErrSnapshotNotFound is returned when a snapshot is not found.
	ErrSnapshotNotFound = errors.New("snapshot not found")

	// ErrSnapshotScheduleNotFound is returned when a snapshot schedule is not found.
	ErrSnapshotScheduleNotFound = errors.New("snapshot schedule not found")
)
View Source
var ErrLogNotFound = fmt.Errorf("log file not found")

ErrLogNotFound is returned when the requested log file doesn't exist

View Source
var ErrTailNotFound = fmt.Errorf("tail command not found: required for log streaming")

ErrTailNotFound is returned when the tail command is not available

View Source
var ValidTransitions = map[State][]State{

	StateCreated: {
		StateInitializing,
		StateRunning,
		StateShutdown,
	},
	StateInitializing: {
		StateRunning,
		StateShutdown,
	},
	StateRunning: {
		StatePaused,
		StateShutdown,
	},
	StatePaused: {
		StateRunning,
		StateShutdown,
		StateStandby,
	},
	StateShutdown: {
		StateRunning,
		StateStopped,
	},

	StateStopped: {
		StateCreated,
	},
	StateStandby: {
		StatePaused,
		StateStopped,
	},

	StateUnknown: {},
}

ValidTransitions defines allowed single-hop state transitions Based on Cloud Hypervisor's actual state machine plus our additions

Functions

func NewLivenessChecker

func NewLivenessChecker(m Manager) devices.InstanceLivenessChecker

NewLivenessChecker creates a new InstanceLivenessChecker that wraps the instances manager. This adapter allows the devices package to query instance state without a circular import.

func WaitForProcessExit

func WaitForProcessExit(pid int, timeout time.Duration) bool

WaitForProcessExit polls for a process to exit, returns true if exited within timeout. Exported for use in tests.

Types

type AttachVolumeRequest

type AttachVolumeRequest struct {
	MountPath string
	Readonly  bool
}

AttachVolumeRequest is the domain request for attaching a volume (used for API compatibility)

type CreateInstanceRequest

type CreateInstanceRequest struct {
	Name                     string                      // Required
	Image                    string                      // Required: OCI reference
	Platform                 string                      // Optional: target platform as os/arch[/variant]; drives image resolution and auto-pull. Empty means host platform.
	Size                     int64                       // Base memory in bytes (default: 1GB)
	HotplugSize              int64                       // Hotplug memory in bytes (default: 0, set explicitly to enable)
	OverlaySize              int64                       // Overlay disk size in bytes (default: 10GB)
	Vcpus                    int                         // Default 2
	NetworkBandwidthDownload int64                       // Download rate limit bytes/sec (0 = auto, proportional to CPU)
	NetworkBandwidthUpload   int64                       // Upload rate limit bytes/sec (0 = auto, proportional to CPU)
	DiskIOBps                int64                       // Disk I/O rate limit bytes/sec (0 = auto, proportional to CPU)
	Env                      map[string]string           // Optional environment variables
	Tags                     tags.Tags                   // Optional user-defined key-value tags
	NetworkEnabled           bool                        // Whether to enable networking (uses default network)
	NetworkEgress            *NetworkEgressPolicy        // Optional host-mediated egress policy
	Credentials              map[string]CredentialPolicy // Optional host-managed credential brokering policies
	Devices                  []string                    // Device IDs or names to attach (GPU passthrough)
	Volumes                  []VolumeAttachment          // Volumes to attach at creation time
	Hypervisor               hypervisor.Type             // Optional: hypervisor type (defaults to config)
	HypervisorVersion        string                      // Optional: hypervisor version (defaults to configured default)
	GPU                      *GPUConfig                  // Optional: vGPU configuration
	Entrypoint               []string                    // Override image entrypoint (nil = use image default)
	Cmd                      []string                    // Override image cmd (nil = use image default)
	SkipKernelHeaders        bool                        // Skip kernel headers installation (disables DKMS)
	SkipGuestAgent           bool                        // Skip guest-agent installation (disables exec/stat API)
	SnapshotPolicy           *SnapshotPolicy             // Optional snapshot policy defaults for this instance
	AutoStandby              *autostandby.Policy         // Optional automatic standby policy
	HealthCheck              *healthcheck.Policy         // Optional workload health check policy
	RestartPolicy            *restartpolicy.Policy       // Optional whole-instance restart policy
}

CreateInstanceRequest is the domain request for creating an instance

type CreateSnapshotRequest added in v0.1.0

type CreateSnapshotRequest struct {
	Kind        SnapshotKind                        // Required: Standby or Stopped
	Name        string                              // Optional: unique per source instance
	Tags        tags.Tags                           // Optional user-defined key-value tags
	Compression *snapshot.SnapshotCompressionConfig // Optional compression override
}

CreateSnapshotRequest is the domain request for creating a snapshot.

type CredentialInjectAs added in v0.1.0

type CredentialInjectAs struct {
	Header string // Header name to set/mutate
	Format string // Format template containing ${value}
}

CredentialInjectAs describes how the credential is materialized for outbound requests. Header templating is currently supported; future types (e.g., request signing) can extend this.

type CredentialInjectRule added in v0.1.0

type CredentialInjectRule struct {
	Hosts []string           // Optional host patterns (api.example.com, *.example.com); empty means all
	As    CredentialInjectAs // Current v1 injection shape
}

CredentialInjectRule scopes a credential injection policy to destination hosts.

type CredentialPolicy added in v0.1.0

type CredentialPolicy struct {
	Source CredentialSource
	Inject []CredentialInjectRule
}

CredentialPolicy configures one host-managed credential brokering policy.

type CredentialSource added in v0.1.0

type CredentialSource struct {
	Env string // Host env variable name
}

CredentialSource references where real credential material is loaded from.

type EgressEnforcementMode added in v0.1.0

type EgressEnforcementMode string
const (
	EgressEnforcementModeAll           EgressEnforcementMode = "all"
	EgressEnforcementModeHTTPHTTPSOnly EgressEnforcementMode = "http_https_only"
)

type ForkInstanceRequest added in v0.0.7

type ForkInstanceRequest struct {
	Name        string // Required: name for the new forked instance
	FromRunning bool   // Optional: allow forking from Running by auto standby/fork/restore
	TargetState State  // Optional: desired final state of forked instance (Stopped, Standby, Running). Empty means inherit source state.
}

ForkInstanceRequest is the domain request for forking an instance.

type ForkSnapshotRequest added in v0.1.0

type ForkSnapshotRequest struct {
	Name             string          // Required: name for the new instance
	TargetState      State           // Optional
	TargetHypervisor hypervisor.Type // Optional, allowed only for Stopped snapshots
}

ForkSnapshotRequest is the domain request for forking from a snapshot.

type GPUConfig added in v0.0.5

type GPUConfig struct {
	Profile string // vGPU profile name (e.g., "L40S-1Q")
}

GPUConfig contains GPU configuration for instance creation

type HealthCheckController added in v0.1.0

type HealthCheckController struct {
	// contains filtered or unexported fields
}

func NewHealthCheckController added in v0.1.0

func NewHealthCheckController(manager Manager, log *slog.Logger) *HealthCheckController

func (*HealthCheckController) Run added in v0.1.0

type HostTopology

type HostTopology struct {
	ThreadsPerCore int
	CoresPerSocket int
	Sockets        int
}

HostTopology represents the CPU topology of the host machine

type ImageUsageRecorder added in v0.1.0

type ImageUsageRecorder interface {
	MarkUsed(ctx context.Context, imageName, digest string) error
}

ImageUsageRecorder records newly used images before instance metadata is persisted.

type ImageUsageRecorderSetter added in v0.1.0

type ImageUsageRecorderSetter interface {
	SetImageUsageRecorder(recorder ImageUsageRecorder)
}

ImageUsageRecorderSetter configures an optional image usage recorder on the manager.

type IngressResolver

type IngressResolver struct {
	// contains filtered or unexported fields
}

IngressResolver provides instance resolution for the ingress package. It implements ingress.InstanceResolver interface without importing the ingress package to avoid import cycles.

func NewIngressResolver

func NewIngressResolver(manager Manager) *IngressResolver

NewIngressResolver creates a new IngressResolver that wraps an instance manager.

func (*IngressResolver) InstanceExists

func (r *IngressResolver) InstanceExists(ctx context.Context, nameOrID string) (bool, error)

InstanceExists checks if an instance with the given name, ID, or ID prefix exists.

func (*IngressResolver) ResolveInstance

func (r *IngressResolver) ResolveInstance(ctx context.Context, nameOrID string) (string, string, error)

ResolveInstance resolves an instance name, ID, or ID prefix to its canonical name and ID.

func (*IngressResolver) ResolveInstanceIP

func (r *IngressResolver) ResolveInstanceIP(ctx context.Context, nameOrID string) (string, error)

ResolveInstanceIP resolves an instance name, ID, or ID prefix to its IP address.

type Instance

type Instance struct {
	StoredMetadata

	// Derived fields (not stored in metadata.json)
	State               State   // Derived from socket + VMM query + guest boot markers
	StateError          *string // Error message if state couldn't be determined (non-nil when State=Unknown)
	HasSnapshot         bool    // Derived from filesystem check
	BootMarkersHydrated bool    // True when missing boot markers were hydrated from logs in this read
	HealthCheckRuntime  *healthcheck.Runtime
}

Instance represents a virtual machine instance with derived runtime state

func (*Instance) GetHypervisorType

func (i *Instance) GetHypervisorType() string

GetHypervisorType returns the hypervisor type as a string. This implements the middleware.HypervisorTyper interface for OTEL enrichment.

type LifecycleEvent added in v0.1.0

type LifecycleEvent struct {
	Action     LifecycleEventAction
	InstanceID string
	Instance   *Instance
}

LifecycleEvent is a global instance change event stream used by internal consumers such as wait-for-state and background controllers.

type LifecycleEventAction added in v0.1.0

type LifecycleEventAction string

LifecycleEventAction identifies which instance lifecycle action occurred.

const (
	LifecycleEventCreate  LifecycleEventAction = "create"
	LifecycleEventUpdate  LifecycleEventAction = "update"
	LifecycleEventStart   LifecycleEventAction = "start"
	LifecycleEventStop    LifecycleEventAction = "stop"
	LifecycleEventStandby LifecycleEventAction = "standby"
	LifecycleEventRestore LifecycleEventAction = "restore"
	LifecycleEventDelete  LifecycleEventAction = "delete"
	LifecycleEventFork    LifecycleEventAction = "fork"
)

type LifecycleEventConsumer added in v0.1.0

type LifecycleEventConsumer string

LifecycleEventConsumer identifies the internal consumer of lifecycle events. Keep this set bounded for observability label safety.

const (
	LifecycleEventConsumerWaitForState  LifecycleEventConsumer = "wait_for_state"
	LifecycleEventConsumerAutoStandby   LifecycleEventConsumer = "auto_standby"
	LifecycleEventConsumerHealthCheck   LifecycleEventConsumer = "health_check"
	LifecycleEventConsumerRestartPolicy LifecycleEventConsumer = "restart_policy"
)

type ListInstancesFilter added in v0.0.7

type ListInstancesFilter struct {
	State *State    // Filter by instance state
	Tags  tags.Tags // Filter by tag key-value pairs (all must match)
}

ListInstancesFilter contains optional filters for listing instances. All fields are ANDed together: an instance must match every specified filter.

func (*ListInstancesFilter) Matches added in v0.0.7

func (f *ListInstancesFilter) Matches(inst *Instance) bool

Matches returns true if the given instance satisfies all filter criteria.

type ListSnapshotsFilter added in v0.1.0

type ListSnapshotsFilter = snapshot.ListSnapshotsFilter

ListSnapshotsFilter contains optional filters for listing snapshots.

type LogSource

type LogSource string

LogSource represents a log source type

const (
	// LogSourceApp is the guest application log (serial console)
	LogSourceApp LogSource = "app"
	// LogSourceVMM is the Cloud Hypervisor VMM log
	LogSourceVMM LogSource = "vmm"
	// LogSourceHypeman is the hypeman operations log
	LogSourceHypeman LogSource = "hypeman"
)

type Manager

type Manager interface {
	ListInstances(ctx context.Context, filter *ListInstancesFilter) ([]Instance, error)
	ListSnapshots(ctx context.Context, filter *ListSnapshotsFilter) ([]Snapshot, error)
	GetSnapshot(ctx context.Context, snapshotID string) (*Snapshot, error)
	CreateInstance(ctx context.Context, req CreateInstanceRequest) (*Instance, error)
	CreateSnapshot(ctx context.Context, id string, req CreateSnapshotRequest) (*Snapshot, error)
	// GetInstance returns an instance by ID, name, or ID prefix.
	// Lookup order: exact ID match -> exact name match -> ID prefix match.
	// Returns ErrAmbiguousName if prefix matches multiple instances.
	GetInstance(ctx context.Context, idOrName string) (*Instance, error)
	DeleteInstance(ctx context.Context, id string) error
	DeleteSnapshot(ctx context.Context, snapshotID string) error
	ForkInstance(ctx context.Context, id string, req ForkInstanceRequest) (*Instance, error)
	ForkSnapshot(ctx context.Context, snapshotID string, req ForkSnapshotRequest) (*Instance, error)
	StandbyInstance(ctx context.Context, id string, req StandbyInstanceRequest) (*Instance, error)
	RestoreInstance(ctx context.Context, id string) (*Instance, error)
	RestoreSnapshot(ctx context.Context, id string, snapshotID string, req RestoreSnapshotRequest) (*Instance, error)
	StopInstance(ctx context.Context, id string) (*Instance, error)
	StartInstance(ctx context.Context, id string, req StartInstanceRequest) (*Instance, error)
	UpdateInstance(ctx context.Context, id string, req UpdateInstanceRequest) (*Instance, error)
	StreamInstanceLogs(ctx context.Context, id string, tail int, follow bool, source LogSource) (<-chan string, error)
	RotateLogs(ctx context.Context, maxBytes int64, maxFiles int) error
	AttachVolume(ctx context.Context, id string, volumeId string, req AttachVolumeRequest) (*Instance, error)
	DetachVolume(ctx context.Context, id string, volumeId string) (*Instance, error)
	// ListInstanceAllocations returns resource allocations for all instances.
	// Used by the resource manager for capacity tracking.
	ListInstanceAllocations(ctx context.Context) ([]resources.InstanceAllocation, error)
	// ListRunningInstancesInfo returns info needed for utilization metrics collection.
	// Used by the resource manager for VM utilization tracking.
	ListRunningInstancesInfo(ctx context.Context) ([]resources.InstanceUtilizationInfo, error)
	// SetResourceValidator sets the validator for aggregate resource limit checking.
	// Called after initialization to avoid circular dependencies.
	SetResourceValidator(v ResourceValidator)
	// GetVsockDialer returns a VsockDialer for the specified instance.
	GetVsockDialer(ctx context.Context, instanceID string) (hypervisor.VsockDialer, error)
	// SubscribeLifecycleEvents returns the shared internal lifecycle event stream.
	SubscribeLifecycleEvents(consumer LifecycleEventConsumer) (<-chan LifecycleEvent, func())
}

func NewManager

func NewManager(p *paths.Paths, imageManager images.Manager, systemManager system.Manager, networkManager network.Manager, deviceManager devices.Manager, volumeManager volumes.Manager, limits ResourceLimits, defaultHypervisor hypervisor.Type, snapshotDefaults SnapshotPolicy, meter metric.Meter, tracer trace.Tracer, memoryPolicy ...guestmemory.Policy) Manager

NewManager creates a new instances manager. If meter is nil, metrics are disabled. defaultHypervisor specifies which hypervisor to use when not specified in requests.

func NewManagerWithConfig added in v0.1.0

func NewManagerWithConfig(p *paths.Paths, imageManager images.Manager, systemManager system.Manager, networkManager network.Manager, deviceManager devices.Manager, volumeManager volumes.Manager, limits ResourceLimits, defaultHypervisor hypervisor.Type, snapshotDefaults SnapshotPolicy, managerConfig ManagerConfig, meter metric.Meter, tracer trace.Tracer, memoryPolicy ...guestmemory.Policy) Manager

NewManagerWithConfig creates a new instances manager with additional manager settings.

func NewManagerWithConfigE added in v0.1.0

func NewManagerWithConfigE(p *paths.Paths, imageManager images.Manager, systemManager system.Manager, networkManager network.Manager, deviceManager devices.Manager, volumeManager volumes.Manager, limits ResourceLimits, defaultHypervisor hypervisor.Type, snapshotDefaults SnapshotPolicy, managerConfig ManagerConfig, meter metric.Meter, tracer trace.Tracer, memoryPolicy ...guestmemory.Policy) (Manager, error)

NewManagerWithConfigE creates a new instances manager and returns startup errors for optional host services such as the Firecracker UFFD pager.

type ManagerConfig added in v0.1.0

type ManagerConfig struct {
	LifecycleEventBufferSize          int
	FirecrackerSnapshotMemoryBackend  string
	FirecrackerUFFDCacheMaxBytes      int64
	MaxConcurrentRestoresByHypervisor map[hypervisor.Type]int
}

ManagerConfig holds non-resource manager behavior settings.

func (ManagerConfig) Normalize added in v0.1.0

func (c ManagerConfig) Normalize() ManagerConfig

Normalize applies defaults to manager config values.

type Metrics

type Metrics struct {
	// contains filtered or unexported fields
}

Metrics holds the metrics instruments for instance operations.

type NetworkEgressPolicy added in v0.1.0

type NetworkEgressPolicy struct {
	Enabled         bool                  // Whether host-mediated egress policy is enabled
	EnforcementMode EgressEnforcementMode // all (default) blocks direct non-proxy TCP egress, http_https_only blocks only 80/443
}

NetworkEgressPolicy configures host-mediated outbound networking behavior.

type PendingStandbyCompression added in v0.1.0

type PendingStandbyCompression struct {
	Policy    snapshot.SnapshotCompressionConfig
	NotBefore time.Time
}

PendingStandbyCompression stores the effective standby compression plan that should be recovered after a server restart.

type ResourceLimits

type ResourceLimits struct {
	MaxOverlaySize       int64 // Maximum overlay disk size in bytes per instance
	MaxVcpusPerInstance  int   // Maximum vCPUs per instance (0 = unlimited)
	MaxMemoryPerInstance int64 // Maximum memory in bytes per instance (0 = unlimited)
}

ResourceLimits contains configurable resource limits for instances

type ResourceValidator added in v0.0.6

type ResourceValidator interface {
	// ValidateAllocation checks if the requested resources are available.
	// Returns nil if allocation is allowed, or a detailed error describing
	// which resource is insufficient and the current capacity/usage.
	ValidateAllocation(ctx context.Context, vcpus int, memoryBytes int64, networkDownloadBps int64, networkUploadBps int64, diskIOBps int64, diskBytes int64, needsGPU bool) error
	// ReserveAllocation tentatively reserves resources for an in-flight operation.
	// Call FinishAllocation once the operation fails or becomes visible to resource accounting.
	ReserveAllocation(ctx context.Context, instanceID string, vcpus int, memoryBytes int64, networkDownloadBps int64, networkUploadBps int64, diskIOBps int64, diskBytes int64, needsGPU bool) error
	// FinishAllocation removes any pending reservation for the given instance ID.
	FinishAllocation(instanceID string)
}

ResourceValidator validates if resources can be allocated

type RestoreSnapshotRequest added in v0.1.0

type RestoreSnapshotRequest struct {
	TargetState      State           // Optional
	TargetHypervisor hypervisor.Type // Optional, allowed only for Stopped snapshots
}

RestoreSnapshotRequest is the domain request for restoring a snapshot in-place.

type SetSnapshotScheduleRequest added in v0.1.0

type SetSnapshotScheduleRequest = scheduledsnapshots.SetRequest

type Snapshot added in v0.1.0

type Snapshot = snapshot.Snapshot

Snapshot is a centrally stored immutable snapshot resource.

type SnapshotKind added in v0.1.0

type SnapshotKind = snapshot.SnapshotKind

SnapshotKind determines how snapshot data is captured and restored.

type SnapshotPolicy added in v0.1.0

type SnapshotPolicy struct {
	Compression             *snapshot.SnapshotCompressionConfig
	StandbyCompressionDelay *time.Duration
}

SnapshotPolicy defines default snapshot behavior for an instance.

type SnapshotSchedule added in v0.1.0

type SnapshotSchedule = scheduledsnapshots.Schedule

type SnapshotScheduleManager added in v0.1.0

type SnapshotScheduleManager interface {
	SetSnapshotSchedule(ctx context.Context, instanceID string, req SetSnapshotScheduleRequest) (*SnapshotSchedule, error)
	GetSnapshotSchedule(ctx context.Context, instanceID string) (*SnapshotSchedule, error)
	DeleteSnapshotSchedule(ctx context.Context, instanceID string) error
	RunSnapshotSchedules(ctx context.Context) error
}

SnapshotScheduleManager provides schedule operations in addition to core instance APIs.

type SnapshotScheduleRetention added in v0.1.0

type SnapshotScheduleRetention = scheduledsnapshots.Retention

type StandbyInstanceRequest added in v0.1.0

type StandbyInstanceRequest struct {
	Compression      *snapshot.SnapshotCompressionConfig // Optional compression override
	CompressionDelay *time.Duration                      // Optional standby-only compression delay override
}

StandbyInstanceRequest is the domain request for putting an instance into standby.

type StartInstanceRequest added in v0.0.6

type StartInstanceRequest struct {
	Entrypoint []string // Override entrypoint (nil = keep previous/image default)
	Cmd        []string // Override cmd (nil = keep previous/image default)
}

StartInstanceRequest is the domain request for starting a stopped instance

type State

type State string

State represents the instance state

const (
	StateStopped      State = "Stopped"      // No VMM, no snapshot
	StateCreated      State = "Created"      // VMM created but not booted (CH native)
	StateInitializing State = "Initializing" // VM running, guest init in progress
	StateRunning      State = "Running"      // Guest program started and ready
	StatePaused       State = "Paused"       // VM paused (CH native)
	StateShutdown     State = "Shutdown"     // VM shutdown, VMM exists (CH native)
	StateStandby      State = "Standby"      // No VMM, snapshot exists
	StateUnknown      State = "Unknown"      // Failed to determine state (VMM query failed)
)

func (State) CanTransitionTo

func (s State) CanTransitionTo(target State) error

CanTransitionTo checks if a transition from current state to target state is valid

func (State) IsTerminal

func (s State) IsTerminal() bool

IsTerminal returns true if this state represents a terminal transition point

func (State) RequiresVMM

func (s State) RequiresVMM() bool

RequiresVMM returns true if this state requires a running VMM process

func (State) String

func (s State) String() string

String returns the string representation of the state

type StoredMetadata

type StoredMetadata struct {
	// Identification
	Id    string // Auto-generated CUID2
	Name  string
	Image string // OCI reference as supplied by the caller (tag or digest); the display/API value

	// ResolvedImage is the digest-pinned reference (repo@sha256:...) of the
	// image actually resolved at create time. It is the source of truth for
	// locating the rootfs on boot/start/restore, so a mutable tag (last-pull-wins)
	// can't drift the instance to a different image/arch across restarts. Internal;
	// not exposed on the API. Empty for instances created before this field existed
	// (callers fall back to Image via bootImageRef).
	ResolvedImage string

	// Platform is the resolved image platform as os/arch[/variant] (e.g.
	// "linux/amd64"), captured from the pulled image's metadata at create time.
	// Read-only; echoed on the instance API.
	Platform string

	// Resources (matching Cloud Hypervisor terminology)
	Size                     int64 // Base memory in bytes
	HotplugSize              int64 // Hotplug memory in bytes
	OverlaySize              int64 // Overlay disk size in bytes
	Vcpus                    int
	NetworkBandwidthDownload int64 // Download rate limit in bytes/sec (external→VM), 0 = auto
	NetworkBandwidthUpload   int64 // Upload rate limit in bytes/sec (VM→external), 0 = auto
	DiskIOBps                int64 // Disk I/O rate limit in bytes/sec, 0 = auto

	// Configuration
	Env            map[string]string
	Tags           tags.Tags // User-defined key-value tags
	NetworkEnabled bool      // Whether instance has networking enabled (uses default network)
	NetworkEgress  *NetworkEgressPolicy
	Credentials    map[string]CredentialPolicy
	IP             string // Assigned IP address (empty if NetworkEnabled=false)
	MAC            string // Assigned MAC address (empty if NetworkEnabled=false)

	// Attached volumes
	Volumes []VolumeAttachment // Volumes attached to this instance

	// Timestamps (stored for historical tracking)
	CreatedAt time.Time
	StartedAt *time.Time // Boot epoch start time (set on create/start; preserved across standby restore)
	StoppedAt *time.Time // Last time VM was stopped

	// Boot progress markers (derived from guest serial log sentinels and persisted)
	ProgramStartedAt  *time.Time // Set when guest program handoff/start boundary is reached
	GuestAgentReadyAt *time.Time // Set when guest-agent is ready (unless skip_guest_agent=true)

	// Versions
	KernelVersion string // Kernel version (e.g., "ch-v6.12.9")

	// Hypervisor configuration
	HypervisorType    hypervisor.Type // Hypervisor type (e.g., "cloud-hypervisor")
	HypervisorVersion string          // Hypervisor version (e.g., "v51.1")
	HypervisorPID     *int            // Hypervisor process ID (may be stale after host restart)

	// Firecracker UFFD snapshot restore metadata.
	FirecrackerSnapshotCacheKey           string
	FirecrackerUseUFFDOnNextRestore       bool
	FirecrackerUFFDSessionID              string
	FirecrackerUFFDPagerVersion           string
	FirecrackerDeferredSnapshotMemoryPath string

	// Paths
	SocketPath string // Path to API socket
	DataDir    string // Instance data directory

	// vsock configuration
	VsockCID    int64  // Guest vsock Context ID
	VsockSocket string // Host-side vsock socket path

	// Attached devices (GPU passthrough)
	Devices []string // Device IDs attached to this instance

	// GPU configuration (vGPU mode)
	GPUProfile  string // vGPU profile name (e.g., "L40S-1Q")
	GPUMdevUUID string // mdev device UUID

	// Command overrides (like docker run <image> <command>)
	Entrypoint []string // Override image entrypoint (nil = use image default)
	Cmd        []string // Override image cmd (nil = use image default)

	// Boot optimizations
	SkipKernelHeaders bool // Skip kernel headers installation (disables DKMS)
	SkipGuestAgent    bool // Skip guest-agent installation (disables exec/stat API)

	// EnableRosetta attaches an Apple Rosetta virtio-fs share so the guest can
	// execute x86-64 binaries. vz on Apple silicon only. Derived internally when
	// the image platform differs from the host; not a user-facing field.
	EnableRosetta bool

	// Snapshot policy defaults for this instance.
	SnapshotPolicy *SnapshotPolicy

	// Pending standby compression plan for the latest standby snapshot.
	// Persisted so server restarts can recover delayed or interrupted jobs.
	PendingStandbyCompression *PendingStandbyCompression

	// Automatic standby policy driven by host-observed inbound TCP activity.
	AutoStandby *autostandby.Policy

	// Workload health check policy. Health is reported separately from lifecycle state.
	HealthCheck *healthcheck.Policy

	// Whole-instance restart supervision policy and runtime status.
	RestartPolicy *restartpolicy.Policy
	RestartStatus restartpolicy.Status

	// Shutdown configuration
	StopTimeout int // Grace period in seconds for graceful stop (0 = use default 5s)

	// Exit information (populated from serial console sentinel when VM stops)
	ExitCode    *int   // App exit code, nil if VM hasn't exited
	ExitMessage string // Human-readable description of exit (e.g., "command not found", "killed by signal 9 (SIGKILL) - OOM")

	// Cumulative time spent in each lifecycle phase. Updated at every state
	// transition by transition orchestration sites (create/start/stop/standby/
	// restore/fork). Consumers use Snapshot() to read live values that include
	// time accrued in the current phase. See lib/instances/phasetracking.
	Phases phasetracking.Tracker `json:"phases,omitempty"`
}

StoredMetadata represents instance metadata that is persisted to disk

type UpdateInstanceRequest added in v0.1.0

type UpdateInstanceRequest struct {
	Env              map[string]string     // Updated environment variables (merged with existing)
	AutoStandby      *autostandby.Policy   // Replaces the persisted auto-standby policy when non-nil
	HealthCheck      *healthcheck.Policy   // Replaces the persisted health check policy when non-nil
	RestartPolicy    *restartpolicy.Policy // Replaces the persisted restart policy when non-nil
	RestartPolicySet bool                  // True when restart policy was present in the update request
}

UpdateInstanceRequest is the domain request for updating mutable instance properties.

type VolumeAttachment

type VolumeAttachment struct {
	VolumeID    string // Volume ID
	MountPath   string // Mount path in guest
	Readonly    bool   // Whether mounted read-only
	Overlay     bool   // If true, create per-instance overlay for writes (requires Readonly=true)
	OverlaySize int64  // Size of overlay disk in bytes (max diff from base)
}

VolumeAttachment represents a volume attached to an instance

type WaitForStateResult added in v0.1.0

type WaitForStateResult struct {
	State      State
	StateError *string
	TimedOut   bool
}

WaitForStateResult is the outcome of a WaitForState call.

func WaitForState added in v0.1.0

func WaitForState(ctx context.Context, mgr Manager, inst *Instance, targetState State, timeout time.Duration) (*WaitForStateResult, error)

WaitForState subscribes to lifecycle events for the instance and waits until it reaches targetState, a terminal/error state is detected, the timeout expires, or the context is cancelled. A polling fallback (every 5s) guards against missed or dropped events.

Directories

Path Synopsis
Package phasetracking accumulates cumulative time-in-phase for instance lifecycle phases.
Package phasetracking accumulates cumulative time-in-phase for instance lifecycle phases.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL