ch

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 9, 2026 License: Apache-2.0 Imports: 16 Imported by: 0

Documentation

Overview

Package ch drives a single cloud-hypervisor instance over its REST api-socket: pause, snapshot, resume against a running VMM (e.g. the socket kata creates at /run/vc/vm/<id>/clh-api.sock), plus relaunching a fresh VMM from a snapshot directory for restore.

This is the snapshot/restore half of the ateom-microvm model: kata owns RUN (boot the micro-VM + run the OCI container), and ateom drives the CH REST API underneath for suspend (pause+snapshot) and owns the bare-CH relaunch for restore (see LaunchVMM + RestoreWithNetFDs in restorefds.go). The REST wire format is the one cloud-hypervisor documents for snapshot/restore.

Index

Constants

View Source 
const (
	// MemRestoreOnDemand faults pages in as the guest touches them, so an idle
	// restored guest holds its working set rather than its whole snapshot.
	MemRestoreOnDemand = "OnDemand"
	// MemRestoreEager reads the snapshot's populated extents up front. It registers
	// no userfaultfd, so nothing prefaults and nothing gates a later snapshot.
	MemRestoreEager = "Copy"
)

Guest RAM restore modes accepted by vm.restore.

Variables

This section is empty.

Functions

func MergeDeltaIntoBase 

func MergeDeltaIntoBase(ctx context.Context, baseFile, deltaFile string) error

MergeDeltaIntoBase overlays deltaFile's populated pages onto baseFile in place and leaves the complete merged snapshot at deltaFile's path — the same result as MergeSparseOverlay, but WITHOUT copying baseFile's working set on every suspend.

baseFile is the per-actor restore staging file (restore-state/memory-ranges), demand-paged only by the now-paused CH we are about to tear down and discarded afterward. So rather than `cp`-ing its whole working set (e.g. ~150MiB of a 2GiB guest, ~0.8s on the suspend critical path), we rename baseFile next to deltaFile, overlay deltaFile's (small) faulted pages onto it, and swap it into deltaFile's place — turning an O(working-set) copy into an O(delta) write plus two renames.

baseFile and deltaFile are siblings under the actor dir (restore-state/ and checkpoint-state/), so the renames are same-filesystem (metadata-only). If they straddle a mount boundary (EXDEV) it falls back to the copying MergeSparseOverlay (baseFile is untouched until the first rename succeeds).

func MergeSparseOverlay 

func MergeSparseOverlay(ctx context.Context, baseFile, deltaFile, outFile string) error

MergeSparseOverlay reconstructs a COMPLETE memory snapshot from an OnDemand (userfaultfd) restore. CH's new snapshot (deltaFile) contains only the pages the guest faulted in since the OnDemand restore; every other page is unchanged from the snapshot it restored FROM (baseFile). So the complete current memory = baseFile, with deltaFile's populated pages overlaid.

It writes outFile = a sparse copy of baseFile, then overlays every DATA region of deltaFile (located via SEEK_DATA/SEEK_HOLE, so holes — the un-faulted pages — are skipped) at the same byte offsets. baseFile and deltaFile MUST be flat images of identical size and layout (CH memory-ranges of the same guest + CH version), which holds across a restore/snapshot of one actor. This is a Firecracker-style differential snapshot implemented on top of CH (which has no native diff snapshot): it keeps OnDemand's fast, non-densifying restore while still producing complete, re-restorable snapshots for the suspend/resume chain.

func SnapshotURL 

func SnapshotURL(dir string) string

SnapshotURL returns the file:// URL cloud-hypervisor expects for a snapshot destination or restore source directory.

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client talks to one cloud-hypervisor VMM over its unix api-socket.

func LaunchVMM 

func LaunchVMM(ctx context.Context, o LaunchVMMOptions) (*exec.Cmd, *Client, error)

LaunchVMM starts a cloud-hypervisor process with only an api-socket (no VM) and waits until it answers. Use Client.RestoreWithNetFDs to then restore a snapshot that has fd-backed net devices. The caller owns cmd.

func NewClient 

func NewClient(apiSocket string) *Client

NewClient returns a Client bound to a cloud-hypervisor api-socket path. The socket need not exist yet; use WaitReady to block until the VMM answers.

func (*Client) AddNetWithFDs 

func (c *Client) AddNetWithFDs(ctx context.Context, mac string, numQueues int, fds []int) error

AddNetWithFDs hotplugs a virtio-net device into a freshly-created (pre-boot or running) VM, passing the tap FDs via SCM_RIGHTS — the boot-path analog of RestoreWithNetFDs. kata adds net this way between vm.create and vm.boot (clh.go vmAddNetPut). mac may be empty (CH assigns one); numQueues should be 2*queuePairs (rx+tx) and len(fds) == queuePairs.

func (*Client) BootVM 

func (c *Client) BootVM(ctx context.Context) error

BootVM boots a created VM via /api/v1/vm.boot (transitions Created -> Running).

func (*Client) CreateVM 

func (c *Client) CreateVM(ctx context.Context, cfg VmConfig) error

CreateVM creates (but does not boot) the VM from cfg via /api/v1/vm.create. The VMM must already be up (LaunchVMM). After this the VM is in "Created".

func (*Client) Info 

func (c *Client) Info() VMMInfo

Info returns what the VMM last reported about itself, zero until a successful Ping or WaitReady. A Client belongs to one actor's VMM and is used from that actor's goroutine, so this needs no synchronization.

func (*Client) Pause 

func (c *Client) Pause(ctx context.Context) error

Pause pauses the running guest (quiescing it before snapshot). Idempotent: already-paused is success (CH itself 500s on pausing a paused VM, which would otherwise wedge checkpoint retries after a partial earlier attempt).

func (*Client) Ping 

func (c *Client) Ping(ctx context.Context) (VMMInfo, error)

Ping reports what the VMM says about itself, or an error if the api-socket does not answer vmm.ping.

func (*Client) RestoreWithNetFDs 

func (c *Client) RestoreWithNetFDs(ctx context.Context, sourceDir string, nets []RestoredNet, memMode string) error

RestoreWithNetFDs issues vm.restore for a snapshot dir, passing fresh tap FDs for the snapshot's fd-backed net devices via SCM_RIGHTS on the api-socket (the only way CH accepts net FDs on restore; mirrors ch-remote's send_with_fds). The VM comes back paused; call Resume after.

memMode selects guest-RAM restore: "" / "Copy" = eager copy (CH default), or "OnDemand" = userfaultfd demand-paging. OnDemand keeps the (memfd-backed) guest memory SPARSE — it only faults in the pages the guest touches, instead of eager copy densifying the whole memfd — so a subsequent snapshot writes just the working set (fast) instead of full RAM. Confirmed on CH v52: the REST RestoreConfig accepts memory_restore_mode (enum Copy|OnDemand) alongside the SCM_RIGHTS net_fds, so ondemand + fd-backed net DO compose over REST (an earlier note claimed memory_restore_mode was CLI-only; that was a pre-v52 limitation). NOTE: with OnDemand, CH demand-pages from the snapshot's memory file for the VM's whole lifetime, so sourceDir must stay present until the actor is torn down.

func (*Client) Resume 

func (c *Client) Resume(ctx context.Context) error

Resume resumes a paused guest (after snapshot or restore).

func (*Client) Shutdown 

func (c *Client) Shutdown(ctx context.Context) error

Shutdown best-effort tears down the VM and the VMM process behind the socket.

func (*Client) Snapshot 

func (c *Client) Snapshot(ctx context.Context, destDir string) error

Snapshot writes the (paused) guest's state to destDir as a CH snapshot (config.json + state.json + memory-ranges). The guest must be paused first.

func (*Client) State 

func (c *Client) State(ctx context.Context) (string, error)

State returns the VM state as reported by vm.info (e.g. "Running", "Paused").

func (*Client) WaitReady 

func (c *Client) WaitReady(ctx context.Context, deadline time.Duration) (VMMInfo, error)

WaitReady blocks until the api-socket answers vmm.ping or the deadline passes, returning what that answer said. Callers get the VMM's version for free this way: the handshake already happens before every boot and restore, so nothing has to run the binary again to ask.

type ConsoleConfig 

type ConsoleConfig struct {
	Mode string `json:"mode"`
	File string `json:"file,omitempty"`
}

ConsoleConfig is a serial/console device. Mode "Off" disables it; "File" with File set captures the guest console (for boot debugging); "Tty" to a pty.

type CpusConfig 

type CpusConfig struct {
	BootVcpus int32 `json:"boot_vcpus"`
	MaxVcpus  int32 `json:"max_vcpus"`
}

CpusConfig sets the boot/max vCPU counts.

type DiskConfig 

type DiskConfig struct {
	Path      string `json:"path"`
	Readonly  bool   `json:"readonly"`
	Direct    bool   `json:"direct"`
	NumQueues int32  `json:"num_queues,omitempty"`
	QueueSize int32  `json:"queue_size,omitempty"`
	ImageType string `json:"image_type,omitempty"`
}

DiskConfig is one virtio-blk disk. The only disk is the kata guest image (/dev/vda, read-only); the actor rootfs is an overlay served over virtio-fs, not a disk. NumQueues/QueueSize mirror kata's clh (num_queues = vcpus, queue_size = 1024).

type FsConfig 

type FsConfig struct {
	Tag        string `json:"tag"`
	Socket     string `json:"socket"`
	NumQueues  int32  `json:"num_queues,omitempty"`
	QueueSize  int32  `json:"queue_size,omitempty"`
	PciSegment int32  `json:"pci_segment,omitempty"`
}

FsConfig is a virtio-fs device backed by a vhost-user (virtiofsd) socket. The overlay rootfs path uses it as the RO lower; the guest mounts it via the FsTag.

type LaunchVMMOptions 

type LaunchVMMOptions struct {
	// Binary is the cloud-hypervisor executable (defaults to "cloud-hypervisor").
	Binary string
	// APISocket is the api-socket path the new VMM should listen on.
	APISocket string
	// Stdout/Stderr receive the VMM's output.
	Stdout, Stderr interface{ Write([]byte) (int, error) }
}

LaunchVMMOptions configures starting a bare VMM (no VM) for an FD-passing restore.

type MemoryConfig 

type MemoryConfig struct {
	Size   int64 `json:"size"`
	Shared bool  `json:"shared"`
}

MemoryConfig sets guest RAM. Shared=true makes CH back RAM with a memfd, which is what lets vm.snapshot write a SPARSE image (the memory-only snapshot the rest of ateom relies on).

type PayloadConfig 

type PayloadConfig struct {
	Kernel  string `json:"kernel"`
	Cmdline string `json:"cmdline"`
}

PayloadConfig points at the guest kernel + its cmdline (initramfs/firmware unused: the kata guest boots from a virtio-blk image disk, root=/dev/vda1).

type PlatformConfig 

type PlatformConfig struct {
	NumPciSegments int32 `json:"num_pci_segments,omitempty"`
}

PlatformConfig sets VM-wide platform options. NumPciSegments must be >1 when a virtio-fs device sits on a non-zero PCI segment (kata puts fs on segment 1).

type RestoredNet 

type RestoredNet struct {
	// ID is the device id from the snapshot's config.json (e.g. "_net1").
	ID string
	// FDs are open tap fds (one per queue pair) for CH to adopt.
	FDs []int
}

RestoredNet identifies one fd-backed network device in a snapshot and the fresh tap FDs to back it with on restore. kata boots CH virtio-net devices from tap FDs, so the snapshot's config requires net_fds on restore (RestoreMissingRequiredNetId otherwise); CH reopens the device on the FDs we pass over the api-socket via SCM_RIGHTS.

type RngConfig 

type RngConfig struct {
	Src string `json:"src"`
}

RngConfig sets the entropy source (kata uses /dev/urandom).

type SnapshotNetDevice 

type SnapshotNetDevice struct {
	// ID is the CH device id (e.g. "_net1").
	ID string
	// QueuePairs is the number of tap FDs the device needs (num_queues/2).
	QueuePairs int
	// MAC is the guest-visible MAC address of the device.
	MAC string
}

SnapshotNetDevice describes one net device found in a CH snapshot's config.json. Restore must supply net_fds for every one of them.

func SnapshotNetDevices 

func SnapshotNetDevices(snapshotDir string) ([]SnapshotNetDevice, error)

SnapshotNetDevices parses a CH snapshot's config.json and returns its net devices, in order.

type VMMInfo 

type VMMInfo struct {
	Version      string   `json:"version"`
	BuildVersion string   `json:"build_version"`
	Features     []string `json:"features"`
}

VMMInfo is what vmm.ping reports about the running VMM. Version is a semver ("53.0.0"); BuildVersion is the release tag it was built from ("v53.0").

func (VMMInfo) AdvancesGuestClockOnRestore 

func (i VMMInfo) AdvancesGuestClockOnRestore() bool

AdvancesGuestClockOnRestore reports whether this VMM repairs the guest clock across a restore by itself.

It reports false when the version cannot be read or parsed, which is the safe direction: a caller that wrongly believes the VMM corrects the clock leaves the guest reading a stale time after every resume, with no error to show for it.

func (VMMInfo) PrefaultsUnconditionally 

func (i VMMInfo) PrefaultsUnconditionally() bool

PrefaultsUnconditionally reports whether this VMM prefaults an OnDemand restore.

It reports true when the version cannot be read or parsed. The two ways to be wrong are not equal: choosing OnDemand on an affected version leaves the guest unable to pass its readiness probe, because the prefault storm starves it, while choosing eager on an unaffected one merely costs memory. Callers should log when they fall back on an unknown version, since that cost is otherwise invisible.

type VmConfig 

type VmConfig struct {
	Cpus     CpusConfig      `json:"cpus"`
	Memory   MemoryConfig    `json:"memory"`
	Payload  PayloadConfig   `json:"payload"`
	Disks    []DiskConfig    `json:"disks,omitempty"`
	Fs       []FsConfig      `json:"fs,omitempty"`
	Rng      *RngConfig      `json:"rng,omitempty"`
	Serial   *ConsoleConfig  `json:"serial,omitempty"`
	Console  *ConsoleConfig  `json:"console,omitempty"`
	Vsock    *VsockConfig    `json:"vsock,omitempty"`
	Platform *PlatformConfig `json:"platform,omitempty"`
}

VmConfig is the body of /api/v1/vm.create — the subset of cloud-hypervisor's VmConfig ateom sets to boot the kata guest. Modeled on kata's clh driver (src/runtime/virtcontainers/clh.go). vm.create + vm.boot are issued with PUT.

type VsockConfig 

type VsockConfig struct {
	Cid    int64  `json:"cid"`
	Socket string `json:"socket"`
}

VsockConfig is the hybrid-vsock the kata-agent listens on. Cid is the guest CID (kata uses 3); Socket is the host unix socket (kata.VsockSocketPath) that ateom then dials (DialAgent) to drive the agent.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.