uffdpager

UFFD Snapshot Pager

The UFFD pager lets Firecracker restore snapshot memory lazily. A restored VM gets a per-session UFFD socket, and page faults are served from the snapshot memory file by a local pager process.

The pager is local to the host. It is not backed by Redis or another external cache because page faults are latency-sensitive and the kernel-facing UFFD socket is local. The process keeps one shared in-memory page cache, bounded by hypervisor.firecracker_uffd_cache_max_bytes. Cache entries are keyed by a snapshot cache key plus page offset, so restore sessions with the same propagated snapshot cache key can reuse hot pages without starting one pager per snapshot.

UFFD is opt-in through hypervisor.firecracker_snapshot_memory_backend=uffd. The default backend remains file. Enabling UFFD does not change already running VMs; it only changes future Firecracker snapshot restores. If a restore uses UFFD, the VM is pinned to the pager session created for that restore until the instance stops, is deleted, or otherwise closes the session.

The pager version is declared in lib/uffdpager/VERSION. Installed Linux hosts run versioned systemd units named hypeman-uffd@<version>.service. Hypeman connects to the pager version from VERSION, so regular Hypeman releases do not start a new pager unless the UFFD pager version changes. Older pager versions are drained: they reject new sessions but continue serving existing sessions until those sessions close. Systemd runs the pager through the dedicated hypeman-uffd-pager binary rather than an alternate API-server mode. UFFD requires that systemd unit template to be installed; Hypeman does not launch an unmanaged pager subprocess if the template is missing.

Control HTTP API

Hypeman talks to the pager over Unix HTTP at <data_dir>/uffd/<version>/control.sock:

• GET /health
• GET /stats
• POST /sessions
• POST /sessions/{id}/close
• POST /sessions/{id}/complete
• POST /drain

POST /sessions/{id}/complete populates every outstanding page of a session from the backing file and then unregisters userfaultfd, so the restored VM stops depending on this pager and keeps running on resident memory. The VM is never paused and its network is untouched. Completion reads the whole remaining memory image, so it is paced by the caller; the request is bounded by its context rather than a fixed timeout. Unregister happens only after a full populate, because once a range is unregistered the kernel zero-fills any still-absent page.

Firecracker does not use this control socket directly. Each restore session gets its own Unix socket that receives Firecracker's UFFD file descriptor and memory regions.

The instance health check currently verifies that the pager version is healthy; it does not prove that a particular UFFD session still exists after an unplanned pager restart. If a pager disappears while serving active UFFD VMs, those VMs should be treated as unhealthy and recycled.