overlay

package
v0.17.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 17, 2026 License: MPL-2.0 Imports: 22 Imported by: 0

Documentation

Overview

Package overlay provisions the drives of an overlay volume: a read-only EROFS base shared by every microvm of one image, and a writable ext4 disk per microvm. Service implements ports.VolumeService over them.

A base is one EROFS file built from the layers of an OCI image. The build never unpacks the image. Each layer is converted to an EROFS file from its tar stream, and the layer files are merged into one image. The build is deterministic: two builds of one image with one profile give files with the same sha256.

A base is keyed by the rootfs chain ID of the image and the name of the profile that fixes the build inputs. A writable disk is a reflink of a template formatted once per size. The on-host layout under the state root is:

bases/<sha256>.img                                 mode 0400
bases/by-source/<chain-id>/<profile> -> ../../<sha256>.img
bases/.locks/<key>.lock                            one flock per key
templates/rw-<size-mb>.img                         ext4, mode 0600
templates/.locks/rw-<size-mb>.lock                 one flock per size
scratch/                                           build temp
vm/<ns>/<name>/<uid>/volumes/root.base -> ../../../../../bases/<sha256>.img
vm/<ns>/<name>/<uid>/volumes/root.rw               reflink of a template

The modification time of a by-source link is the last use of its base. The sweep removes a base that no microvm links to once that time is older than the retention period. Every lock is a flock, not a mutex, because the daemon runs more than one set of ports.

The package imports no containerd code. The layers come from a LayerSource, which infrastructure/containerd implements.

Index

Constants

View Source
const (

	// MkfsExt4 is the default mkfs.ext4, found on PATH.
	MkfsExt4 = "mkfs.ext4"
)
View Source
const ProfileNameErofsV1 = "erofs-v1"

ProfileNameErofsV1 is the name of the first base profile.

Variables

View Source
var (
	// ErrReflinkNotSupported reports a state directory whose filesystem
	// cannot clone files with FICLONE. It is a configuration error: there is
	// no fallback to a full copy, because that would hide a misprovisioned
	// host.
	ErrReflinkNotSupported = errors.New("filesystem does not support reflinks")
	// ErrDiskTooSmall reports a writable disk size under the minimum.
	ErrDiskTooSmall = errors.New("writable disk size is under the minimum")
)

Functions

func BaseUUID

func BaseUUID(chainID digest.Digest, profile string) uuid.UUID

BaseUUID is the filesystem UUID of a merged base. It is a version 5 UUID of "<chain-id>/<profile>", for example "sha256:0123.../erofs-v1", in the namespace of the package.

func CheckMkfsErofs

func CheckMkfsErofs(ctx context.Context, path string) error

CheckMkfsErofs runs the mkfs.erofs at path with -V and checks that it reports the erofs-utils version of the erofs-v1 profile. It is the startup probe of the base builder.

func ClearScratch

func ClearScratch(stateRoot string) error

ClearScratch removes everything under scratch/ of the state root. The daemon calls it once at startup: a build that a crash interrupted left its files there, and the kernel dropped its lock.

func LayerUUID

func LayerUUID(diffID digest.Digest) uuid.UUID

LayerUUID is the filesystem UUID of the EROFS file built from one layer. It is a version 5 UUID of the diff ID string, for example "sha256:0123...", in the namespace of the package.

func ProbeReflink(dir string) error

ProbeReflink checks that FICLONE works in dir. It writes a small file, clones it and removes both. The error wraps ErrReflinkNotSupported when the filesystem of dir cannot clone.

Types

type Base

type Base struct {
	// Digest is the sha256 of the base file in hex.
	Digest string
	// Path is the absolute path of the base file, bases/<sha256>.img.
	Path string
	// ChainID is the rootfs chain ID of the image the base was built from.
	ChainID digest.Digest
	// Profile is the name of the profile the base was built with.
	Profile string
}

Base describes a built base on the host.

type Builder

type Builder struct {
	// contains filtered or unexported fields
}

Builder builds and finds the bases of overlay volumes under a state root.

func NewBuilder

func NewBuilder(stateRoot, mkfsPath string, source LayerSource) (*Builder, error)

NewBuilder returns a Builder that keeps bases under stateRoot, runs the mkfs.erofs at mkfsPath and reads image layers from source. It creates the bases and scratch directories. The profile is erofs-v1.

func (*Builder) Build

func (b *Builder) Build(ctx context.Context, in LayerInput) (*Base, error)

Build returns the base of the image. It builds the base when the index has none for the image's chain ID and the profile. One flock per key serialises builds across processes: a second caller waits and then finds the finished base. A found base has its last-use time refreshed.

type Config

type Config struct {
	// StateRoot is the state directory of the daemon. Bases, templates,
	// scratch files and the volumes of each microvm live under it.
	StateRoot string
	// MkfsErofsPath is the pinned mkfs.erofs the base builder runs.
	MkfsErofsPath string
	// MkfsExt4Path is the mkfs.ext4 that formats writable disk templates.
	// An empty value means MkfsExt4 on PATH.
	MkfsExt4Path string
	// DefaultRWSizeMB is the size of a writable disk in megabytes when the
	// spec gives none.
	DefaultRWSizeMB int32
	// BaseRetention is how long a base with no microvm link is kept after
	// its last use.
	BaseRetention time.Duration
}

Config configures the overlay volume service.

type DiffIDError

type DiffIDError struct {
	// LayerIndex is the position of the layer, lowest layer first from 0.
	LayerIndex int
	// Want is the diff ID the source claimed.
	Want digest.Digest
	// Got is the digest of the stream the source gave.
	Got digest.Digest
}

DiffIDError reports a layer whose decompressed tar stream does not hash to the diff ID its source claimed. The chain ID, and so the base key, is derived from the claimed diff IDs, so a mismatch would record a base under a key that does not identify its bytes. The build fails instead.

func (*DiffIDError) Error

func (e *DiffIDError) Error() string

type Entry

type Entry struct {
	// Key is the chain ID and profile of the link.
	Key baseKey
	// Digest is the sha256 the link target names.
	Digest string
	// Path is the absolute path of the base file the link points to. The
	// file can be missing.
	Path string
	// LastUse is the modification time of the link, which Touch refreshes.
	LastUse time.Time
}

Entry is one by-source link of the index: the key, the base it points to and the time of its last use.

type Ext4Error

type Ext4Error struct {
	// Args is the command line without the program path.
	Args []string
	// Stderr is the standard error of the command, trimmed.
	Stderr string
	// Err is the error exec returned.
	Err error
}

Ext4Error reports a mkfs.ext4 run that exited with an error. Stderr holds what the command printed.

func (*Ext4Error) Error

func (e *Ext4Error) Error() string

func (*Ext4Error) Unwrap

func (e *Ext4Error) Unwrap() error

type HardlinkError

type HardlinkError struct {
	// LayerIndex is the position of the layer, lowest layer first from 0.
	LayerIndex int
	// DiffID is the diff ID of the layer.
	DiffID digest.Digest
	// Entry is the path of the hardlink entry in the layer.
	Entry string
	// Target is the path the entry links to.
	Target string
}

HardlinkError reports a layer entry that is a hardlink to a path the layer does not hold. The merge cannot link into a lower layer, so the build fails.

func (*HardlinkError) Error

func (e *HardlinkError) Error() string

type ImageLayers

type ImageLayers struct {
	// ChainID is the OCI chain ID of the diff IDs, lowest layer first.
	ChainID digest.Digest
	// Layers holds the layers, lowest first.
	Layers []Layer
}

ImageLayers is the rootfs of an image as a chain of layers.

type Layer

type Layer struct {
	// DiffID is the sha256 digest of the uncompressed tar stream of the
	// layer. The builder checks the stream against it.
	DiffID digest.Digest
	// Open returns a new uncompressed tar stream of the layer. The caller
	// closes the stream. Open can be called more than once.
	Open func(ctx context.Context) (io.ReadCloser, error)
}

Layer is one layer of an image.

type LayerInput

type LayerInput struct {
	// ImageRef is the OCI image reference, for example
	// ghcr.io/liquidmetal-dev/ubuntu:24.04.
	ImageRef string
	// Owner identifies the VM the layers are read for. The containerd
	// implementation holds the image content under this owner's lease.
	Owner string
}

LayerInput names the image whose layers a LayerSource returns.

type LayerSource

type LayerSource interface {
	// Layers pulls the image if needed and returns its rootfs chain ID and
	// one uncompressed tar stream per layer, lowest first.
	Layers(ctx context.Context, in LayerInput) (*ImageLayers, error)
}

LayerSource gives the builder the layers of an image.

The builder trusts no source for the content of a layer. It hashes each stream as it converts it and fails the build when the digest is not the layer's DiffID. The chain ID is derived from the diff IDs, so a recorded base is keyed by the identity of the bytes that were built, whatever the source claimed.

type MkfsError

type MkfsError struct {
	// Args is the command line without the program path.
	Args []string
	// Stderr is the standard error of the command, trimmed.
	Stderr string
	// Err is the error exec returned.
	Err error
}

MkfsError reports a mkfs.erofs run that exited with an error. Stderr holds what the command printed.

func (*MkfsError) Error

func (e *MkfsError) Error() string

func (*MkfsError) Unwrap

func (e *MkfsError) Unwrap() error

type Profile

type Profile struct {
	// Name is the profile name. It is part of the base key.
	Name string
	// ErofsUtilsVersion is the version mkfs.erofs must report. The builder
	// refuses any other version.
	ErofsUtilsVersion string
	// BlockSize is the EROFS block size in bytes. It is passed explicitly
	// because the default of mkfs.erofs is the page size of the build host.
	BlockSize int
}

Profile fixes every input of a base build. A base is keyed by the chain ID of its image and the name of the profile. A change to any input is a new profile name and so a new base identity.

func ErofsV1

func ErofsV1() Profile

ErofsV1 returns the profile erofs-v1: erofs-utils 1.9.4, 4096-byte blocks, no compression, a zero build time with the file times of the layers kept, and the UUIDs of LayerUUID and BaseUUID.

type Reference

type Reference struct {
	// Digest is the sha256 of the base the link resolves to.
	Digest string
	// Link is the path of the VM's root.base symlink.
	Link string
}

Reference is a VM's link to a base.

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service is the overlay implementation of ports.VolumeService.

func NewService

func NewService(cfg Config, source LayerSource) (*Service, error)

NewService returns a Service over the state root of cfg that builds bases from source. It creates the directories of the on-host layout.

func (*Service) Create

Create builds or finds the base of the image, links it as root.base in the volumes directory of the microvm, and reflinks the writable disk template of the requested size to root.rw. A root.rw that exists is kept: it can hold guest writes. A root.base that resolves to the base is kept.

func (*Service) Release

func (s *Service) Release(ctx context.Context, ref ports.VolumeRef) error

Release removes the writable disk and the base link of the microvm. A file that is already gone is a success. It then sweeps unused bases.

func (*Service) Sweep

func (s *Service) Sweep(ctx context.Context) error

Sweep removes every base that no microvm links to and whose last use is older than the retention period. It takes the per-key lock before it removes a base and rechecks the references and the last use under the lock, so a Create that is linking the base wins. A base file with no by-source link is removed when it is older than the retention period and unreferenced.

type VersionError

type VersionError struct {
	// Path is the mkfs.erofs that was run.
	Path string
	// Reported is the version line mkfs.erofs printed.
	Reported string
	// Want is the version the profile requires.
	Want string
}

VersionError reports a mkfs.erofs whose version is not the one the profile fixes.

func (*VersionError) Error

func (e *VersionError) Error() string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL