provision

package
v0.17.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 17, 2026 License: MPL-2.0 Imports: 21 Imported by: 0

Documentation

Overview

Package provision contains the business logic used to bootstrap a host for running flintlock: installing Firecracker/Cloud Hypervisor/containerd/ flintlockd, setting up their systemd services, and configuring a devicemapper thinpool. It has no dependency on the CLI framework used to expose it - that wiring lives in internal/command/provision.

Index

Constants

View Source
const (
	// DefaultVersion is used to indicate that the latest release of a
	// component should be installed.
	DefaultVersion = "latest"

	// DefaultBranch is the branch used when fetching raw files (such as
	// systemd unit files) from GitHub repositories.
	DefaultBranch = "main"

	// InstallPath is the directory that downloaded binaries are installed to.
	InstallPath = "/usr/local/bin"

	// FirecrackerBin is the name of the firecracker binary.
	FirecrackerBin = "firecracker"
	// FirecrackerRepo is the GitHub repository firecracker releases are published to.
	FirecrackerRepo = "firecracker-microvm/firecracker"
	// FirecrackerVersionEnv is the environment variable used to override the
	// default firecracker version to install.
	FirecrackerVersionEnv = "FIRECRACKER"

	// CloudHypervisorBin is the name of the cloud-hypervisor binary.
	CloudHypervisorBin = "cloud-hypervisor-static"
	// CloudHypervisorRepo is the GitHub repository cloud-hypervisor releases are published to.
	CloudHypervisorRepo = "cloud-hypervisor/cloud-hypervisor"
	// CloudHypervisorVersionEnv is the environment variable used to override
	// the default cloud-hypervisor version to install.
	CloudHypervisorVersionEnv = "CLOUD_HYPERVISOR"

	// ContainerdBin is the name of the containerd binary.
	ContainerdBin = "containerd"
	// ContainerdRepo is the GitHub repository containerd releases are published to.
	ContainerdRepo = "containerd/containerd"
	// ContainerdVersionEnv is the environment variable used to override the
	// default containerd version to install.
	ContainerdVersionEnv = "CONTAINERD"

	// FlintlockBin is the name of the flintlockd binary.
	FlintlockBin = "flintlockd"
	// FlintlockRepo is the GitHub repository flintlock releases are published to.
	FlintlockRepo = "liquidmetal-dev/flintlock"
	// FlintlockVersionEnv is the environment variable used to override the
	// default flintlockd version to install.
	FlintlockVersionEnv = "FLINTLOCK"

	// FlintlockdServiceFile is the path the flintlockd systemd unit is installed to.
	FlintlockdServiceFile = "/etc/systemd/system/flintlockd.service"
	// FlintlockdConfigPath is the path the flintlockd config file is written to.
	FlintlockdConfigPath = "/etc/opt/flintlockd/config.yaml"

	// ThinpoolProfilePath is the directory LVM thinpool profiles are written to.
	ThinpoolProfilePath = "/etc/lvm/profile"
	// DefaultThinpool is the name used for a direct-lvm backed thinpool.
	DefaultThinpool = "flintlock"
	// DefaultDevThinpool is the name used for a loopback backed thinpool.
	DefaultDevThinpool = "flintlock-dev"
	// DataSparseSize is the size of the sparse file backing the devpool data device.
	DataSparseSize = "100G"
	// MetadataSparseSize is the size of the sparse file backing the devpool metadata device.
	MetadataSparseSize = "10G"

	// SectorSize is the sector size (in bytes) used when calculating the devpool thin-pool table.
	SectorSize = 512
	// DataBlockSize is the data block size (in 512-byte sectors) used when creating the devpool thin-pool.
	DataBlockSize = 128
	// LowWaterMark is the free-space threshold (in 512-byte sectors) that triggers a dm-event for the devpool thin-pool.
	LowWaterMark = 32768
)
View Source
const (
	// DefaultFstabPath is the file mounts are persisted in.
	DefaultFstabPath = "/etc/fstab"
	// DefaultStateDirLoopSizeGB is the size of the sparse file behind a
	// loop-backed state directory when the caller gives none.
	DefaultStateDirLoopSizeGB = 20
	// StateDirFilesystem is the filesystem type the provisioner creates
	// for the state directory.
	StateDirFilesystem = reflink.FilesystemXFS
)
View Source
const LVMProfile = `activation {
thin_pool_autoextend_threshold=80
thin_pool_autoextend_percent=20
}
`

LVMProfile is the content written to a thinpool's LVM profile.

Variables

View Source
var (
	// ErrChecksumMismatch is returned when a downloaded file does not match
	// the sha256 published next to it. The file is not installed.
	ErrChecksumMismatch = errors.New("sha256 mismatch")

	// ErrErofsUtilsArch is returned when the host is not linux/amd64, the
	// only architecture flintlock releases ship mkfs.erofs for. arm64 is
	// tracked in https://github.com/liquidmetal-dev/flintlock/issues/1317.
	ErrErofsUtilsArch = errors.New("flintlock releases ship mkfs.erofs for linux/amd64 only " +
		"(arm64 is tracked in https://github.com/liquidmetal-dev/flintlock/issues/1317)")

	// ErrNoReflink is returned when the state directory is on a filesystem
	// that cannot clone files with FICLONE and no device or loop file was
	// given to create one.
	ErrNoReflink = errors.New("state directory has no reflink support")
)
View Source
var AptPackages = []string{"thin-provisioning-tools", "lvm2", "xfsprogs", "git", "curl", "wget"}

AptPackages are the apt packages required to run flintlock.

Functions

func AllDevPool

func AllDevPool(runner *Runner, paths ContainerdPaths, thinpool string) error

AllDevPool sets up a loopback-device backed thin-pool named thinpool+"-thinpool" using paths.PoolData/PoolMetadata as the backing sparse files, matching do_all_devpool.

func AllDirectLVM

func AllDirectLVM(runner *Runner, diskPath, thinpool string) error

AllDirectLVM sets up a direct-lvm backed thinpool on diskPath.

func AllFlintlock

func AllFlintlock(ctx context.Context, runner *Runner, opts AllFlintlockOptions) error

AllFlintlock installs, configures and starts flintlockd, resolving the gRPC address/interface/port from the host when they are not supplied, matching do_all_flintlock.

func ApplyLVMProfile

func ApplyLVMProfile(runner *Runner, thinpool string) error

ApplyLVMProfile creates (if necessary) and applies the LVM profile for thinpool.

func AssociateLoopDevice

func AssociateLoopDevice(runner Executor, sparseFile string) (string, error)

AssociateLoopDevice returns the loop device associated with sparseFile, creating one if none exists yet, matching associate_loop_device.

func BuildContainerdConfig

func BuildContainerdConfig(paths ContainerdPaths, thinpool string) string

BuildContainerdConfig renders the containerd config.toml content for the given paths and thinpool name. An empty thinpool means the host has no pool: the devmapper snapshotter section is left out and containerd runs with its native snapshotter only.

func BuildFlintlockdConfig

func BuildFlintlockdConfig(settings map[string]string) string

BuildFlintlockdConfig renders settings as YAML, matching write_flintlockd_config's output.

func BuildFlintlockdSettings

func BuildFlintlockdSettings(s FlintlockdSettings) map[string]string

BuildFlintlockdSettings returns the flintlockd config settings for the given options, matching write_flintlockd_config's auto-generated options.

func BuildFstabLine added in v0.17.0

func BuildFstabLine(spec, dir, fsType, options string) string

BuildFstabLine renders one fstab entry. Both fsck fields are 0: xfs is checked by its own tools, not by fsck at boot.

func BuildThinPoolTable

func BuildThinPoolTable(lengthSectors int64, metadev, datadev string) string

BuildThinPoolTable renders the dmsetup table line for a thin-pool backed by metadev/datadev, matching create_dev_thinpool's thinp_table.

func Confirm

func Confirm(in io.Reader, out io.Writer, msg string) bool

Confirm asks the user to confirm msg via in, returning true if they answered "y". Unattended callers should skip calling this altogether, matching the script's get_user_confirmation.

func ContainerdReleaseBinName

func ContainerdReleaseBinName(tag, arch string) string

ContainerdReleaseBinName returns the name of the containerd release tarball for the given tag and architecture.

func CreateDevThinPool

func CreateDevThinPool(runner *Runner, thinpool, datadev, metadev string) error

CreateDevThinPool creates (or reloads) the loopback-backed thin-pool device named thinpool from metadev/datadev.

func CreateLogicalVolume

func CreateLogicalVolume(runner *Runner, volumeGroup string) error

CreateLogicalVolume creates and converts the thinpool data/metadata logical volumes into a thin-pool, doing nothing if they already exist.

func CreatePhysicalVolume

func CreatePhysicalVolume(runner *Runner, diskPath string) error

CreatePhysicalVolume creates an LVM physical volume on diskPath, doing nothing if one already exists.

func CreateSparseFile

func CreateSparseFile(path, size string) error

CreateSparseFile creates an empty file of the given size at path, doing nothing if the file already exists, matching create_sparse_file.

func CreateVolumeGroup

func CreateVolumeGroup(runner *Runner, diskPath, thinpool string) error

CreateVolumeGroup creates an LVM volume group named thinpool on diskPath, doing nothing if one already exists.

func DownloadFile

func DownloadFile(ctx context.Context, url, destPath string, perm os.FileMode) error

DownloadFile downloads url and writes it to destPath with the given permissions, replacing the script's "wget -O" based install_release_bin.

func DownloadURL

func DownloadURL(repo, tag, bin string) string

DownloadURL returns the URL of a binary attached to a repository's release.

func DownloadVerified added in v0.17.0

func DownloadVerified(ctx context.Context, url, sumURL, destPath string, perm os.FileMode) error

DownloadVerified downloads url to destPath with the given permissions after checking its sha256 against the digest published at sumURL. The sum file has the sha256sum format: the hex digest, then whitespace and the file name. The download goes to a temporary file next to destPath and is moved into place only when the digest matches, so a mismatch never leaves a file at destPath.

func EnsureKVM

func EnsureKVM() error

EnsureKVM checks that /dev/kvm exists, returning an error if it doesn't.

func ErofsUtilsAssetName added in v0.17.0

func ErofsUtilsAssetName(erofsVersion, arch string) string

ErofsUtilsAssetName returns the name of the mkfs.erofs release asset for the given erofs-utils version and normalised (amd64/arm64) architecture. hack/scripts/build_erofs_utils.sh produces the asset under this name and the release workflow attaches it to the flintlock release together with a "<name>.sha256" file.

func ExtractTarGz

func ExtractTarGz(ctx context.Context, url, destDir string) error

ExtractTarGz downloads the gzipped tarball at url and extracts it into destDir, replacing the script's "curl | tar xz" based install_release_tar.

func FetchServiceFile

func FetchServiceFile(ctx context.Context, repo, service, dest string) error

FetchServiceFile downloads the named systemd unit file from repo's default branch and writes it to dest. Callers that go on to edit dest (e.g. via ReplaceRequires or appendExecStartArg) should call ReloadSystemd themselves once all edits are done, rather than reloading here with an unedited unit.

func FindFreeDisk

func FindFreeDisk(runner *Runner) (string, error)

FindFreeDisk naively finds a spare block device which is not mounted or partitioned. It is not safe to rely on in production - callers should prefer an explicit disk name, matching the script's find_free_disk.

func InstallAptPackages

func InstallAptPackages(runner *Runner) error

InstallAptPackages installs AptPackages via apt.

func InstallCloudHypervisor

func InstallCloudHypervisor(ctx context.Context, runner *Runner, version, normalisedArch string) error

InstallCloudHypervisor downloads and installs the given (or latest) version of cloud-hypervisor for the given normalised (amd64/arm64) architecture to InstallPath.

func InstallContainerd

func InstallContainerd(ctx context.Context, runner *Runner, version, arch string) error

InstallContainerd downloads and installs the given (or latest) version of containerd for arch to InstallPath.

func InstallErofsUtils added in v0.17.0

func InstallErofsUtils(ctx context.Context, runner *Runner, version, arch string) error

InstallErofsUtils downloads the pinned mkfs.erofs (defaults.ErofsUtilsVersion) attached to the given (or latest) flintlock release for arch, verifies it against the published sha256 and installs it at defaults.MkfsErofsPath. A system mkfs.erofs is left alone. The installed binary must report the pinned version.

Releases ship the binary for linux/amd64 only. Any other arch is refused with ErrErofsUtilsArch before anything is downloaded.

func InstallFirecracker

func InstallFirecracker(ctx context.Context, runner *Runner, version, normalisedArch string) error

InstallFirecracker downloads and installs the given (or latest) version of firecracker for the given normalised (amd64/arm64) architecture to InstallPath.

func InstallFlintlockd

func InstallFlintlockd(ctx context.Context, runner *Runner, version, arch string) error

InstallFlintlockd downloads and installs the given (or latest) version of flintlockd for arch to InstallPath.

func LatestReleaseTag

func LatestReleaseTag(ctx context.Context, repo string) (string, error)

LatestReleaseTag returns the tag of the latest release of the given "owner/repo" GitHub repository.

func LookupAddress

func LookupAddress(runner *Runner, iface string) (string, error)

LookupAddress returns the private IPv4 address of the host associated with the given interface, replacing the script's awk/grep pipeline over "ip route show".

func LookupInterface

func LookupInterface(runner *Runner) (string, error)

LookupInterface returns the interface of the default route, replacing the script's "ip route show | awk '/default/ {print $5}'".

func MakeContainerdDirs

func MakeContainerdDirs(paths ContainerdPaths) error

MakeContainerdDirs creates the directories containerd needs before it can start.

func MergeConfigFile

func MergeConfigFile(settings map[string]string, configFile string) error

MergeConfigFile merges the "key: value" lines of a user-supplied flintlockd config file into settings, overriding any auto-generated values with the same key.

func MonitorLVMProfile

func MonitorLVMProfile(runner *Runner, thinpool string) error

MonitorLVMProfile tries (up to 5 times) to ensure the lvm profile for thinpool is monitored, matching the script's monitor_lvm_profile.

func NormaliseArch

func NormaliseArch(unameArch string) (string, error)

NormaliseArch maps a uname -m style architecture name to the amd64/arm64 naming used by flintlock/cloud-hypervisor/containerd release artefacts.

func ParseAddressForInterface

func ParseAddressForInterface(ipRouteShowOutput, iface string) string

ParseAddressForInterface extracts the private IPv4 "src" address of the route belonging to iface from the output of "ip route show".

func ParseDefaultInterface

func ParseDefaultInterface(ipRouteShowOutput string) string

ParseDefaultInterface extracts the default route's interface name from the output of "ip route show".

func ParseSHA256Sum added in v0.17.0

func ParseSHA256Sum(content string) (string, error)

ParseSHA256Sum returns the lower-case hex digest from the first field of a sha256sum style file ("<digest> <name>").

func RawURL

func RawURL(repo, fileName string) string

RawURL returns the URL of a file at the root of a repository's default branch.

func ReloadSystemd

func ReloadSystemd(runner Executor) error

ReloadSystemd reloads the systemd manager configuration so unit file changes on disk take effect.

func ReplaceFstabEntry added in v0.17.0

func ReplaceFstabEntry(fstab, dir, line string) string

ReplaceFstabEntry returns fstab with every entry for the mount point dir replaced by line. When fstab has no entry for dir, line is appended.

func ReplaceRequires

func ReplaceRequires(path, service string) error

ReplaceRequires rewrites the "Requires=" line of a systemd unit file at path so it requires the given service, matching the script's use of sed to point flintlockd.service at the correct (possibly "-dev" tagged) containerd service.

func ResolveTag

func ResolveTag(ctx context.Context, repo, tag string) (string, error)

ResolveTag returns tag as-is unless it is DefaultVersion, in which case the latest release tag for repo is looked up and returned instead.

func StartContainerdService

func StartContainerdService(ctx context.Context, runner *Runner, paths ContainerdPaths) error

StartContainerdService fetches the containerd systemd unit, points it at paths.ConfigPath and starts it.

func StartFlintlockdService

func StartFlintlockdService(ctx context.Context, runner *Runner, containerdSystemdSvc string) error

StartFlintlockdService fetches the flintlockd systemd unit, points its Requires= at the given containerd service, and starts it.

func StartService

func StartService(runner *Runner, service string) error

StartService enables the given systemd service and starts it, or restarts it if it's already active so a changed unit/config is applied.

func UnameArch

func UnameArch(normalisedArch string) (string, error)

UnameArch maps a normalised amd64/arm64 architecture name back to the uname -m style naming used in firecracker's and cloud-hypervisor's own release artefacts (e.g. firecracker-v1.7.0-x86_64.tgz, cloud-hypervisor-static-aarch64).

func UpdateFstab added in v0.17.0

func UpdateFstab(path, dir, line string) error

UpdateFstab replaces every entry in the fstab at path whose mount point is dir with line, or appends line when there is none. A missing file is created.

func VersionFromEnv

func VersionFromEnv(envVar string) string

VersionFromEnv returns the value of envVar if set, otherwise DefaultVersion. It matches provision.sh's use of e.g. FIRECRACKER_VERSION="${FIRECRACKER:=$DEFAULT_VERSION}" to let a component's default version be overridden via an environment variable.

func WriteContainerdConfig

func WriteContainerdConfig(paths ContainerdPaths, thinpool string) error

WriteContainerdConfig writes the containerd config.toml for the given paths and thinpool. An empty thinpool leaves the devmapper section out.

func WriteFlintlockdConfig

func WriteFlintlockdConfig(settings map[string]string, configFile string) error

WriteFlintlockdConfig writes settings, optionally merged with configFile, to FlintlockdConfigPath.

Types

type AllFlintlockOptions

type AllFlintlockOptions struct {
	Version              string
	Address              string
	ParentIface          string
	BridgeName           string
	Insecure             bool
	ConfigFile           string
	Port                 string
	Arch                 string
	ContainerdStateDir   string
	ContainerdSystemdSvc string
}

AllFlintlockOptions bundles the options needed to install, configure and start flintlockd, matching do_all_flintlock's parameters.

type ContainerdPaths

type ContainerdPaths struct {
	ConfigPath   string
	RootDir      string
	StateDir     string
	ServiceFile  string
	SystemdSvc   string
	DevMapperDir string
	PoolMetadata string
	PoolData     string
}

ContainerdPaths holds the various state paths used by containerd. A "-dev" tagged set is used in development environments so a dev containerd instance never collides with a production one on the same host.

func AllContainerd

func AllContainerd(
	ctx context.Context, runner *Runner, version, thinpool, arch string, dev bool,
) (ContainerdPaths, error)

AllContainerd installs, configures and starts containerd for the given version and thinpool, tagging state paths with "-dev" when dev is true.

func BuildContainerdPaths

func BuildContainerdPaths(dev bool) ContainerdPaths

BuildContainerdPaths returns the containerd state paths to use, tagging them with "-dev" when dev is true.

type Executor added in v0.17.0

type Executor interface {
	Run(name string, args ...string) error
	Output(name string, args ...string) (string, error)
}

Executor runs host commands. *Runner is the real implementation. Tests substitute a fake that records the commands it is asked to run.

type FlintlockdSettings

type FlintlockdSettings struct {
	ContainerdSocket string
	Address          string
	Port             string
	ParentIface      string
	BridgeName       string
	Insecure         bool
}

FlintlockdSettings describes the options used to build a flintlockd config file.

type Runner

type Runner struct {
	Stdout io.Writer
	Stderr io.Writer
}

Runner is the single seam provisioning logic uses to execute host commands (systemctl, apt, lvm2 tools, dmsetup, ...). Keeping every exec.Command call behind this type means the rest of the package never shells out directly.

func NewRunner

func NewRunner(stdout, stderr io.Writer) *Runner

NewRunner returns a Runner that streams command output to the given writers.

func (*Runner) Contains

func (r *Runner) Contains(substr, name string, args ...string) bool

Contains reports whether the output of name with args contains substr. It is a convenience for the "if already exists, do nothing" checks the original provisioning script uses around pvdisplay/vgdisplay/lvdisplay/lvs.

func (*Runner) Output

func (r *Runner) Output(name string, args ...string) (string, error)

Output executes name with args and returns its trimmed stdout.

func (*Runner) Run

func (r *Runner) Run(name string, args ...string) error

Run executes name with args, streaming its output to the runner's writers.

type StateDir added in v0.17.0

type StateDir struct {
	Exec      Executor
	Out       io.Writer
	FstabPath string
	Probe     func(dir string) error
	FSType    func(dir string) (string, error)
}

StateDir prepares the flintlockd state directory so that files on it can be cloned with FICLONE. Host commands go through Exec and the filesystem checks through Probe and FSType, so tests can replace all three.

func NewStateDir added in v0.17.0

func NewStateDir(exec Executor, out io.Writer) *StateDir

NewStateDir returns a StateDir that runs commands through exec, writes progress to out and uses the real reflink probe and /etc/fstab.

func (*StateDir) Setup added in v0.17.0

func (s *StateDir) Setup(opts StateDirOptions) error

Setup verifies or creates the state directory as opts describes. See StateDirOptions for the three cases.

type StateDirOptions added in v0.17.0

type StateDirOptions struct {
	// Dir is the state directory. It becomes the mount point when a
	// filesystem is created.
	Dir string
	// Device is a block device or logical volume to format as XFS.
	Device string
	// LoopFile is a sparse file to format as XFS and mount through a loop
	// device. It is meant for development and e2e hosts.
	LoopFile string
	// SizeGB is the size of LoopFile when it is created.
	SizeGB int
	// Force formats a device or loop file that already holds a filesystem.
	Force bool
}

StateDirOptions describes how to prepare the flintlockd state directory. With neither Device nor LoopFile set the directory is only verified.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL