README
¶
ermite-runner
The CI runner of ermite: runs GitHub Actions workflows, one microVM per job.
Read-only mirror. Development happens at git.bnema.dev; issues and pull requests are closed here.
ermite-runner fetches jobs from an ermite server and runs workflows written for GitHub Actions (.github/workflows/*.yml). Each job runs in its own sandbox, destroyed when the job ends.
Isolation modes
| Mode | Sandbox | Needs |
|---|---|---|
vm |
A Firecracker microVM per job | A usable /dev/kvm |
container |
A container per job with its own Docker daemon | A rootful Docker Engine |
ERMITE_RUNNER_ISOLATION=auto is the default. It picks vm when /dev/kvm is usable and Firecracker, the guest kernel and the base image exist. Otherwise it picks container. The chosen mode and the reasons are logged at start.
Details: docs/sandboxes.md.
Requirements
- A Linux host. The runner runs as root.
ip,nftandmkfs.ext4(e2fsprogs 1.43 or newer).- Go, to build the runner.
- For
vm:/dev/kvm,firecrackerandjailerinPATH, a guest kernel and a base image. - For
container: a rootful Docker Engine and the job image. sysbox is optional: without it the runner uses a privileged container.
docs/images.md explains how to build the kernel and the images.
Build
The binary must be static:
CGO_ENABLED=0 go build -trimpath -o ermite-runner .
Register and run
-
On the ermite server, create a registration token:
ermite token create my-runner --preset runner -
Install the binary, the unit and the settings:
sudo install -m 0755 ermite-runner /usr/local/bin/ermite-runner sudo install -m 0644 contrib/ermite-runner.service /etc/systemd/system/ sudo install -d -m 0700 /etc/ermite-runner sudo install -m 0600 /dev/stdin /etc/ermite-runner/env <<'ENV' ERMITE_RUNNER_URL=https://ermite.example.com ERMITE_RUNNER_NAME=runner-1 ERMITE_RUNNER_LABELS=ubuntu-latest,ubuntu-26.04 ERMITE_RUNNER_LOG_FORMAT=json ENV -
Register once. The token goes through standard input, so it is in no command line:
printf %s "$TOKEN" | sudo systemd-run --pipe --wait \ -p EnvironmentFile=/etc/ermite-runner/env \ -p StateDirectory=ermite-runner -p StateDirectoryMode=0700 \ /usr/local/bin/ermite-runner register --token-stdin -
Start the service:
sudo systemctl enable --now ermite-runner journalctl -u ermite-runner -f
ermite-runner run polls the server for jobs and runs each one in a sandbox, up to ERMITE_RUNNER_CAPACITY jobs at a time. On SIGTERM it cancels the running jobs, reports them and removes their sandboxes.
To check an installation without a server, run ermite-runner sandbox-check. It runs one stub script in the selected mode.
More on the service, logs and shutdown: docs/service.md.
Commands
| Command | Purpose |
|---|---|
ermite-runner register [--token-stdin] [--token T] [--replace] |
Register with the server and store the runner token |
ermite-runner run |
Fetch and run jobs |
ermite-runner sandbox-check [script] |
Run one stub script in a sandbox |
ermite-runner version |
Print the version |
Key settings
| Variable | Default | Meaning |
|---|---|---|
ERMITE_RUNNER_URL |
Base URL of the ermite server. Must be https:// |
|
ERMITE_RUNNER_NAME |
host name | Runner name |
ERMITE_RUNNER_LABELS |
Extra labels, comma separated | |
ERMITE_RUNNER_ISOLATION |
auto |
auto, vm or container |
ERMITE_RUNNER_CAPACITY |
1 |
Jobs in parallel |
ERMITE_RUNNER_JOB_CPUS |
2 |
CPUs per job |
ERMITE_RUNNER_JOB_MEMORY |
4G |
Memory per job |
ERMITE_RUNNER_JOB_DISK |
64G |
Scratch disk per job |
ERMITE_RUNNER_STATE_DIR |
/var/lib/ermite-runner |
State directory |
All settings: docs/configuration.md.
Security notes
- Run only one runner per host.
- The registration token can read the code and secrets of every repository. Pass it through standard input.
- Without sysbox,
containermode uses a privileged container, which is equivalent to host root. SetERMITE_RUNNER_DENY_PRIVILEGED=1to refuse it. - The state directory and all its parents must be owned by root and not writable by others.
Documentation
- Configuration: every environment variable
- Sandboxes:
vmandcontainermodes, network policy, cleanup - Service: systemd unit, logs, shutdown
- Images: job image, guest kernel
- Workflows: job and service containers, cancellation, annotations
- Cache and artifacts: Actions cache and artifacts v4
- Node.js externals: Node.js for JavaScript actions
- Development: build, tests
License
MIT. See LICENSE. Third-party attributions: THIRD_PARTY.md.
Documentation
¶
Overview ¶
Command ermite-runner runs CI jobs for an ermite server. Each job runs in its own Firecracker micro-VM, or in its own container when the host has no KVM, and executes workflows compatible with GitHub Actions.
Subcommands: version; register stores a runner token obtained from ermite; run fetches jobs from ermite and runs each in a sandbox; init and exec run inside a sandbox (guest PID 1, and the executor); sandbox-check runs one stub script through the selected isolation mode and prints its output.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
internal
|
|
|
actionscache
Package actionscache is the host-side cache of action archives (plan CI-4, D8).
|
Package actionscache is the host-side cache of action archives (plan CI-4, D8). |
|
cachesvc
Package cachesvc is the GitHub Actions cache service (API v1) of the runner (plan CI-3, D8).
|
Package cachesvc is the GitHub Actions cache service (API v1) of the runner (plan CI-3, D8). |
|
cmdrun
Package cmdrun runs host commands (ip, nft, mkfs.ext4) behind a small interface so that callers can be tested with a fake.
|
Package cmdrun runs host commands (ip, nft, mkfs.ext4) behind a small interface so that callers can be tested with a fake. |
|
dnsfwd
Package dnsfwd is a small DNS forwarder for job sandboxes (stdlib only).
|
Package dnsfwd is a small DNS forwarder for job sandboxes (stdlib only). |
|
execstream
Package execstream is the framed protobuf stream between the host process of ermite-runner and the executor running inside a sandbox.
|
Package execstream is the framed protobuf stream between the host process of ermite-runner and the executor running inside a sandbox. |
|
executor
Package executor runs the steps of a GitHub Actions job inside a sandbox.
|
Package executor runs the steps of a GitHub Actions job inside a sandbox. |
|
externals
Package externals provides the tools that jobs expect to find outside the image, today the Node.js builds of actions/runner (node20, node24 and the alpine variants for job containers), so that a runner upgrade or a new Node.js version never needs an image rebuild.
|
Package externals provides the tools that jobs expect to find outside the image, today the Node.js builds of actions/runner (node20, node24 and the alpine variants for job containers), so that a runner upgrade or a new Node.js version never needs an image rebuild. |
|
guest
Package guest is the code that runs inside a sandbox: `ermite-runner init` (the guest PID 1) and `ermite-runner exec` (the executor).
|
Package guest is the code that runs inside a sandbox: `ermite-runner init` (the guest PID 1) and `ermite-runner exec` (the executor). |
|
host
Package host is the host side of ermite-runner: it registers the runner with ermite, fetches jobs, runs each one in a sandbox through the executor, and reports logs, steps and the result with the job token.
|
Package host is the host side of ermite-runner: it registers the runner with ermite, fetches jobs, runs each one in a sandbox through the executor, and reports logs, steps and the result with the job token. |
|
host/hosttest
Package hosttest is an in-memory RunnerService for tests that run the real host loop against a real sandbox.
|
Package hosttest is an in-memory RunnerService for tests that run the real host loop against a real sandbox. |
|
netpolicy
Package netpolicy generates and applies the nftables policy of job sandboxes.
|
Package netpolicy generates and applies the nftables policy of job sandboxes. |
|
resultssvc
Package resultssvc is the GitHub Actions results service of the runner, the part of it that artifacts v4 use (plan CI-4 C).
|
Package resultssvc is the GitHub Actions results service of the runner, the part of it that artifacts v4 use (plan CI-4 C). |
|
sandbox
Package sandbox defines the isolation boundary of a CI job: one Sandbox per job, either a Firecracker micro-VM (package vm) or a container with its own Docker daemon (package container).
|
Package sandbox defines the isolation boundary of a CI job: one Sandbox per job, either a Firecracker micro-VM (package vm) or a container with its own Docker daemon (package container). |
|
sandbox/container
Package container is the container sandbox, for hosts without KVM: one container per job with its own Docker daemon inside (the sysbox runtime when the daemon lists it, else a privileged container), on its own bridge network and with its own scratch disk.
|
Package container is the container sandbox, for hosts without KVM: one container per job with its own Docker daemon inside (the sysbox runtime when the daemon lists it, else a privileged container), on its own bridge network and with its own scratch disk. |
|
sandbox/disk
Package disk creates the per-job scratch disks (sparse ext4 files) and, for the container mode, loop-mounts them.
|
Package disk creates the per-job scratch disks (sparse ext4 files) and, for the container mode, loop-mounts them. |
|
sandbox/drivers
Package drivers wires the sandbox drivers into sandbox.Select.
|
Package drivers wires the sandbox drivers into sandbox.Select. |
|
sandbox/netsetup
Package netsetup is the host-side network plumbing shared by both sandbox drivers: the nftables policy (package netpolicy), the DNS forwarder (package dnsfwd) and IPv4 forwarding.
|
Package netsetup is the host-side network plumbing shared by both sandbox drivers: the nftables policy (package netpolicy), the DNS forwarder (package dnsfwd) and IPv4 forwarding. |
|
sandbox/sandboxtest
Package sandboxtest has helpers shared by the tests of the sandbox drivers.
|
Package sandboxtest has helpers shared by the tests of the sandbox drivers. |
|
sandbox/vm
Package vm is the micro-VM sandbox: one Firecracker VM per job.
|
Package vm is the micro-VM sandbox: one Firecracker VM per job. |
|
tcprelay
Package tcprelay forwards TCP connections accepted on a sandbox's gateway address to one fixed target.
|
Package tcprelay forwards TCP connections accepted on a sandbox's gateway address to one fixed target. |
|
urlorigin
Package urlorigin gives the one canonical form of the origin of an http(s) URL that the host and the executor both compare: the archive token of an action goes only to the origin the host named, so the two sides must agree on what "the same origin" means, whatever the spelling of the URL.
|
Package urlorigin gives the one canonical form of the origin of an http(s) URL that the host and the executor both compare: the archive token of an action goes only to the origin the host named, so the two sides must agree on what "the same origin" means, whatever the spelling of the URL. |
Click to show internal directories.
Click to hide internal directories.