runtime

package
v0.2.7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 26, 2026 License: MIT Imports: 34 Imported by: 0

Documentation

Index

Constants

View Source
const Namespace = "ephemerd"

Namespace is the containerd namespace every runner container, and every image the runtime pulls, lives in. Exported so the image collector can be pointed at it without duplicating the string.

View Source
const WindowsToolCache = `C:\hostedtoolcache`

WindowsToolCache is the RUNNER_TOOL_CACHE directory for Windows job containers.

The GitHub runner resolves its tool cache from, in order, RUNNER_TOOL_CACHE, RUNNER_TOOLSDIRECTORY, AGENT_TOOLSDIRECTORY, agent.ToolsDirectory, and only then falls back to <runner root>\_work\_tool (Runner.Common, HostContext.GetDirectory, WellKnownDirectory.Tools). Every action that caches a toolchain — actions/setup-go, setup-node, setup-python — looks there and nowhere else. PATH is irrelevant to them: an image that installs Go and puts it on PATH still makes setup-go download and extract a second copy.

The default is unusable for us. On Windows the whole runner root is a per-job copy of the host runner directory that ephemerd maps into the container (see withRunnerMount and copyDirForJob); the runner creates _work inside it. So <runner root>\_work\_tool is host storage reached over a VSMB share into the Hyper-V utility VM, and it is created fresh per job. Two consequences: anything we bake into the image under C:\actions-runner is shadowed by the mapped directory, and small-file writes there are slow enough that extracting the ~15,000-file Go toolchain zip blows past actions/setup-go's 8 minute timeout. Pointing the tool cache at a path inside the image fixes both — the contents survive because nothing is mounted over them, and a cache hit means no extraction at all.

C:\hostedtoolcache mirrors the GitHub-hosted Windows images and the /opt/hostedtoolcache we already hand to Gitea/Forgejo runners in pkg/forgerunner.BuildEnv. images/runner-ci-windows/Dockerfile seeds it; keep the two in sync.

Variables

This section is empty.

Functions

func CleanOldLogs

func CleanOldLogs(logDir string, maxAge time.Duration, log *slog.Logger)

CleanOldLogs removes job log files older than maxAge from the log directory. Called on startup and periodically to prevent unbounded disk usage.

func CleanOrphanJobDirs added in v0.1.4

func CleanOrphanJobDirs(dataDir string, keep map[string]struct{}, log *slog.Logger)

CleanOrphanJobDirs removes leftover per-job workdirs under <dataDir>/jobs/, skipping any whose directory name is in keep (the set of currently-running job IDs). It is the belt-and-suspenders half of the leak fix: per-job removal in Destroy handles the normal path, but a crash / SIGKILL / kill -9 of ephemerd (or a job dir left by a version that predates the per-job removal) skips Destroy entirely, so we sweep on startup as well.

At worker startup nothing is running yet, so callers pass a nil/empty keep and every jobs/* dir is an orphan. keep is provided for symmetry and future callers (e.g. a periodic sweep) that must not touch live jobs.

func ImportImagesTo

func ImportImagesTo(ctx context.Context, c *client.Client, paths []string, snapshotter string, log *slog.Logger)

ImportImagesTo imports a list of OCI image tarballs into the given containerd client. Used to import deferred Linux images into a VM's containerd after the VM is ready.

Types

type Config

type Config struct {
	Client       *client.Client
	RunnerDir    string // host path to extracted runner binary
	RunnerMount  string // container path to mount runner at
	DefaultImage string // override default container image (auto-detected if empty)
	ImagesDir    string // directory containing pre-downloaded OCI image tarballs to import on startup
	LogDir       string // directory for per-job container logs
	DataDir      string // ephemerd data directory (used for dind socket paths)
	// ContainerDataDir is the path containerd/runc see for the DataDir.
	// On Linux this matches DataDir. On Darwin the host DataDir is shared
	// into the Linux VM via virtio-fs at a different path (e.g.
	// /mnt/ephemerd), and any bind-mount sources that reference the
	// DataDir must be rewritten to that VM-side path. When empty, falls
	// back to DataDir.
	ContainerDataDir string
	DindEnabled      bool // mount a fake Docker socket into each container
	// DindAllowPrivileged is forwarded to each per-job dind.Server.
	// When false, requests carrying HostConfig.Privileged=true or
	// HostConfig.CapAdd are rejected with HTTP 403. See
	// config.DindConfig.AllowPrivileged for the threat model.
	DindAllowPrivileged bool
	// RegistryMirror routes image pulls through a LAN pull-through cache.
	// Nil (the zero value) means no mirror: PullImage builds exactly the
	// containerd pull call it built before the feature existed. Forwarded
	// to each per-job dind.Server so the hot dind pull path is covered too.
	RegistryMirror *registrymirror.Mirror
	CacheProxyEnv  []string // extra env vars from cache proxies (e.g., GOPROXY=...)
	// CacheProxyMounts are read-only bind mounts requested by cache proxies
	// for toolchains that cannot be redirected with an env var. The Cargo
	// proxy uses this to place a generated .cargo/config.toml at the
	// container's filesystem root, where Cargo's ancestor-directory config
	// search finds it for any workspace path.
	CacheProxyMounts []proxies.Mount
	// Rlimits sets POSIX resource limits on each runner container's OCI
	// process. Zero values fall back to the containerd default (1024).
	// Applies on Linux only; ignored on Windows (HCS uses a different model).
	Rlimits config.RuntimeRlimits
	// AllowNewPrivileges permits privilege escalation via execve in the
	// runner container (NoNewPrivileges=false), which is what makes
	// `sudo` work. See config.RuntimeConfig.AllowNewPrivileges.
	//
	// NOTE: this is a resolved value — callers pass
	// cfg.Runtime.ResolvedAllowNewPrivileges(), whose default is true.
	// The zero value here is false, i.e. the hardened setting, so a
	// construction site that forgets this field breaks `sudo` rather
	// than silently loosening the sandbox.
	AllowNewPrivileges bool
	// LinuxRuntime is the containerd runtime handler for Linux job
	// containers — "io.containerd.runc.v2" (default) or
	// "io.containerd.kata.v2" for VM-isolated jobs. Empty means runc, so
	// a construction site that forgets this field keeps today's behavior
	// rather than failing to create containers. Ignored on Windows, which
	// always uses io.containerd.runhcs.v1.
	//
	// Callers pass config.LinuxRunnerToml.ContainerdRuntime().
	LinuxRuntime string
	Network      *networking.Manager
	// WindowsMemoryBytes is the memory limit for Hyper-V isolated Windows
	// runner containers. Zero leaves the OCI spec field unset, which gives
	// the HCS default (~1 GB) — too small for MSVC builds. Caller should
	// pass config.WindowsRunnerToml.MemoryBytes() which defaults to 4 GB.
	WindowsMemoryBytes uint64
	// WindowsCPUs is the virtual CPU count for Hyper-V isolated Windows
	// runner containers. Zero leaves the OCI spec field unset. Caller
	// should pass config.WindowsRunnerToml.CPUCount() which defaults to 2.
	WindowsCPUs uint64
	// BuildKit is the shared embedded BuildKit solver handed to each per-job
	// dind.Server for `docker build` support. Optional; nil means `docker build`
	// falls back to the platform default (buildah on Linux, 501 elsewhere).
	BuildKit *buildkit.Server
	// OnTaskStarted is invoked synchronously by Create after the container
	// task has successfully Start()ed. Nil means no hook. Used to wire
	// per-container resource samplers into the metrics endpoint; see
	// docs/arch/container-metrics.md.
	OnTaskStarted func(env *RunnerEnv)
	// OnTaskDestroy is invoked synchronously by Destroy before the
	// container is torn down. Symmetric with OnTaskStarted.
	OnTaskDestroy func(env *RunnerEnv)
	// ImageGC evicts LRU container images when the node is under disk
	// pressure. Nil disables it. The runtime consults it before pulling
	// an image and before creating a runner environment — a periodic
	// timer alone loses the race a single job can win by pulling a
	// multi-gigabyte toolchain image between ticks.
	ImageGC *imagegc.Collector
	Log     *slog.Logger
}

Config for the container runtime.

type CreateConfig

type CreateConfig struct {
	ID    string // unique job identifier (container name, dind socket path)
	Image string // OCI image reference (empty = use default)

	// Provider is the forge provider name (e.g. "github", "gitea") that
	// queued the job. Together with Repo it's used to scope dind's
	// per-repo image cache. Empty disables caching for this job.
	Provider string

	// Repo is the forge-native repo path (e.g. "owner/repo"). Together
	// with Provider it's used to scope dind's per-repo image cache.
	// Empty disables caching for this job.
	Repo string

	// JITConfig is the base64-encoded JIT config for GitHub runners.
	// Passed as "--jitconfig <value>" to the runner entrypoint.
	// Mutually exclusive with Entrypoint.
	JITConfig string

	// Env holds extra environment variables injected into the container.
	// Used by Gitea/Forgejo to pass instance URL, runner token, etc.
	Env map[string]string

	// Entrypoint overrides the container's process args.
	// When set, used instead of the default "--jitconfig" entrypoint.
	// When nil and JITConfig is set, uses the GitHub "--jitconfig" mode.
	// When nil and JITConfig is empty, uses the image's default CMD.
	Entrypoint []string
}

CreateConfig holds parameters for creating a runner environment.

type RunnerEnv

type RunnerEnv struct {
	ID        string
	Provider  string       // forge provider that queued the job (e.g. "github")
	Repo      string       // forge-native repo path (e.g. "owner/repo")
	Netns     string       // network namespace path (Linux only)
	RunnerDir string       // per-job runner copy, cleaned up on destroy
	Dind      *dind.Server // per-job fake Docker daemon (nil if disabled)
	Container client.Container
	Task      client.Task
}

RunnerEnv represents a running runner environment.

type Runtime

type Runtime struct {
	// contains filtered or unexported fields
}

Runtime manages container lifecycle for runner environments.

func New

func New(cfg Config) (*Runtime, error)

New creates a container runtime manager.

func (*Runtime) CleanOrphans

func (r *Runtime) CleanOrphans(ctx context.Context) error

CleanOrphans removes any leftover containers and snapshots from a previous ephemerd run. This should be called on startup before the scheduler starts accepting jobs.

STARTUP ONLY. It kills and deletes EVERY container in the runtime namespace on the assumption that nothing legitimate is running yet. Calling it while jobs are in flight would tear those jobs down. The periodic equivalent is SweepOrphans, which never touches containers.

func (*Runtime) Client

func (r *Runtime) Client() *client.Client

Client returns the underlying containerd client. Used by the in-VM debug-exec HTTP server so the Windows host can poke into running containers (kindest/node, buildkit) without leaving the VM.

func (*Runtime) Create

func (r *Runtime) Create(ctx context.Context, cfg CreateConfig) (*RunnerEnv, error)

Create provisions an ephemeral runner environment.

func (*Runtime) Destroy

func (r *Runtime) Destroy(ctx context.Context, env *RunnerEnv) error

Destroy tears down a runner environment completely.

func (*Runtime) ImportImages

func (r *Runtime) ImportImages(ctx context.Context) (deferred []string, err error)

ImportImages loads pre-downloaded OCI image tarballs from the images directory. Each tarball is inspected for its target OS. Images matching the host OS are imported into the host containerd and unpacked immediately. Images targeting a different OS (e.g. Linux images on a Windows host) are returned as deferred paths — the caller should import them into the appropriate VM's containerd after the VM is ready using ImportImagesTo.

On Linux, all images are imported directly (no deferral).

func (*Runtime) LogDir

func (r *Runtime) LogDir() string

LogDir returns the configured per-job log directory (empty if logs go to stdio).

func (*Runtime) PullImage

func (r *Runtime) PullImage(ctx context.Context, ref string) error

PullImage ensures the runner image is available locally. Serialized with a mutex to avoid concurrent pulls contending on the content store (which produces noisy lock errors).

func (*Runtime) SetTaskHooks

func (r *Runtime) SetTaskHooks(onStarted, onDestroy func(*RunnerEnv))

SetTaskHooks installs (or replaces) the OnTaskStarted / OnTaskDestroy callbacks. Used by main.go to wire metrics-sampler registration after both the runtime and the dispatch server have been constructed — constructor-order makes a plain Config-time hook awkward because the dispatch server depends on the runtime.

func (*Runtime) SweepOrphans added in v0.1.8

func (r *Runtime) SweepOrphans(ctx context.Context) error

SweepOrphans removes per-job state that no existing container owns: leftover runner-dir copies under <data-dir>/runners/job-*, per-job workdirs under <data-dir>/jobs/*, and writable container snapshots.

Unlike CleanOrphans it never touches containers, tasks or networking, so it is safe to run on a timer while jobs are in flight. That matters because the leaks it cleans up are produced by crashes and partially failed creates, which a startup-only sweep leaves to accumulate for the entire uptime of a long-lived daemon.

Cost is one container list plus one readdir per directory plus a snapshotter walk — cheap enough for a ~60s cadence.

func (*Runtime) Wait

func (r *Runtime) Wait(ctx context.Context, env *RunnerEnv) (uint32, error)

Wait blocks until the runner environment's task exits. Returns the exit status code.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL