Documentation
¶
Overview ¶
Package dind implements a fake Docker Engine API server.
Each job gets its own server instance listening on a Unix socket. The socket is bind-mounted into the job container at /var/run/docker.sock. Docker CLI calls inside the container are translated into containerd operations on the host — no real Docker daemon.
Sibling containers created through this API can opt into the full docker --privileged elevation stack (all caps, all devices, seccomp and apparmor unconfined, writable sysfs/cgroupfs) when the host is configured to allow it. See Server.allowPrivileged and config.DindConfig.AllowPrivileged for the threat model — the short version is that on Windows and macOS hosts the dind containerd lives inside a managed Linux VM, so an escape stays in that VM; on a Linux host with no VM fence the default is to deny privileged requests.
Index ¶
- Constants
- func CacheNamespace(provider, repo string) string
- func CachePrune(ctx context.Context, c *client.Client, maxAge time.Duration, log *slog.Logger) error
- func CleanupJobNamespace(ctx context.Context, c *client.Client, ns string, log *slog.Logger)
- func CleanupStaleDindNamespaces(ctx context.Context, c *client.Client, log *slog.Logger)
- func MirrorImageToCache(ctx context.Context, c *client.Client, jobNS, cacheNS, imageName string, ...) error
- func RefreshLastAccessed(ctx context.Context, c *client.Client, cacheNS, imageName string, ...)
- type Config
- type Server
Constants ¶
const DindCacheNamespacePrefix = "ephemerd-dind-cache-"
DindCacheNamespacePrefix prefixes every per-repo image cache namespace.
Full namespace name format:
ephemerd-dind-cache-<provider>-<sanitized-repo>
Examples:
ephemerd-dind-cache-github-ephpm_ephpm ephemerd-dind-cache-gitea-ephpm_ephpm (distinct from the github one) ephemerd-dind-cache-gitlab-acme_platform_api (nested GitLab groups OK)
Provider + repo together form the privacy boundary: two different forges with same-named repos do NOT share a cache, and two different orgs on the same forge get separate caches keyed by the full `owner/repo` path.
const DindNamespacePrefix = "ephemerd-dind-"
DindNamespacePrefix is the prefix every per-job containerd namespace the dind subsystem creates. Each running ephpm-style job has its own namespace (e.g. "ephemerd-dind-ephemerd-github-ephpm-fast_shannon") so containers, images, and leases from one job can't pin disk against another's.
const LastAccessedLabel = "ephemerd.io/last-accessed"
LastAccessedLabel records the most recent time an Image record in a cache namespace was touched (pull or container-create). The pruner uses this for LRU eviction. RFC3339-formatted, UTC.
Variables ¶
This section is empty.
Functions ¶
func CacheNamespace ¶
CacheNamespace returns the containerd namespace name used to cache image metadata for a given (provider, repo) pair. Both inputs are sanitized so the result is always a valid containerd namespace identifier (regex: ^[A-Za-z0-9]+(?:[._-]+[A-Za-z0-9]+)*$).
Provider should be the value from providers.Provider.Name() (e.g. "github", "gitea"). Repo is the forge-native repo path (e.g. "owner/repo" on GitHub or "group/subgroup/project" on GitLab); path separators are mapped to underscores so the namespace identifier stays valid. Empty provider or repo returns "" — callers should treat that as "caching disabled for this job".
func CachePrune ¶
func CachePrune(ctx context.Context, c *client.Client, maxAge time.Duration, log *slog.Logger) error
CachePrune walks every per-repo cache namespace and evicts Image records whose LastAccessedLabel (or CreatedAt fallback for records pre-dating the label) is older than maxAge. Empty cache namespaces are deleted entirely. Containerd's content GC reclaims the unreferenced blobs after this runs.
Returns nil and logs warnings on partial failures — the next pass will retry whatever didn't clean up this time.
func CleanupJobNamespace ¶
CleanupJobNamespace removes everything inside a per-job dind namespace and then the namespace metadata bucket itself. Safe to call multiple times and safe to call on a namespace that contains stragglers from prior crashes — each step logs and continues on error rather than bailing partway through.
Order matters:
- Containers (with their tasks + snapshots) — releases the overlayfs upperdirs that hold container rootfs writes.
- Images — drops the gc.ref labels that pin manifest+config+layer blobs.
- Leases — releases any explicit content holds (buildkit etc. take these during pulls/builds).
- Snapshots — orphan snapshots from layers that were unpinned in step 2 don't get reclaimed by containerd's async GC fast enough; the NamespaceService.Delete in step 6 would fail with FailedPrecondition until those snapshots are gone. Walk the snapshotter and remove them explicitly.
- Content blobs — same story as snapshots; the async GC won't have swept by the time we try to delete the namespace.
- NamespaceService.Delete — drops the metadata bucket. Will only succeed if 1-5 left the namespace truly empty; on failure we log and leave the bucket so a subsequent boot's CleanupStaleDindNamespaces can retry.
func CleanupStaleDindNamespaces ¶
CleanupStaleDindNamespaces enumerates every namespace matching DindNamespacePrefix and runs CleanupJobNamespace on each. Intended to be called once at ephemerd worker-mode startup to clean up after crashed or killed jobs from a previous boot — the same Server.Stop path would have done this on a graceful shutdown but a runner timeout / SIGKILL skips it.
func MirrorImageToCache ¶
func MirrorImageToCache(ctx context.Context, c *client.Client, jobNS, cacheNS, imageName string, log *slog.Logger) error
MirrorImageToCache copies an Image record from the per-job namespace into the per-repo cache namespace (creating it if needed), refreshing the LastAccessedLabel on the cache record. The underlying content blobs are already in the global content store from the original pull; this only adds metadata so the cache record's gc.ref labels keep the content alive after the per-job namespace is cleaned up.
Returns nil if the cache namespace name is empty (no provider/repo set).
func RefreshLastAccessed ¶
func RefreshLastAccessed(ctx context.Context, c *client.Client, cacheNS, imageName string, log *slog.Logger)
RefreshLastAccessed bumps the LastAccessedLabel on a cached image. Called from the container-create path when a job references an image that's already in the cache (no pull happens, but the image is in use). Silently no-ops if the image isn't in the cache.
Types ¶
type Config ¶
type Config struct {
// JobID is the unique job identifier.
JobID string
// Provider is the forge provider name ("github", "gitea", "forgejo",
// "gitlab", "woodpecker") for the job. Used together with Repo to
// build the per-repo image cache namespace; if empty, image caching
// across jobs is disabled and every pull is cold for this job.
Provider string
// Repo is the forge-native repo path (e.g. "owner/repo" on GitHub
// or "group/subgroup/project" on GitLab). Used together with Provider
// to build the per-repo image cache namespace; if empty, image
// caching across jobs is disabled.
Repo string
// DataDir is the ephemerd data directory. The socket and temp layers
// are stored under <DataDir>/jobs/<JobID>/docker/.
DataDir string
// Client is the containerd client for image pulls and container ops.
Client *client.Client
// Network is the networking manager for attaching sibling containers
// to the CNI bridge. May be nil if networking is not available.
Network *networking.Manager
// BuildKit is the shared embedded BuildKit solver. When non-nil,
// POST /build routes through handleImageBuildBuildkit. When nil, the
// platform default (buildah on Linux, 501 elsewhere) is used.
BuildKit *buildkit.Server
// RunnerNetNS is the path to the runner container's net namespace
// (e.g. /proc/<pid>/ns/net). Required for port forwarding — when a
// dind sibling exposes ports via -p, we install iptables DNAT rules
// in this namespace so the runner sees 127.0.0.1:hostPort routed to
// the sibling's container IP. Empty disables port forwarding.
RunnerNetNS string
// AllowPrivileged controls whether sibling containers may opt into
// the full elevation stack via HostConfig.Privileged or via
// HostConfig.CapAdd. When false, requests carrying either are
// rejected with HTTP 403. See config.DindConfig.AllowPrivileged for
// the threat model.
AllowPrivileged bool
Log *slog.Logger
}
Config for creating a per-job fake Docker daemon.
type Server ¶
type Server struct {
// contains filtered or unexported fields
}
Server is a per-job fake Docker daemon.
func (*Server) Endpoint ¶
Endpoint returns the value a container should set DOCKER_HOST to in order to reach this fake daemon. Linux/macOS return the unix socket path directly (e.g. "unix:///var/run/docker.sock" once bind-mounted); Windows returns a "tcp://<gateway-ip>:<port>" URI pointing at a TCP listener bound on the HCN NAT gateway so containers on the NAT can reach it without any mount.
func (*Server) SetRunnerNetNS ¶
SetRunnerNetNS records the runner container's net namespace path so the dind server can install iptables DNAT rules for port-bound siblings. Must be called after the runner task starts (PID is known) and before any docker create from inside the runner.
func (*Server) SetRunnerRootfs ¶
func (s *Server) SetRunnerRootfs(snapshotKey string, runnerRootfsPath string, bindMappings map[string]string)
SetRunnerRootfs registers the runner container's snapshot, rootfs mount path, and the non-rootfs bind table ephemerd installed into it, so that subsequent docker create requests from inside the runner can have their -v sources translated from the runner's mount namespace to real host paths.
snapshotKey is the containerd snapshot name (typically "<runnerID>-snapshot" in the runtime's "ephemerd" namespace). Kept for the (now-fallback) layer walk used by the unit tests; production resolution goes through runnerRootfsPath.
runnerRootfsPath is the host-namespace path where runc has mounted the runner's merged overlay. Caller typically obtains this by calling os.Readlink("/proc/<runner-pid>/root") after the runner task starts — runc sets up the overlay before exec, and the symlink reads back to the bundle's rootfs in the host's namespace. Required for the merged view: without it, sibling binds for paths split across image layers (e.g. /home/runner/externals) bind incomplete trees.
bindMappings keys are container destination paths (what the runner sees, e.g. "/var/run/docker.sock"); values are host source paths (what the dind daemon hands to containerd). The map is copied so the caller may continue to mutate it.
Must be called after the runner task starts (rootfs is mounted) and before any docker create from inside the runner. Pairs with SetRunnerNetNS in the runtime, called at the same point in startup.
func (*Server) SocketPath ¶
SocketPath returns the host-side Unix socket path (Linux/macOS). Empty on Windows — use Endpoint() to get the DOCKER_HOST value instead.
func (*Server) Start ¶
Start begins serving the fake Docker API.
Linux/macOS: listens on a per-job unix socket at <DataDir>/jobs/<jobID>/docker/d.sock. Windows: listens on TCP on the HCN NAT gateway IP (picked from networking.Manager) so Hyper-V-isolated runner containers on the same NAT can reach it without a runhcs bind mount (which isn't supported) or named pipe sharing (which needs HCS config for isolated containers).
Source Files