Documentation
¶
Overview ¶
Package buildkit wires an in-process BuildKit solver into ephemerd.
The server type holds a *control.Controller configured with:
- a containerd worker pointed at ephemerd's embedded containerd
- the Dockerfile frontend plus the gateway.v0 frontend
- bbolt-backed cache and history stores under <dataDir>/buildkit
Callers interact with the server through the Build method, which accepts a high-level BuildOpts describing a Docker-style build request and returns a progress stream. The Docker-API translation layer lives in pkg/dind.
Index ¶
- type Config
- type GCConfig
- type Server
- func (s *Server) Build(ctx context.Context, opt client.SolveOpt, statusCh chan *client.SolveStatus) (*client.SolveResponse, error)
- func (s *Server) Client(ctx context.Context) (*client.Client, error)
- func (s *Server) Close() error
- func (s *Server) ContainerdNamespace() string
- func (s *Server) Prune(ctx context.Context, rule client.PruneInfo) (int64, error)
- func (s *Server) SessionManager() *session.Manager
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
// DataDir is where BuildKit stores its cache, history, and content.
// Typically <ephemerd data dir>/buildkit.
DataDir string
// ContainerdAddress is the address of ephemerd's embedded containerd
// gRPC endpoint. On Linux this is a unix socket path; on Windows it's
// a named pipe (e.g. "npipe:////./pipe/ephemerd-containerd").
ContainerdAddress string
// ContainerdNamespace is the containerd namespace buildkit should use
// for image and content storage. Defaults to "buildkit" if empty.
ContainerdNamespace string
// Snapshotter selects the containerd snapshotter. Defaults to "overlayfs"
// on Linux and "windows" on Windows.
Snapshotter string
// Network manages container networking. Required on Windows so build
// containers get an HCN NAT endpoint (otherwise RUN steps that hit the
// internet exit immediately). Ignored on platforms where buildkit's
// default network providers already work.
Network *networking.Manager
// GC bounds the on-disk build cache. The zero value produces NO prune
// rules, which is how BuildKit reads "never garbage-collect" — see
// GCConfig for what that cost us in production. Callers should pass a
// configured policy.
GC GCConfig
// Log receives structured logging from the buildkit server.
Log *slog.Logger
}
Config configures an embedded BuildKit Server.
type GCConfig ¶ added in v0.1.8
type GCConfig struct {
// Disabled turns BuildKit's garbage collection off entirely, restoring
// the previous (unbounded) behavior. Only useful for debugging a
// cache-correctness problem.
Disabled bool
// ReservedBytes is cache that is never collected, even when idle. This
// is the warm floor: below it BuildKit keeps everything, so the common
// "same repo builds again an hour later" case still hits cache.
ReservedBytes int64
// MaxUsedBytes is the hard ceiling on total build cache. Anything above
// it is collected regardless of age.
MaxUsedBytes int64
// MinFreeBytes makes GC collect whatever it must to keep at least this
// much free space on the filesystem, overriding ReservedBytes. This is
// the arm that saves the node when something else (runner images, job
// workdirs) is consuming the disk.
MinFreeBytes int64
// KeepDuration is the age after which cache records are collected once
// usage is above ReservedBytes.
KeepDuration time.Duration
// EphemeralKeepDuration and EphemeralMaxUsedBytes bound the cheaply
// reproducible record types — local build contexts, RUN --mount=cache
// mounts and git checkouts. Re-creating those costs a copy, not a
// network round trip, so they are collected far more eagerly than
// layer cache. Mirrors the first rule of BuildKit's own default policy.
EphemeralKeepDuration time.Duration
EphemeralMaxUsedBytes int64
}
GCConfig bounds the on-disk BuildKit build cache.
WHY THIS EXISTS: BuildKit only garbage-collects when its worker is constructed with a non-empty GC policy — control.Controller's gc() is literally `if policy := w.GCPolicy(); len(policy) > 0 { w.Prune(...) }`. ephemerd built its worker without one, so the shared "buildkit" containerd namespace grew without bound: every `docker build` in every CI job added cache records, snapshots and `containerd.io/gc.flat` leases that nothing ever released. Measured on a production node: 76 images, 302 snapshots, 481 leases and ~44 GB of a 116 GB disk, spanning 49 long-dead jobs going back two and a half weeks. That, not runner image retention, is what filled the disk until QEMU froze the VM.
The goal is a WARM BUT BOUNDED cache. Pruning aggressively would defeat the point of a build cache and push the cost onto the network, which we are separately trying to reduce; leaving it unbounded is what caused the outage. So: a floor that is never collected, a ceiling that always is, and a free-space guard that overrides both.
func (GCConfig) Enabled ¶ added in v0.1.8
Enabled reports whether this config produces any prune rule at all — i.e. whether the build cache is bounded.
func (GCConfig) PruneInfo ¶ added in v0.1.8
PruneInfo renders the config as the BuildKit prune rules a worker's GC policy is made of. Pure — no disk access, no clock — so the resulting rule set is unit-testable without standing up a worker.
Returns nil when collection is disabled or nothing is configured, which is exactly the "never collect" state that caused the leak; callers that want a bounded cache must supply a non-zero bound.
Rules are evaluated in order and are complementary: the first keeps ephemeral records from ever becoming a large share of the cache, the second ages records out, and the third is the unconditional size bound.
THE THIRD RULE IS LOAD-BEARING AND MUST NOT CARRY A KeepDuration. BuildKit applies KeepDuration as an absolute exemption within a rule — cache/manager.go's pruneOnce skips every record whose lastUsedAt is newer than now-KeepDuration BEFORE it looks at MaxUsedSpace. So a rule that sets both an age and a size bound does not mean "collect anything over the cap, and additionally anything older than the age"; it means "collect only records older than the age, and only once over the cap". With the default 168h that leaves a week's worth of build cache completely exempt from the ceiling, which on a node that rebuilds the same images several times a day is indistinguishable from having no ceiling at all. Measured in a Linux VM against a real containerd + BuildKit: 18 builds of a unique 200 MB layer with reserved=1 GiB, max_used=2 GiB and keep_duration=168h grew the cache to 4.4 GB and climbing; the identical workload with the size bound applied unconditionally settled at 2.1 GB. BuildKit's own DefaultGCPolicy has the same shape — its last two rules carry no KeepDuration for exactly this reason.
type Server ¶
type Server struct {
// contains filtered or unexported fields
}
Server hosts an in-process BuildKit Controller and the supporting objects (session manager, worker controller, caches) it needs. Callers interact with the controller through a buildkit client.Client obtained from the Client() method; the client dials an in-process bufconn listener that the Controller serves on, so no network socket is exposed.
func NewServer ¶
NewServer constructs and initializes an embedded BuildKit server. The returned Server is ready to accept Build calls but does not expose a network listener; it is used in-process only.
func (*Server) Build ¶
func (s *Server) Build(ctx context.Context, opt client.SolveOpt, statusCh chan *client.SolveStatus) (*client.SolveResponse, error)
Build performs a Docker-style build using the embedded BuildKit solver. Progress events are written to statusCh as they arrive; statusCh is closed by the underlying solve when the build terminates. Build itself blocks until the solve completes and returns the solve response.
The caller constructs SolveOpt from Docker build options (this is the translation layer pkg/dind owns). def is nil when using a frontend like dockerfile.v0 — the frontend loads the definition from the build context supplied via SolveOpt.LocalMounts.
func (*Server) Client ¶
Client returns a buildkit client.Client connected to the in-process Controller via bufconn. The returned Client is not safe for concurrent use across different callers — construct one per request/goroutine and Close it when done.
func (*Server) Close ¶
Close signals the Controller to shut down gracefully, stops the in-process gRPC server, and releases worker resources. Safe to call multiple times.
func (*Server) ContainerdNamespace ¶ added in v0.1.8
ContainerdNamespace reports the containerd namespace build results and cache records land in. Exposed so per-job teardown (pkg/dind) can find and remove that job's build artifacts without hard-coding "buildkit".
func (*Server) Prune ¶ added in v0.1.8
Prune runs BuildKit's cache prune through its own cache manager and returns the number of bytes released.
It must go through BuildKit rather than deleting containerd records directly: BuildKit's bbolt cache DB keeps its own references to the snapshots backing each cache record, so containerd-level deletion leaves the snapshots pinned and reclaims nothing. (Confirmed the hard way on a production node — image records and leases were gone and the space did not come back until the snapshots were removed too.)
rule is a single BuildKit prune rule. The zero rule prunes everything not currently in use (a full `docker builder prune`); passing GCConfig.PruneInfo()'s bounding rule performs the same bounded collection the worker does automatically, on demand.
BuildKit's Prune RPC takes one rule per call, so callers wanting a multi-rule policy applied should call this once per rule.
func (*Server) SessionManager ¶
SessionManager exposes the session manager so callers (pkg/dind) can hijack incoming POST /session HTTP streams into session gRPC.
Source Files