Documentation
¶
Overview ¶
Package docker holds the local docker-compose shell-out primitives the stack / project / migrate commands drive. Seams (RunComposeFn / RunComposeEnvFn) let tests stub the shell-out.
Index ¶
- Constants
- Variables
- func BuilderEnv(builder string) []string
- func BuilderName(project string) string
- func ComposeProjectName(root string) string
- func EnsureBuilder(root, project, cap string) string
- func FileArgs(root string) []string
- func PruneBuilderCache(root, builder, cap string) string
- type Reclaimed
Constants ¶
const ComposeFile = "compose.w17.yaml"
ComposeFile is the tool-owned compose file. The sibling `compose.yaml` is the developer's extension point and `include:`s this one.
const DefaultCacheCap = "1GB"
DefaultCacheCap is how much build cache one project may keep.
⚠️ Deliberately SMALLER than one build's worth, which reverses the reasoning this constant shipped with. It used to be 10 GB, justified by "one build's records came to roughly 7 GB, so anything below that turns every build into a cold one". Both halves of that have since failed:
- the 7 GB was measured on a tree whose build CONTEXT was ~2.5 GB, because a developer's `node_modules` and `.next` were being uploaded into a w17 build. Measured again on a consumer's bundle from a clean checkout: one build is 1.6 GB of records, of which the context is 92 MB;
- a per-PROJECT cap does not bound a MACHINE. Twenty workspaces times any number is a product, not a limit, and a shared box does not care whose gigabytes they are. 20 x 10 GB is 200 GB on a 100 GB disk, which is not a tuning problem, it is arithmetic that cannot come out.
So the trade is taken in the other direction, on purpose and on instruction: a COLD BUILD IS CHEAPER THAN A FULL DISK. A colleague could not work at all because the machine had no space left, and no build time saved is worth that. At this cap a build keeps its base-image layers and little else, so expect the Go compile to re-run; `--cache-cap` raises it for one project that genuinely needs a warm loop, which is the right place for that decision because the person raising it is the person whose disk it is.
Variables ¶
var ( RunDockerFn = realRunDocker CaptureDockerFn = realCaptureDocker )
RunDockerFn / CaptureDockerFn are the plain-`docker` seams, beside the `docker compose` ones above. Separate because a prune is not a compose operation: compose has no verb for it, and the filters below are docker's.
var CaptureComposeFn = realCaptureCompose
CaptureComposeFn is the seam tests stub; production runs realCaptureCompose. It captures a compose subcommand's stdout (e.g. `ps --format json`) for programmatic inspection rather than streaming it to the terminal. Stderr is discarded — callers treat any error as "no data" and degrade gracefully.
var RunComposeEnvFn = realRunComposeEnv
RunComposeEnvFn is the seam tests stub; production runs realRunComposeEnv.
var RunComposeFn = realRunCompose
RunComposeFn is the seam tests stub; production runs realRunCompose.
Functions ¶
func BuilderEnv ¶
BuilderEnv is what a compose build runs with so it lands on the project's own builder.
func BuilderName ¶
BuilderName is the builder a project builds on.
func ComposeProjectName ¶
ComposeProjectName asks compose what this project is called, or "" when it cannot be asked.
Exported because the builder is named after it too: one string answers both "whose builder is this" and "whose images are these", so the two cannot drift into pruning a builder for one project and images for another.
func EnsureBuilder ¶
EnsureBuilder makes sure this project has its OWN buildx builder, and returns its name — or "" when one could not be had, in which case the caller builds on the default builder exactly as before.
Why a project gets its own builder ¶
Because the shared one cannot say whose cache is whose, and that is not a gap in docker. Measured on a live daemon: a cache record carries an id, parents, timestamps, a size, a usage count and a `shared` flag — and no owner. `buildx prune --filter` has no label key. So on the shared builder there is no filter that reclaims a project's own cache and nothing else.
The first instinct — leave it shared, because pruning a shared cache would throw away layers other projects would hit — turned out to be wrong in the important direction. Broken down by size, the cache is:
3.07 GB + 2.49 GB + 2.49 GB "local source for context" ← ONE project's 2.29 GB [build 4/6] COPY . . ← that project's 1.79 GB go build exec mount ← that project's kB … 67 MB WORKDIR, base-image bits ← cross-project
The heavily reused records (16, 29, 31 uses) are the TRIVIAL ones. The gigabytes are per-project, and they are marked `shared` only because one w17 project builds several bundles from one context — sharing WITHIN a project, which a per-project builder keeps completely. What is lost is the base-image layers: tens of MB, pulled once per builder.
So: ownership of gigabytes, at a cost of tens of megabytes and one buildkit container.
The cap is set twice, on purpose ¶
The builder is created with a buildkit GC policy (`gckeepstorage`), which bounds growth even for builds that never go through w17ctl — a plain `docker compose build` still lands on this builder if BUILDX_BUILDER is set in the shell.
That policy is NOT a hard ceiling: measured against a 2 GB target, repeated builds read 0.85 → 1.9 → 3.16 → 2.32 → 4.0 GB. It trims lazily and overshoots between passes. So PruneBuilderCache also runs an explicit `prune --max-used-space` after each build, which brought that same 4.0 GB to 1.69 GB — under the cap, synchronously, at a point the developer controls.
func FileArgs ¶
FileArgs returns the `-f <file>` docker compose should be driven with for the project at root.
Without it, compose picks its own default — `compose.yaml` in the working directory — and in a repo that had one BEFORE w17 arrived, that is somebody else's file. A consumer's `stack up` started building their unrelated services and failed pulling an image w17 never heard of. Two correct decisions produced it: codegen refuses to overwrite an existing compose.yaml, and the stack is driven from the project root. Neither is wrong; the intersection had no way to say "just mine".
So the choice is made here rather than left to discovery:
- compose.yaml INCLUDES ComposeFile → it is the stub (or a developer's extension of it), and driving it is what brings both in;
- otherwise → ComposeFile alone. A compose.yaml that does not include ours is not ours to start.
For a greenfield project, where codegen wrote the stub itself, this is the behaviour that was already there.
func PruneBuilderCache ¶
PruneBuilderCache holds one project's builder to its cap, and reports the size afterwards.
Only ever the project's OWN builder: `--builder` is not optional here, and without a name this does nothing. `docker buildx prune` with no builder acts on the default one, which is shared with every other project on the machine — see EnsureBuilder for why that is the one thing this must never do.
Types ¶
type Reclaimed ¶
type Reclaimed struct {
// Project is the compose project the prune was scoped to. Empty when the
// name could not be resolved, in which case nothing was pruned: a prune
// that cannot say whose images it is removing must not run.
Project string
// Images is how many untagged images belonging to Project were removed.
Images int
// BuildCache and BuildCacheReclaimable are docker's OWN strings, passed
// through rather than parsed. Reporting them in docker's formatting means
// there is no size parser here to drift from the one that produced them.
//
// They describe the DEFAULT builder — the shared one — and are reported only
// when this project is not on its own builder. Once it is, its cache is
// bounded and the machine-wide figure is somebody else's business.
BuildCache string
BuildCacheReclaimable string
// Builder is the project's own builder, empty when it is building on the
// shared default one. BuilderCache is that builder's total after the cap
// was applied.
Builder string
BuilderCache string
Cap string
}
Reclaimed is what one post-build reclaim did and what it deliberately did not.
func ReclaimAfterBuild ¶
ReclaimAfterBuild removes the untagged images THIS project's build just orphaned, and reports what it did not touch.
Why a rebuild leaves anything behind at all ¶
A generated project's compose stanzas carry `build:` with no `image:`, so compose names each image `<project>-<service>` and a rebuild REPLACES that tag. The layers the old tag pointed at survive as an untagged image that nothing references and nothing reclaims. Build daily and that is daily growth, which is the complaint this exists for.
Why it is scoped to the project, and why that is not optional ¶
`docker image prune` without a filter removes every dangling image on the daemon. A developer's daemon is not theirs alone — the machine this was written on runs three other projects' stacks, one of whose images is 3.64GB — and a build command that reaches outside its own project is a build command nobody can afford to run twice.
So the filter is `label=com.docker.compose.project=<name>`, and the name comes from asking compose (`config --format json`) rather than deriving it. Compose honours COMPOSE_PROJECT_NAME and normalises directory names; a second implementation of that rule would disagree with the first one eventually, and the disagreement would be a prune aimed at the wrong project.
If the name cannot be resolved, NOTHING is pruned. A prune that cannot say whose images it removes has no business running.
What it deliberately leaves alone ¶
- VOLUMES. They hold the dev database. `docker rm` without `-v` once turned leaked containers into leaked volumes on this machine, and the opposite mistake — reclaiming a volume — costs somebody their data, which no disk saving justifies.
- The BUILD CACHE, which is both the largest item and the one that makes the next build fast. It also has no project filter, so capping it is a decision about the whole machine and cannot be made from inside one project's build. Reported, never touched.
- TAGGED images, including this project's previous versions. Unused is not unwanted.
Source Files