Documentation
¶
Overview ¶
Package nixstore implements push/pull of a nix-store tarball as an OCI image layer, against a remote registry like GHCR.
Background: the CI workflow caches the populated `/nix` Docker volume between runs by serializing it as a single OCI layer. CELL-292 made this work via the `crane` CLI, but `crane append`'s stdin handler had behavior that caused six iterations of CI breakage (re-gzipping pre-gzipped input, buffering uncompressed stdin to disk, …). This package replaces those CLI calls with direct uses of `github.com/google/go-containerregistry/pkg/v1/{stream,tarball}`, giving deterministic encoding + true streaming + a single Go code path shared between local tests and CI.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ProgressTick = 5 * time.Second
ProgressTick controls how often Push emits an in-flight progress line. Set short in tests; CI uses the default.
var ProgressWriter io.Writer = os.Stderr
ProgressWriter receives periodic progress lines from Push. Defaults to os.Stderr so CI logs surface upload activity; tests can swap it for a buffer. Concurrent writes are serialized inside Push.
Functions ¶
func Pull ¶
Pull resolves srcRef, downloads the LAST layer of the image, and extracts the gzipped tarball into dstDir. Archive entries are written relative to dstDir, with stripComponents leading path elements stripped per `tar --strip-components=N` semantics (e.g. with stripComponents=1, archive entry `nix/store/abc/file` becomes `dstDir/store/abc/file`). Symlinks and file modes are preserved.
We only extract the topmost layer because the production workflow builds cache images as `<busybox-base> + <single nix-store tar layer>` — extracting earlier layers would pollute dstDir with the base image's filesystem.
The decompression stream is consumed lazily as tar entries are read, so peak memory stays near the gzip window size (~32 KB) regardless of layer size.
func PullToDockerVolume ¶
PullToDockerVolume streams the LAST layer of srcRef into the named Docker volume by spawning `docker run -i alpine sh -c 'cd /dest && tar -x --strip-components=N'` and feeding the (gunzipped) tar stream over stdin. Use this when the destination is a Docker volume that the current process can't directly mount (the standard workflow case — the volume is owned by the host docker daemon).
stripComponents has the same meaning as Pull: leading path elements to strip from each archive entry.
We pull alpine implicitly via docker run; this matches the existing workflow's approach and keeps the call site identical to what the CI publish step does today.
func Push ¶
Push streams an uncompressed tar from r through to dstRef as a single OCI tar+gzip layer atop baseRef.
Streaming guarantees:
- Bytes flow: r → gzip writer → registry chunked upload. No disk buffer, no temp file. Peak memory is the gzip window (~32 KB).
- Digest is computed incrementally during the upload (via `pkg/v1/stream.NewLayer`), then finalized in the PUT-with-digest call that completes the OCI chunked upload — the registry protocol doesn't require the digest upfront.
- The on-wire layer is SINGLE-gzipped. `stream.NewLayer` treats its input as uncompressed and gzips exactly once, regardless of what bytes the caller provides. The caller MUST provide raw (uncompressed) tar — pre-gzipped input would round-trip through a redundant gzip layer.
Why this exists: `crane append --new_layer -` produced double-gzipped layers when fed `tar -czf -` because crane's stdin handler doesn't detect gzip magic; the CLI workaround was to stage a tarball file (which crane DOES detect). This function bypasses both surprises by using the underlying go-containerregistry library directly.
Types ¶
This section is empty.
Source Files