README
¶
oci
Pure-Go, read-only OCI / Docker image filesystem for the
go-filesystems family. It overlays an
image's tar layers — honouring whiteouts, opaque directories, hardlinks and
symlinks — and serves the merged rootfs through
go-filesystems/interface. No
container runtime, no cgo.
Why
A container image is a filesystem: an ordered stack of tar layers that overlay
into a rootfs. This driver reads that rootfs natively — to mkfs it into an
ext4 image, serve it read-only over virtio-fs, or inspect it — without pulling
in containerd, a runtime, or any C toolchain.
Install
go get github.com/go-filesystems/oci
Usage
import "github.com/go-filesystems/oci"
// From an OCI image layout directory…
fsys, err := oci.OpenLayout("path/to/oci-layout")
// …or a `docker save` / OCI archive tarball:
// fsys, err := oci.OpenTarball("image.tar")
if err != nil {
return err
}
defer fsys.Close()
data, err := fsys.ReadFile("/etc/os-release")
entries, err := fsys.ListDir("/usr/bin")
st, err := fsys.Stat("/bin/sh")
target, err := fsys.ReadLink("/bin/sh") // if it is a symlink
fsys satisfies filesystem.Filesystem. For a multi-arch index, select a
manifest with OpenSelect/OpenDescriptor; to read from a custom blob store,
implement BlobSource and call Open.
The image is read only: WriteFile, MkDir, DeleteFile, DeleteDir and
Rename return ErrReadOnly.
Layer compression
gzip and uncompressed layers are built in (stdlib only). Other codecs are
opt-in with no extra dependency — register them yourself:
oci.RegisterDecompressor(oci.MediaTypeLayerTarZstd, func(r io.Reader) (io.Reader, error) {
return zstd.NewReader(r) // your pure-Go zstd of choice
})
Untrusted input
An image is untrusted input: a few hundred bytes of gzip can declare a layer that
expands to gigabytes, and a tar header costs far more memory than its 512 bytes.
Every input-driven allocation is bounded before it happens, and exceeding a
bound returns an error wrapping safeio.ErrTooLarge. Defaults (DefaultLimits()):
| Bound | Default | Why |
|---|---|---|
manifest, index, index.json, manifest.json |
16 MiB | the distribution spec asks for at least 4 MB |
| raw layer blob | 4 GiB | |
| regular-file bytes: per file / per layer / whole image | 4 GiB / 8 GiB / 16 GiB | decompression bomb |
| decompressed tar stream of one layer | 16 GiB | headers and skipped entries count |
| bytes a layer claims beyond what it carries (sparse holes) | 64 MiB | 199 bytes of gzip declared 1 GiB |
| tree nodes (files, directories, implied directories) | 2 Mi | a node costs about 1.1 KB |
| names and link targets, whole image | 512 MiB | a 2 KiB layer held a 1 MiB path |
| one path or link target | 4096 bytes | Linux PATH_MAX |
| layers / index nesting | 1024 / 16 |
Open an image with tighter ceilings:
src, err := oci.TarballLimits("image.tar", oci.Limits{MaxArchiveTotal: 1 << 30})
img, err := oci.OpenSelectLimits(src, oci.Selector{}, oci.Limits{
MaxImageUncompressed: 1 << 30,
MaxEntries: 200_000,
})
A zero Limits field takes its default. Digests name a directory in an OCI
layout (blobs/<alg>/<hex>), so an algorithm outside the image-spec grammar
(../..) is rejected rather than followed. FuzzApplyLayer and FuzzOpenTarball
exercise the layer folder and the archive reader.
License
BSD-3-Clause © the go-filesystems/oci authors.
Documentation
¶
Overview ¶
Package oci presents an OCI/Docker image as a read-only github.com/go-filesystems/interface Filesystem by overlaying the image's tar layers with overlayfs semantics. It is pure Go (CGO_ENABLED=0) and pulls no third-party dependencies for its core: gzip and plain layers are handled by the standard library, and any other compression (e.g. zstd) is supported through an injectable Decompressor registry.
Index ¶
- Constants
- Variables
- func RegisterDecompressor(mediaType string, d Decompressor)
- type BlobSource
- type Decompressor
- type FS
- func Open(src BlobSource) (*FS, error)
- func OpenDescriptor(src BlobSource, top descriptor, sel Selector) (*FS, error)
- func OpenLayout(dir string) (*FS, error)
- func OpenSelect(src BlobSource, sel Selector) (*FS, error)
- func OpenSelectLimits(src BlobSource, sel Selector, lim Limits) (*FS, error)
- func OpenTarball(p string) (*FS, error)
- func (f *FS) Close() error
- func (f *FS) DeleteDir(path string) error
- func (f *FS) DeleteFile(path string) error
- func (f *FS) ListDir(path string) ([]filesystem.DirEntry, error)
- func (f *FS) MkDir(path string, perm os.FileMode) error
- func (f *FS) ReadFile(path string) ([]byte, error)
- func (f *FS) ReadLink(path string) (string, error)
- func (f *FS) Rename(oldPath, newPath string) error
- func (f *FS) Stat(path string) (filesystem.Stat, error)
- func (f *FS) WriteFile(path string, data []byte, perm os.FileMode) error
- type Limits
- type Selector
Constants ¶
const ( // Uncompressed tar layers. MediaTypeLayerTar = "application/vnd.oci.image.layer.v1.tar" MediaTypeDockerLayerTar = "application/vnd.docker.image.rootfs.diff.tar" // gzip-compressed tar layers. MediaTypeLayerTarGzip = "application/vnd.oci.image.layer.v1.tar+gzip" MediaTypeDockerLayerTarGzip = "application/vnd.docker.image.rootfs.diff.tar.gzip" // zstd-compressed tar layers (no built-in decompressor; register one). MediaTypeLayerTarZstd = "application/vnd.oci.image.layer.v1.tar+zstd" // Manifest / index media types we resolve. MediaTypeImageManifest = "application/vnd.oci.image.manifest.v1+json" MediaTypeImageIndex = "application/vnd.oci.image.index.v1+json" MediaTypeDockerManifest = "application/vnd.docker.distribution.manifest.v2+json" MediaTypeDockerManifestList = "application/vnd.docker.distribution.manifest.list.v2+json" MediaTypeImageConfig = "application/vnd.oci.image.config.v1+json" MediaTypeDockerImageConfig = "application/vnd.docker.container.image.v1+json" )
Built-in OCI / Docker layer media types.
const ( FileTypeUnknown uint8 = iota FileTypeRegular FileTypeDir FileTypeSymlink FileTypeHardlink FileTypeChar FileTypeBlock FileTypeFifo )
FileType constants returned by DirEntry.FileType() and node.ftype. The go-filesystems/interface package does not define these, so we define a local mapping that mirrors the POSIX d_type / tar typeflag taxonomy.
Variables ¶
var ErrReadOnly = errors.New("oci: read-only filesystem")
ErrReadOnly is returned by every mutating method of FS. An OCI image filesystem is immutable: it is a read-only overlay of the image's layers.
Functions ¶
func RegisterDecompressor ¶
func RegisterDecompressor(mediaType string, d Decompressor)
RegisterDecompressor registers d for the given layer media type, replacing any previous registration. Pass a media type such as MediaTypeLayerTarZstd to enable zstd without a core dependency.
Types ¶
type BlobSource ¶
type BlobSource interface {
// Blob returns a reader for the blob identified by digest. The caller
// must Close the returned reader. An error wrapping fs.ErrNotExist is
// returned when the digest is unknown.
Blob(digest string) (io.ReadCloser, error)
}
BlobSource provides content-addressed access to an image's blobs (manifests, configs and layers) by their digest string (e.g. "sha256:abc...").
func OCILayout ¶
func OCILayout(dir string) BlobSource
OCILayout returns a BlobSource backed by an OCI image layout directory, i.e. a directory containing an "oci-layout" marker, "index.json" and a "blobs/<alg>/<hex>" content store.
func Tarball ¶
func Tarball(p string) (BlobSource, error)
Tarball returns a BlobSource backed by a `docker save` / OCI archive tar file on disk.
func TarballFS ¶
func TarballFS(fsys fs.FS) (BlobSource, error)
TarballFS returns a BlobSource backed by an archive tar exposed through an fs.FS. The archive is expected at the path "image.tar"; if absent, the first (and only) regular file is used. Most callers should prefer Tarball.
func TarballLimits ¶ added in v0.4.0
func TarballLimits(p string, lim Limits) (BlobSource, error)
TarballLimits is Tarball with an explicit resource budget for the archive (MaxArchiveMember, MaxArchiveTotal, MaxEntries, MaxMetadata, MaxPathLen, MaxSparseHoles). A zero field takes the default.
type Decompressor ¶
Decompressor wraps a raw layer blob reader and returns a reader that yields the decompressed tar stream. Implementations must not assume ownership of r; closing the underlying blob is the caller's responsibility.
type FS ¶
type FS struct {
// contains filtered or unexported fields
}
FS is a read-only github.com/go-filesystems/interface Filesystem view over an OCI/Docker image's merged layer tree.
ReadFile strategy: file contents are buffered into the in-memory merged tree at Open time (each regular file's bytes are read once while its owning layer is decompressed). This keeps ReadFile a pure in-memory lookup, makes hardlink resolution trivial (both names share the target's buffered bytes), and avoids holding blob file handles open after Open returns. The trade-off is memory proportional to the uncompressed image size; for the disk-image-as-filesystem use cases this driver targets that is acceptable and matches how the sibling drivers buffer their trees.
func Open ¶
func Open(src BlobSource) (*FS, error)
Open resolves the single image addressed by src (the first manifest in the index, or a multi-arch index's first matching manifest) and returns a read-only FS over its merged layers. To select a specific manifest from a multi-arch index, pass a Selector via OpenSelect.
func OpenDescriptor ¶
func OpenDescriptor(src BlobSource, top descriptor, sel Selector) (*FS, error)
OpenDescriptor opens an image given an explicit top-level descriptor, allowing use of a bare BlobSource that does not embed index discovery.
func OpenLayout ¶
OpenLayout is a convenience wrapper: OpenLayout(dir) == Open(OCILayout(dir)).
func OpenSelect ¶
func OpenSelect(src BlobSource, sel Selector) (*FS, error)
OpenSelect is like Open but selects a manifest from a multi-arch index by digest or platform.
func OpenSelectLimits ¶ added in v0.4.0
func OpenSelectLimits(src BlobSource, sel Selector, lim Limits) (*FS, error)
OpenSelectLimits is OpenSelect with an explicit resource budget for the image: use it to open an untrusted image with tighter ceilings than the defaults. A zero field of lim takes the default (see DefaultLimits). A limit that is exceeded fails with an error wrapping safeio.ErrTooLarge before the memory is spent.
func OpenTarball ¶
OpenTarball is a convenience wrapper around Tarball.
func (*FS) DeleteFile ¶
DeleteFile always returns ErrReadOnly.
func (*FS) ListDir ¶
func (f *FS) ListDir(path string) ([]filesystem.DirEntry, error)
ListDir returns the entries of the directory at path, sorted by name.
func (*FS) ReadFile ¶
ReadFile returns the contents of the regular file (or hardlink to one) at path.
type Limits ¶ added in v0.4.0
type Limits struct {
// MaxBlob bounds one raw (compressed) layer blob, in bytes.
MaxBlob int64
// MaxManifest bounds a manifest, index, index.json or manifest.json.
MaxManifest int64
// MaxLayerUncompressed bounds the regular-file bytes of one layer.
MaxLayerUncompressed int64
// MaxImageUncompressed bounds the regular-file bytes of the whole image.
MaxImageUncompressed int64
// MaxFile bounds one regular file.
MaxFile int64
// MaxLayerStream bounds the decompressed tar stream of one layer.
MaxLayerStream int64
// MaxSparseHoles bounds the bytes a layer may claim beyond what it carries.
MaxSparseHoles int64
// MaxMetadata bounds the names and link targets of the whole image.
MaxMetadata int64
// MaxEntries bounds the nodes of the merged tree.
MaxEntries int64
// MaxPathLen bounds one entry name or link target, in bytes.
MaxPathLen int
// MaxLayers bounds the number of layers.
MaxLayers int
// MaxIndexDepth bounds index-of-index nesting.
MaxIndexDepth int
// MaxArchiveMember bounds one member of a docker-save / OCI archive.
MaxArchiveMember int64
// MaxArchiveTotal bounds all the members of an archive together.
MaxArchiveTotal int64
}
Limits is the resource budget for opening one untrusted image. A zero field takes the package default (see DefaultLimits); a negative one refuses everything of that kind. Pass it to OpenSelectLimits or TarballLimits.
func DefaultLimits ¶ added in v0.4.0
func DefaultLimits() Limits
DefaultLimits returns the default budget: generous for real images, finite for hostile ones.
type Selector ¶
type Selector struct {
// Digest, if non-empty, selects the manifest descriptor whose digest
// matches exactly.
Digest string
// OS / Architecture / Variant, if set, select by platform. Empty fields
// are treated as wildcards.
OS string
Architecture string
Variant string
}
Selector picks one manifest from a multi-manifest index.
Source Files