artifact

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 7, 2026 License: Apache-2.0 Imports: 46 Imported by: 0

Documentation

Overview

Package artifact packages a zygote as an OCI artifact and moves it through a registry, so every home that holds a grant for a template warms byte-identical zygote pages from one content-addressed source.

Layout (one manifest, three layers, all titled): 

zygote       the executable, application/vnd.fiberd.zygote.bin.v1
config.json  argv, arch, kernel and libc of the build host, digests,
             and how images.tar is sealed
images.tar   CRIU images of the zygote taken right after READY: the
             parent pages that fiber deltas are computed against.
             Sealed to the fleet's wrap key (wrap.go), never plain.
             Or a Hyperlight guest's snapshot (snapshot.go), under
             its own media type, and then no zygote

The manifest digest is the template digest a grant carries.

Index

Constants

View Source 
const (
	ArtifactType    = "application/vnd.fiberd.zygote.v1"
	MediaTypeBinary = "application/vnd.fiberd.zygote.bin.v1"
	MediaTypeConfig = "application/vnd.fiberd.zygote.config.v1+json"
	MediaTypeImages = "application/vnd.fiberd.zygote.criu.v1.tar"
	// MediaTypeHyperlight is images.tar holding a Hyperlight guest's
	// snapshot, a tree a helper restores the warm template from, rather
	// than CRIU images (ImagesFormatHyperlight).
	MediaTypeHyperlight = "application/vnd.fiberd.zygote.hyperlight.v1.tar"

	// ImagesFormatHyperlight is Config.ImagesFormat for a Hyperlight
	// snapshot. Its artifact needs no zygote, since the helper restores
	// the guest from the snapshot and runs no executable of the template's.
	ImagesFormatHyperlight = "hyperlight"
)
View Source 
const (
	ArtifactTypeDelta  = "application/vnd.fiberd.delta.v1"
	ArtifactTypeParent = "application/vnd.fiberd.parent.v1"

	AnnotationWBytes  = "io.fiberd.w_bytes"
	AnnotationParent  = "io.fiberd.parent_sha256"
	AnnotationHome    = "io.fiberd.home"
	AnnotationFence   = "io.fiberd.fence"
	AnnotationGrant   = "io.fiberd.grant_uid"
	AnnotationSession = "io.fiberd.session"
	// AnnotationGeneration is the parked fiber's generation, so a home
	// that claims the delta resumes it as the next one of its lineage.
	AnnotationGeneration = "io.fiberd.generation"
)
View Source 
const (
	// LinkStatic is an ELF executable with no program interpreter.
	LinkStatic = "static"
	// LinkDynamic is anything else: an ELF with an interpreter, or a
	// script, which needs its interpreter from the root filesystem.
	LinkDynamic = "dynamic"
)

How the zygote executable is linked. A home whose backend runs the template inside a root filesystem of its own (gVisor, runc) cannot load a dynamically linked executable unless that filesystem holds the libc it was built against, which the home cannot tell. A static executable needs nothing from it. Build records the fact so the home can refuse at warm, with a clear error, instead of failing inside the sandbox.

View Source 
const (
	AnnotationArch      = "io.fiberd.arch"
	AnnotationKernel    = "io.fiberd.kernel"
	AnnotationLibc      = "io.fiberd.libc"
	AnnotationBackend   = "io.fiberd.backend"
	AnnotationTemplates = "io.fiberd.templates"
)

Annotations for platform facts on delta and parent manifests.

View Source 
const (
	AnnotationEngineClass   = "io.fiberd.engine_class"
	AnnotationEngineLayout  = "io.fiberd.engine_layout"
	AnnotationEngineVersion = "io.fiberd.engine_version"
	AnnotationGPUName       = "io.fiberd.gpu_name"
	AnnotationGPUCC         = "io.fiberd.gpu_cc"
	AnnotationGPUDriver     = "io.fiberd.gpu_driver"
)

Annotations for the device engine a parked fiber's slice was saved by, on a delta of a grant with an engine.

View Source 
const (
	ParityExact  = "exact"  // must be identical
	ParitySeries = "series" // kernel only: same major.minor
	ParityOff    = "off"    // not compared
)

Parity levels for one fact.

View Source 
const (
	AnnotationDomain  = "io.fiberd.domain"
	AnnotationExpires = "io.fiberd.expires"
	AnnotationSealKey = "io.fiberd.seal_key"
)
View Source 
const (
	AnnotationSignature = "io.fiberd.signature"
	AnnotationSigner    = "io.fiberd.signer"
)
View Source 
const FileScheme = "file://"

Variables

View Source 
var ErrEntryName = errors.New("artifact: archive entry is not a plain file name")

ErrEntryName means an images archive or a file registry manifest names a file that is not a plain file name, as TarDir and PushDir write them.

View Source 
var ErrExpired = errors.New("artifact: sealed content has expired")

ErrExpired means a sealed delta is past the expiry it was sealed with.

View Source 
var ErrParity = errors.New("platform parity")

ErrParity is wrapped by every mismatch.

View Source 
var ErrSealed = errors.New("artifact: sealed content does not open")

ErrSealed means a sealed file does not open with this home's key for the domain and session asked for, or it was changed after sealing.

View Source 
var ErrUntrusted = errors.New("artifact: not signed by a trusted key")

ErrUntrusted means a directory artifact is unsigned, signed by a key the home does not trust, or signed over content it no longer holds.

View Source 
var Strict = Parity{Kernel: ParityExact, Libc: ParityExact}

Strict requires everything to match exactly.

Functions

func Build 

func Build(ctx context.Context, o BuildOptions) (string, error)

Build runs the zygote the way a home does (control channel on fd 3), waits for READY, checkpoints it with criu --leave-running, seals the images to SealTo, and writes the artifact directory. Returns the packed manifest digest. The plaintext images and their tar are removed once sealed. With SkipImages it only checks that the zygote reaches READY, which needs no criu, and with NoSmoke as well it runs nothing. 

SkipImages  NoSmoke   runs the zygote   criu   images.tar
false       any       yes               yes    yes, sealed to SealTo
true        false     yes, to READY     no     no
true        true      no                no     no

func Delete 

func Delete(ctx context.Context, ref string) (removed bool, err error)

Delete removes the manifest at ref (a tag or digest). Registries must allow deletes (registry:2: REGISTRY_STORAGE_DELETE_ENABLED=true). A missing manifest is not an error: the goal is that it be gone. A tag is deleted as a tag first (some registries keep tag entries when the manifest is deleted by digest), then the manifest by digest.

removed reports whether this call took the reference away. Two deleters of one tag race on the registry's one delete, and only the one whose delete found it wins. A claim of a parked session is that delete, so a caller that is told false lost the claim.

func GenerateSealKey 

func GenerateSealKey() (*jose.JSONWebKey, error)

GenerateSealKey returns a fresh master seal key as a symmetric JWK.

func HostInfo 

func HostInfo() (arch, kernel, libc string)

HostInfo fills the parity fields for the running host.

func ImagesDir 

func ImagesDir(dir string) string

ImagesDir is where images.tar is unpacked in a pulled artifact.

func IsFileRef 

func IsFileRef(ref string) bool

IsFileRef reports whether ref names a file registry.

func KernelSeries 

func KernelSeries(release string) string

KernelSeries is the "major.minor" of a release string ("6.10.14-linuxkit" -> "6.10"): the level at which the syscall and image ABI usually holds.

func LibcVersion 

func LibcVersion(bin string) string

LibcVersion is the libc the ldd called bin names.

func Linking 

func Linking(path string) string

Linking reports how the executable at path is linked. Anything that is not a static ELF, including a file that cannot be read, is dynamic: the conservative answer, since a home refuses what it cannot vouch for.

func OpenImages 

func OpenImages(src, dst string, k *SealKey, s ImagesSeal) error

OpenImages opens the sealed tar at src into dst, written owner-only, using the fleet key derived from k and the metadata config.json carries. The file is refused before any of it is decrypted when its header does not say what s says, and the plaintext is refused after when it does not hash to s.SHA256. On error dst is removed.

func OpenParentDir 

func OpenParentDir(dir string, k *SealKey, sha string) error

OpenParentDir replaces every sealed file in dir by its plaintext. It refuses any file not sealed with k as the parent sha, or sealed as a delta or as author images, or changed since. On error, dir holds a mix of sealed and plain files, and the caller must remove it. On success the caller still loads dir with the delta codec and checks that it hashes to sha.

func OpenSnapshot 

func OpenSnapshot(dir string, cfg Config, k *SealKey, dst string) error

OpenSnapshot opens the sealed Hyperlight snapshot of the verified artifact in dir with the fleet key derived from k, checks it hashes to what cfg names, and unpacks it into dst for one warm alone, which must not exist. A copy of its own, so nothing another warm or a later resolve does reaches it, and read-only once made: files 0400 and directories 0500, fiberd's alone. The plaintext tar is made beside dst and is gone when it returns. On any error dst is removed. RemoveSnapshot removes it.

Read-only is by mode. It keeps a process without CAP_DAC_OVERRIDE from writing the template by mistake, and is no boundary: one running as fiberd's user could make it writable, and anything else that user can write besides.

func Pack 

func Pack(ctx context.Context, dir string) (string, error)

Pack computes the manifest for an artifact directory and records its digest in the DIGEST side file (never inside a layer, or the digest would change itself). It is deterministic for a given directory: the created annotation is the build time from the config, not now.

func PullDir 

func PullDir(ctx context.Context, ref, dst string) (string, error)

PullDir downloads the artifact at ref into dst (files land under their titles) and returns the manifest digest.

func Push 

func Push(ctx context.Context, dir, ref string, cred auth.CredentialFunc) (string, error)

Push uploads the artifact directory to ref ("host/repo:tag") and returns the manifest digest. cred answers a registry that asks for a login, and is only read if one does. A nil cred pushes anonymously, and a registry named with http:// is never given a login.

func PushDir 

func PushDir(ctx context.Context, dir, ref, artifactType string, annotations map[string]string, signer *jose.JSONWebKey) (string, error)

PushDir uploads every regular file in dir (logs excluded) as the artifact at ref, with the annotations on the manifest. It signs with signer when that is set, and returns the manifest digest.

func ReadDigest 

func ReadDigest(dir string) (string, error)

ReadDigest returns the digest Pack recorded.

func RemoveSnapshot 

func RemoveSnapshot(dst string) error

RemoveSnapshot removes a snapshot OpenSnapshot opened, giving its directories back the write bit a removal needs first, and any plaintext tar an open left beside it. A missing dst is no error.

func Repository 

func Repository(ref string) (*remote.Repository, string, error)

Repository is the remote.Repository for callers that need it directly.

func SealDir 

func SealDir(src, dst string, k *SealKey, c SealContext) error

SealDir writes every file PushDir would push from src, sealed for c, into dst under the same names. A delta always expires, so a zero Expires is refused here rather than sealed as a file nothing opens.

func SealParentDir 

func SealParentDir(src, dst string, k *SealKey, sha string) error

SealParentDir writes every file PushDir would push from src, sealed as the parent checkpoint sha, into dst under the same names. src must be the store entry whose codec hash is sha.

func TarDir 

func TarDir(src, dst string) error

TarDir writes every regular file directly under src into the tar archive dst (flat, reproducible). Untar unpacks such an archive into dst and fails with ErrEntryName on any entry that is not a plain name.

func TarSnapshot 

func TarSnapshot(src, dst string) error

TarSnapshot writes the tree at src into the tar at dst, directories and regular files only, in a stable order with fixed times and owners, so one tree always tars to the same bytes. Anything else in the tree (a symlink, a device, a FIFO) refuses the tar rather than be dropped, and so do a src that is itself a symlink, which would tar as nothing, and a tree that holds no file.

func UnpackImages 

func UnpackImages(dir string, cfg Config, k *SealKey) error

UnpackImages opens the sealed images.tar of the verified artifact in dir with the fleet key derived from k, checks it hashes to what cfg names, and unpacks it into ImagesDir(dir), owner-only. The unpack goes beside and is swapped in, so a parent already mapped from the old files keeps them until it is closed. Without k it says the home needs the fleet's seal key.

func Untar 

func Untar(src, dst string) error

func ZygotePath 

func ZygotePath(dir string) string

ZygotePath is where the executable lives in an artifact directory.

Types

type BuildOptions 

type BuildOptions struct {
	Zygote string   // path to the executable
	Args   []string // arguments the home will pass
	Out    string   // artifact directory to create
	// SkipImages builds an artifact without a CRIU checkpoint (no criu on
	// this host, or a template that does not need deltas). The zygote is
	// still run to READY once, as a smoke test, unless NoSmoke.
	SkipImages bool
	// NoSmoke skips that run, for a template whose warm-up is slow or one
	// this host cannot run, such as a gVisor-mode template that waits on
	// /proc/gvisor/checkpoint, another architecture's, or any off Linux.
	NoSmoke bool
	// SealTo is the fleet's public wrap key (fleet-pub.json, which
	// grant-issuer keygen -alg A256GCM writes). images.tar is sealed to
	// it, and a build with images refuses to run without it, so no
	// template memory is ever packed in the clear. A build with
	// SkipImages needs none.
	SealTo       *WrapPublicKey
	CRIU         criu.Options
	ReadyTimeout time.Duration
	// Platform is the build host the artifact records. Empty fields are
	// detected (ldd for the libc on Linux), so a caller that names all
	// three runs nothing. Backend and Templates are not recorded.
	Platform Platform
	// HyperlightSnapshot is the directory of a Hyperlight guest's
	// snapshot to carry as the artifact's images instead of CRIU images
	// of a zygote (ImagesFormatHyperlight), sealed to SealTo. Zygote must
	// then be empty, and Platform must name the kernel and libc a
	// Hyperlight home reports, the helper's facts (hyperlight.PlatformOf),
	// since the snapshot depends on the helper and not on this host.
	HyperlightSnapshot string
}

BuildOptions describe how to turn a zygote executable into an artifact.

type Config 

type Config struct {
	Args         []string `json:"args"`
	Arch         string   `json:"arch"`
	Kernel       string   `json:"kernel"`
	Libc         string   `json:"libc"`
	ZygoteSHA256 string   `json:"zygote_sha256"`
	// Linking is LinkStatic or LinkDynamic (linking.go). A home whose
	// backend runs the template inside a root filesystem of its own
	// refuses a dynamic one, since it cannot tell whether that
	// filesystem holds the libc. Empty in a config written before the
	// fact was recorded, which counts as dynamic.
	Linking string `json:"linking,omitempty"`
	// Images says how images.tar is sealed. Nil when the artifact has no
	// images. An artifact never carries images in the clear.
	Images *ImagesSeal `json:"images,omitempty"`
	// ImagesFormat is what images.tar holds: empty for CRIU images of
	// the zygote, ImagesFormatHyperlight for a Hyperlight snapshot.
	ImagesFormat string    `json:"images_format,omitempty"`
	BuiltAt      time.Time `json:"built_at"`
	// Digest is the manifest digest. It is never stored in config.json
	// (it is a layer); Pull fills it from the transfer, and ReadDigest
	// reads the DIGEST side file Pack and Pull write.
	Digest string `json:"-"`
}

Config describes the zygote and the host it was checkpointed on. The parity fields gate cross-host resume: a delta over these pages only restores where the kernel and libc match.

func Pull 

func Pull(ctx context.Context, ref, dst string) (Config, error)

Pull downloads the artifact at repo@digest (or repo:tag) into dst. The manifest digest is verified by the transfer, and the directory must pack to it, as Reverify holds it at every warm. Sealed images stay sealed. A home opens them with UnpackImages.

func ReadConfig 

func ReadConfig(dir string) (Config, error)

ReadConfig loads config.json from an artifact directory.

func Reverify 

func Reverify(ctx context.Context, dir, digest string) (Config, error)

Reverify re-checks a template cache entry a previous Pull left. The directory must still pack to digest and its zygote still hash to its config. A home then unpacks the verified images.tar afresh with UnpackImages. Without this, anything that can write the cache could change what every later warm runs.

func (Config) HasImages 

func (c Config) HasImages() bool

HasImages reports whether the artifact carries a sealed images.tar.

func (Config) HasZygote 

func (c Config) HasZygote() bool

HasZygote reports whether the artifact carries an executable. A Hyperlight snapshot artifact never does: the helper restores the guest and runs no executable of the template's, and a fiberd from before the format, which would run its own guest in the snapshot's place, cannot pull an artifact without one.

func (Config) Platform 

func (c Config) Platform() Platform

Platform returns the build host recorded in an artifact config.

type File 

type File struct {
	Name   string `json:"name"`
	Digest string `json:"digest"`
	Size   int64  `json:"size"`
}

File is one file of a directory artifact as its manifest lists it.

type ImagesSeal 

type ImagesSeal struct {
	KeyID   string `json:"wrap_key"`
	Enc     []byte `json:"enc"`
	Wrapped []byte `json:"wrapped_key"`
	SHA256  string `json:"sha256"`
}

ImagesSeal is what config.json carries about a sealed images.tar. A home needs all four to open it, and each is also bound into the sealed file.

func SealImages 

func SealImages(src, dst string, pub *WrapPublicKey) (ImagesSeal, error)

SealImages seals the plaintext tar at src into dst for the fleet key pub and returns what config.json must carry. src is read twice, once to hash it and once to seal it, so a 512 MiB tar never sits in memory. The name images.tar is bound into the file key whatever dst is called.

type Keys 

type Keys struct {
	// Signer is the private Ed25519 JWK pushes are signed with.
	Signer *jose.JSONWebKey
	// Trust holds the public keys whose signatures are accepted, by kid.
	// Signer's own public half is always accepted.
	Trust []jose.JSONWebKey
	// Seal is the master key deltas are sealed under (seal.go).
	Seal *SealKey
}

Keys is what a home signs with and what it accepts.

func LoadKeys 

func LoadKeys(signerPath, trustPath string) (Keys, error)

LoadKeys reads the signing key (a private Ed25519 JWK, as `grant-issuer keygen -alg EdDSA` writes) and, when trustPath is set, a JWKS of further public keys to accept.

func (Keys) Verify 

func (k Keys) Verify(loc Located) error

Verify checks that a located artifact is signed by a trusted key.

func (Keys) VerifyDir 

func (k Keys) VerifyDir(dir, artifactType string, annotations map[string]string) error

VerifyDir checks a signature against the files in dir, for artifacts that arrive as files rather than from a registry (an import).

type Located 

type Located struct {
	Digest       string
	ArtifactType string
	Annotations  map[string]string
	Files        []File
}

Located describes an artifact found in a registry without pulling it.

func Resolve 

func Resolve(ctx context.Context, ref string) (Located, bool, error)

Resolve fetches only the manifest at ref (repo:tag or repo@digest). found is false when the registry has nothing there.

type Parity 

type Parity struct {
	Kernel string // ParityExact (default), ParitySeries or ParityOff
	Libc   string // ParityExact (default) or ParityOff
}

Parity says how strictly a home requires a checkpoint's platform to match its own. The architecture, the backend and the template cache path are always required to match; there is no level at which a foreign instruction set, another mechanism's image format or a missing template executable can restore. The zero value is strict: exact kernel release and exact libc.

func ParseParity 

func ParseParity(s string) (Parity, error)

ParseParity reads the -parity flag: "strict", "off", or a comma list of kernel=exact|series|off and libc=exact|off.

func (Parity) Check 

func (p Parity) Check(host, want Platform) error

Check compares the platform a checkpoint was made on with the host it would restore on, at this parity level.

func (Parity) String 

func (p Parity) String() string

type Platform 

type Platform struct {
	Arch   string `json:"arch"`
	Kernel string `json:"kernel"` // uname -r
	Libc   string `json:"libc"`   // "(gnu libc) 2.36"
	// Backend is the sandbox mechanism that made the checkpoint ("proc",
	// "gvisor", ...). Empty for a template artifact (any backend that
	// reads the format may warm it) and for facts recorded before it.
	Backend string `json:"backend,omitempty"`
	// Templates is the template cache path, for a backend whose restore
	// reopens a cached template executable by that path (proc). Empty
	// for any other, and for a session whose template is not cached.
	Templates string `json:"templates,omitempty"`
}

Platform is what a checkpoint's pages assume about the host that made them.

func Host 

func Host() Platform

Host is the platform this process runs on. It runs ldd on Linux, so a caller that already knows its facts completes a Platform instead.

func PlatformFromAnnotations 

func PlatformFromAnnotations(m map[string]string) (Platform, bool)

PlatformFromAnnotations reads a platform back; ok is false when the manifest carries none (a publisher from before the parity gate).

func (Platform) Annotate 

func (p Platform) Annotate(m map[string]string)

Annotate adds the platform to a manifest annotation map.

func (Platform) Complete 

func (p Platform) Complete() Platform

Complete returns p with the Arch, Kernel and Libc it leaves empty detected on this host. A platform that names all three is returned as is, so no detection runs for it. That is the seam a unit test uses, because the libc comes from ldd on Linux.

func (Platform) String 

func (p Platform) String() string

type SealContext 

type SealContext struct {
	Domain  string    `json:"domain"`
	Session string    `json:"session"`
	Fence   string    `json:"fence,omitempty"`
	Expires time.Time `json:"expires"`
}

SealContext is what a sealed file is bound to. A delta's Expires is the instant it stops opening. A parent's is zero, which means never, and a zero expiry is accepted for parent contexts only.

func OpenDir 

func OpenDir(dir string, k *SealKey, domain, session string, now time.Time) (SealContext, error)

OpenDir replaces every sealed file in dir by its plaintext. It refuses any file not sealed with k for domain and session, or expired at now, and a directory whose files were sealed for different contexts, so one delta cannot be assembled from files of several parks. It returns the one context the files were sealed with. On error, dir holds a mix of sealed and plain files, and the caller must remove it.

func ParentContext 

func ParentContext(sha string) SealContext

ParentContext is the context a parent checkpoint is sealed for. The zero Expires means never.

type SealKey 

type SealKey struct {
	ID  string
	Key []byte // 32 bytes
}

SealKey is the master key deltas are sealed under.

func LoadSealKey 

func LoadSealKey(path string) (*SealKey, error)

LoadSealKey reads a master seal key (a 32-byte symmetric JWK with a kid, as GenerateSealKey makes).

func SealKeyFromJWK 

func SealKeyFromJWK(jwk *jose.JSONWebKey) (*SealKey, error)

SealKeyFromJWK takes the master seal key out of its JWK.

type WrapKey 

type WrapKey struct {
	ID string
	// contains filtered or unexported fields
}

WrapKey is the fleet's X25519 key pair. The private half is derived on a home from the master and never serialised.

func DeriveWrapKey 

func DeriveWrapKey(k *SealKey) (*WrapKey, error)

DeriveWrapKey derives the fleet wrap pair from the master seal key. The same master always gives the same pair and the same key id.

func (*WrapKey) Public 

func (w *WrapKey) Public() *WrapPublicKey

Public returns the half authors get.

func (*WrapKey) PublicJWK 

func (w *WrapKey) PublicJWK() ([]byte, error)

PublicJWK is the public half as an OKP X25519 JWK with a kid, what grant-issuer keygen writes to fleet-pub.json.

type WrapPublicKey 

type WrapPublicKey struct {
	ID string
	// contains filtered or unexported fields
}

WrapPublicKey is the public half authors seal images to.

func LoadWrapPublicKey 

func LoadWrapPublicKey(path string) (*WrapPublicKey, error)

LoadWrapPublicKey reads fleet-pub.json.

func ParseWrapPublicKey 

func ParseWrapPublicKey(b []byte) (*WrapPublicKey, error)

ParseWrapPublicKey reads an OKP X25519 public JWK. The kid must be the one its public key gives, so a mislabelled file is refused.

func (*WrapPublicKey) JWK 

func (p *WrapPublicKey) JWK() ([]byte, error)

JWK renders the public key as an OKP X25519 JWK.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.