fiberd

fiberd

Platforms often need many instances of the same workload, started on demand. The workload may be a container, an agent or a plain process. Each new instance usually boots, loads its code and fills its own memory, and often waits on a control plane first, even though it is a copy of one that already runs. fiberd creates those instances from one warm copy, fast and densely, inside whatever already places the workload, such as Kubernetes or Slurm. It is a reference implementation, not yet production-ready, and production-readiness.md lists what is missing.

What fiberd does

• A template starts once and stays warm.
• Each fiber is a cheap fork or snapshot restore of that template, so it starts in milliseconds.
• An issuer grants a block of capacity once, as a signed grant.
• The home checks the grant locally and mints fibers from it without calling back.

Why fiberd

fiberd works with the scheduler, not instead of it. Fast starts from a warm copy are not new either. Android's zygote forks warm processes, and Firecracker and Lambda SnapStart restore snapshots. fiberd's part is making that safe to run at scale.

• Placement once, instances locally. The scheduler places a signed block of capacity once. The home creates instances inside it with no API write or scheduler call per instance, so the scheduler sees one placement instead of one object per instance.
• Every instance stays checkable. Each fiber's fence ties it to its grant and to fiberd's epoch, so revocation and restarts reach every fiber. Each operation lands in a hash-chained audit log with signed checkpoints. A caller that misses gets a typed answer that says whether to back off or fall back.
• One contract, any scheduler, any isolation. The same grant works on Kubernetes, Slurm or a bare host, and over plain processes, runc, gVisor, Hyperlight or Cloud Hypervisor.
• Sessions move. A parked session can resume on another home, verified and sealed to its tenant. A home given a drain budget parks its sessions when it stops, rather than losing them.

The fastest path, proc and runc, shares the host kernel, so it suits trusted code. Untrusted code runs on gVisor, Hyperlight or Cloud Hypervisor, where a fiber is a snapshot restore.

fiberd verbs

• Clone returns a fiber for a grant, anonymous or by session name.
• Park saves a session and frees its fiber's memory.
• Release ends a fiber.
• Discard deletes a parked session.
• Watch streams a summary of each grant.

protocol.md is the wire contract for all verbs.

Reading path

Read in this order, and look up any term in the glossary.

1. Quickstart to see it run.
2. Architecture for how it is designed.
3. Security for the trust boundaries.
4. The operator guides, which are Operating on Kubernetes, Networking, Resources and Production readiness.
5. The design docs for each module in depth.
6. The examples for fiberd in real environments.
7. Protocol for the wire contract, and Benchmarks for measured results.

Install

Each release has an archive per platform, named fiberd_<version>_<os>_<arch>.tar.gz, for linux or darwin on amd64 or arm64. The libfiberzygote archives hold the zygote library. Check the checksum and provenance before you unpack.

gh release download v0.1.0 -R Azure/fiberd -p fiberd_0.1.0_linux_amd64.tar.gz -p SHA256SUMS
shasum -a 256 --ignore-missing -c SHA256SUMS
gh attestation verify fiberd_0.1.0_linux_amd64.tar.gz -R Azure/fiberd
tar -xzf fiberd_0.1.0_linux_amd64.tar.gz
./fiberd -version

The release publishes three images, each with the same version tags.

• ghcr.io/azure/fiberd holds fiberd, criu and the zygote library.
• ghcr.io/azure/fiberd-runc adds the runc fiberd was built and tested against.
• ghcr.io/azure/fiberd-gvisor adds the runsc fiberd was built and tested against.

The two variants also hold the empty mount points of a sandbox rootfs at /usr/share/fiberd/rootfs. A deployment on the runc or gVisor runtime builds on one of them and adds its workload.

gh attestation verify oci://ghcr.io/azure/fiberd:0.1.0 -R Azure/fiberd

Releases after v0.1.0 also publish the Python SDK to PyPI. Its sdist and wheels are release assets too, under the same SHA256SUMS and provenance.

pip install 'fiberd[client]'

Start

make build
make conform-stub   # the conformance suite on the in-memory runtime, on any OS
make conform-proc   # the same on real fibers, in the Linux dev container (hack/dev)

The quickstart walks through each call. To write your own template, read sdk/c/template/README.md for C or sdk/python/README.md for Python, and the zygote design.

Callers can use the Go client, Python client, or TypeScript client for Node.js.

Examples

• Kubernetes is a core home. A grant Pod runs fiberd's own image with fiberd -home kubernetes. The controller that turns each CapacityGrant custom resource into a signed grant and one grant Pod is a separate project, fiberd-k8s-controller.
• Slurm runs fiberd inside a job allocation.
• Knative is an activator, the component that holds requests while a service scales from zero, here with fibers in Hyperlight micro-VMs.
• Kata-shaped shim is a containerd shim, the plug-in that starts a Pod's containers. It has the shape of the Kata Containers shim without using Kata, and its containers are fibers.
• Agent Substrate is a worker for a platform that starts, suspends and resumes actors in worker Pods. Its actors are fibers.

Contributing

CI runs the unit tests (make test) on linux/amd64 and macOS as a non-root user. The integration suites under tests/ and each example's tests/ build only with -tags integration and run as root in the dev container (make integration-tests). Run make test and make lint before sending a change.

make integration-tests and make linux-test rebuild the TypeScript client first when its dependencies are installed (npm ci --ignore-scripts --prefix sdk/typescript, Node.js 22.12 or later). Without them they leave out the TypeScript suite and say so.

The dev image

hack/dev/run.sh, and every make target that uses the dev container, pulls the image from ghcr.io/azure/fiberd-dev. The tag is a hash of docker/dev/Dockerfile and the files it copies, printed by hack/dev/image-tag.sh. The dev-image workflow publishes that tag when those files change on main.

When the tag is missing or the registry cannot be reached, hack/dev/run.sh builds the image locally. A change to those files therefore tests its own image. Set FIBERD_DEV_BUILD=1 to force a local build.

When the workflow fails, jobs keep working by building locally, which needs snapshot.debian.org. Fix the cause or wait out the outage, then re-run the failed jobs or start the workflow by hand. The workflow skips a tag that is already published, so refreshing what the image holds takes a change to one of those files.

License

Apache-2.0.