kata

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 9, 2026 License: Apache-2.0 Imports: 23 Imported by: 0

Documentation

Overview

Package kata holds the helpers ateom uses to boot and drive a kata guest in a cloud-hypervisor micro-VM without the kata shim: ateom boots cloud-hypervisor itself (see internal/ch), then drives the stock kata-agent over its hybrid-vsock ttrpc API (DialAgent / AgentClient) to create the sandbox and run each container on its host-merged rootfs (overlay_linux.go).

It also renders the kata configuration.toml (for the agent kernel_params + guest sizing) from runtime-fetched assets (config.go) and sweeps leftover per-sandbox host-side state (CleanupSandboxState).

Index

Constants

View Source 
const DefaultCPUPeriodUS = 100000

DefaultCPUPeriodUS is the CFS period assumed when a spec carries a quota but no period. It matches the kernel default and the period atelet encodes against.

View Source 
const (
	// FsTag is the virtio-fs tag kata uses for the shared filesystem. The CH fs
	// device Tag and the agent mount Source must both be this value.
	FsTag = "kataShared"
)

Variables

This section is empty.

Functions

func BindIntoShare 

func BindIntoShare(ctx context.Context, src, id, rel string) error

BindIntoShare bind-mounts a host directory at SharedDir(id)/<name>, so the ONE virtiofsd serves it to the guest as a subtree of the kataShared mount (--announce-submounts presents it to the guest as its own filesystem).

This is THE pattern for exposing another host directory to the guest — the durable-dir and CSI volumes ride it today: a separate share would otherwise pay for its own virtiofsd (a process, a vhost socket, an fs device in every snapshot config and a restore-time revival of all three) per actor, forever. A bind costs one mount, and teardown already covers it: CleanupSandboxState lazily detaches every mount under the sandbox dir first, and will not remove a dir whose mounts it could not all detach, so the source directory (which may belong to atelet, as the volume dirs do) is never deleted through a live bind. Callers must stage binds before StartVirtiofsd, both to keep the served tree complete from the first request and because find-paths migration re-opens a restored guest's open files by path at reconnect.

rel is the mount's path relative to the share root: a top-level name sits beside the per-container <cid>/... entries, so it must not collide with a container id (the durable/csi subtrees); a path inside a container's own subtree (the image volumes' <cid>/volumes/<name>) cannot collide at all.

func CLHSocketPath 

func CLHSocketPath(id string) string

CLHSocketPath returns the default cloud-hypervisor API socket path for the sandbox with the given id (the per-sandbox runtime dir). ateom records the actual api-socket it launched the VMM on, but uses this as the fallback.

func CleanupSandboxState 

func CleanupSandboxState(ctx context.Context, id string)

CleanupSandboxState removes leftover host-side state for a sandbox id (the virtio-fs shared sandbox dir and the per-VM runtime dir), lazily unmounting anything still mounted underneath them first, and kills orphaned per-sandbox processes. ateom owns the cloud-hypervisor boot directly (no kata shim, no containerd), so a failed Create does not fully self-clean; the deterministic sandbox id (= actor name) then collides on the next attempt: "listen unix .../virtiofsd.sock: bind: address already in use", "Could not bind mount .../shared/sandboxes/<id>/mounts", "directory not empty". Calling this before each run gives a clean slate.

Removal is gated on the unmounting: the shared tree holds bind mounts whose SOURCES belong to someone else (the durable-dir and CSI volume dirs are atelet's — see BindIntoShare), and a RemoveAll that walks into a live bind deletes the actor's data through it. So a dir is removed only once every mount beneath it is known detached; otherwise it is left in place for the next sweep (the stagers tolerate a leftover dir — they unmount stale binds and reuse mountpoints).

func ConsoleLogPath 

func ConsoleLogPath(id string) string

ConsoleLogPath is where the guest's virtio-console (hvc0) is captured: the kernel log from virtio-console probe onwards, plus everything the kata-agent prints.

func DebugConsoleDump 

func DebugConsoleDump(ctx context.Context, vsockPath, cmd string) string

DebugConsoleDump connects to the guest's kata debug console (vsock 1026) and runs cmd, returning its combined output. Diagnostic only (requires debug_console_enabled=true in the kata config). Best-effort: returns the error text on failure rather than failing the caller.

func GuestSharedRootfs 

func GuestSharedRootfs(containerID string) string

GuestSharedRootfs is the in-guest path the kataShared mount exposes a container's merged rootfs at. A container with this as Root.Path makes the agent's setup_bundle bind it to /run/kata-containers/<cid>/rootfs and run the container there — the stock kata flow.

func GuestSharedVolumeDir 

func GuestSharedVolumeDir(containerID, volumeName string) string

GuestSharedVolumeDir is the in-guest path one image volume's contents appear at, beside the container's rootfs in the same kataShared tree.

func ReconstructSharedDirFromImage 

func ReconstructSharedDirFromImage(ctx context.Context, bundleRootfs, restoreID, cid string) error

ReconstructSharedDirFromImage bind-mounts a container's OCI image rootfs at <cid>/rootfs under SharedDir(restoreID) so virtiofsd serves it as the read-only lower. LEGACY restores only: guests from retired guest-tmpfs-upper snapshots hold this plain image tree open (their overlay upper lives inside the restored guest memory), so the share must present the bare image, not a merged overlay. The bind copies nothing on the host. cid is stable across the actor's lineage.

func SerialLogPath 

func SerialLogPath(id string) string

SerialLogPath is where the emulated UART is captured. Only wired up in debug mode, where it carries the early-boot messages hvc0 is too late to see.

func SharedDir 

func SharedDir(id string) string

SharedDir is the host directory virtiofsd serves into the guest as the RO base. Its layout (<cid>/rootfs) is what find-paths re-opens by path on restore.

func SharedVolumeDir 

func SharedVolumeDir(id, containerID, volumeName string) string

SharedVolumeDir is the host path under virtiofsd's served tree that GuestSharedVolumeDir resolves to.

func SpecToAgentPB 

func SpecToAgentPB(s *specs.Spec) *agentpb.Spec

SpecToAgentPB converts an OCI runtime spec into the kata-agent's protobuf Spec (agentpb.Spec) for a CreateContainer ttrpc call. A blind json round-trip does NOT work: agentpb's Spec JSON tags are PascalCase (from oci.proto), while OCI config.json is lowercase.

Only the fields the kata-agent needs to create + start a container are mapped (process, root, mounts, linux namespaces/resources/cgroup/masked+readonly paths). The container rootfs is provided out-of-band as storages; the caller sets the returned spec's Root.Path to the overlay mount point.

func StageImageVolume 

func StageImageVolume(ctx context.Context, src, id, cid, volumeName string) error

StageImageVolume bind-mounts one composed image volume read-only at <cid>/volumes/<name> under SharedDir(id), so virtiofsd exposes it to the guest.

func StageMergedRootfs 

func StageMergedRootfs(ctx context.Context, bundleRootfs, upperBase, restoreID, cid string) error

StageMergedRootfs mounts overlay(lower = the OCI image bundle rootfs, upper/work = the actor's host rootfs-upper dirs for cid) at SharedDir(restoreID)/<cid>/rootfs — the merged tree the ONE virtiofsd serves and the guest runs the container on directly. The host kernel owns the overlay (the canonical ext4-upper case: no special mount options, whiteouts/opaque markers are ordinary trusted.overlay.* metadata in the upper), and the lower stays pristine (overlayfs never writes below).

The merged path is identical on every node — find-paths migration re-opens the guest's open files by path — given a deterministic image unpack plus the upper re-materialized from the snapshot tar (see cmd/ateom-microvm/rootfsupper.go).

func StartVirtiofsd 

func StartVirtiofsd(ctx context.Context, o VirtiofsdOptions) (*exec.Cmd, error)

StartVirtiofsd launches virtiofsd in find-paths migration mode serving o.SharedDir on o.SocketPath, and waits for the socket to appear. The returned cmd outlives the caller's ctx (CH demand-pages from it under the running VM); the caller owns it.

func UnmountMergedRootfs 

func UnmountMergedRootfs(restoreID, cid string)

UnmountMergedRootfs drops one container's merged overlay mount (teardown and failure paths; lazy fallback if busy). Best-effort like the rest of teardown — CleanupSandboxState's sweep catches stragglers on the next boot.

func UpperWorkDirs 

func UpperWorkDirs(upperBase, containerID string) (upper, work string)

UpperWorkDirs returns the HOST overlay upperdir and workdir for one container under the actor's rootfs-upper base dir: SIBLING directories, <cid>/fs and <cid>/work. Both properties are load-bearing — the kernel requires upperdir and workdir on the same filesystem and rejects a nested workdir — and the layout is also the snapshot tar's entry layout, so a change here breaks both every overlay mount and every existing snapshot. Covered by regression tests.

func VMDir 

func VMDir(id string) string

VMDir is the per-sandbox runtime dir (holds the cloud-hypervisor API socket and the hybrid-vsock socket).

func VirtiofsdSocketPath 

func VirtiofsdSocketPath(id string) string

VirtiofsdSocketPath is the vhost-user-fs socket CH connects to for the fs device.

func VsockSocketPath 

func VsockSocketPath(id string) string

VsockSocketPath is the hybrid-vsock socket the CH snapshot's vsock device references; CH recreates the listener here on restore.

func WithAgentDebug 

func WithAgentDebug(kernelParams string) string

WithAgentDebug appends agent.log=debug so the guest kata-agent emits debug-level logs (including the failing path on errors) over its vsock log channel. Idempotent.

func WithDebugConsole 

func WithDebugConsole(kernelParams string) string

WithDebugConsole appends the kata-agent debug-console kernel parameters so the guest agent binds a root debug shell on vsock port 1026, which DebugConsoleDump connects to for in-guest diagnostics. Both params are required: agent.debug_console enables the console and agent.debug_console_vport=1026 makes the agent bind it on the vsock port (the agent only binds a vsock listener when the vport is > 0). Idempotent.

Types

type AgentClient 

type AgentClient struct {
	// contains filtered or unexported fields
}

AgentClient is a thin ttrpc client for the kata-agent RPCs ateom drives directly. ateom owns the cloud-hypervisor boot (no kata shim) and drives the kata-agent over ttrpc itself: alongside UpdateInterface/UpdateRoutes for guest networking, it issues CreateContainer/StartContainer to assemble the container rootfs directly, instead of relying on the kata runtime's hooks (ShareRootFilesystem) to emit the storages. It dials the agent through CH's hybrid-vsock unix socket — the same channel the kata shim would use.

func DialAgent 

func DialAgent(ctx context.Context, vsockPath string) (*AgentClient, error)

DialAgent connects to the kata-agent through the hybrid-vsock socket at vsockPath (VsockSocketPath(id)): plain-text "CONNECT <port>" handshake with the VMM, then ttrpc over the stream.

func (*AgentClient) AddARPNeighbors 

func (a *AgentClient) AddARPNeighbors(ctx context.Context, neighbors []*agentpb.ARPNeighbor) error

AddARPNeighbors installs static ARP entries in the guest — used to pin the gateway (169.254.17.1) to its FIXED MAC so a restored guest's frozen neighbor entry stays valid across pods. Mirrors grpc.AgentService/AddARPNeighbors.

func (*AgentClient) Close 

func (a *AgentClient) Close() error

Close shuts the ttrpc client and underlying connection.

func (*AgentClient) CreateContainer 

func (a *AgentClient) CreateContainer(ctx context.Context, req *agentpb.CreateContainerRequest) error

CreateContainer asks the agent to create a container: mount its storages (in order) and build the rootfs, then fork the parked init process. This is the hook point — the agent mounts storages[] (here: a bind of the virtio-fs lower followed by the disk-backed-upper overlay) before init_rootfs consumes the rootfs. Mirrors grpc.AgentService/CreateContainer (returns google.protobuf.Empty).

func (*AgentClient) CreateSandbox 

func (a *AgentClient) CreateSandbox(ctx context.Context, req *agentpb.CreateSandboxRequest) error

CreateSandbox establishes the agent's sandbox context (sandbox id, hostname, sandbox pidns) before any container is created. The kata shim normally issues this once at VM boot; ateom (no shim) must call it itself so the agent has a sandbox to attach containers to. Storages carries the shared virtio-fs mount (the overlay lowers); each container's rootfs is assembled per-container. Mirrors grpc.AgentService/CreateSandbox (returns google.protobuf.Empty).

func (*AgentClient) CreateSandboxForActor 

func (a *AgentClient) CreateSandboxForActor(ctx context.Context, opts CreateSandboxOpts) error

CreateSandboxForActor creates the guest sandbox with the kataShared virtio-fs mount (the merged rootfs trees, durable volumes, CSI volumes, and system-info volumes every container runs on). Mirrors kata startSandbox.

func (*AgentClient) ReadStderr 

func (a *AgentClient) ReadStderr(ctx context.Context, containerID, execID string, max uint32) ([]byte, error)

ReadStderr reads up to max bytes from the container process's stderr. Same semantics as ReadStdout (unary, loop-until-error). Mirrors grpc.AgentService/ReadStderr.

func (*AgentClient) ReadStdout 

func (a *AgentClient) ReadStdout(ctx context.Context, containerID, execID string, max uint32) ([]byte, error)

ReadStdout reads up to max bytes from the container process's stdout. It is a unary RPC (NOT a server stream): each call returns whatever bytes the agent has buffered (up to max), so callers loop until it returns an error — the agent returns an error/EOF-like status once the stream ends (container exit / connection close). Mirrors grpc.AgentService/ReadStdout. The kata-agent keys the stream by ExecId, which ateom sets equal to ContainerId.

func (*AgentClient) SignalProcess 

func (a *AgentClient) SignalProcess(ctx context.Context, containerID, execID string, signal uint32) error

SignalProcess sends signal to a process in the guest. Targeting the container's init process (ExecId == ContainerId) delivers the signal to the workload; an empty execID delivers it to ALL processes in the container. Used during graceful shutdown to propagate SIGTERM into the actor. Mirrors grpc.AgentService/SignalProcess (returns google.protobuf.Empty).

func (*AgentClient) StartContainer 

func (a *AgentClient) StartContainer(ctx context.Context, containerID string) error

StartContainer execs the container's init process (pivots into the rootfs the storages assembled). Mirrors grpc.AgentService/StartContainer.

func (*AgentClient) StartRootfsContainer 

func (a *AgentClient) StartRootfsContainer(ctx context.Context, cid string, spec *specs.Spec) error

StartRootfsContainer creates + starts one container on the shared merged rootfs — the stock kata flow: the agent's setup_bundle binds shared/<cid>/rootfs to /run/kata-containers/<cid>/rootfs and the container runs there. Writable: the host-side overlay upper receives the writes (the guest mounts no overlay itself).

func (*AgentClient) StatsContainer 

func (a *AgentClient) StatsContainer(ctx context.Context, containerID string) (*agentpb.CgroupStats, error)

StatsContainer returns the guest cgroup accounting for one container, as the kata-agent reads it from inside the guest. Mirrors grpc.AgentService/StatsContainer.

It returns only the cgroup half of the response; the network counters alongside it are per-guest-interface rather than per-container and are not what ateom reports. A nil return with a nil error means the agent answered without cgroup stats for this container — it has no accounting for it, which is a normal state for one that has exited. What that means for the actor is the caller's call: the summing in GetWorkloadStats folds it in as a zero contribution, because a gone container consumes nothing from here on.

Safe to call while the stdout/stderr forwarding goroutines are reading over the same client: ttrpc multiplexes concurrent calls over the one connection, which is already what those goroutines rely on.

func (*AgentClient) UpdateInterface 

func (a *AgentClient) UpdateInterface(ctx context.Context, iface *agentpb.Interface) error

UpdateInterface configures a guest network interface (the kata shim's job, which ateom does itself). The agent matches the link by HwAddr, then applies the name/IP/MTU. Mirrors grpc.AgentService/UpdateInterface (returns the resulting Interface).

func (*AgentClient) UpdateRoutes 

func (a *AgentClient) UpdateRoutes(ctx context.Context, routes []*agentpb.Route) error

UpdateRoutes replaces the guest's route table with routes (the agent flushes and re-adds). Pass the connected (scope-link) route AND the default route so the gateway stays reachable. Mirrors grpc.AgentService/UpdateRoutes.

func (*AgentClient) WaitProcess 

func (a *AgentClient) WaitProcess(ctx context.Context, containerID, execID string) (int32, error)

WaitProcess blocks until the identified guest process exits and returns its exit status (mimics waitpid(2)). Used during graceful shutdown to confirm the actor has stopped before ateom tears the VM down. Mirrors grpc.AgentService/WaitProcess.

type CreateSandboxOpts 

type CreateSandboxOpts struct {
	SandboxID string
	Hostname  string
}

type KataConfig 

type KataConfig struct {
	// MemoryMiB is the guest RAM size ([hypervisor.clh] default_memory).
	MemoryMiB int
	// VCPUs is the guest vCPU count ([hypervisor.clh] default_vcpus).
	VCPUs int
	// KernelParams is the guest kernel command line ([hypervisor.clh]
	// kernel_params): the kata-agent parameters (agent.log, the systemd target,
	// etc.). ateom appends these to the cloud-hypervisor payload cmdline, since
	// there is no kata shim to inject them.
	KernelParams string
}

KataConfig holds the values ateom reads from a kata configuration.toml. ateom owns the cloud-hypervisor boot and points it at the runtime-fetched asset paths directly, so the only things it needs from the config are the guest sizing and the agent kernel command line.

func ParseConfig 

func ParseConfig(base []byte, memDefault, vcpuDefault int) (KataConfig, error)

ParseConfig reads the guest sizing and kernel_params from a kata configuration.toml. memDefault/vcpuDefault are substituted when the key is absent or non-positive (kata also accepts default_vcpus = -1 meaning "all host CPUs", which ateom does not support).

type StreamReader 

type StreamReader struct {
	// contains filtered or unexported fields
}

StreamReader adapts the agent's repeated ReadStdout/ReadStderr unary calls into an io.Reader, so the consumer can pump the container's output through the shared actorlog forwarder like any other stream. Each Read issues one RPC with Len set to len(p); on RPC error (the agent signals EOF/container-exit via an error status) it returns io.EOF so the consuming goroutine terminates cleanly. The reader stops when its context is cancelled OR the underlying ttrpc connection is closed (both surface as RPC errors), so it never outlives the AgentClient.

func NewStdioReader 

func NewStdioReader(ctx context.Context, ac *AgentClient, containerID, execID string, stderr bool) *StreamReader

NewStdioReader returns an io.Reader over the container's stdout (stderr=false) or stderr (stderr=true). execID equals containerID (ateom sets ExecId == ContainerId when it creates the container).

func (*StreamReader) Read 

func (r *StreamReader) Read(p []byte) (int, error)

Read issues a single ReadStdout/ReadStderr RPC for up to len(p) bytes, copying the returned data into p. It returns io.EOF on any RPC error so the consumer stops cleanly when the container exits or the connection closes.

type VirtiofsdOptions 

type VirtiofsdOptions struct {
	Binary     string // virtiofsd executable; defaults to "virtiofsd"
	SocketPath string // vhost-user socket CH connects to (VirtiofsdSocketPath)
	SharedDir  string // directory to serve (SharedDir(id))
	Log        io.Writer
}

VirtiofsdOptions configures StartVirtiofsd.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.