runtime

package
v0.6.10 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 2, 2026 License: MIT Imports: 7 Imported by: 0

Documentation

Overview

Package runtime defines the abstraction the service layer uses to drive a sandbox container, regardless of which OCI runtime executes it. The first implementation is pkg/docker.Client, which talks to the local Docker daemon and can drive either runc or runsc (gVisor) depending on per-sandbox / host-default selection. Future implementations (e.g. native runsc without Docker, Firecracker, WASM) plug in behind the same interface.

Methods that are intrinsically Docker-API-shaped — exec hijacking, the /events stream — intentionally stay off this interface and on the concrete *docker.Client. Pulling them through the abstraction would only buy clutter; gVisor under Docker uses the same exec/event paths anyway.

Index

Constants

This section is empty.

Variables

View Source
var ErrEgressFloorUnavailable = errors.New("runtime: the host firewall is off; sandboxes can reach the cloud metadata endpoints")

ErrEgressFloorUnavailable means the runtime has no host firewall (network rules are off), so nothing on the host keeps its sandboxes off the cloud metadata endpoints.

View Source
var ErrInstanceNotFound = errors.New("runtime: instance not found")

ErrInstanceNotFound is a runtime's definitive "no such instance": Docker answered 404, containerd answered NotFound. Drivers wrap it so callers can tell it from a runtime that could not answer at all.

View Source
var ErrNodeGuardUnavailable = errors.New("runtime: the host firewall is off; sandboxes can reach every service on this node")

ErrNodeGuardUnavailable means the runtime has no host firewall (network rules are off), so nothing keeps its sandboxes off the node's services.

Functions

func InstanceGone added in v0.6.9

func InstanceGone(state *models.SandboxRuntimeState, err error) bool

InstanceGone reports whether an Inspect result proves the instance does not exist. Only that may justify deleting a sandbox's row: an unknown answer (a timeout, an unreadable netns, a shim that did not reply) once did, and reconcile deleted live sandboxes on it.

func NotifyAddressKnown added in v0.6.9

func NotifyAddressKnown(ctx context.Context, ip string)

NotifyAddressKnown tells ctx's observer, if any, the sandbox's address.

func NotifyReleasingAddress added in v0.6.9

func NotifyReleasingAddress(ctx context.Context, ip string)

NotifyReleasingAddress tells ctx's observer, if any, that ip is about to be given back.

func WithAddressObserver added in v0.6.9

func WithAddressObserver(ctx context.Context, o AddressObserver) context.Context

WithAddressObserver carries o on ctx for a runtime Create or Destroy.

Types

type AddressObserver added in v0.6.9

type AddressObserver interface {
	// AddressKnown reports the sandbox's address, at most once per create.
	// The driver calls it only after the request's network rules (the
	// block-all DROP of a hostname-filtered create) are installed on ip and
	// before the workload can run, and ip must be the address the create
	// returns: the observer may act on ip at once.
	AddressKnown(ip string)
	// ReleasingAddress is called once the workload can no longer send from
	// ip and before the driver gives ip back (a failed create's cleanup, a
	// destroy), so the address never reaches another sandbox while state
	// keyed by it for this one is left. It returns when that state is gone
	// or its removal is queued for retry.
	ReleasingAddress(ip string)
}

AddressObserver is how a create learns its sandbox's address before the runtime finishes building the sandbox, and how anything keyed by that address is told before the runtime gives the address back.

The egress gateway is the reason it exists. A hostname-filtered create attaches the sandbox to the gateway by address; a driver that knows the address early (a prepaid netns slot, a pause-netns slot, a Firecracker TAP slot) reports it so the attach overlaps the rest of the build instead of waiting for it (plans/egress-domain-filtering.md §8.2). A driver that learns the address only from the finished create (Docker's own IPAM) reports nothing, and the create attaches afterwards as before.

It rides the create's (or destroy's) context, so drivers and callers that don't use it are untouched; a context without one makes both calls no-ops.

func AddressObserverFrom added in v0.6.9

func AddressObserverFrom(ctx context.Context) AddressObserver

AddressObserverFrom returns the observer ctx carries, or nil.

type ContainerRuntime added in v0.5.3

type ContainerRuntime interface {
	Runtime

	// PushAllowedPorts updates the in-container toolbox's allowlist of ports
	// reachable through its proxy. Best-effort; callers log on failure.
	PushAllowedPorts(ctx context.Context, containerIP, toolboxToken string, ports []int) error

	// ClearNetworkRules releases any per-IP network rules previously attached
	// to a sandbox. Used by the event-driven path when a container exits or
	// is destroyed out-of-band; Destroy() handles this during normal teardown.
	ClearNetworkRules(containerIP string) error

	// ApplyNetworkBlockAll installs the per-IP egress DROP rule for sandboxes
	// created with NetworkBlockAll=true. Idempotent — safe to call on every
	// Start and reconcile pass. Used to reapply the rule after Stop+Start
	// cycles (which clear it on the stop event) and after host-side state
	// loss (iptables flush, daemon restart that rebuilds chains).
	ApplyNetworkBlockAll(containerIP string) error

	// ApplyNetworkBlockIngress installs the per-IP ingress DROP rule. Used
	// by the network-quota enforcer when net_bytes_in_limit is crossed.
	// Idempotent at the netrules layer. Distinct from the egress block so
	// quota enforcement composes with NetworkBlockAll independently on each
	// direction.
	ApplyNetworkBlockIngress(containerIP string) error

	// ClearNetworkBlockIngress removes the per-IP ingress DROP rule. The
	// caller is responsible for not racing this against quota state — the
	// service layer only clears when limits are raised above current usage.
	ClearNetworkBlockIngress(containerIP string) error

	// ClearNetworkBlockEgress removes the per-IP egress DROP rule. Symmetric
	// to ApplyNetworkBlockAll for callers that need to walk back a quota-
	// driven block without also undoing NetworkBlockAll. Today the underlying
	// rule is shared, so the service must consult NetworkBlockAll before
	// invoking this on a quota-clear path.
	ClearNetworkBlockEgress(containerIP string) error

	// ApplyEgressPolicy installs the selective-egress CIDR policy (E2B
	// network.allowOut / denyOut). At most one of allowCIDRs / denyCIDRs is
	// non-empty. Comment-tagged so it composes with, rather than collides
	// with, the NetworkBlockAll / quota DROP. Idempotent — reapplied on Start
	// and reconcile just like ApplyNetworkBlockAll.
	ApplyEgressPolicy(containerIP string, allowCIDRs, denyCIDRs []string) error

	// ClearEgressPolicy removes the rules ApplyEgressPolicy installed for the
	// same (containerIP, allowCIDRs, denyCIDRs). Callers pass the policy from
	// the sandbox row so cleanup is exact and the blanket DROP is untouched.
	ClearEgressPolicy(containerIP string, allowCIDRs, denyCIDRs []string) error
}

func AsContainerRuntime added in v0.5.3

func AsContainerRuntime(rt Runtime) (ContainerRuntime, bool)

AsContainerRuntime returns the network-rule surface when rt implements it. WASM and other host-mediated runtimes return false.

type EgressFloorSetter added in v0.6.6

type EgressFloorSetter interface {
	SetEgressFloor(ctx context.Context, cidrs []netip.Prefix) error
	// ReassertEgressFloor puts back a floor something removed; repaired
	// reports that it had to.
	ReassertEgressFloor(ctx context.Context) (repaired bool, err error)
}

EgressFloorSetter is implemented by container runtimes that can install the operator's node-wide deny floor for every sandbox on their bridge (plans/egress-domain-filtering.md §5.10 PC-2). It works without the egress gateway, which carries its own copy of the floor.

type EgressHolder added in v0.6.6

type EgressHolder interface {
	ApplyEgressHold(containerIP string) error
	ClearEgressHold(containerIP string) error
}

EgressHolder is implemented by container runtimes that can install the fail-closed egress hold: a comment-tagged DROP of its own that quota and limits code never touch (plans/egress-domain-filtering.md CEO D16). Only a successful gateway attach clears it.

type IPOwnerResolver added in v0.6.6

type IPOwnerResolver interface {
	IPOwner(ctx context.Context, ip string) (sandboxID string, err error)
}

ContainerRuntime extends Runtime with per-IP network rules and the in-container toolbox port allowlist. Docker and Firecracker satisfy both; WASM satisfies only Runtime and uses host-mediated sockets instead. IPOwnerResolver is implemented by container runtimes that can report which sandbox currently holds a bridge IP. Event-driven rule clears consult it so a late stop or destroy event for an old owner never strips the rules of a sandbox that has since been given the same IP — which would leave the new sandbox unrestricted (plans/egress-domain-filtering.md P0-6). It catches the window before the new owner's store row records the IP. An empty id means no sandbox holds the IP (free or parked pool slots included).

type IdentityStarter added in v0.6.6

type IdentityStarter interface {
	StartWithIdentity(ctx context.Context, containerRef, sandboxID, toolboxToken string) (*models.SandboxRuntimeState, error)
}

IdentityStarter is a Runtime whose start must hand the container its sandbox identity again. A Docker or containerd container adopted from the warm pool keeps the pool's parked env for life (Docker can't change a container's env; containerd adoption relabels the container and leaves its spec alone), so after every stop its toolboxd boots parked and serves nothing until it is adopted again with the sandbox's ID and toolbox token. A Firecracker guest restored from its stopped snapshot is adopted again over vsock: its toolboxd serves nothing without the token.

func AsIdentityStarter added in v0.6.6

func AsIdentityStarter(rt Runtime) (IdentityStarter, bool)

AsIdentityStarter returns the identity-aware start when rt implements it. Callers fall back to Start when this returns false.

type NetRulesReporter added in v0.6.9

type NetRulesReporter interface {
	NetRulesEnabled() bool
}

NetRulesReporter is implemented by drivers that enforce a sandbox's network policy (block-all, CIDR allow/deny lists, byte-cap blocks, the egress gateway's fail-closed hold) with host firewall rules: docker, containerd and Firecracker. NetRulesEnabled is false when that firewall is off (SB_ENABLE_NETWORK_RULES=false, or a non-Linux host such as macOS local mode). Every rule call is then a silent no-op, so the service refuses those policies instead of storing one nothing enforces.

type NetworkBlockReporter added in v0.5.6

type NetworkBlockReporter interface {
	ApplyNetworkBlockAllReport(containerIP string) (inserted bool, err error)
}

NetworkBlockReporter is the reporting variant of ContainerRuntime.ApplyNetworkBlockAll: same install, but it also says whether the rule was actually missing. Kept as a separate optional interface rather than widening ApplyNetworkBlockAll's signature so drivers that can't answer the question (Firecracker, which doesn't implement per-IP rules at all) stay untouched.

Only reconcile needs the answer — it reapplies on every pass, so the raw call count is a constant, not a drift signal.

func AsNetworkBlockReporter added in v0.5.6

func AsNetworkBlockReporter(cr ContainerRuntime) (NetworkBlockReporter, bool)

AsNetworkBlockReporter returns the reporting surface when cr implements it. Callers must fall back to plain ApplyNetworkBlockAll when this returns false — the healing behavior is mandatory, the metric is not.

type NodeGuardSetter added in v0.6.9

type NodeGuardSetter interface {
	SetNodeGuard(ctx context.Context, allow []netrules.NodePort) error
	// ReassertNodeGuard puts back a guard something removed; repaired
	// reports that it had to.
	ReassertNodeGuard(ctx context.Context) (repaired bool, err error)
}

NodeGuardSetter is implemented by runtimes whose sandboxes reach the node over a host bridge or TAP subnet (containers, gVisor, Firecracker): it default-denies every sandbox on that subnet from the node itself, except allow (pkg/docker/netrules ChainAerolvmNode). WASM and isolate sandboxes dial from host processes and are held off the node by the dial guard instead (egresspolicy.CheckNode).

type Runtime

type Runtime interface {
	// Create provisions and starts a managed container. The caller owns the
	// sandbox ID. The runtime sets it as the container's canonical name so
	// the sandbox ID is the end-to-end identifier (the container ID is an
	// internal detail). The chosen OCI runtime is read from
	// req.Runtime; the implementation is expected to substitute its own
	// default when req.Runtime is empty.
	Create(ctx context.Context, req models.CreateSandboxRequest, sandboxID, toolboxToken string, hostMounts []mounts.ContainerBind) (*models.SandboxRuntimeState, error)

	// Start boots a previously created container and waits for the in-container
	// toolbox to become reachable.
	Start(ctx context.Context, containerRef string) (*models.SandboxRuntimeState, error)

	// Stop sends a graceful stop with a default timeout.
	Stop(ctx context.Context, containerRef string) error

	// Destroy removes the container and clears any per-IP network rules.
	// Passing nil is a no-op — convenient for cleanup paths that may not yet
	// have a sandbox record.
	Destroy(ctx context.Context, sandbox *models.Sandbox) error

	// CreateSnapshot commits a managed container into a reusable local image
	// reference and returns the resulting image ID.
	CreateSnapshot(ctx context.Context, containerRef, imageRef string) (string, error)

	// Resize updates CPU/memory caps on a running container. DiskGB is not
	// resizable on a live container; resize-up at create time is honored
	// per-runtime.
	Resize(ctx context.Context, containerRef string, req models.ResizeSandboxRequest) error

	// Inspect returns the current runtime state for a single container.
	//
	// A result says one of three things, and callers that delete anything
	// must tell them apart (see InstanceGone):
	//   - a state: the instance exists. An instance whose address could not
	//     be read this time is still a state (Started, empty ContainerIP);
	//   - gone: an error wrapping ErrInstanceNotFound, or (nil, nil) from a
	//     driver whose registry is in-process memory (firecracker, wasm,
	//     isolate), where a registry miss is the definitive answer;
	//   - unknown: any other error. The runtime could not answer; the
	//     instance may well be running.
	Inspect(ctx context.Context, containerRef string) (*models.SandboxRuntimeState, error)

	// ListManaged returns the runtime state of every container managed by
	// this sandboxd, keyed by sandbox ID. Used by reconcile.
	ListManaged(ctx context.Context) (map[string]*models.SandboxRuntimeState, error)

	// Ping verifies the underlying runtime daemon is reachable.
	Ping(ctx context.Context) error

	// RemoveImage GCs an image from the runtime's local store. 404/409 are
	// treated as success — see implementation notes.
	RemoveImage(ctx context.Context, imageRef string) error
}

Runtime is the core contract the service layer requires of any sandbox runtime driver. Implementations must be safe for concurrent use; the service serializes per-sandbox operations but issues calls for different sandboxes in parallel.

Network-rule methods (per-IP iptables, in-container toolbox allowlists) live on ContainerRuntime instead — WASM satisfies only Runtime (see plans/wasm-runtime.md D17).

type ToolboxAddresser added in v0.6.2

type ToolboxAddresser interface {
	ToolboxAddress(ctx context.Context, sandbox *models.Sandbox) (string, error)
}

ToolboxAddresser reports the host:port at which sandboxd reaches a sandbox's toolboxd when that is not ContainerIP:ToolboxPort — Docker with SB_DOCKER_TOOLBOX_LOOPBACK publishes it on a 127.0.0.1 port that changes on every start. Optional so drivers whose toolbox sits at the guest IP (Firecracker, containerd) stay untouched.

func AsToolboxAddresser added in v0.6.2

func AsToolboxAddresser(rt Runtime) (ToolboxAddresser, bool)

AsToolboxAddresser returns the addressing surface when rt implements it. Callers fall back to ContainerIP:ToolboxPort when this returns false.

type WritableLayerSizer added in v0.6.9

type WritableLayerSizer interface {
	WritableLayerBytes(ctx context.Context, containerRef string) (int64, error)
}

WritableLayerSizer is a container engine that can say how much a sandbox has written to its writable layer. Neither containerd's overlay snapshotter nor Docker's overlay2 on a stock ext4 root enforces a size, so the disk watchdog (Service.runDiskLimitWatch) stops a sandbox that has written past its DiskGB.

Directories

Path Synopsis
Package firecracker is the native Firecracker runtime driver.
Package firecracker is the native Firecracker runtime driver.
Package isolate is the V8-isolate (Workers-model) runtime driver — the fifth implementation of internal/runtime.Runtime, after pkg/docker (docker + gvisor), internal/runtime/firecracker, and internal/runtime/wasm (plans/isolate-runtime.md).
Package isolate is the V8-isolate (Workers-model) runtime driver — the fifth implementation of internal/runtime.Runtime, after pkg/docker (docker + gvisor), internal/runtime/firecracker, and internal/runtime/wasm (plans/isolate-runtime.md).
Package resolvconf builds the resolv.conf a sandbox gets: the host's upstream resolvers, without loopback stubs the sandbox can't reach.
Package resolvconf builds the resolv.conf a sandbox gets: the host's upstream resolvers, without loopback stubs the sandbox can't reach.
Package wasm is the WASM/WASI runtime driver — the third implementation of internal/runtime.Runtime (after pkg/docker and internal/runtime/firecracker).
Package wasm is the WASM/WASI runtime driver — the third implementation of internal/runtime.Runtime (after pkg/docker and internal/runtime/firecracker).
statekv
Package statekv implements the host KV capability for durable WASM sandboxes (plans/wasm-runtime.md §4.6).
Package statekv implements the host KV capability for durable WASM sandboxes (plans/wasm-runtime.md §4.6).
toolhost
Package toolhost implements the host-side toolbox for WASM sandboxes (plans/wasm-runtime.md §3).
Package toolhost implements the host-side toolbox for WASM sandboxes (plans/wasm-runtime.md §3).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL