host

package
v0.21.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 30, 2026 License: AGPL-3.0 Imports: 42 Imported by: 0

Documentation

Overview

Package host drives stock workerd inside a gVisor/runsc sandbox via clrk's extracted pkg/sandbox.Runtime. It is the artifact a lifecycle controller (APO-796 ServiceManager) runs: it reconstructs a workerd config from a compute-API bundle and serves fetch over an HTTP socket (M1 backend mode).

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func BuildResidentConfig

func BuildResidentConfig(in ResidentConfigInput) (string, error)

BuildResidentConfig renders the static textual capnp config for the one resident workerd: a single dispatcher worker (WorkerLoader + manager service binding) behind the http socket. Pure and deterministic.

func BuildWorkerdConfig

func BuildWorkerdConfig(in BuildInput) (string, error)

BuildWorkerdConfig renders the textual capnp config that `workerd serve` consumes. It is a pure, deterministic function: identical input yields byte-identical output. Module `embed` paths are emitted relative to the config file, which the host co-locates with the extracted modules layer so each path is just Module.Path.

func BundleImageRef

func BundleImageRef(b computev1alpha1.BundleRef) (string, error)

BundleImageRef builds the digest-pinned OCI reference for a bundle, exported for the ServiceManager control plane (pkg/workerd/manager), which pulls a revision's modules to inline into the WorkerLoader payload.

func CleanModulePath

func CleanModulePath(p string) string

CleanModulePath normalizes a tar entry name / Module.Path to a stable lookup key: a leading "./" is stripped and the path is lexically cleaned, so the builder's "./index.js" and a manifest's "index.js" resolve identically. The ServiceManager resolver uses it to map BundleManifest.Modules[i].Path to the keys FetchBundleModules returns.

func FetchBundle

FetchBundle pulls a bundle's BundleManifest and module bytes in one pass: one repository, one auth exchange, one OCI manifest resolve serving both. It is the fetch entry point for the ServiceManager control plane (pkg/workerd/manager), which inlines a revision's module bytes into the WorkerLoader payload rather than mounting the bundle rootfs.

func FetchBundleManifest

FetchBundleManifest fetches the OCI config blob — the JSON-encoded BundleManifest (media type application/vnd.apoxy.dev.service.config.v1+json) — for a bundle. The sandbox ImageStore extracts the rootfs but does not surface this config blob, so the host fetches it directly. (R1.)

Pull credentials are derived from the BundleRef itself; callers that also need the module bytes should use FetchBundle, which shares one registry session for both.

func FetchBundleModules

func FetchBundleModules(ctx context.Context, b computev1alpha1.BundleRef) (map[string][]byte, error)

FetchBundleModules pulls a bundle's modules layer (media type application/vnd.apoxy.dev.service.modules.v1.tar+gzip) and returns each regular file's bytes keyed by its cleaned in-layer path (Module.Path).

Unlike the per-revision sandbox path, which extracts the modules layer into the jail, the ServiceManager dispatcher model never mounts a customer bundle: the manager reads the bytes here and inlines them into the WorkerLoader payload the dispatcher pulls. The returned map is keyed to match BundleManifest.Modules[i].Path; the caller maps Path -> Module.Name.

Pull credentials are derived from the BundleRef itself; callers that also need the BundleManifest should use FetchBundle, which shares one registry session for both.

func Main

func Main()

Main is the workerd-host entry point, invoked after sandbox.DispatchRunsc(). It installs the PID-1 reaper, constructs the runtime, optionally ensures a single resident from flags (the standalone 625-e acceptance harness), and blocks until signalled. A full lifecycle controller (APO-796 ServiceManager) drives Ensure/Stop on a Runtime instead of using this loop.

func SetPlatformPullTLS

func SetPlatformPullTLS(fn PlatformPullTLSFunc)

SetPlatformPullTLS installs the platform-registry TLS source for bundle pulls. Call once at process startup; nil clears it.

func StartChildReaper

func StartChildReaper()

StartChildReaper installs a SIGCHLD-driven reaper for orphan child processes.

workerd-host runs as PID 1 in its container. When `runsc create` spawns the Sentry+gofer and exits, those processes are re-parented to PID 1 (us) and become zombies when they exit. Without reaping, `runsc wait`'s kill(pid,0) liveness probe sees a zombie as alive for the full 2-minute backoff. To avoid racing the core's own cmd.Wait(), the reaper consults sandbox.ShouldSkipReap and skips PIDs the core is actively waiting on.

Types

type BuildInput

type BuildInput struct {
	// Manifest is the bundle's BundleManifest (modules + compat + assets).
	Manifest computev1alpha1.BundleManifest
	// Config is the revision's serving config (runtime overrides, bindings, env).
	Config computev1alpha1.ServiceConfigSpec
	// Socket is the listening socket workerd binds.
	Socket SocketSpec
	// AssetsDir is the in-jail absolute path the assets layer extracted to.
	// Used only when Manifest.AssetsPrefix is set.
	AssetsDir string
	// Secrets maps binding names (Binding.Name, for bindings of type secret)
	// to their resolved values. The caller resolves them from the referenced
	// SecretStores; rendering fails on a secret binding with no entry here.
	Secrets map[string]string
}

BuildInput is the input to BuildWorkerdConfig.

type Config

type Config struct {
	// StateDir is runsc's --root (one subdirectory per sandbox).
	StateDir string
	// RootDir is the host staging area for generated config and per-sandbox
	// netconfig.
	RootDir string
	// ImageBaseDir is where OCI bundle images are pulled and extracted.
	ImageBaseDir string
}

Config constructs a Runtime.

type DNSApplier

type DNSApplier interface {
	// ApplyDNS installs the resident sandbox's VPC name plane atomically.
	// Idempotent, last-writer-wins by Generation; returns the generation now
	// in effect.
	ApplyDNS(apply DNSApply) (uint64, error)
}

DNSApplier is the optional name-plane extension of ResidentRuntime, mirroring EgressApplier: the manager's per-tenant DNSConfig gRPC sink probes for it with a type assertion.

type DNSApply

type DNSApply struct {
	// Zones are the DNS zones the resident's resolver answers authoritatively
	// for (NXDOMAIN for unbound names within them).
	Zones []string
	// Bindings is the full desired binding set; it replaces the prior set
	// atomically. Their Reachable prefixes also feed the egress bridge's SSRF
	// carve-out.
	Bindings []vpcdns.Binding
	// Generation orders applies; a push older than the last applied one for
	// the sandbox is ignored. Independent of the egress plane's generation.
	Generation uint64
}

DNSApply is one VPC name-plane push for a sandbox — the Go shape of the DNSConfig/ApplyDNS request (api/workerd/v1) the manager's infra watch sends.

type EgressApplier

type EgressApplier interface {
	// ApplyEgress installs the resident sandbox's egress config atomically.
	// Idempotent, last-writer-wins by Generation; returns the generation now
	// in effect (the request's if applied, the newer retained one if the
	// request was stale).
	ApplyEgress(apply EgressApply) (uint64, error)
}

EgressApplier is the optional egress-config extension of ResidentRuntime: the manager's per-tenant EgressConfig gRPC sink (APO-723) probes for it with a type assertion, mirroring how callers probe the sandbox core for sandbox.EgressController. It is not part of ResidentRuntime so existing fakes and non-egress drivers keep compiling.

type EgressApply

type EgressApply struct {
	// Services is the full desired set of per-Service egress planes for the
	// resident; it replaces the prior set atomically.
	Services []sandbox.ServiceEgress
	// InvocationID is stamped on egress connections for attribution.
	InvocationID string
	// Generation orders applies; a push older than the last applied one for
	// the sandbox is ignored.
	Generation uint64
}

EgressApply is one compiled egress config push for a sandbox — the Go shape of the EgressConfig/ApplyEgress request (api/workerd/v1) the backplane's egress reconciler sends.

type EgressState

type EgressState struct {
	// Services is the full set of per-Service egress planes for the resident.
	Services []sandbox.ServiceEgress
	// InvocationID is stamped on egress connections for attribution.
	InvocationID string
	// Generation is the config generation this state was applied at. It lives
	// here — not beside the caller — so the guard shares the state's exact
	// lifecycle: a recreated sandbox starts from a fresh zero-generation state
	// and can never report a generation whose config was dropped with the old
	// sandbox.
	Generation uint64

	// DNSZones and DNSBindings are the resident's VPC name plane (the
	// DNSConfig/ApplyDNS push): the zones its DNS listener answers
	// authoritatively for and the bindings workers may resolve — whose
	// Reachable prefixes also back the egress bridge's SSRF carve-out.
	// DNSGeneration orders name-plane applies independently of Generation:
	// the two planes have independent pushers (project apiserver vs infra
	// watch) with independent counters.
	DNSZones      []string
	DNSBindings   []vpcdns.Binding
	DNSGeneration uint64
}

EgressState is the recorded egress configuration of one sandbox — what the config plane (APO-723) has applied, held for the egress data path (the forwarder installer / worker egress bridge, APO-713/APO-722) to consume. It mirrors clrk's worker sandbox EgressState, except state is keyed per compute Service: the resident hosts every Service of its project, and each Service selects its egress gateway independently.

type PlatformPullTLSFunc

type PlatformPullTLSFunc func(registryHost string) (*tls.Config, bool)

PlatformPullTLSFunc returns the TLS client configuration for pulls from registryHost, or ok=false when the host is not the platform registry. Set by the embedding data plane (SetPlatformPullTLS) so edge services can authenticate platform-registry pulls with the client certs they already hold. Deliberately keyed on the registry host: platform credentials must never be presented to a customer's BYO registry.

type PullCredentials

type PullCredentials = auth.Credential

PullCredentials authenticate bundle pulls against a private registry, using the docker/oras credential model directly: Username+Password drive basic auth and the standard token-service exchange; RefreshToken drives an OAuth2 exchange (ACR-style identity tokens); AccessToken is sent as a bearer as-is. The zero value means anonymous.

func BundlePullCredentials

func BundlePullCredentials(b computev1alpha1.BundleRef) (PullCredentials, error)

BundlePullCredentials extracts the pull credentials a BundleRef carries. Inline credentials are honored (PasswordData, raw bytes, wins over Password when both are set). CredentialsRef cannot be resolved here — there is no secret store to dereference it against yet — so it fails loudly rather than silently degrading to an anonymous pull that 401s at the registry. Admission rejects credentialsRef for the same reason (validateBundle); this guard covers objects that predate that check.

type Resident

type Resident struct {
	Tenant    string
	Revision  string
	SandboxID sandbox.SandboxID
	Socket    SocketSpec
	Phase     sandbox.SandboxPhase
	// SandboxIP is the in-Sentry container IP the workerd socket binds on.
	// There is no host route to it — reaching the worker goes through
	// InboundSocket — but it is surfaced here for per-tenant isolation
	// assertions and for the lifecycle owner (APO-796).
	SandboxIP netip.Addr

	// InboundSocket is the host AF_UNIX socket path that fronts the in-Sentry
	// worker via the APO-694 ingress forwarder. An Envoy upstream cluster
	// (APO-628) — or the acceptance test — dials this to reach the worker's
	// fetch handler. Set once the resident is Running; empty if the socket
	// is non-HTTP.
	InboundSocket string
}

Resident tracks one live (tenant) slot.

type ResidentConfig

type ResidentConfig struct {
	// Tenant is the project UUID this resident serves; empty for the
	// single-project topologies (apoxy dev, dedicated mode). It keys the
	// sandbox id and the inbound socket path via pkg/workerd/names, so the
	// gateway's per-project resident cluster dials the matching socket.
	Tenant string

	// StateDir is runsc's --root.
	StateDir string
	// RootDir is the host staging area for the generated dispatcher config.
	RootDir string
	// ImageBaseDir is where the stock workerd image is pulled and extracted.
	ImageBaseDir string

	// WorkerdImage is the stock upstream workerd OCI image the resident runs.
	// The dispatcher source is inlined into the config, so this image carries no
	// customer code — only the workerd binary.
	WorkerdImage string
	// ListenAddr is the dispatcher's http socket bind address (workerd syntax,
	// e.g. "*:8080"). Defaults to defaultResidentListenAddr.
	ListenAddr string
	// ControlHostAddr is the HOST loopback TCP address (e.g. "127.0.0.1:2024")
	// the manager's control HTTP server listens on. The clrk control forwarder
	// dials it for each connection the dispatcher opens to ControlForwardAddr.
	ControlHostAddr string
	// ControlForwardAddr is the in-sandbox TCP address the dispatcher dials for
	// the control channel; the clrk control forwarder routes it to
	// ControlHostAddr. Defaults to defaultControlForwardAddr.
	ControlForwardAddr string

	// OverlayNetnsPathFunc maps a tenant to the bind-mount path of that
	// project's VPC network namespace (where the VTEP TUN and overlay routes
	// live), or "" when the tenant has none. When it yields a path, the egress
	// bridge dials overlay (Apoxy VPC ULA) destinations from inside that
	// namespace. The path is opaque to this package — naming conventions
	// (e.g. /var/run/netns/vpc-<projectID>) belong to the embedder. nil (the
	// default) keeps the current behavior: all egress dials from the pod netns.
	OverlayNetnsPathFunc func(tenant string) string
}

ResidentConfig constructs a ResidentHost.

type ResidentConfigInput

type ResidentConfigInput struct {
	// SocketAddr is the address the dispatcher's http socket binds — where the
	// inbound forwarder (APO-694) delivers Envoy's requests. workerd syntax:
	// "*:8080", "127.0.0.1:8080", or "unix:/path.sock".
	SocketAddr string
	// ManagerAddr is the address of the external manager service the dispatcher
	// fetches worker definitions from (the control channel). Typically a donated
	// AF_UNIX socket, e.g. "unix:/run/workerd-host/control.sock".
	ManagerAddr string
}

ResidentConfigInput is the input to BuildResidentConfig.

type ResidentFactory

type ResidentFactory struct {
	// contains filtered or unexported fields
}

ResidentFactory constructs per-tenant ResidentHosts over ONE shared sandbox core. The core must be shared because its state dir (runsc --root), image store, and host cgroup are process-wide; in particular the core's cleanup purges the ENTIRE state dir — every tenant's resident — which is why orphan reaping lives here as a boot-only operation (CleanupOrphans) instead of on the per-tenant ResidentRuntime surface.

func NewResidentFactory

func NewResidentFactory(base ResidentConfig) (*ResidentFactory, error)

NewResidentFactory builds the shared sandbox core from base. Base carries the process-wide config (StateDir/RootDir/ImageBaseDir/WorkerdImage/ListenAddr/ ControlForwardAddr); Tenant and ControlHostAddr are per-resident and filled in by NewResident.

func (*ResidentFactory) CleanupOrphans

func (f *ResidentFactory) CleanupOrphans(ctx context.Context) error

CleanupOrphans reaps sandboxes left behind by a previous host incarnation. It purges the whole shared state dir, so it must run exactly once at process start, before any resident exists; a call after the first NewResident is refused rather than trusted to be safe.

func (*ResidentFactory) NewResident

func (f *ResidentFactory) NewResident(tenant, controlHostAddr string) (ResidentRuntime, error)

NewResident constructs the resident host for a tenant, listening for control connections on controlHostAddr. It is a pure constructor: get-or-create semantics and lifecycle (who calls EnsureResident/Stop when) belong to the caller (pkg/workerd/manager.ResidentManager, which fakes this seam in tests — hence the interface return).

type ResidentHost

type ResidentHost struct {
	// contains filtered or unexported fields
}

ResidentHost owns one tenant's resident workerd. It stages the static dispatcher config, runs it in a single sandbox with the inbound forwarder and the manager control socket, and exposes idempotent lifecycle for the resident reconciler. Construct via ResidentFactory.NewResident so all residents share one sandbox core.

func (*ResidentHost) ApplyDNS

func (h *ResidentHost) ApplyDNS(apply DNSApply) (uint64, error)

ApplyDNS implements DNSApplier for the resident, delegating to the egress core's atomic whole-state apply. Like the egress plane, the applied generation lives inside the recorded state and dies with the sandbox; EnsureResident carries the last-known bindings across a self-heal recreation at generation 0 so name resolution doesn't fall back to upstream-only until the next push.

func (*ResidentHost) ApplyEgress

func (h *ResidentHost) ApplyEgress(apply EgressApply) (uint64, error)

ApplyEgress implements EgressApplier for the resident: it delegates to the egress core's atomic whole-state apply for this tenant's resident sandbox. The applied generation lives inside the recorded state, so it is dropped with the sandbox: a self-healed (recreated) resident starts from a fresh zero-generation state and the reconciler's next push — whatever its generation — lands the config again. To avoid a deny-all gap in that window, EnsureResident re-applies the last-known service planes at generation 0 across a recreation (the reset generation preserves the re-land property above). This is the worker-side sink of the egress config plane (APO-723), consumed by the egress data path (APO-713/APO-722).

func (*ResidentHost) EnsureResident

func (h *ResidentHost) EnsureResident(ctx context.Context) (*ResidentInstance, error)

EnsureResident implements ResidentRuntime.

func (*ResidentHost) Stop

func (h *ResidentHost) Stop(ctx context.Context) error

Stop implements ResidentRuntime.

type ResidentInstance

type ResidentInstance struct {
	SandboxID sandbox.SandboxID
	// InboundSocket is the host AF_UNIX path that fronts the dispatcher's http
	// socket via the inbound forwarder; the backplane's resident Envoy cluster
	// dials it. Empty until Running.
	InboundSocket string
	// SandboxIP is the in-Sentry container IP (diagnostics / isolation asserts).
	SandboxIP netip.Addr
}

ResidentInstance is the running resident, surfaced to the lifecycle owner.

type ResidentRef

type ResidentRef struct {
	// Tenant is the isolation slot; one live workerd per tenant.
	Tenant string
	// Revision is the ServiceRevision name — the reload key.
	Revision string
	// Bundle is the digest-pinned OCI artifact to run.
	Bundle computev1alpha1.BundleRef
	// Config is the serving config (runtime overrides, bindings, env, mode).
	Config computev1alpha1.ServiceConfigSpec
	// Socket is the listening socket workerd binds.
	Socket SocketSpec
}

ResidentRef is the desired state of one resident workerd instance.

type ResidentRuntime

type ResidentRuntime interface {
	// EnsureResident brings this tenant's resident up if it is not already, and
	// is idempotent: a second call while the resident is up returns the same
	// instance without recreating the sandbox.
	EnsureResident(ctx context.Context) (*ResidentInstance, error)
	// Stop drains and tears down the resident, including its staged config.
	Stop(ctx context.Context) error
}

ResidentRuntime is the surface the ServiceManager resident reconciler drives. ResidentHost implements it over the gVisor core; tests fake it on any platform.

The interface deliberately has NO Cleanup: the underlying sandbox core's cleanup purges the entire runsc state dir — every tenant's resident — so it is a process-wide, boot-only operation owned by ResidentFactory (CleanupOrphans), not something a per-tenant driver can be handed.

type Runtime

type Runtime struct {
	// contains filtered or unexported fields
}

Runtime is the workerd policy layer over the tenant-neutral sandbox.Runtime. It owns bundle->config reconstruction, the workerd `serve` argv, the config mount, and per-tenant resident lifecycle with make-before-break reload. APO-796 (ServiceManager) drives many residents through this type; 625's own cmd/workerd-host drives one.

func NewRuntime

func NewRuntime(cfg Config) (*Runtime, error)

NewRuntime constructs the workerd host runtime. The concrete sandbox core is platform-specific: linux builds the gVisor Manager; other platforms return an unsupported-platform error. The core is wrapped with the recording egress controller (APO-723) so the egress config plane has a sink; the neutral core itself stays egress-free.

func (*Runtime) Cleanup

func (r *Runtime) Cleanup(ctx context.Context) error

Cleanup reaps orphan sandboxes left by a previous host incarnation.

func (*Runtime) Ensure

func (r *Runtime) Ensure(ctx context.Context, want ResidentRef) (*Resident, error)

Ensure reconciles the tenant slot to want. No resident -> pull+create+start. A resident on a different revision -> make-before-break reload (start the new one, then drain the old). Same revision -> no-op. Idempotent.

func (*Runtime) List

func (r *Runtime) List() []*Resident

List returns a snapshot of the live residents.

func (*Runtime) Stop

func (r *Runtime) Stop(ctx context.Context, tenant string) error

Stop gracefully drains the tenant's resident and forgets the slot.

type SocketKind

type SocketKind int

SocketKind selects the workerd listening-socket variant Envoy talks to.

const (
	// HTTPSocket is the standard workerd `http` socket used by backend-mode
	// Services (M1). Envoy connects to it as an upstream cluster (APO-628).
	HTTPSocket SocketKind = iota
	// FilterSocket is the capnpFilter ext_proc socket used by filter-mode
	// Services. Scaffolding only in M1; full semantics are APO-629 (M2).
	FilterSocket
)

func (SocketKind) String

func (k SocketKind) String() string

String returns the socket kind's short name.

type SocketSpec

type SocketSpec struct {
	Kind SocketKind
	// Addr is the listen address in workerd syntax: "*:8080" or
	// "127.0.0.1:8080" for TCP, or "unix:/path/to.sock" for a UDS.
	Addr string
}

SocketSpec describes the listening socket workerd binds and Envoy dials.

type WorkerDefinition

type WorkerDefinition struct {
	CompatibilityDate  string                   `json:"compatibilityDate"`
	CompatibilityFlags []string                 `json:"compatibilityFlags,omitempty"`
	MainModule         string                   `json:"mainModule"`
	Modules            map[string]moduleContent `json:"modules"`
	Env                map[string]string        `json:"env,omitempty"`
}

WorkerDefinition is the per-isolate payload the manager serves to the dispatcher's WorkerLoader callback. It marshals to workerd's WorkerCode JSON shape: { compatibilityDate, compatibilityFlags, mainModule, modules, env }.

func BuildWorkerDefinition

func BuildWorkerDefinition(
	manifest computev1alpha1.BundleManifest,
	cfg computev1alpha1.ServiceConfigSpec,
	source map[string][]byte,
	secrets map[string]string,
) (WorkerDefinition, error)

BuildWorkerDefinition renders the WorkerCode payload for one ServiceRevision. source maps each Module.Name to its raw bytes (read from the extracted bundle by the manager); BuildWorkerDefinition stays pure and filesystem-free so it is table-testable. The first esModule is the entrypoint (mainModule). secrets maps binding names (type=secret) to resolved values; see BuildInput.Secrets.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL