Documentation
¶
Overview ¶
Package host drives stock workerd inside a gVisor/runsc sandbox via clrk's extracted pkg/sandbox.Runtime. It is the artifact a lifecycle controller (APO-796 ServiceManager) runs: it reconstructs a workerd config from a compute-API bundle and serves fetch over an HTTP socket (M1 backend mode).
Index ¶
- func BuildResidentConfig(in ResidentConfigInput) (string, error)
- func BuildWorkerdConfig(in BuildInput) (string, error)
- func BundleImageRef(b computev1alpha1.BundleRef) (string, error)
- func CleanModulePath(p string) string
- func FetchBundle(ctx context.Context, b computev1alpha1.BundleRef) (computev1alpha1.BundleManifest, map[string][]byte, error)
- func FetchBundleManifest(ctx context.Context, b computev1alpha1.BundleRef) (computev1alpha1.BundleManifest, error)
- func FetchBundleModules(ctx context.Context, b computev1alpha1.BundleRef) (map[string][]byte, error)
- func Main()
- func SetPlatformPullTLS(fn PlatformPullTLSFunc)
- func StartChildReaper()
- type BuildInput
- type Config
- type DNSApplier
- type DNSApply
- type EgressApplier
- type EgressApply
- type EgressState
- type PlatformPullTLSFunc
- type PullCredentials
- type Resident
- type ResidentConfig
- type ResidentConfigInput
- type ResidentFactory
- type ResidentHost
- type ResidentInstance
- type ResidentRef
- type ResidentRuntime
- type Runtime
- type SocketKind
- type SocketSpec
- type WorkerDefinition
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func BuildResidentConfig ¶
func BuildResidentConfig(in ResidentConfigInput) (string, error)
BuildResidentConfig renders the static textual capnp config for the one resident workerd: a single dispatcher worker (WorkerLoader + manager service binding) behind the http socket. Pure and deterministic.
func BuildWorkerdConfig ¶
func BuildWorkerdConfig(in BuildInput) (string, error)
BuildWorkerdConfig renders the textual capnp config that `workerd serve` consumes. It is a pure, deterministic function: identical input yields byte-identical output. Module `embed` paths are emitted relative to the config file, which the host co-locates with the extracted modules layer so each path is just Module.Path.
func BundleImageRef ¶
func BundleImageRef(b computev1alpha1.BundleRef) (string, error)
BundleImageRef builds the digest-pinned OCI reference for a bundle, exported for the ServiceManager control plane (pkg/workerd/manager), which pulls a revision's modules to inline into the WorkerLoader payload.
func CleanModulePath ¶
CleanModulePath normalizes a tar entry name / Module.Path to a stable lookup key: a leading "./" is stripped and the path is lexically cleaned, so the builder's "./index.js" and a manifest's "index.js" resolve identically. The ServiceManager resolver uses it to map BundleManifest.Modules[i].Path to the keys FetchBundleModules returns.
func FetchBundle ¶
func FetchBundle(ctx context.Context, b computev1alpha1.BundleRef) (computev1alpha1.BundleManifest, map[string][]byte, error)
FetchBundle pulls a bundle's BundleManifest and module bytes in one pass: one repository, one auth exchange, one OCI manifest resolve serving both. It is the fetch entry point for the ServiceManager control plane (pkg/workerd/manager), which inlines a revision's module bytes into the WorkerLoader payload rather than mounting the bundle rootfs.
func FetchBundleManifest ¶
func FetchBundleManifest(ctx context.Context, b computev1alpha1.BundleRef) (computev1alpha1.BundleManifest, error)
FetchBundleManifest fetches the OCI config blob — the JSON-encoded BundleManifest (media type application/vnd.apoxy.dev.service.config.v1+json) — for a bundle. The sandbox ImageStore extracts the rootfs but does not surface this config blob, so the host fetches it directly. (R1.)
Pull credentials are derived from the BundleRef itself; callers that also need the module bytes should use FetchBundle, which shares one registry session for both.
func FetchBundleModules ¶
func FetchBundleModules(ctx context.Context, b computev1alpha1.BundleRef) (map[string][]byte, error)
FetchBundleModules pulls a bundle's modules layer (media type application/vnd.apoxy.dev.service.modules.v1.tar+gzip) and returns each regular file's bytes keyed by its cleaned in-layer path (Module.Path).
Unlike the per-revision sandbox path, which extracts the modules layer into the jail, the ServiceManager dispatcher model never mounts a customer bundle: the manager reads the bytes here and inlines them into the WorkerLoader payload the dispatcher pulls. The returned map is keyed to match BundleManifest.Modules[i].Path; the caller maps Path -> Module.Name.
Pull credentials are derived from the BundleRef itself; callers that also need the BundleManifest should use FetchBundle, which shares one registry session for both.
func Main ¶
func Main()
Main is the workerd-host entry point, invoked after sandbox.DispatchRunsc(). It installs the PID-1 reaper, constructs the runtime, optionally ensures a single resident from flags (the standalone 625-e acceptance harness), and blocks until signalled. A full lifecycle controller (APO-796 ServiceManager) drives Ensure/Stop on a Runtime instead of using this loop.
func SetPlatformPullTLS ¶
func SetPlatformPullTLS(fn PlatformPullTLSFunc)
SetPlatformPullTLS installs the platform-registry TLS source for bundle pulls. Call once at process startup; nil clears it.
func StartChildReaper ¶
func StartChildReaper()
StartChildReaper installs a SIGCHLD-driven reaper for orphan child processes.
workerd-host runs as PID 1 in its container. When `runsc create` spawns the Sentry+gofer and exits, those processes are re-parented to PID 1 (us) and become zombies when they exit. Without reaping, `runsc wait`'s kill(pid,0) liveness probe sees a zombie as alive for the full 2-minute backoff. To avoid racing the core's own cmd.Wait(), the reaper consults sandbox.ShouldSkipReap and skips PIDs the core is actively waiting on.
Types ¶
type BuildInput ¶
type BuildInput struct {
// Manifest is the bundle's BundleManifest (modules + compat + assets).
Manifest computev1alpha1.BundleManifest
// Config is the revision's serving config (runtime overrides, bindings, env).
Config computev1alpha1.ServiceConfigSpec
// Socket is the listening socket workerd binds.
Socket SocketSpec
// AssetsDir is the in-jail absolute path the assets layer extracted to.
// Used only when Manifest.AssetsPrefix is set.
AssetsDir string
// Secrets maps binding names (Binding.Name, for bindings of type secret)
// to their resolved values. The caller resolves them from the referenced
// SecretStores; rendering fails on a secret binding with no entry here.
Secrets map[string]string
}
BuildInput is the input to BuildWorkerdConfig.
type Config ¶
type Config struct {
// StateDir is runsc's --root (one subdirectory per sandbox).
StateDir string
// RootDir is the host staging area for generated config and per-sandbox
// netconfig.
RootDir string
// ImageBaseDir is where OCI bundle images are pulled and extracted.
ImageBaseDir string
}
Config constructs a Runtime.
type DNSApplier ¶
type DNSApplier interface {
// ApplyDNS installs the resident sandbox's VPC name plane atomically.
// Idempotent, last-writer-wins by Generation; returns the generation now
// in effect.
ApplyDNS(apply DNSApply) (uint64, error)
}
DNSApplier is the optional name-plane extension of ResidentRuntime, mirroring EgressApplier: the manager's per-tenant DNSConfig gRPC sink probes for it with a type assertion.
type DNSApply ¶
type DNSApply struct {
// Zones are the DNS zones the resident's resolver answers authoritatively
// for (NXDOMAIN for unbound names within them).
Zones []string
// Bindings is the full desired binding set; it replaces the prior set
// atomically. Their Reachable prefixes also feed the egress bridge's SSRF
// carve-out.
Bindings []vpcdns.Binding
// Generation orders applies; a push older than the last applied one for
// the sandbox is ignored. Independent of the egress plane's generation.
Generation uint64
}
DNSApply is one VPC name-plane push for a sandbox — the Go shape of the DNSConfig/ApplyDNS request (api/workerd/v1) the manager's infra watch sends.
type EgressApplier ¶
type EgressApplier interface {
// ApplyEgress installs the resident sandbox's egress config atomically.
// Idempotent, last-writer-wins by Generation; returns the generation now
// in effect (the request's if applied, the newer retained one if the
// request was stale).
ApplyEgress(apply EgressApply) (uint64, error)
}
EgressApplier is the optional egress-config extension of ResidentRuntime: the manager's per-tenant EgressConfig gRPC sink (APO-723) probes for it with a type assertion, mirroring how callers probe the sandbox core for sandbox.EgressController. It is not part of ResidentRuntime so existing fakes and non-egress drivers keep compiling.
type EgressApply ¶
type EgressApply struct {
// Services is the full desired set of per-Service egress planes for the
// resident; it replaces the prior set atomically.
Services []sandbox.ServiceEgress
// InvocationID is stamped on egress connections for attribution.
InvocationID string
// Generation orders applies; a push older than the last applied one for
// the sandbox is ignored.
Generation uint64
}
EgressApply is one compiled egress config push for a sandbox — the Go shape of the EgressConfig/ApplyEgress request (api/workerd/v1) the backplane's egress reconciler sends.
type EgressState ¶
type EgressState struct {
// Services is the full set of per-Service egress planes for the resident.
Services []sandbox.ServiceEgress
// InvocationID is stamped on egress connections for attribution.
InvocationID string
// Generation is the config generation this state was applied at. It lives
// here — not beside the caller — so the guard shares the state's exact
// lifecycle: a recreated sandbox starts from a fresh zero-generation state
// and can never report a generation whose config was dropped with the old
// sandbox.
Generation uint64
// DNSZones and DNSBindings are the resident's VPC name plane (the
// DNSConfig/ApplyDNS push): the zones its DNS listener answers
// authoritatively for and the bindings workers may resolve — whose
// Reachable prefixes also back the egress bridge's SSRF carve-out.
// DNSGeneration orders name-plane applies independently of Generation:
// the two planes have independent pushers (project apiserver vs infra
// watch) with independent counters.
DNSZones []string
DNSBindings []vpcdns.Binding
DNSGeneration uint64
}
EgressState is the recorded egress configuration of one sandbox — what the config plane (APO-723) has applied, held for the egress data path (the forwarder installer / worker egress bridge, APO-713/APO-722) to consume. It mirrors clrk's worker sandbox EgressState, except state is keyed per compute Service: the resident hosts every Service of its project, and each Service selects its egress gateway independently.
type PlatformPullTLSFunc ¶
PlatformPullTLSFunc returns the TLS client configuration for pulls from registryHost, or ok=false when the host is not the platform registry. Set by the embedding data plane (SetPlatformPullTLS) so edge services can authenticate platform-registry pulls with the client certs they already hold. Deliberately keyed on the registry host: platform credentials must never be presented to a customer's BYO registry.
type PullCredentials ¶
type PullCredentials = auth.Credential
PullCredentials authenticate bundle pulls against a private registry, using the docker/oras credential model directly: Username+Password drive basic auth and the standard token-service exchange; RefreshToken drives an OAuth2 exchange (ACR-style identity tokens); AccessToken is sent as a bearer as-is. The zero value means anonymous.
func BundlePullCredentials ¶
func BundlePullCredentials(b computev1alpha1.BundleRef) (PullCredentials, error)
BundlePullCredentials extracts the pull credentials a BundleRef carries. Inline credentials are honored (PasswordData, raw bytes, wins over Password when both are set). CredentialsRef cannot be resolved here — there is no secret store to dereference it against yet — so it fails loudly rather than silently degrading to an anonymous pull that 401s at the registry. Admission rejects credentialsRef for the same reason (validateBundle); this guard covers objects that predate that check.
type Resident ¶
type Resident struct {
Tenant string
Revision string
SandboxID sandbox.SandboxID
Socket SocketSpec
Phase sandbox.SandboxPhase
// SandboxIP is the in-Sentry container IP the workerd socket binds on.
// There is no host route to it — reaching the worker goes through
// InboundSocket — but it is surfaced here for per-tenant isolation
// assertions and for the lifecycle owner (APO-796).
SandboxIP netip.Addr
// InboundSocket is the host AF_UNIX socket path that fronts the in-Sentry
// worker via the APO-694 ingress forwarder. An Envoy upstream cluster
// (APO-628) — or the acceptance test — dials this to reach the worker's
// fetch handler. Set once the resident is Running; empty if the socket
// is non-HTTP.
InboundSocket string
}
Resident tracks one live (tenant) slot.
type ResidentConfig ¶
type ResidentConfig struct {
// Tenant is the project UUID this resident serves; empty for the
// single-project topologies (apoxy dev, dedicated mode). It keys the
// sandbox id and the inbound socket path via pkg/workerd/names, so the
// gateway's per-project resident cluster dials the matching socket.
Tenant string
// StateDir is runsc's --root.
StateDir string
// RootDir is the host staging area for the generated dispatcher config.
RootDir string
// ImageBaseDir is where the stock workerd image is pulled and extracted.
ImageBaseDir string
// WorkerdImage is the stock upstream workerd OCI image the resident runs.
// The dispatcher source is inlined into the config, so this image carries no
// customer code — only the workerd binary.
WorkerdImage string
// ListenAddr is the dispatcher's http socket bind address (workerd syntax,
// e.g. "*:8080"). Defaults to defaultResidentListenAddr.
ListenAddr string
// ControlHostAddr is the HOST loopback TCP address (e.g. "127.0.0.1:2024")
// the manager's control HTTP server listens on. The clrk control forwarder
// dials it for each connection the dispatcher opens to ControlForwardAddr.
ControlHostAddr string
// ControlForwardAddr is the in-sandbox TCP address the dispatcher dials for
// the control channel; the clrk control forwarder routes it to
// ControlHostAddr. Defaults to defaultControlForwardAddr.
ControlForwardAddr string
// OverlayNetnsPathFunc maps a tenant to the bind-mount path of that
// project's VPC network namespace (where the VTEP TUN and overlay routes
// live), or "" when the tenant has none. When it yields a path, the egress
// bridge dials overlay (Apoxy VPC ULA) destinations from inside that
// namespace. The path is opaque to this package — naming conventions
// (e.g. /var/run/netns/vpc-<projectID>) belong to the embedder. nil (the
// default) keeps the current behavior: all egress dials from the pod netns.
OverlayNetnsPathFunc func(tenant string) string
}
ResidentConfig constructs a ResidentHost.
type ResidentConfigInput ¶
type ResidentConfigInput struct {
// SocketAddr is the address the dispatcher's http socket binds — where the
// inbound forwarder (APO-694) delivers Envoy's requests. workerd syntax:
// "*:8080", "127.0.0.1:8080", or "unix:/path.sock".
SocketAddr string
// ManagerAddr is the address of the external manager service the dispatcher
// fetches worker definitions from (the control channel). Typically a donated
// AF_UNIX socket, e.g. "unix:/run/workerd-host/control.sock".
ManagerAddr string
}
ResidentConfigInput is the input to BuildResidentConfig.
type ResidentFactory ¶
type ResidentFactory struct {
// contains filtered or unexported fields
}
ResidentFactory constructs per-tenant ResidentHosts over ONE shared sandbox core. The core must be shared because its state dir (runsc --root), image store, and host cgroup are process-wide; in particular the core's cleanup purges the ENTIRE state dir — every tenant's resident — which is why orphan reaping lives here as a boot-only operation (CleanupOrphans) instead of on the per-tenant ResidentRuntime surface.
func NewResidentFactory ¶
func NewResidentFactory(base ResidentConfig) (*ResidentFactory, error)
NewResidentFactory builds the shared sandbox core from base. Base carries the process-wide config (StateDir/RootDir/ImageBaseDir/WorkerdImage/ListenAddr/ ControlForwardAddr); Tenant and ControlHostAddr are per-resident and filled in by NewResident.
func (*ResidentFactory) CleanupOrphans ¶
func (f *ResidentFactory) CleanupOrphans(ctx context.Context) error
CleanupOrphans reaps sandboxes left behind by a previous host incarnation. It purges the whole shared state dir, so it must run exactly once at process start, before any resident exists; a call after the first NewResident is refused rather than trusted to be safe.
func (*ResidentFactory) NewResident ¶
func (f *ResidentFactory) NewResident(tenant, controlHostAddr string) (ResidentRuntime, error)
NewResident constructs the resident host for a tenant, listening for control connections on controlHostAddr. It is a pure constructor: get-or-create semantics and lifecycle (who calls EnsureResident/Stop when) belong to the caller (pkg/workerd/manager.ResidentManager, which fakes this seam in tests — hence the interface return).
type ResidentHost ¶
type ResidentHost struct {
// contains filtered or unexported fields
}
ResidentHost owns one tenant's resident workerd. It stages the static dispatcher config, runs it in a single sandbox with the inbound forwarder and the manager control socket, and exposes idempotent lifecycle for the resident reconciler. Construct via ResidentFactory.NewResident so all residents share one sandbox core.
func (*ResidentHost) ApplyDNS ¶
func (h *ResidentHost) ApplyDNS(apply DNSApply) (uint64, error)
ApplyDNS implements DNSApplier for the resident, delegating to the egress core's atomic whole-state apply. Like the egress plane, the applied generation lives inside the recorded state and dies with the sandbox; EnsureResident carries the last-known bindings across a self-heal recreation at generation 0 so name resolution doesn't fall back to upstream-only until the next push.
func (*ResidentHost) ApplyEgress ¶
func (h *ResidentHost) ApplyEgress(apply EgressApply) (uint64, error)
ApplyEgress implements EgressApplier for the resident: it delegates to the egress core's atomic whole-state apply for this tenant's resident sandbox. The applied generation lives inside the recorded state, so it is dropped with the sandbox: a self-healed (recreated) resident starts from a fresh zero-generation state and the reconciler's next push — whatever its generation — lands the config again. To avoid a deny-all gap in that window, EnsureResident re-applies the last-known service planes at generation 0 across a recreation (the reset generation preserves the re-land property above). This is the worker-side sink of the egress config plane (APO-723), consumed by the egress data path (APO-713/APO-722).
func (*ResidentHost) EnsureResident ¶
func (h *ResidentHost) EnsureResident(ctx context.Context) (*ResidentInstance, error)
EnsureResident implements ResidentRuntime.
type ResidentInstance ¶
type ResidentInstance struct {
SandboxID sandbox.SandboxID
// InboundSocket is the host AF_UNIX path that fronts the dispatcher's http
// socket via the inbound forwarder; the backplane's resident Envoy cluster
// dials it. Empty until Running.
InboundSocket string
// SandboxIP is the in-Sentry container IP (diagnostics / isolation asserts).
SandboxIP netip.Addr
}
ResidentInstance is the running resident, surfaced to the lifecycle owner.
type ResidentRef ¶
type ResidentRef struct {
// Tenant is the isolation slot; one live workerd per tenant.
Tenant string
// Revision is the ServiceRevision name — the reload key.
Revision string
// Bundle is the digest-pinned OCI artifact to run.
Bundle computev1alpha1.BundleRef
// Config is the serving config (runtime overrides, bindings, env, mode).
Config computev1alpha1.ServiceConfigSpec
// Socket is the listening socket workerd binds.
Socket SocketSpec
}
ResidentRef is the desired state of one resident workerd instance.
type ResidentRuntime ¶
type ResidentRuntime interface {
// EnsureResident brings this tenant's resident up if it is not already, and
// is idempotent: a second call while the resident is up returns the same
// instance without recreating the sandbox.
EnsureResident(ctx context.Context) (*ResidentInstance, error)
// Stop drains and tears down the resident, including its staged config.
Stop(ctx context.Context) error
}
ResidentRuntime is the surface the ServiceManager resident reconciler drives. ResidentHost implements it over the gVisor core; tests fake it on any platform.
The interface deliberately has NO Cleanup: the underlying sandbox core's cleanup purges the entire runsc state dir — every tenant's resident — so it is a process-wide, boot-only operation owned by ResidentFactory (CleanupOrphans), not something a per-tenant driver can be handed.
type Runtime ¶
type Runtime struct {
// contains filtered or unexported fields
}
Runtime is the workerd policy layer over the tenant-neutral sandbox.Runtime. It owns bundle->config reconstruction, the workerd `serve` argv, the config mount, and per-tenant resident lifecycle with make-before-break reload. APO-796 (ServiceManager) drives many residents through this type; 625's own cmd/workerd-host drives one.
func NewRuntime ¶
NewRuntime constructs the workerd host runtime. The concrete sandbox core is platform-specific: linux builds the gVisor Manager; other platforms return an unsupported-platform error. The core is wrapped with the recording egress controller (APO-723) so the egress config plane has a sink; the neutral core itself stays egress-free.
func (*Runtime) Ensure ¶
Ensure reconciles the tenant slot to want. No resident -> pull+create+start. A resident on a different revision -> make-before-break reload (start the new one, then drain the old). Same revision -> no-op. Idempotent.
type SocketKind ¶
type SocketKind int
SocketKind selects the workerd listening-socket variant Envoy talks to.
const ( // HTTPSocket is the standard workerd `http` socket used by backend-mode // Services (M1). Envoy connects to it as an upstream cluster (APO-628). HTTPSocket SocketKind = iota // FilterSocket is the capnpFilter ext_proc socket used by filter-mode // Services. Scaffolding only in M1; full semantics are APO-629 (M2). FilterSocket )
func (SocketKind) String ¶
func (k SocketKind) String() string
String returns the socket kind's short name.
type SocketSpec ¶
type SocketSpec struct {
Kind SocketKind
// Addr is the listen address in workerd syntax: "*:8080" or
// "127.0.0.1:8080" for TCP, or "unix:/path/to.sock" for a UDS.
Addr string
}
SocketSpec describes the listening socket workerd binds and Envoy dials.
type WorkerDefinition ¶
type WorkerDefinition struct {
CompatibilityDate string `json:"compatibilityDate"`
CompatibilityFlags []string `json:"compatibilityFlags,omitempty"`
MainModule string `json:"mainModule"`
Modules map[string]moduleContent `json:"modules"`
Env map[string]string `json:"env,omitempty"`
}
WorkerDefinition is the per-isolate payload the manager serves to the dispatcher's WorkerLoader callback. It marshals to workerd's WorkerCode JSON shape: { compatibilityDate, compatibilityFlags, mainModule, modules, env }.
func BuildWorkerDefinition ¶
func BuildWorkerDefinition( manifest computev1alpha1.BundleManifest, cfg computev1alpha1.ServiceConfigSpec, source map[string][]byte, secrets map[string]string, ) (WorkerDefinition, error)
BuildWorkerDefinition renders the WorkerCode payload for one ServiceRevision. source maps each Module.Name to its raw bytes (read from the extracted bundle by the manager); BuildWorkerDefinition stays pure and filesystem-free so it is table-testable. The first esModule is the entrypoint (mainModule). secrets maps binding names (type=secret) to resolved values; see BuildInput.Secrets.