Documentation
¶
Overview ¶
Package workers ports packages/chord/src/next-node/workers: Chord v2's workers component (slice 10 §3, §4), a worker process per address of a host kind, started on demand, stopped when idle, never restarted by itself, its plugins shipped from the server; `WorkerMain` is the worker's side. Pi's barrel `index.ts` is designed out: Go imports the package, and what the barrel exported is exported here (`CrossedError`, `ProcessLock`, `WorkerMain` and its types, `CreateWorkers` and its types).
A worker is a child process, not a goroutine ¶
The observable contract of a worker is isolation, and a goroutine has none of it:
- crash containment: a worker's crash (`os.Exit`, a signal, a panic, running out of memory) ends that worker only; the server sees its exit code or signal (`worker exited (code 7)`, `worker exited (SIGKILL)`) and fails the address. A panic in a goroutine ends the whole Go process.
- kill: a worker that answers nothing (Pi's blocked event loop) is killed with everything it spawned (its process group, a job object on Windows). A goroutine cannot be killed.
- lock semantics: one live worker per address across processes, held by the kernel for the holder's lifetime (workers_lock.go), released by its death, however it dies.
- output: its stdout and stderr are its own, piped to the server, the stderr tail kept for the failure report.
A worker is therefore this program's own executable (or any program that calls `WorkerMain`), started with its IPC channel (a socketpair on Unix, a named pipe on Windows; workers_channel.go) beside its stdout and stderr pipes. `Entry` is the executable and `ExecArgv` its arguments (Node's `execArgv` and module). A test starts its own test binary again.
Async contract (AGENTS.md "TypeScript async/Promise parity") ¶
- Ordering: the control messages and the link's frames share one ordered stream, told to listeners in order by one goroutine; a write never blocks its sender. Per address, starts and stops run one after another (a chain of done channels); an event of a child is recorded on that child as it happens, in the order its source tells it, and a late event of an old child never changes a new one. A process's exit handler runs before anyone waiting on the exit resumes.
- Cancellation: `context.Context` is `AbortSignal`. `Close` cancels starts waiting in the app's `Start`, for `started` or for the plugins; the caller's context ends only its own wait. A worker's context aborts at shutdown.
- Shutdown draining: `Close` stops every child (`shutdown`, a 5 s grace, then the tree is killed), clears the timers and joins every goroutine the component owns. A worker runs its closers in reverse order, then closes its NodeHost, forcing its own exit after 3 s.
- Unawaited Promises are owned tasks (the component's task group, the worker's reload loop), never naked goroutines.
Index ¶
- func WorkerMain(fn func(start WorkerStart, ctx context.Context) (*WorkerResult, error), ...)
- type AuthorizeFailedError
- type BatchWorkersError
- type BusyWatchable
- type CrossedCause
- type CrossedError
- type CrossedFailure
- type CrossedObject
- type CrossedReport
- type Exit
- type ForkOptions
- type FromWorker
- type FromWorkerKind
- type ProcessEvents
- type ProcessLockHandle
- type ReloadedPlugins
- type ToWorker
- type ToWorkerKind
- type WorkerExitedError
- type WorkerFailedError
- type WorkerLoad
- type WorkerMainOptions
- type WorkerPlugins
- type WorkerProcess
- type WorkerResult
- type WorkerStart
- type WorkerState
- type WorkerStatus
- type Workers
- func (c *Workers) Close(ctx context.Context) error
- func (c *Workers) Retry(address contracts.Address, _ context.Context) error
- func (c *Workers) State() state.SourceHandle
- func (c *Workers) Step(name string) plugins.RolloutStep
- func (c *Workers) Stop(address contracts.Address, ctx context.Context) error
- type WorkersError
- type WorkersOptions
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func WorkerMain ¶
func WorkerMain(fn func(start WorkerStart, ctx context.Context) (*WorkerResult, error), options WorkerMainOptions)
WorkerMain runs this process as a worker: fn once the start message arrived and the lock is taken. An error from fn (or a panic in it) sends `fatal` (redacted) and exits. It does not return.
Types ¶
type AuthorizeFailedError ¶
AuthorizeFailedError (`authorize_failed`): the app's `authorize` failed: the hold is refused (`forbidden`), the error stays private.
type BatchWorkersError ¶
type BatchWorkersError struct{ state.BatchError }
BatchWorkersError is a `BatchError` of the state's batch.
type BusyWatchable ¶
type BusyWatchable interface {
Value() (delta.JsonValue, error)
Watch(ctx context.Context) (chord.WatchHandle[delta.JsonValue], error)
}
BusyWatchable is a JSON watchable of numbers: a reference's member, or a source.
type CrossedCause ¶
type CrossedCause struct {
Kind string
Error CrossedError
}
CrossedCause is a failure cause as it crosses; Kind is the cause's kind.
type CrossedError ¶
CrossedError is an error as it crosses: its message, and a ChordError's code and reason.
type CrossedFailure ¶
type CrossedFailure struct {
Plugin string
Error CrossedError
}
CrossedFailure is a plugin that failed to load, its error crossed.
type CrossedObject ¶
type CrossedObject struct {
Object string
Kind string
Cause *CrossedCause
}
CrossedObject is an object of a reload report as it crosses; Cause is set only for a parked one.
type CrossedReport ¶
type CrossedReport struct {
Plugins ReloadedPlugins
Failed []CrossedFailure
Objects []CrossedObject
}
CrossedReport is a reload report as it crosses: its errors crossed.
type Exit ¶
Exit is how a worker process ended: its exit code, or the signal that ended it; the other is nil (Node's null).
type ForkOptions ¶
type ForkOptions struct {
Entry string
ExecArgv []string
Env map[string]string
// OnOutput is its output; without it, mirrored to this process's own.
OnOutput func(stream string, chunk []byte)
}
ForkOptions is how a worker process is started: the bootstrap options of the workers component.
type FromWorker ¶
type FromWorker struct {
Kind FromWorkerKind
Idle bool
Report *CrossedReport
Error *CrossedError
}
FromWorker is what a worker sends the server: Idle with FromWorkerReady and FromWorkerIdle, Report with FromWorkerReady and a FromWorkerReloaded that applied, Error with FromWorkerFatal and a FromWorkerReloaded that failed.
type FromWorkerKind ¶
type FromWorkerKind string
FromWorkerKind tags a FromWorker.
const ( FromWorkerStarted FromWorkerKind = "started" FromWorkerReady FromWorkerKind = "ready" FromWorkerIdle FromWorkerKind = "idle" FromWorkerFatal FromWorkerKind = "fatal" FromWorkerReloaded FromWorkerKind = "reloaded" )
type ProcessEvents ¶
type ProcessEvents struct {
Control func(message FromWorker)
// LinkLost: its link was closed by this side (the heartbeat, a protocol error) while the child may live.
LinkLost func()
// LinkClosed: its link was closed by the child's side (it exits, or it closed its IPC channel and may live on).
LinkClosed func()
Exited func(exit Exit)
}
ProcessEvents is what a worker process tells as it happens.
type ProcessLockHandle ¶
type ProcessLockHandle struct {
// contains filtered or unexported fields
}
ProcessLockHandle is a held lock: its listening socket and the connections of the waiters, kept open and never answered. Release frees it (the process's exit does too).
func ProcessLock ¶
func ProcessLock(name string, ctx context.Context) (*ProcessLockHandle, error)
ProcessLock takes the lock name, waiting while another process holds it (until ctx aborts: it returns its cause). It returns what listening returns besides "in use", and for a stale socket file (macOS, the BSDs) an error naming it.
func (*ProcessLockHandle) Release ¶
func (lock *ProcessLockHandle) Release() error
Release closes the lock's socket and its waiters' connections (they see them close) once it is gone; releasing again returns the first result.
type ReloadedPlugins ¶
ReloadedPlugins is the plugins of a reload report: names added, changed and removed.
type ToWorker ¶
type ToWorker struct {
Kind ToWorkerKind
Address contracts.Address
Start delta.JsonValue
Scripts []plugins.PluginScript
Lock string
}
ToWorker is what the server sends a worker: Address, Start (JSON), Scripts and Lock only with ToWorkerStart; Scripts also with ToWorkerReload.
type ToWorkerKind ¶
type ToWorkerKind string
ToWorkerKind tags a ToWorker.
const ( ToWorkerStart ToWorkerKind = "start" ToWorkerShutdown ToWorkerKind = "shutdown" ToWorkerReload ToWorkerKind = "reload" )
type WorkerExitedError ¶
WorkerExitedError (`worker_exited`): an exit nobody asked for.
type WorkerFailedError ¶
WorkerFailedError (`worker_failed`): Error is the full error (a start's throw, a crash, the child's `fatal`), for the app's logs, never public.
type WorkerLoad ¶
type WorkerLoad struct {
Kind string
// Core is the app's own facets that are JavaScript: modules whose default export is the facet, loaded into the NodeHost
// before the plugins (they import Pi's chord by its package name, `@earendil-works/chord/next`).
Core []plugins.CoreFacet
// GoCore are the app's own facets that are Go: they load on the worker's Go host, and the local types the NodeHost's
// facets use are named by Bridged.
GoCore []host.HostFacet
// Bridged are the Go core facets whose local types the NodeHost's core facets and plugins use (plugins.BridgedFacet): the
// NodeHost loads a proxy facet of each, in the same closure as the plugins. A BridgedFacet without a Host is the worker's.
Bridged []plugins.BridgedFacet
// HostKind is the kind of the worker's address, the one the server routes to this worker.
HostKind *contracts.HostKind
// Principal is who the NodeHost acts as, and how the worker's Go host stamps it; Peer is who the worker's Go host is for
// the NodeHost (the server, whose holds it relays).
Principal data.Principal
Peer data.Principal
}
WorkerLoad is what WorkerPlugins.Load takes besides the host.
The plugins of a worker run in a Node subprocess (docs/plan/chord-plugins-node-evaluator.md): Pi's worker host loads them in process, here the worker's Go host is the relay the server's holds pass through, and the host that serves the worker's address, the app's core facets and the plugins' is the NodeHost, the child of the worker's Go host. Load makes it, routes the worker's address kind to its link on the worker's Go host (`Route`, before `Load` of that host: the app's host has no address of its own, which the route would not reach) and loads the Go host.
type WorkerMainOptions ¶
type WorkerMainOptions struct {
// HostPackages is the host packages by specifier: the module files the NodeHost imports once and the plugins share.
HostPackages plugins.HostPackages
// Node is how the worker's Node subprocess is started (its output goes to this process's own unless OnOutput says
// otherwise, as a worker's does).
Node plugins.NodeOptions
// Redact is what crosses as `fatal` for an error (never the original message unless this returns it).
Redact func(err any) CrossedError
OnError func(err any)
}
WorkerMainOptions is what WorkerMain takes besides the app's function.
type WorkerPlugins ¶
type WorkerPlugins struct {
// contains filtered or unexported fields
}
WorkerPlugins is the worker's plugins: the shipped scripts of its kind, loaded on its host; the server's reloads applied to it.
func (*WorkerPlugins) Load ¶
func (p *WorkerPlugins) Load(h host.Host, options WorkerLoad, ctx context.Context) (host.LoadReport, error)
Load loads the app's Go core facets (options.GoCore) on h (a host without an address, not loaded yet), routes the worker's address kind to the link of the NodeHost on h (`Route`, before `Load` of h), starts the NodeHost of the worker's address with the start message's scripts of Kind, the JavaScript Core and a proxy for each of options.Bridged (the Go core facets whose local types the NodeHost's facets use; a BridgedFacet without a Host is h's), and returns the NodeHost's report. The Go host is loaded first: the proxies ask for their Go objects as they activate, inside the NodeHost's own load. When the NodeHost ends before the worker is told to stop, the worker ends with it: it exits with its exit code, or sends `fatal` with the exception it died of. Its close is the worker's last: shutdown closes it after the app's closers, those registered after Load too, so the app's host closes the link to the server before the NodeHost ends and fails a reload in flight (Pi's host closer closes the link before its reload rejects).
type WorkerProcess ¶
type WorkerProcess struct {
Pid int
StartedAt int64
// Link is the link to the child's host, made at the fork.
Link link.Link
// contains filtered or unexported fields
}
WorkerProcess is one started worker process.
func NewWorkerProcess ¶
func NewWorkerProcess(options ForkOptions, events ProcessEvents) (*WorkerProcess, error)
NewWorkerProcess starts the worker process options describe and tells events as they happen.
func (*WorkerProcess) Exit ¶
func (p *WorkerProcess) Exit() Exit
Exit is how the process ended; it is valid once Exited is closed.
func (*WorkerProcess) Exited ¶
func (p *WorkerProcess) Exited() <-chan struct{}
Exited is closed when the process ended (every process it spawned is killed then); Exit is its result.
func (*WorkerProcess) Kill ¶
func (p *WorkerProcess) Kill()
Kill kills the child and every process it spawned; it does nothing once the process ended.
func (*WorkerProcess) Send ¶
func (p *WorkerProcess) Send(message ToWorker)
Send sends a control message; it is dropped when the child is gone (its exit tells).
func (*WorkerProcess) StderrTail ¶
func (p *WorkerProcess) StderrTail() string
StderrTail is the last bytes it wrote to stderr; empty when it wrote none.
type WorkerResult ¶
type WorkerResult struct {
Busy BusyWatchable
// Held is the number of holds the app's own code keeps on the NodeHost's objects (a reference it reads Busy through): the
// NodeHost counts them as references like the server's, and they do not keep the worker busy.
Held int
}
WorkerResult is what the app's function returns: extra keep-alive counts (busy while any is nonzero).
type WorkerStart ¶
type WorkerStart struct {
Address contracts.Address
// Start is the start data the server's `start(address)` gave.
Start delta.JsonValue
// Parent is the link to the server: connect it before anything slow (`started` goes out then).
Parent link.Link
Plugins *WorkerPlugins
// OnClose registers a closer: closers run in reverse order at shutdown.
OnClose func(closer func(ctx context.Context) error)
}
WorkerStart is what the app's function gets.
type WorkerState ¶
type WorkerState struct {
Status WorkerStatus
Pid int
StartedAt int64
Error CrossedError
At int64
}
WorkerState is one address's worker (R5): running or on its way, or failed (until `Retry` or a rollout) with a public error: a fixed message (`worker failed to start`, `worker exited (SIGSEGV)`) or the child's redacted `fatal`. Pid and StartedAt are set for a worker that lives, Error and At for a failed one.
type WorkerStatus ¶
type WorkerStatus string
WorkerStatus is where a worker stands.
const ( WorkerStarting WorkerStatus = "starting" WorkerRunning WorkerStatus = "running" WorkerStopping WorkerStatus = "stopping" WorkerFailed WorkerStatus = "failed" )
type Workers ¶
type Workers struct {
// contains filtered or unexported fields
}
Workers is the workers component.
func CreateWorkers ¶
func CreateWorkers(h host.Host, kind *contracts.HostKind, options WorkersOptions, ctx context.Context) (*Workers, error)
CreateWorkers creates the workers component on h: it routes kind itself (`host.Route`, before `Load`).
func (*Workers) Close ¶
Close cancels starts, stops every worker; new starts reject `released/closed`.
func (*Workers) Retry ¶
Retry clears address's failure and suspension: it starts again now if held (`RouteHandle.Retry`).
func (*Workers) State ¶
func (c *Workers) State() state.SourceHandle
State is per address key its WorkerState; no entry when stopped or never started. A JSON watchable.
type WorkersError ¶
type WorkersError interface {
// contains filtered or unexported methods
}
WorkersError is what the component reports (and the state's watch listeners that threw): one of WorkerFailedError, AuthorizeFailedError, WorkerExitedError and BatchWorkersError.
type WorkersOptions ¶
type WorkersOptions struct {
// Kind is the facet kind its workers load ("worker"), shipped by the plugins component.
Kind string
// Entry is the worker's executable (it calls WorkerMain), ExecArgv its arguments and Env its environment.
Entry string
ExecArgv []string
Env map[string]string
// Authorize is the route's `authorize`: whether principal may reach address.
Authorize func(principal data.Principal, address contracts.Address) (bool, error)
// Principal is who the worker at address is (stamped on its link).
Principal func(address contracts.Address) (data.Principal, error)
// Start is the worker's start data (JSON), before each start; abortable (`Close`), an error fails the start.
Start func(address contracts.Address, ctx context.Context) (delta.JsonValue, error)
// Lock is the lock one live worker of address holds (across processes).
Lock func(address contracts.Address) (string, error)
// IdleMs is how long a worker idles before it is stopped: 5 minutes by default.
IdleMs func() time.Duration
Plugins *plugins.Plugins
// OnOutput is a worker's output; without it, mirrored to this process's own.
OnOutput func(address contracts.Address, stream string, chunk []byte)
OnError func(err WorkersError)
// contains filtered or unexported fields
}
WorkersOptions is what CreateWorkers takes. Kind, Entry, ExecArgv and Env are bootstrap; the functions are called per use.