plugins

package
v0.7.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 5, 2026 License: MIT Imports: 46 Imported by: 0

Documentation

Overview

Package plugins ports packages/chord/src/next-node/plugins: the plugins component of Chord v2 (slice 7c, design §21 "Loading plugins and the reload loop"), the plugin directories found, each facet built, evaluated, loaded and reloaded per plugin, watched and rolled out.

Upstream builds a facet into a script with esbuild and evaluates it on Node's `vm` in the process of the host that loads it. Go builds with esbuild's own Go API (BuildFacet: a 1:1 port of build.ts), and has no JavaScript runtime in process (AGENTS.md), so the host that loads the plugins is Pi's own chord host in a Node subprocess, started by a PluginRuntime (a NodeHost) whose host is the child of a Go host over the chord link wire, as a forked worker is of its server (the app routes the NodeHost's kind to NodeHost.Link, or accepts it); Pi's evaluate.ts, stack.ts and runtime.ts run there, vendored verbatim under node/ (docs/plan/chord-plugins-node-evaluator.md).

The files (plugins_<stem>.go) mirror upstream's: build.ts is plugins_build.go (and plugins_sourcemap.go, the inline source maps stack.ts also reads), discover.ts, install.ts and watcher.ts are ported, plugins.ts is the Plugins component, runtime.ts is the Go client of the subprocess (plugins_runtime.go, plugins_node.go); evaluate.ts and stack.ts are the subprocess's. Nothing in the package waits on the loop: the watcher's methods run in the host's sections, and the file events of its watches enter the loop through the batch's section (state.Batch.UseSection) as a timer or a link read does.

Index

Constants

View Source
const SocketOwnerEnv = "PIG_CHORD_SOCKET_OWNER"

defaultReadyTimeout is how long Load waits for the process to start and connect (chord/host's loader waits as long). SocketOwnerEnv names the environment variable that tells a process who started it, for the directories its NodeHosts make: the process that starts a worker sets it to a NewSocketOwner and removes what the worker left (RemoveSocketDirsOf); the worker takes it out of its environment at start (AdoptSocketOwner).

Variables

View Source
var ErrSyncCallback = errors.New("plugins: a synchronous callback cannot call into its NodeHost")

ErrSyncCallback is the error of a NodeHost call made from a synchronous callback: the process's host thread is blocked until the callback returns, so the call would wait for ever. It is returned (never waited for) by every NodeHost operation that waits for the process, when its context is the one a synchronous callback received.

Functions

func AdoptSocketOwner

func AdoptSocketOwner()

AdoptSocketOwner takes SocketOwnerEnv out of the environment, as the worker's channel variable is taken: this process's NodeHosts name their directories by it from now on, and no program this process starts sees it.

func InstallDependencies

func InstallDependencies(plugin Plugin, ctx context.Context, options InstallOptions) error

InstallDependencies installs plugin's `dependencies` into its own `node_modules` (`npm install --omit=dev --legacy-peer-deps --ignore-scripts --no-audit --no-fund` in its real directory). It blocks until the command ends and returns an error carrying the command's output when it fails, and the abort's reason (context.Cause) when ctx aborts (the command is sent SIGTERM, as Node's `spawn` signal does).

func NewSocketOwner

func NewSocketOwner() string

NewSocketOwner is a value for SocketOwnerEnv unique to one start: 8 characters, short because it is part of every socket path, which a unix address holds only 104 bytes of (macOS).

func RemoveSocketDirsOf

func RemoveSocketDirsOf(tempDir, owner string)

RemoveSocketDirsOf removes the socket directories in tempDir that a process started with SocketOwnerEnv=owner left behind, once that process has ended. A process killed, or exiting, while its NodeHost was still starting cannot clean up after itself (no one closes that host): its parent does. The owner is unique to one start, so a directory of another process (one that shares the temporary directory from another pid namespace) is never named.

Types

type Asset

type Asset struct {
	Name  string
	Bytes []byte
}

Asset is a browser build's asset, by content-hashed file name.

type BridgedFacet

type BridgedFacet struct {
	// Name is the Go core facet's name: its plugin name on the Go host, and the proxy's on the Node host.
	Name string
	// Host is the Go host that loaded the facet: the objects of Contracts live there, and a proxy object holds one by
	// `Host.Get`.
	Host host.Host
	// Contracts are the local types the Go core facet serves that plugins use. A contract with a keyed member is refused: a
	// keyed member is the document handle of its object and cannot be forwarded.
	Contracts []*contracts.Contract
	// Failed is why the Go core facet failed on the Go host, nil when it loaded or reloaded. The proxy is then shipped as a
	// failed facet: the Node host keeps the proxy it has loaded (none at startup) and, by Pi's closure rule, the plugins that
	// need what its new code serves keep their old code.
	Failed error
}

BridgedFacet is a Go core facet whose local types plugins on a Node host may use: the Node host loads a proxy facet named Name that serves Contracts, and its objects forward to Host's own (RuntimeLoad.Bridged).

type BuildInput

type BuildInput struct {
	File string
	Hash string
}

BuildInput is a file a build read, with the hash of the bytes it handed its tool (the watcher's inputs).

type BuildOptions

type BuildOptions struct {
	// HostPackages are the host packages' specifiers (`@earendil-works/chord/next`, `react`, `react/jsx-runtime`,
	// `@app/core`): external.
	HostPackages []string
	// Platform is where the script runs.
	Platform Platform
	// Plugins are the discovered plugins: a file of one of them may be imported (a copy).
	Plugins []Plugin
	// PublicPath is where the app serves a browser build's assets: the URL prefix their imports evaluate to (nil: `/`).
	PublicPath *string
}

BuildOptions is what a build takes besides the plugin and the kind.

type Built

type Built struct {
	Facets  []BuiltFacet
	Plugins map[string]Plugin
	Removed []string
	// contains filtered or unexported fields
}

Built is a build of some plugins, plain data (a copy of it reloads the same): every facet of each, the plugins as its discovery saw them by name (what a reload evaluates with), and the plugins no longer discovered. Only the pointer Build returned clears marks when rolled out: a copy carries none.

type BuiltFacet

type BuiltFacet struct {
	Kind   BuiltKind
	Plugin string
	Facet  string
	// Source is the script: the builder's output, whatever its Evaluator takes.
	Source string
	// Digest is a hash of Source: the same on every machine and checkout for the same files.
	Digest string
	// Inputs is every plugin file it read (its own and copied ones; not packages), absolute paths, with their hashes.
	Inputs []BuildInput
	// Copied is the files of other plugins inlined into it, absolute paths.
	Copied []string
	// CSS is a browser build's stylesheet, when it imports CSS.
	CSS *CSS
	// Assets is a browser build's assets, by content-hashed file name.
	Assets []Asset
	// Errors is why the build failed, each naming its file and line where there is one.
	Errors []string
}

BuiltFacet is a built facet: its script and what it used, or the build's errors. Plain data. Plugin and Facet (the kind) are set for both kinds; Source, Digest, Inputs, Copied, CSS and Assets for BuiltFacetScript, Errors for BuiltFacetFailed.

func BuildFacet

func BuildFacet(plugin Plugin, kind string, options BuildOptions, ctx context.Context) (BuiltFacet, error)

BuildFacet builds plugin's facet of kind (one esbuild build from cold): a script wrapped as `(function (require, module, exports) { … })`, or the errors that refuse it (an import the rules of §2 do not admit, a syntax error, `import.meta`, `__dirname`, `__filename`, a literal dynamic `import("x")`, a top-level `await`), each naming its file and line. A computed `import(x)` becomes a call of the script's `require`. It returns an error only when ctx is aborted before the build (its cause, as `throwIfAborted`), or when the script's own source map cannot be read.

type BuiltKind

type BuiltKind string

BuiltKind tags a BuiltFacet.

const (
	// BuiltFacetScript is a build that made a script.
	BuiltFacetScript BuiltKind = "script"
	// BuiltFacetFailed is a build that failed, with its errors.
	BuiltFacetFailed BuiltKind = "failed"
)

type CSS

type CSS struct {
	Name string
	Text string
}

CSS is a browser build's stylesheet.

type ContractData

type ContractData struct {
	Name        string            `json:"name"`
	Version     int               `json:"version"`
	Fingerprint string            `json:"fingerprint"`
	Normal      *delta.JsonObject `json:"normal"`
}

ContractData is a contract as it crosses to a Node host: its name, its current version and fingerprint, and Normal, the normal form the fingerprint hashes (`{ kind, params, members }` for a local contract, `{ kind, params, variants }` for a remote one: contracts.NormalForm). The Node host rebuilds the contract from it and refuses one that does not hash to Fingerprint. A remote contract's `HostAddress` is a Go function and does not cross; the history of an evolved contract does not either (the version a holder holds is the one that crosses).

func ContractDataOf

func ContractDataOf(contract *contracts.Contract) (*ContractData, error)

ContractDataOf is the ContractData of contract's current version. It fails for a value that is not a declared contract (contracts.ContractDataOf).

type ContractDescription

type ContractDescription struct {
	Kind        contracts.Kind `json:"kind"`
	Name        string         `json:"name"`
	Version     int            `json:"version"`
	Fingerprint string         `json:"fingerprint"`
}

ContractDescription is a contract as the Node host rebuilt it from ContractData: its kind, name, version and fingerprint as that host's Pi computes them.

type CoreFacet

type CoreFacet struct {
	Name   string
	Module string
}

CoreFacet is one of the app's own facets that is JavaScript: its name (a plugin name) and the module whose default export is the facet, a file the subprocess imports.

type Discovery

type Discovery struct {
	Plugins  []Plugin
	Problems []DiscoveryProblem
}

Discovery is what discovery found: the plugins by name, and the directories it reported.

func DiscoverPlugins

func DiscoverPlugins(roots []string, options DiscoveryOptions, ctx context.Context) (Discovery, error)

DiscoverPlugins lists every root one level deep: a directory with a valid `package.json` is a plugin (a missing root is empty). A directory without a `package.json`, with an invalid one or an invalid name, with two files of one facet kind, or whose name an earlier root (or an earlier directory of the same root) has already, is reported. Plugins sorted by name. It returns the abort's reason (context.Cause) when ctx aborts, and the error of a directory it cannot read.

type DiscoveryOptions

type DiscoveryOptions struct {
	Kinds []string
}

DiscoveryOptions is what discovery takes besides the roots: the app's facet kinds (`["server", "worker", "web", "tui"]`).

type DiscoveryProblem

type DiscoveryProblem struct {
	Path  string
	Error string
}

DiscoveryProblem is a directory under a root that is not a plugin, or a plugin left out, and why.

type DiscoverySnapshot

type DiscoverySnapshot struct {
	Set      int
	Manifest int
}

DiscoverySnapshot is the generations of the latest change of the plugin set and of a `package.json` when a discovery started.

type GoExport

type GoExport struct {
	// contains filtered or unexported fields
}

GoExport is one export of a GoPackage: a Go function (GoFunc) or a JSON value (GoValue). The zero value is no export.

func GoFunc

func GoFunc(fn GoFunction) GoExport

GoFunc is an export that is a Go function. In the plugin it is an asynchronous function `(args, context?) => Promise`: it calls fn with the JSON `args` and resolves with fn's JSON result; a Go error rejects it (a *data.ChordError as a ChordError, any other as an Error with its message). `context` is the request's chord Context or an AbortSignal: its abort rejects the call with the abort's reason at once and cancels fn's context. Calls run concurrently, each on a goroutine of its own; fn's context also ends when the process does.

func GoValue

func GoValue(value delta.JsonValue) GoExport

GoValue is an export that is a JSON value: the plugin reads it as the value itself, parsed once per process. Its Go value follows json.Marshal (a *delta.JsonObject keeps its key order).

type GoFunction

type GoFunction func(ctx context.Context, args delta.JsonValue) (delta.JsonValue, error)

GoFunction is a function of the Go app that a plugin calls: its JSON arguments in, its JSON result out. ctx ends when the plugin's request aborts (the call's context argument on the Node side), and when the process ends.

type GoPackage

type GoPackage struct {
	Specifier string
	Exports   map[string]GoExport
}

GoPackage is a host package the Go app offers: the plugins' `require(Specifier)` (and an app module's `import`) answers a module with these exports. Specifier is the name the plugins' builds keep external (BuildOptions.HostPackages, a plugin's `chord.external`).

type HostPackages

type HostPackages map[string]string

HostPackages are the host packages a script's imports are answered with, by specifier: the file of the module (an absolute path) the subprocess imports once, and shares between the scripts of its host. `@earendil-works/chord/next` is always Pi's own, the subprocess's; naming it here replaces nothing.

type InstallOptions

type InstallOptions struct {
	NpmCommand []string
}

InstallOptions is what InstallDependencies takes besides the plugin: the npm command (`["npm"]` by default, e.g. `["pnpm"]`). A nil NpmCommand is upstream's undefined (npm); an empty, non-nil one is refused.

type KindDefinition

type KindDefinition struct {
	Name         string
	Platform     Platform
	HostPackages []string
}

KindDefinition is one facet kind: Node or browser builds, and the host packages (by specifier) its builds keep external. Name is the kind (`server`, `worker`, `web`): upstream's record key, which Go keeps as a field so the kinds stay in the order the app gave them.

type MarkSnapshot

type MarkSnapshot struct {
	Plugins map[string]int
	Names   []string
}

MarkSnapshot is the generation of each dirty plugin's latest change when a build started: what its rollout may clear. Plugins is by name (a plugin with no mark is 0), Names its keys in the order they were asked, as the Map they are.

type NodeError

type NodeError struct {
	Name    string
	Message string
	Stack   string
	Cause   *NodeError
	Chord   *data.ChordError
}

NodeError is an error thrown in a subprocess, as data: its message, its name, and its stack with the positions of plugin scripts mapped through their source maps (`notes/helper.ts:3:8`), as `mapStack` gives it. Cause is its `cause` (the chain goes on through the causes' own; a cause that is no Error is an error of its string). Chord is set when it is a ChordError (`aborted` after a reload's commit carries `details.committed`). Unwrap gives both, so errors.As finds a *data.ChordError anywhere in the chain.

func (*NodeError) Error

func (err *NodeError) Error() string

func (*NodeError) Unwrap

func (err *NodeError) Unwrap() []error

Unwrap returns the ChordError the subprocess threw, if it threw one, and the error's cause, if it has one.

type NodeExit

type NodeExit struct {
	// Err is the process's exit error (an *exec.ExitError), nil for an exit with code 0.
	Err error
	// Fatal is the uncaught exception or rejection the process reported before it ended, nil if it reported none.
	Fatal *NodeError
	// Stderr is the last of the process's stderr, which includes what it printed for a Fatal.
	Stderr string
	// Code is the process's exit code: 1 after an uncaught exception, n after `process.exit(n)`, -1 when a signal ended it.
	Code int
	// Signal is Node's name of the signal that ended the process (`SIGKILL`), empty when it exited.
	Signal string
	// Reason is why the Go side ended the process, when it did: its link closed (a heartbeat that found the host silent, a
	// closed link) and it did not exit within the stop grace. Empty when the process ended by itself.
	Reason string
}

pig additive (D223): how the subprocess ended; Pi's in-process plugins end the app instead. NodeExit is why a subprocess ended before it was asked to: its exit, what it reported as fatal, and the last of its stderr. Pi's plugins run in the app's process, where an uncaught exception ends the app with code 1, `process.exit(n)` with n and a loop stalls it; here they end or stall only the subprocess, and the NodeExit is what Node's `exit` event (`code`, `signal`) gives for it.

func (*NodeExit) Error

func (exit *NodeExit) Error() string

func (*NodeExit) Unwrap

func (exit *NodeExit) Unwrap() []error

Unwrap returns the exit error and the fatal error.

type NodeHost

type NodeHost struct {
	// contains filtered or unexported fields
}

NodeHost is a Node subprocess whose host loaded the plugins of one kind: Pi's chord host with Pi's plugin runtime, linked to a Go host. It is never restarted by itself: when the process ends, the link closes, every pending request fails with a *NodeExit and Exited closes.

func (*NodeHost) Activity

func (h *NodeHost) Activity() host.ActivityCounts

Activity is the keep-alive counts of the process's host as last reported (NodeOptions.OnActivity gives each change).

func (*NodeHost) Close

func (h *NodeHost) Close(ctx context.Context) error

Close asks the process to close its host and exit, and waits for it. A process that does not answer within the stop grace is killed, as is one still running its close after its closeTimeout and the grace, or when ctx ends. It returns the exit that was a crash, if the process ended that way before.

func (*NodeHost) Describe

func (h *NodeHost) Describe(contract *contracts.Contract, ctx context.Context) (ContractDescription, error)

Describe is what the Node host makes of contract: the contract it rebuilds from ContractDataOf(contract), described with the fingerprint its own Pi hashes the rebuilt declaration to. It equals contract's own when the two sides agree on the normal form; the Node host's `get` refuses a contract whose rebuilt fingerprint differs. It blocks until the answer.

func (*NodeHost) Err

func (h *NodeHost) Err() error

Err is why the process ended before Close asked it to (a *NodeExit), or nil.

func (*NodeHost) Exited

func (h *NodeHost) Exited() <-chan struct{}

Exited is closed when the process ended.

func (*NodeHost) ExtractTelemetry

func (h *NodeHost) ExtractTelemetry(carrier telemetry.TelemetryCarrier) telemetry.TelemetryContext

ExtractTelemetry is `propagation.extract(carrier)` of the app's adapter for a carrier a plugin made (its `inject` of a span of this process's telemetry, which this resolves to the adapter's own carrier of that span) or any carrier of the adapter's: the parent whose spans continue that trace. It is what a Go function that a plugin calls with a carrier uses to continue the plugin's trace. The no-op parent is returned when the host has no adapter or the adapter propagates nothing, and the adapter's root for a carrier it does not know (as the adapter's own extract does).

func (*NodeHost) Get

func (h *NodeHost) Get(ctx context.Context, contract *contracts.Contract, params any, options host.GetOptions) (*NodeReference, error)

Get is the Go form of Pi's `server.get(Index)` for a local type the Node host's plugins serve: a reference once its object is live, as `host.Get` of a Go host. It sends `get` with ContractDataOf(contract); the Node host declares the same contract (a different fingerprint is refused) and runs `host.get` with the request's context, so what it does while the object is not live is Pi's: it waits while the reference is `loading`, `failed { transient }` or `failed { waiting }` until options.ReadyTimeout (the host's `readyTimeout` by default), then fails with the last cause, which crosses as a *NodeError wrapping the ChordError (`unavailable`, `aborted`, ...); a broken failure fails at once. It blocks, and aborting ctx aborts the request there. Only a local contract is held this way: a remote one is held through a Go host that routes its kind to the node (NodeHost.Link). params is the type's params object (a *delta.JsonObject).

func (h *NodeHost) Link() link.Link

Link is the link to the process's host, the other end of the one that host connects to its parent (Pi's worker `connect`s a link to its server). The app attaches it to a Go host as the server of a worker does: `Route` of the host's kind, whose Resolve returns it (with the principal the app stamps the host with), or `Accept` with LinkOptions{Principal, Trusted: true}. It is one link, made when the process connected. It is attached lazily: the first use of it by the Go host (OnMessage or Send) makes the process's host connect its end and start its heartbeat (RuntimeLoad.Heartbeat), which closes a link silent for its window and ends the process with it. A NodeHost whose link nothing uses has no link and no heartbeat: it stays alive until Close, and the link can be attached at any later time. A closed link, from either end, ends the process's channel and the process.

func (*NodeHost) ReloadOptions

func (h *NodeHost) ReloadOptions(report host.ReloadReport) host.ReloadOptions

ReloadOptions are the plugins this Node host's reload report says failed (their new code did not build or evaluate, or the closure kept them old), as the options a Go host's Reload takes: its facets named like those plugins (the Go side's own form of what the plugins serve) keep their loaded code too, and the Go host's closure rule keeps every facet that needs what their new code would have served old. The Go core facets of RuntimeLoad.Bridged are not in it: they failed on the Go host, not on the Node host.

func (*NodeHost) Revoke

func (h *NodeHost) Revoke(revocation delta.JsonValue, ctx context.Context) error

Revoke is `host.revoke(revocation)` on the process's host (Pi's `Host.revoke`, which Go's host.Host.Revoke mirrors): every remote object of the host whose principal its `revokedBy` (RuntimeLoad.RevokedBy) says the revocation revokes ends at once, whatever references remain (holders get `forbidden`, running calls are aborted, an activation is abandoned), and its child links close. A Go host that routes to this NodeHost sends the revocation down the route itself (host.Host.Revoke), which the process's host applies with its own `revokedBy`; Revoke is for the one that did not (a host that accepted the link as a child's, or an app that revokes a worker directly). It returns once the process applied the revocation, and the error of its `revokedBy` when that failed. It must not be called from a synchronous callback (ErrSyncCallback).

func (*NodeHost) SetBridged

func (h *NodeHost) SetBridged(facets []BridgedFacet) error

SetBridged replaces the Go core facets the next reloads ship as proxies (RuntimeLoad.Bridged is the first set): the app calls it when its Go core facets changed or failed on the Go host, then reloads the plugins (Plugins.Reload, PluginRuntime.Reload) so the Node host applies the set with the plugins' own, by Pi's closure rule. A facet whose contracts did not change is the same proxy (its script's digest is the same). A facet that failed without a Host or Contracts of its own keeps the ones it had: the proxy the Node host keeps loaded goes on forwarding to the Go host's old code. It fails for facets that cannot be bridged (BridgedFacet).

type NodeHostError

type NodeHostError struct {
	Kind string
	// Object is the object's type, with its params key when it has params.
	Object  string
	Failure data.FailureCause
	Err     error
	// Native and Routine are where a reported part sits on its object (close_failed, close_timeout: Native;
	// routine_failed: Routine).
	Native  host.Position
	Routine host.Position
	// Address is a failed route's address (resolve_failed).
	Address string
	// Member is the exposed state member of a derive_failed, if it is one.
	Member string
	// Job and ID name a job's run (job_failed, job_finish_failed, job_interrupted, job_dropped); Restarts is a
	// job_interrupted's count.
	Job      string
	ID       string
	Restarts int
	// WaitingFor, State and Since are a parked_long's: its first blocker, that blocker's state ("loading" or "failed") and
	// since when (Date.now() time).
	WaitingFor string
	State      string
	Since      int64
	// Registry names the registry of a registry error; Message and Plugins are a registry_conflict's, Plugin, Type (empty
	// for a facet) and Version a contribution's.
	Registry string
	Message  string
	Plugins  []string
	Plugin   string
	Type     string
	Version  int
}

NodeHostError is what the host of a subprocess reported in the background (upstream's `HostError`, which Go's host.HostError mirrors): the Kind and the fields that kind lists, as typed values; the others are zero. Err is the error of the kinds that carry a plain one (a *NodeError, its Cause chain included), Failure the cause of the kinds that carry a failure (activation_failed, object_failed). Kind is the host's: the kinds of host.HostError, of the batch (listener_error, delivery_failed) and of the registries (registry_conflict, registry_failed, contribution_failed, contribution_refused).

type NodeKeyed

type NodeKeyed struct {
	// contains filtered or unexported fields
}

NodeKeyed is a keyed member of a NodeReference.

func (*NodeKeyed) Get

func (k *NodeKeyed) Get(ctx context.Context, key string) (value delta.JsonValue, found bool, err error)

Get is the document at key, found false when absent: a round trip. It blocks, and aborting ctx aborts it there.

func (*NodeKeyed) Pin

func (k *NodeKeyed) Pin(ctx context.Context, key string) (*NodePinnedKey, error)

Pin pins the document at key of the member on the Node host and returns it, nil when there is no such document: it blocks until the Node host pinned, aborting ctx aborts it there. The pin is released by its Release and with the reference.

func (*NodeKeyed) Watch

Watch is a watch of the document at key, nil when absent; it ends with the reference.

type NodeMember

type NodeMember struct {
	// contains filtered or unexported fields
}

NodeMember is a state member of a NodeReference.

func (*NodeMember) Value

func (m *NodeMember) Value() (delta.JsonValue, error)

Value is the member's current value as the Node host's reference reads it: a round trip. The error is what the getter there throws (`not_ready` before the first binding, the failure's error, `released`), as a *NodeError wrapping the ChordError.

func (*NodeMember) Watch

Watch is a watch of the member that ends when the reference is released (`ended`), when the Node host ends it (its end is the Node host's: `ended`, `failed`), when it is stopped, or when ctx aborts. It blocks until the Node host acquired the watch, which starts at the current value and then gets one frame per change, in order.

type NodeOptions

type NodeOptions struct {
	// ExecArgv are Node's own flags for the subprocess (`--max-old-space-size=...`), after the one this component passes
	// (`--disable-warning=ExperimentalWarning`).
	ExecArgv []string
	// CacheDir is where the embedded runtime is extracted (once per content): `pig` under os.UserCacheDir() by default.
	CacheDir string
	// ReadyTimeout is how long finding Node (each `--version` probe also stops at 10 s) and, after it, starting the process may
	// each take, the second until it is connected and said it started; 30 s by default.
	ReadyTimeout *time.Duration
	// OnOutput gets what the process writes. Without it, the process's stdout and stderr go to this process's own.
	OnOutput func(stream NodeStream, chunk []byte)
	// OnActivity gets the keep-alive counts of the process's host (`host.activity`): its objects' references, calls, jobs and
	// lingering objects, once at the start and at each change. It runs on the goroutine that reads the channel and must not
	// block. A supervisor that stops idle workers reads them here: a Go host that routes to a NodeHost counts none of its
	// objects' work.
	OnActivity func(host.ActivityCounts)
	// OnError gets what the process's host reports in the background (a failed activation, a protocol error, a close past its
	// timeout, ...): upstream's `onError`, with every field of the kind (NodeHostError).
	OnError func(NodeHostError)
	// Command makes the command that starts a program (Node, to find its version and to run the subprocess): os/exec's
	// CommandContext by default. A platform that cannot execute a file in the app's data directory starts it through its own
	// loader here; the chord packages know no platform.
	Command func(ctx context.Context, name string, args ...string) *exec.Cmd
	// pig additive (D223): the stop grace and the spare process (KeepWarm) are the embedding program's settings.
	// StopGrace is how long a process whose channel ended, or that was told to close, gets to exit before it is killed (Pi's
	// workers wait GRACE = 5 s: workers.ts); 5 s by default. A process stalled in a loop cannot read its channel, so this is how
	// long the Go side waits after it closed the link (a heartbeat that found the host silent, for one) before it kills it. A
	// process that answers a Close with `closing` is running its host's close: Pi ends every object at once, then drains and
	// closes them one by one, each step bounded by `closeTimeout`, and the app's store closes after it. Close then waits one
	// closeTimeout (RuntimeLoad.CloseTimeout or 30 s) plus this grace for the exit, not one per object.
	StopGrace *time.Duration
	// KeepWarm makes the PluginRuntime start a spare process in the background each time a Load has taken its process, so
	// the next Load finds one up (PluginRuntime.Warm). Close ends it.
	KeepWarm bool
}

NodeOptions is how Node subprocesses are started. The Node they run on is found by discoverNode (additive record D221): PIG_NODE, then `node` on PATH, then the install locations of version managers and installers (see discoverNode for the order and the list), the first that is Node 22.19.0 or newer.

type NodePinnedKey

type NodePinnedKey struct {
	// contains filtered or unexported fields
}

NodePinnedKey is a key of a keyed member held on the Node host (Pi's `pinKey`): the document at the key stays loaded and followed across the object's attempts until Release. Additive Go API.

func (*NodePinnedKey) Get

Get is the pinned document's current value.

func (*NodePinnedKey) Release

func (p *NodePinnedKey) Release()

Release releases the pin (idempotent): a notification, lost with a process that ended.

type NodeReference

type NodeReference struct {
	// contains filtered or unexported fields
}

NodeReference is a reference to an object of a local type that the plugins of a Node host serve: host.Reference's shape for the members a local type has (`Member`, `Keyed`, `Call`, `State`, `Release`), held through the Node host's own `host.get`. Additive Go API: it is what the app's Go code uses where Pi's app `get`s a local type in process.

Its reads, calls and watches are round trips to the Node host (a state member's `Value` is a read of the Node host's current value). Values cross as JSON. After Release, and after the process ended, they fail.

func (*NodeReference) Call

func (r *NodeReference) Call(method string, args delta.JsonValue, ctx context.Context) (delta.JsonValue, error)

Call calls the method of the object (nil args for none) and blocks until it answers. It runs while the reference is live and parks while it is loading (the Node host's reference parks it up to the `readyTimeout` of the Get), as a call on a reference does; aborting ctx aborts it there. A method the contract does not have is an error; after Release, or when the process ended, it fails (with the exit, for a process that ended).

func (*NodeReference) Keyed

func (r *NodeReference) Keyed(name string) *NodeKeyed

Keyed is the keyed member name, nil when the contract has none.

func (*NodeReference) Member

func (r *NodeReference) Member(name string) *NodeMember

Member is the state member name, nil when the contract has none.

func (*NodeReference) Release

func (r *NodeReference) Release()

Release releases the reference (idempotent): the Node host releases its own, so its object may end when it was the last; the watches of the reference end `ended`. It does not wait for the Node host, whose own release follows every request sent before it.

func (*NodeReference) State

State is the reference's state: `loading`, `live`, `failed` or `ended`, as the Node host's reference shows it; Value is the latest the Node host sent (the changes arrive in order on the control channel) and a watch gets every change. It is `loading` once the process ended and `ended` (`released`) after Release.

type NodeStream

type NodeStream string

NodeStream is the stream of a process's output a chunk came from.

const (
	// NodeStdout is the process's standard output.
	NodeStdout NodeStream = "stdout"
	// NodeStderr is the process's standard error.
	NodeStderr NodeStream = "stderr"
)

type Platform

type Platform string

Platform is where a built script runs: Node (a server, a worker, a TUI) or a browser (a page).

const (
	// PlatformNode is a build for Node.
	PlatformNode Platform = "node"
	// PlatformBrowser is a build for a browser.
	PlatformBrowser Platform = "browser"
)

type Plugin

type Plugin struct {
	Name string
	// Dir is its directory's real path (symlinks followed).
	Dir string
	// Facets are its facet files by kind, absolute paths; none for a contracts-only plugin.
	Facets map[string]string
	// Dependencies are the packages in its `dependencies`: bundled into the facets that import them.
	Dependencies []string
	// External are the packages in its `chord.external`: kept external in Node builds, loaded from its `node_modules`.
	External []string
}

Plugin is a plugin on disk: its name (its `package.json` name), its real directory, its facet files and npm packages. Every list is non-nil, as upstream's arrays are. Facets has one entry per kind the plugin has a file for; a kind order is the order of DiscoveryOptions.Kinds.

type PluginRuntime

type PluginRuntime struct {
	// contains filtered or unexported fields
}

PluginRuntime starts the Node subprocesses that host plugins, and loads and reloads them with complete sets of shipped scripts. It is Pi's `createPluginRuntime()`.

func NewPluginRuntime

func NewPluginRuntime(node NodeOptions) *PluginRuntime

NewPluginRuntime is upstream's `createPluginRuntime()`: how its subprocesses are started.

func (*PluginRuntime) Close

func (r *PluginRuntime) Close(ctx context.Context) error

Close closes the runtime's spare process, and ends and joins the starts it made on its own. The hosts that Loads returned are their callers': Close does not close them. It waits for the spare's process to end and returns the error of closing it; when ctx ends first it returns the cause, and the process, a start still under way included, is killed as soon as it is up. After Close, Load and Warm return an error. It is safe to call more than once.

func (*PluginRuntime) Load

Load starts a Node subprocess whose host evaluates options.Scripts (the complete set of options.Kind) with options.Modules and loads them with options.Core. A plugin whose build or evaluation failed is left out and reported with the host's report (the failures of the scripts first), each error a *NodeError whose Stack names the plugin's files and lines. It returns the host's error as it is: the subprocess is stopped then.

The host is the child of the Go side, as a forked worker is of its server (next-node/workers/main.ts): it connects the link to its parent when the Go side first uses the other end. The app attaches that end, NodeHost.Link, to a Go host as Pi's app does: a route of the host's kind (`host.Route`, whose Resolve returns it) or `host.Accept`, at any time after Load returns: the heartbeat (RuntimeLoad.Heartbeat) runs from that first use. A NodeHost needs no Go host to load, so one Go host can own any number of them, and a Go host that has a parent of its own can load plugins.

func (*PluginRuntime) Reload

func (r *PluginRuntime) Reload(h *NodeHost, scripts []PluginScript, ctx context.Context) (host.ReloadReport, error)

Reload reloads h (loaded by Load) with scripts, the complete set now, after its earlier reloads: unchanged plugins (the same digest) as the same facets, new scripts evaluated, a plugin whose build or evaluation failed passed as its loaded facet (an added one left out; both given to the host's closure rule), loaded plugins not in scripts left out. What the host committed is the loaded set, also when the reload returns `aborted` after its commit (a *NodeError wrapping the ChordError, whose details.committed says what the commit applied). It blocks until the report, and then until the frames the reload caused have been delivered to the Go host the link is attached to (NodeHost.Link): the process wrote them before its reply, the reader handed each to the host before it decoded the reply, and the host's deliveries have entered their listeners when Reload returns, as a host's own Reload leaves its watchers' listeners entered. ctx ends that last wait, not the report.

func (*PluginRuntime) Warm

func (r *PluginRuntime) Warm(ctx context.Context) error

Warm starts a spare process, if the runtime has none, and returns once it is up to `started` with Pi's modules imported (NodeOptions.ReadyTimeout bounds the start, ctx ends it). The next Load takes it, so the start is off that Load's critical path; a Load that finds the spare still starting waits for it, which is no longer than starting its own. There is at most one spare: Warm with one (or one starting) waits for it and returns nil. A failed start returns its error and leaves no spare. The spare is the runtime's until a Load takes it: Close closes it. A Warm after Close returns an error. pig additive (D223): a spare process started ahead of a Load.

type PluginScript

type PluginScript struct {
	Plugin Plugin
	Built  BuiltFacet
}

PluginScript is one plugin's facet of a kind as shipped to a host (plain data): its build and its plugin record.

type Plugins

type Plugins struct {
	// contains filtered or unexported fields
}

Plugins is the plugins component.

func CreatePlugins

func CreatePlugins(options PluginsOptions, _ context.Context) *Plugins

CreatePlugins creates the plugins component (and its plugin runtime) and starts the watcher. Discovery runs at first use.

func (*Plugins) Build

func (c *Plugins) Build(names []string, ctx context.Context) (*Built, error)

Build builds every facet of names (and of plugins discovered since the last build; discovery runs again first when the plugin set changed): plain data. It changes no current set: a rollout ships it.

func (*Plugins) Close

func (c *Plugins) Close()

Close stops the watcher, closes the spare Node process of its runtime (PluginRuntime.Close) and waits for what it started. The hosts it loaded are their owners': it closes none. The spare's close error is dropped: its process never loaded anything, and Close, as Pi's `plugins.close()`, returns nothing.

func (*Plugins) Current

func (c *Plugins) Current(kind string) ([]PluginScript, bool)

Current is kind's current set now, without blocking (false before its first use): a rollout step reads it as the rollout applied its build, before another rollout can change it.

func (*Plugins) Discovery

func (c *Plugins) Discovery() Discovery

Discovery is the plugins of the last discovery.

func (*Plugins) InstallDependencies

func (c *Plugins) InstallDependencies(name string, ctx context.Context) error

InstallDependencies installs plugin name's dependencies (InstallDependencies): its install mark clears, and it is dirty.

func (*Plugins) Load

func (c *Plugins) Load(options PluginsLoad, ctx context.Context) (*NodeHost, host.LoadReport, error)

Load starts a Node subprocess that hosts options.Kind's current set, evaluated with options.Modules, and options.Core (the app's own facets): a plugin whose build or evaluation fails is left out and reported with the host's report. The app attaches the returned host's Link to a Go host (PluginRuntime.Load).

func (*Plugins) Marks

func (c *Plugins) Marks() PluginsState

Marks is the marks now.

func (*Plugins) Reload

func (c *Plugins) Reload(h *NodeHost, built *Built, ctx context.Context) (host.ReloadReport, error)

Reload applies built to the current set of h's kind (loaded by Load) and reloads h with that complete set, after its earlier reloads (PluginRuntime.Reload). The host's report.

func (*Plugins) Rollout

func (c *Plugins) Rollout(steps []RolloutStep, built *Built, ctx context.Context) (RolloutResult, error)

Rollout runs steps in order with built, applying it to the current set of each step's kinds right before that step, and stops at one that returns an error (Stopped); failed plugins and hosts do not stop it. It clears the dirty marks of the built plugins applied on every host of every step (not failed on any, every built kind of theirs shipped by a step) that did not change since the build; given a copy of a build (not the pointer Build returned), it clears none.

func (*Plugins) Scripts

func (c *Plugins) Scripts(kind string, ctx context.Context) ([]PluginScript, error)

Scripts is the complete current set of kind: every discovered plugin's facet of it (built on first use), as the last rollout or reload shipping kind left it; what a worker starts with.

func (*Plugins) State

func (c *Plugins) State() chord.Watchable[delta.JsonValue]

State is the marks, `{ dirty, install, set, base }`, a JSON watchable.

func (*Plugins) Step

func (c *Plugins) Step(name string, h *NodeHost) (RolloutStep, error)

Step is a rollout step named name: Reload(h, built), one host named name.

func (*Plugins) Warm

func (c *Plugins) Warm(ctx context.Context) error

Warm starts a spare Node process for the next Load (PluginRuntime.Warm).

type PluginsError

type PluginsError struct {
	Kind  PluginsErrorKind
	Error error
}

PluginsError is what the component reports.

type PluginsErrorKind

type PluginsErrorKind string

PluginsErrorKind tags a PluginsError: a failure of the batch of the marks (the kinds of state.BatchError) or of the watcher.

const (
	// PluginsListenerError is a watch listener of the marks that failed.
	PluginsListenerError PluginsErrorKind = PluginsErrorKind(state.BatchListenerError)
	// PluginsDeliveryFailed is a frame delivery of the marks that failed.
	PluginsDeliveryFailed PluginsErrorKind = PluginsErrorKind(state.BatchDeliveryFailed)
	// PluginsWatchFailed is the watcher that cannot watch: reported once, after which the app builds on demand.
	PluginsWatchFailed PluginsErrorKind = "watch_failed"
)

type PluginsLoad

type PluginsLoad struct {
	Kind string
	// Core are the app's own facets that are JavaScript (RuntimeLoad.Core); Go facets are Bridged.
	Core []CoreFacet
	// Modules are the host packages the scripts' imports are answered with, by specifier; kept for the host's reloads.
	Modules HostPackages
	// Address, Principal, Peer and Heartbeat are the Node host's address and principal, who the Go host that attaches its
	// link is, and the host's heartbeat (RuntimeLoad).
	Address   *contracts.Address
	Principal data.Principal
	Peer      data.Principal
	Heartbeat *host.HeartbeatOptions
	// CloseTimeout, ReadyTimeout and Compression are the Node host's (RuntimeLoad).
	CloseTimeout *time.Duration
	ReadyTimeout *time.Duration
	Compression  bool
	// Store is the Node host's store (RuntimeLoad).
	Store *StoreOptions
	// GoPackages, Telemetry, PrincipalKey and RevokedBy are the app's callbacks the Node host calls (RuntimeLoad).
	GoPackages   []GoPackage
	Telemetry    telemetry.TelemetryContext
	PrincipalKey *data.PrincipalKeyFunction
	RevokedBy    host.RevokedBy[delta.JsonValue]
	// Bridged are the app's Go core facets whose local types the plugins use (RuntimeLoad.Bridged).
	Bridged []BridgedFacet
}

PluginsLoad is what Plugins.Load takes.

type PluginsOptions

type PluginsOptions struct {
	// Roots are the directories plugins are listed in, in order: the first root's plugin wins a duplicate name.
	Roots []string
	// Kinds are the app's facet kinds (one file per kind in a plugin), each with where its builds run and the host packages
	// they keep external, in the app's order.
	Kinds []KindDefinition
	// Base is the app's base code: a change sets `base` (the app restarts its hosts; Chord restarts nothing).
	Base []string
	// PublicPath is where the app serves browser builds' assets: the URL prefix their imports evaluate to (nil: `/`).
	PublicPath *string
	// Install is how InstallDependencies runs npm.
	Install InstallOptions
	// OnError gets the watcher's failure to watch (reported once) and the marks' watch listeners that failed; it runs in a
	// section of the component's loop and must not block.
	OnError func(PluginsError)
	// Node is how the Node subprocesses that host the plugins are started (docs/plan/chord-plugins-node-evaluator.md).
	Node NodeOptions
}

PluginsOptions is what CreatePlugins takes.

type PluginsState

type PluginsState struct {
	Dirty   []string
	Install []string
	Set     bool
	Base    bool
}

PluginsState is the marks: plugins to rebuild, plugins to install, whether to discover again, whether base code changed. Its value as a JSON watchable is the object `{ dirty, install, set, base }`.

type RolloutHost

type RolloutHost struct {
	Kind   RolloutHostKind
	Host   string
	Report host.ReloadReport
	Error  error
}

RolloutHost is what a rollout step did on one of its hosts (Host named by the step).

type RolloutHostKind

type RolloutHostKind string

RolloutHostKind tags a RolloutHost.

const (
	// RolloutReported is a host that reloaded: Report.
	RolloutReported RolloutHostKind = "report"
	// RolloutStopped is a host that stopped cleanly during the step (it gets the current set at its next start).
	RolloutStopped RolloutHostKind = "stopped"
	// RolloutErrored is a host that crashed or whose reload failed (every plugin of the step's kinds in the build counts as
	// failed there): Error.
	RolloutErrored RolloutHostKind = "error"
)

type RolloutResult

type RolloutResult struct {
	Steps   []RolloutStepResult
	Stopped *StoppedStep
}

RolloutResult is what a rollout did: each step's hosts in order, and the step it stopped at, if it stopped.

type RolloutStep

type RolloutStep struct {
	Name  string
	Kinds []string
	// Reload applies built to the step's hosts and returns once they did. applied is the current set of each of the step's
	// kinds as THIS rollout left it (Pi's step reads `plugins.current(kind)` in the synchronous part that follows its
	// `ship`, before another rollout can change it: no await lies between them there; a Go step cannot read it atomically
	// with the ship, so it is handed over).
	Reload func(built *Built, applied map[string][]PluginScript, ctx context.Context) (StepResult, error)
}

RolloutStep is what a rollout step does: reloads its hosts with the facets of its kinds. An error stops the rollout.

type RolloutStepResult

type RolloutStepResult struct {
	Name  string
	Hosts []RolloutHost
}

RolloutStepResult is one step's hosts, in a RolloutResult.

type RuntimeLoad

type RuntimeLoad struct {
	Kind string
	// Core are the app's own facets that are JavaScript, loaded into the subprocess host before the plugins. An app's Go facet is
	// loaded on a Go host, and its local types reach the plugins through Bridged.
	Core []CoreFacet
	// Scripts is the complete set of Kind.
	Scripts []PluginScript
	// Modules are the host packages the scripts' imports are answered with; kept for the host's reloads.
	Modules HostPackages
	// Address is the subprocess host's address (`createHost`'s `address`), nil for a client: the one a remote type's
	// `hostAddress` places objects at, and the one the app routes to the host's link.
	Address *contracts.Address
	// Principal is who the subprocess host acts as for its own requests (`createHost`'s `principal`); none for a client. The
	// app stamps the host with it when it attaches the link (RouteOptions.Resolve's, LinkOptions').
	Principal data.Principal
	// Peer is who the Go host that attaches the link is, as the subprocess's host stamps it: the `principal` of its
	// `connect` (trusted), as the server's is to a forked worker. Required.
	Peer data.Principal
	// Heartbeat is the subprocess host's heartbeat (`createHost`'s `heartbeat`): it closes a link silent for Close, 30 s by
	// default, its handshake included, and the process with it. It runs for the link from the first use of NodeHost.Link by
	// the Go host; a host nothing attached has no clock.
	Heartbeat *host.HeartbeatOptions
	// CloseTimeout and ReadyTimeout are the subprocess host's `closeTimeout` and `readyTimeout` (host.HostOptions'): how long
	// a native's close and an ending object's running calls are waited for (30 s by default), and how long an activation
	// attempt and a `get` may take until the object is live (60 s). Nil: the host's defaults.
	CloseTimeout *time.Duration
	ReadyTimeout *time.Duration
	// Compression makes the subprocess host compress as Pi's Node hosts do (`createHost({ compression: nodeCompression })`):
	// the JSON parts above 16 KiB it originates are deflated, and the deflated parts it owns inflated in place. The Go
	// host the link is attached to compresses its own parts when it is created with host.HostOptions.Compression
	// (nextnode.NodeCompression): a compressor is a host's, not a link's, and a host without one inflates what it gets.
	Compression bool
	// Store is the subprocess host's store (`createHost`'s `store`); nil: none (StoreOptions).
	Store *StoreOptions
	// GoPackages are the host packages the app offers as Go functions and JSON values (GoPackage): the scripts' imports of
	// each Specifier are answered with a module of asynchronous functions that call the Go functions, and the values. Kept
	// for the host's reloads, as Modules are.
	GoPackages []GoPackage
	// Telemetry is the app's telemetry adapter (`createHost`'s `telemetry`): the subprocess host builds a proxy
	// TelemetryContext whose spans and metrics are ordered data events, and each is replayed into this adapter on the goroutine
	// that reads the channel, so the adapter sees them in the order the host made them. The work of the host (a request that
	// arrives over the link, a reload) runs under the proxy, so a plugin's `ctx.telemetry` spans and measurements land in the
	// adapter, the spans started there roots. `propagation.inject` of a span in the process makes a carrier that
	// NodeHost.ExtractTelemetry resolves to the adapter's own carrier of that span; a carrier that is not of that kind is the
	// adapter's, which extracts it. A span still open when the process ends ends with an error status. Nil: the no-op adapter.
	Telemetry telemetry.TelemetryContext
	// PrincipalKey is the key of a principal in identities (`createHost`'s `principalKey`), so the same function can be the Go
	// host's. The subprocess host calls it synchronously (its worker thread owns a second connection to the Go side): a call
	// takes tens of microseconds (BenchmarkNodeSyncCall), so it keeps the key by the principal's canonical JSON, for the 4096
	// principals it asked for last, and asks once per principal where Pi asks once per principal object. The function must be
	// pure, as Pi's own cache of the key assumes. Nil: canonical JSON, computed in the subprocess.
	PrincipalKey *data.PrincipalKeyFunction
	// RevokedBy is what revokes a principal (`createHost`'s `revokedBy`): the subprocess host calls it synchronously, once per
	// principal object it holds (as Pi's host does, and never memoized), when a revocation reaches it (NodeHost.Revoke, or a Go
	// host's own `Revoke` over a route to this host). A returned error is Pi's throw. Nil: nothing revokes.
	RevokedBy host.RevokedBy[delta.JsonValue]
	// Bridged are the app's own facets that are loaded on a Go host and serve local types the plugins use: each is a facet
	// of the plugins' closure on the subprocess host, a proxy that serves the same contracts and forwards to the Go host's
	// own objects (BridgedFacet), so a plugin that needs such a type binds to it, and Pi's closure rule and reload rules
	// treat the Go core facets as they treat any facet. NodeHost.SetBridged changes the set for the reloads.
	Bridged []BridgedFacet
}

RuntimeLoad is what PluginRuntime.Load takes.

type StepResult

type StepResult struct {
	Hosts []RolloutHost
}

StepResult is what a rollout step's reload returns: one result per host (D12).

type StoppedStep

type StoppedStep struct {
	Step  string
	Error error
}

StoppedStep is the step a rollout stopped at (its reload returned an error), and that error.

type StoreOptions

type StoreOptions struct {
	// Path is the SQLite file, as `openNodeSqliteDatabase`'s `path`: its directory is created, the file is opened in WAL
	// mode, and ":memory:" is a database no later host sees. A relative path is the subprocess's, so the app gives an
	// absolute one.
	Path string
}

pig additive (D223): the Node host's store; Stock PiG gives none unless the embedding program sets it. StoreOptions gives a Node host the store its persisted documents and tables live in: Pi's `createHost({ store })`, where the app opens `openStore(sqliteBackend(await openNodeSqliteDatabase(path)))` over its own SQLite file and closes it after the host. The subprocess opens that store itself, with Pi's own `next/store` over `node:sqlite` (Pi's `sqlite/node.ts`), and closes the host and then the store when the NodeHost is closed.

A store belongs to one open host (Pi: "one open host per store"), so this store is the Node host's own, a different one from the store of the Go host the host's link is attached to: documents and tables a plugin persists are in this file, read by Pi's code in the subprocess, and the Go host's store is not shared over the link. A host loaded without a store keeps Pi's error: a type that declares a persisted document or a table fails to activate with `persist needs a store`.

A failure of the store after it opened (`storage_failed` in Pi) closes the Node host's host and reaches NodeOptions.OnError as a NodeHostError of that kind, once.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL