Documentation
¶
Overview ¶
Package wasm is Ether's hosted plugin runtime: it runs plugin code inside the Ether process, sandboxed, with no ports and no operating-system access. SPEC section 5.2 is the contract.
Three kinds of hosted plugin are supported, all pure Go (CGO_ENABLED=0):
- JavaScript, on QuickJS-NG compiled to WebAssembly and executed by wazero (via github.com/fastschema/qjs; see THIRD_PARTY_NOTICES.md).
- TypeScript, compiled to JavaScript by Ether with esbuild's Go API (see Compile), then run as JavaScript.
- Raw WebAssembly modules, executed directly by wazero.
The QuickJS WASI instance has no preopened filesystem, environment or arguments. Its std and os modules are disabled. Raw modules cannot import WASI. Host capabilities such as HTTP and folders are available only through Ether's permission-checked host calls. Studio previews use the same runtime.
Request ABI (all kinds) ¶
A plugin answers HTTP-shaped JSON requests. The request is {"method","path","query","headers","body"} and the response is {"status","headers","body"}. In JavaScript the plugin defines handle(request) -> response with plain objects (the TypeScript SDK's plugin({...}) router produces this); host services are on the ether global (ether.kv.get, ether.http.fetch, ether.log, ether.abort, ...). Global TextDecoder (UTF-8 labels, {stream, fatal, ignoreBOM}) and TextEncoder.encode are backed by native Go UTF-8 helpers (__ether_text_decode and __ether_text_encode).
Raw WASM ABI ¶
A raw module must export "memory", "alloc" and "handle":
alloc(size i32) -> i32 // guest allocator; host writes input here handle(ptr i32, len i32) -> i64 // returns (outPtr<<32)|outLen
Input and output are the request/response JSON above, UTF-8, in linear memory. The host copies the output before the next call, so the guest may reuse its buffers. Optional hot-path exports:
rank_key(candPtr, candLen, cfgPtr, cfgLen i32) -> i64 cond_match(candPtr, candLen, opPtr, opLen, valPtr, valLen i32) -> i32 (0 or 1)
where candidate/config/value are JSON and operator is a plain string. Allowed imports are only "ether.log(ptr,len)", "ether.abort(code i32)" (never returns) and "ether.host_call(namePtr,nameLen,argsPtr,argsLen,outPtr,outCap i32) -> i32" which runs a host service named like "kv.get" with a JSON argument array, writes the JSON result into outPtr and returns its length, or a negative error code: -1 error, -2 forbidden, -3 governance (unit budget), -4 the result did not fit in outCap (the return is then -(needed) if needed > 4, see ErrHostResultTooBig handling in raw.go). Any other import (WASI included) makes the module refuse to load. On the hot path host_call always returns -2: ranking code may only log and abort.
Governance ¶
Ported from appkit-sandbox semantics. Every call has a wall deadline, a memory cap (default 32 MiB, maximum 64 MiB) and a unit budget. Units are charged per host call from a cost table plus 1 unit per 100us of wall time, checked at host calls and at return. Outcomes are classified as ok, error, timeout, forbidden, aborted or governance; traps and timeouts poison the instance (it is discarded, never reused) and are counted in Stats so the plugin host can apply backoff and disable a failing plugin.
Hot path ¶
RankBatch and CondMatch run ranking steps and rule conditions in separate instances with no host services, a deterministic environment (Date fixed at the epoch, Math.random a seeded generator reset per batch), a default budget of 2ms per candidate and 50ms per batch, neutral results (rank 0, condition false) on exhaustion, and memoization keyed by (plugin id, version, config hash, release id).
The budgets bound the plugin's own time. Handing over the facts is host work and is not charged. Time is read from the monotonic clock and, when that says a budget is spent, from the thread's CPU clock, so time the thread spent descheduled on a busy host is not charged either. A batch stops starting candidates when its budget is spent; what it finished is memoized, so the next batch for the same releases continues from there. Separately, a hard deadline of twice the batch budget (Limits.hotKill) kills an instance whose code stops returning. The budgets are stated in reference-machine time; the engine measures its own speed once per process and scales them (see HotScale), so a slow CPU or a race-detector build does not cost a light plugin its ranking.
These helpers support offline acceptance tests across package boundaries. Runtime callers use the production APIs in the other files.
Index ¶
- Constants
- Variables
- func ChargeUnits(ctx context.Context, units int64) error
- func DataReadBytes(memoryBytes uint64) int64
- func HostResultBytes(memoryBytes uint64) int64
- func HotScale() float64
- func ViewerRequestPath(path string) bool
- func WaitBackground(ctx context.Context, busy func() bool) error
- type Candidate
- type Diagnostic
- type Engine
- type Error
- type Fault
- type Host
- type HostFunc
- type Limits
- type Outcome
- type Plugin
- func (p *Plugin) Budgets() (perCandidate, perBatch time.Duration)
- func (p *Plugin) Call(ctx context.Context, req Request) (Response, error)
- func (p *Plugin) CallDetailed(ctx context.Context, req Request) (Response, Outcome, error)
- func (p *Plugin) Close() error
- func (p *Plugin) CondBatch(ctx context.Context, cands []Candidate, operator string, value json.RawMessage) ([]bool, Outcome, error)
- func (p *Plugin) CondMatch(ctx context.Context, cand Candidate, operator string, value json.RawMessage) (match bool, out Outcome, err error)
- func (p *Plugin) HasCondMatch() bool
- func (p *Plugin) HasHandle() bool
- func (p *Plugin) HasRankKey() bool
- func (p *Plugin) ID() string
- func (p *Plugin) RankBatch(ctx context.Context, cands []Candidate, config json.RawMessage) (scores []int64, out Outcome, err error)
- func (p *Plugin) Stats() Stats
- func (p *Plugin) Version() string
- type Request
- type Response
- type Stats
- type Status
Constants ¶
const ( // DefaultRankPerCandidate and DefaultRankBatch are the hot-path budgets // (SPEC §5.2). A manifest may lower them, never raise them. DefaultRankPerCandidate = 2 * time.Millisecond DefaultRankBatch = 50 * time.Millisecond // WallUnitInterval is the wall time worth one unit. WallUnitInterval = 100 * time.Microsecond // BaseCallUnits is charged once per handle call. BaseCallUnits = 10 // ArchiveReadUnitsPerMember is charged per requested item in archive.read // on top of archive.read's 50-unit base call cost, so large batches pay // proportional governance units instead of dodging per-call budgets. ArchiveReadUnitsPerMember = 2 // ArchiveExtractUnitsPerMiB is charged per started MiB of uncompressed // archive member streamed by archive.extract on top of its base call cost. ArchiveExtractUnitsPerMiB = 10 // MaxMemoryBytes is the hard ceiling for a plugin instance. MaxMemoryBytes = 4 << 30 // MinDataReadBytes and MaxDataReadBytes bound the per-call readBytes grant // derived from memoryBytes: clamp(memoryBytes/4, 8 MiB, 512 MiB). MinDataReadBytes = 8 << 20 MaxDataReadBytes = 512 << 20 // MaxResponseBytes caps a plugin response. MaxResponseBytes = 8 << 20 // MaxFetchBodyBytes caps a direct HTTP response before JSON encoding. MaxFetchBodyBytes = 10 << 20 // MaxHostResultBytes is the baseline host-result floor; actual per-call // host results scale with HostResultBytes(memoryBytes) so base64-encoded // data reads up to readBytes always fit their JSON envelope. MaxHostResultBytes = 4 << 20 )
Variables ¶
var DefaultCosts = map[string]int64{
"log": 1, "runtime.limits": 1, "kv.get": 5, "kv.put": 10, "kv.delete": 5, "kv.list": 10,
"secret.get": 5, "http.fetch": 100, "http.fetchToCache": 100, "http.hash": 100, "cache.read": 20, "archive.list": 50, "archive.open": 50, "archive.read": 50, "archive.extract": 50, "addon.get": 100, "catalog.search": 50, "catalog.get": 20,
"viewers.history": 50, "viewers.setWatchState": 50, "collections.upsert": 50,
"kinds.upsert": 50, "kinds.delete": 50, "kinds.items": 20,
"sources.resolve": 100, "cache.put": 50, "cache.open": 20, "frontend.streamURL": 5,
"folders.hashStart": 20, "folders.hashPoll": 20, "folders.hashCancel": 20, "folders.hash": 20, "folders.list": 20, "folders.stat": 20, "notify.owner": 20, "settings.get": 5,
}
Default host-call unit costs, in the spirit of appkit-sandbox's priced host functions. Wall time adds 1 unit per WallUnitInterval; archive.read adds ArchiveReadUnitsPerMember per requested member; archive.extract additionally charges ArchiveExtractUnitsPerMiB per started MiB streamed.
Functions ¶
func ChargeUnits ¶
ChargeUnits charges additional governance units to the active guest call, returning a StatusGovernance *Fault when the call's UnitBudget is exceeded. Outside a governed wasm call it is a no-op.
func DataReadBytes ¶
DataReadBytes computes the per-call byte ceiling for cache.read, archive.open, and archive.read from a plugin's memory grant: clamp(memoryBytes/4, 8 MiB, 512 MiB), treating 0 as the 32 MiB default.
func HostResultBytes ¶
HostResultBytes returns the maximum JSON host-result envelope for a plugin with the given memory grant, including base64 expansion of DataReadBytes plus 1 MiB for JSON framing.
func HotScale ¶
func HotScale() float64
HotScale is how many times slower than the reference machine this process runs hosted plugin code (1 on the reference machine or faster). The first call measures it; it costs a few milliseconds and runs off the hot path, at plugin load.
func ViewerRequestPath ¶
ViewerRequestPath identifies calls on a viewer's browse/search/playback path. Administrative actions, setup and background jobs use owner-granted limits.
Types ¶
type Candidate ¶
type Candidate struct {
ReleaseID string
Facts json.RawMessage
}
Candidate is one release on the hot path. Facts is the CandidateFacts JSON (SPEC 5.8); ReleaseID keys the memo and must be stable per release.
type Diagnostic ¶
type Diagnostic struct {
File string `json:"file,omitempty"`
Line int `json:"line,omitempty"` // 1-based
Column int `json:"column,omitempty"` // 1-based
Text string `json:"text"`
Severity string `json:"severity"` // "error" or "warning"
}
Diagnostic is one esbuild error or warning, flattened for Studio and install reporting.
func Compile ¶
func Compile(files map[string]string, entry string) (js []byte, sourcemap []byte, diagnostics []Diagnostic)
Compile bundles TypeScript (or JavaScript) plugin sources into one IIFE JavaScript file plus an external source map, using esbuild's Go API. Files maps slash paths ("src/index.ts") to contents; entry is one of those paths. Relative imports resolve inside the map only: a plugin cannot reach the filesystem or node_modules. Exports land on the __plugin global, which the JS runtime normalizes onto globalThis, so `export function handle(...)` and `globalThis.handle = ...` both work. A .js entry that is already a plain script (a prebuilt bundle such as an installed plugin's dist/plugin.js: no import, export or CommonJS module use) is checked and returned unchanged; wrapping it again would hide its globals inside a second IIFE. On failure js is nil and diagnostics carries every error.
type Engine ¶
type Engine struct {
// contains filtered or unexported fields
}
Engine compiles and caches guest code and builds Plugins. It owns the wazero runtimes used by raw modules; JavaScript instances manage their own (QuickJS) modules. Safe for concurrent use.
func (*Engine) LoadJS ¶
LoadJS loads a JavaScript plugin (TypeScript authors go through Compile first). A probe instance is built eagerly so syntax errors and a missing handle surface at load.
func (*Engine) LoadWASM ¶
func (e *Engine) LoadWASM(id, version string, module []byte, host Host, lim Limits) (*Plugin, error)
LoadWASM compiles (and caches) a raw module and returns the Plugin. A probe instance is built eagerly so load-time failures (bad imports, missing exports) surface here, not on first call.
type Error ¶
type Error struct {
Status Status
Message string
Trap bool // the instance trapped (panic in guest) or was killed by a deadline
}
Error is the error type returned by Plugin calls. It carries the Outcome so WS7's health loop can read the status, trap flag and cost from one value.
type Fault ¶
Fault is a classified, non-ambient failure. A Host returns a *Fault to make the call end with that status; Code is one of the Status values other than ok (usually forbidden), or an application code the plugin can catch.
type Host ¶
type Host interface {
CallHost(ctx context.Context, name string, args json.RawMessage) (json.RawMessage, error)
}
Host is the plugin-to-Ether service surface (SPEC 5.1.5). Name is a dotted service method: runtime.limits, kv.get, kv.put, kv.delete, kv.list, secret.get, http.fetch, http.fetchToCache, archive.list, archive.open, archive.read, archive.extract, catalog.search, catalog.get, viewers.history, viewers.setWatchState, collections.upsert, kinds.upsert, kinds.delete, kinds.items, sources.resolve, cache.put, cache.open, cache.read, frontend.streamURL, folders.list, folders.stat, notify.owner, log. Args is a JSON array of positional arguments; the result is any JSON value. Return a *Fault to classify the failure (forbidden for a missing or revoked grant).
type HostFunc ¶
type HostFunc func(ctx context.Context, name string, args json.RawMessage) (json.RawMessage, error)
HostFunc adapts a function to Host.
type Limits ¶
type Limits struct {
MemoryBytes uint64 // per instance; default 32 MiB, max wasm32 4 GiB
Deadline time.Duration // per handle call; default 30s
UnitBudget int64 // per handle call; default 10000
Busy func() bool // background work yields while playback or host load is high
PoolSize int // instances per pool; default 2
// Hot-path budgets. A manifest may lower these, never raise them:
// normalize clamps to the defaults below.
RankPerCandidate time.Duration // default (and max) 2ms
RankBatch time.Duration // default (and max) 50ms
Costs map[string]int64
// contains filtered or unexported fields
}
Limits bound one plugin. Zero fields take the defaults in normalize.
type Outcome ¶
type Outcome struct {
Status Status
Units int64 // host-call units plus wall-time units
Duration time.Duration
Trap bool
SlowCandidates int // hot path: candidates that went neutral on budget
MemoHits int // hot path: candidates served from the memo
}
Outcome accounts for one finished call.
type Plugin ¶
type Plugin struct {
// contains filtered or unexported fields
}
Plugin is one loaded hosted plugin. Call serves the request ABI; RankBatch and CondMatch serve the hot path. Safe for concurrent use.
func (*Plugin) Budgets ¶
Budgets reports the plugin's effective hot-path budgets: the manifest's (lowered, never raised), scaled by the engine's measured speed.
func (*Plugin) Call ¶
Call runs the request ABI. The returned error, when non-nil, is a *Error classified for the host's health loop.
func (*Plugin) CallDetailed ¶
CallDetailed is Call plus the Outcome (units, duration, trap flag).
func (*Plugin) CondBatch ¶
func (p *Plugin) CondBatch(ctx context.Context, cands []Candidate, operator string, value json.RawMessage) ([]bool, Outcome, error)
CondBatch checks a batch without memoization, suitable for conformance testing. It applies the same per-candidate and batch limits as ranking.
func (*Plugin) CondMatch ¶
func (p *Plugin) CondMatch(ctx context.Context, cand Candidate, operator string, value json.RawMessage) (match bool, out Outcome, err error)
CondMatch evaluates one rule condition. Budget exhaustion and errors read as false with the reason in the Outcome, matching RankBatch's neutral-on-trouble rule.
func (*Plugin) HasCondMatch ¶
func (*Plugin) HasRankKey ¶
func (*Plugin) RankBatch ¶
func (p *Plugin) RankBatch(ctx context.Context, cands []Candidate, config json.RawMessage) (scores []int64, out Outcome, err error)
RankBatch scores candidates with the plugin's rank_key. Scores align with cands; a candidate that exceeds its per-candidate budget, or that the batch did not reach before its budget was spent, scores the neutral 0, counts as slow and is not memoized, so a later batch for the same releases picks up where this one stopped. Errors are returned only for instance-level failures (trap, kill at the hard deadline); per-candidate slowness never fails the resolve.
type Request ¶
type Request struct {
Op string `json:"op,omitempty"`
ID string `json:"id,omitempty"`
Method string `json:"method"`
Path string `json:"path"`
Query map[string][]string `json:"query,omitempty"`
Headers map[string]string `json:"headers,omitempty"`
Body json.RawMessage `json:"body,omitempty"`
}
Request is the HTTP-shaped plugin request (SPEC 5.1.2 / 5.2).
type Response ¶
type Response struct {
Blob json.RawMessage `json:"blob,omitempty"`
Status int `json:"status"`
Headers map[string]string `json:"headers,omitempty"`
Body json.RawMessage `json:"body,omitempty"`
}
Response is the plugin's answer. Body is the raw JSON the plugin produced as its body field (usually a JSON string), passed through untouched.
type Stats ¶
type Stats struct {
Calls int64
Errors int64
Timeouts int64
Traps int64
Forbidden int64
Aborted int64
Governance int64
SlowCandidates int64
Units int64
}
Stats are the health counters WS7's backoff loop consumes.