wasm

package
v0.3.16 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 20, 2026 License: GPL-3.0 Imports: 23 Imported by: 0

Documentation

Overview

Package wasm is Ether's hosted plugin runtime: it runs plugin code inside the Ether process, sandboxed, with no ports and no operating-system access. SPEC section 5.2 is the contract.

Three kinds of hosted plugin are supported, all pure Go (CGO_ENABLED=0):

  • JavaScript, on QuickJS-NG compiled to WebAssembly and executed by wazero (via github.com/fastschema/qjs; see THIRD_PARTY_NOTICES.md).
  • TypeScript, compiled to JavaScript by Ether with esbuild's Go API (see Compile), then run as JavaScript.
  • Raw WebAssembly modules, executed directly by wazero.

The QuickJS WASI instance has no preopened filesystem, environment or arguments. Its std and os modules are disabled. Raw modules cannot import WASI. Host capabilities such as HTTP and folders are available only through Ether's permission-checked host calls. Studio previews use the same runtime.

Request ABI (all kinds)

A plugin answers HTTP-shaped JSON requests. The request is {"method","path","query","headers","body"} and the response is {"status","headers","body"}. In JavaScript the plugin defines handle(request) -> response with plain objects (the TypeScript SDK's plugin({...}) router produces this); host services are on the ether global (ether.kv.get, ether.http.fetch, ether.log, ether.abort, ...). Global TextDecoder (UTF-8 labels, {stream, fatal, ignoreBOM}) and TextEncoder.encode are backed by native Go UTF-8 helpers (__ether_text_decode and __ether_text_encode).

Raw WASM ABI

A raw module must export "memory", "alloc" and "handle":

alloc(size i32) -> i32                     // guest allocator; host writes input here
handle(ptr i32, len i32) -> i64            // returns (outPtr<<32)|outLen

Input and output are the request/response JSON above, UTF-8, in linear memory. The host copies the output before the next call, so the guest may reuse its buffers. Optional hot-path exports:

rank_key(candPtr, candLen, cfgPtr, cfgLen i32) -> i64
cond_match(candPtr, candLen, opPtr, opLen, valPtr, valLen i32) -> i32 (0 or 1)

where candidate/config/value are JSON and operator is a plain string. Allowed imports are only "ether.log(ptr,len)", "ether.abort(code i32)" (never returns) and "ether.host_call(namePtr,nameLen,argsPtr,argsLen,outPtr,outCap i32) -> i32" which runs a host service named like "kv.get" with a JSON argument array, writes the JSON result into outPtr and returns its length, or a negative error code: -1 error, -2 forbidden, -3 governance (unit budget), -4 the result did not fit in outCap (the return is then -(needed) if needed > 4, see ErrHostResultTooBig handling in raw.go). Any other import (WASI included) makes the module refuse to load. On the hot path host_call always returns -2: ranking code may only log and abort.

Governance

Ported from appkit-sandbox semantics. Every call has a wall deadline, a memory cap (default 32 MiB, maximum 64 MiB) and a unit budget. Units are charged per host call from a cost table plus 1 unit per 100us of wall time, checked at host calls and at return. Outcomes are classified as ok, error, timeout, forbidden, aborted or governance; traps and timeouts poison the instance (it is discarded, never reused) and are counted in Stats so the plugin host can apply backoff and disable a failing plugin.

Hot path

RankBatch and CondMatch run ranking steps and rule conditions in separate instances with no host services, a deterministic environment (Date fixed at the epoch, Math.random a seeded generator reset per batch), a default budget of 2ms per candidate and 50ms per batch, neutral results (rank 0, condition false) on exhaustion, and memoization keyed by (plugin id, version, config hash, release id).

The budgets bound the plugin's own time. Handing over the facts is host work and is not charged. Time is read from the monotonic clock and, when that says a budget is spent, from the thread's CPU clock, so time the thread spent descheduled on a busy host is not charged either. A batch stops starting candidates when its budget is spent; what it finished is memoized, so the next batch for the same releases continues from there. Separately, a hard deadline of twice the batch budget (Limits.hotKill) kills an instance whose code stops returning. The budgets are stated in reference-machine time; the engine measures its own speed once per process and scales them (see HotScale), so a slow CPU or a race-detector build does not cost a light plugin its ranking.

These helpers support offline acceptance tests across package boundaries. Runtime callers use the production APIs in the other files.

Index

Constants

View Source
const (
	// DefaultRankPerCandidate and DefaultRankBatch are the hot-path budgets
	// (SPEC §5.2). A manifest may lower them, never raise them.
	DefaultRankPerCandidate = 2 * time.Millisecond
	DefaultRankBatch        = 50 * time.Millisecond
	// WallUnitInterval is the wall time worth one unit.
	WallUnitInterval = 100 * time.Microsecond
	// BaseCallUnits is charged once per handle call.
	BaseCallUnits = 10
	// ArchiveReadUnitsPerMember is charged per requested item in archive.read
	// on top of archive.read's 50-unit base call cost, so large batches pay
	// proportional governance units instead of dodging per-call budgets.
	ArchiveReadUnitsPerMember = 2
	// ArchiveExtractUnitsPerMiB is charged per started MiB of uncompressed
	// archive member streamed by archive.extract on top of its base call cost.
	ArchiveExtractUnitsPerMiB = 10
	// MaxMemoryBytes is the hard ceiling for a plugin instance.
	MaxMemoryBytes = 4 << 30
	// DefaultPoolSize and MaxPoolSize bound instances per hosted plugin pool.
	DefaultPoolSize = 2
	MaxPoolSize     = 4
	// MinDataReadBytes and MaxDataReadBytes bound the per-call readBytes grant
	// derived from memoryBytes: clamp(memoryBytes/4, 8 MiB, 512 MiB).
	MinDataReadBytes = 8 << 20
	MaxDataReadBytes = 512 << 20
	// MaxResponseBytes caps a plugin response.
	MaxResponseBytes = 8 << 20
	// MaxFetchBodyBytes caps a direct HTTP response before JSON encoding.
	MaxFetchBodyBytes = 10 << 20
	// MaxHostResultBytes is the baseline host-result floor; actual per-call
	// host results scale with HostResultBytes(memoryBytes) so base64-encoded
	// data reads up to readBytes always fit their JSON envelope.
	MaxHostResultBytes = 4 << 20
)

Variables

View Source
var DefaultCosts = map[string]int64{
	"log": 1, "runtime.limits": 1, "kv.get": 5, "kv.put": 10, "kv.delete": 5, "kv.list": 10, "kv.usage": 5, "badges.set": 20, "badges.clear": 10, "badges.usage": 5,
	"secret.get": 5, "http.fetch": 100, "http.fetchToCache": 100, "http.hash": 100, "cache.read": 20, "archive.list": 50, "archive.open": 50, "archive.read": 50, "archive.extract": 50, "addon.get": 100, "catalog.search": 50, "catalog.get": 20,
	"viewers.history": 50, "viewers.setWatchState": 50, "collections.upsert": 50,
	"kinds.upsert": 50, "kinds.delete": 50, "kinds.items": 20, "kinds.visible": 15, "kinds.browse": 20,
	"sources.resolve": 100, "cache.put": 50, "cache.open": 20, "frontend.streamURL": 5,
	"folders.hashStart": 20, "folders.hashPoll": 20, "folders.hashCancel": 20, "folders.hash": 20, "folders.list": 20, "folders.stat": 20, "notify.owner": 20, "settings.get": 5,
}

Default host-call unit costs, in the spirit of appkit-sandbox's priced host functions. Wall time adds 1 unit per WallUnitInterval; archive.read adds ArchiveReadUnitsPerMember per requested member; archive.extract additionally charges ArchiveExtractUnitsPerMiB per started MiB streamed.

View Source
var ErrPoolBusy = errors.New("timed out waiting for a plugin instance")

ErrPoolBusy is wrapped by an *Error when a call times out waiting for a free WASM instance in the pool. Pool contention is not a plugin crash.

Functions

func BackgroundPoolPath

func BackgroundPoolPath(path string) bool

BackgroundPoolPath reports whether path is a background job or sync call bounded to at most PoolSize-1 service instances so interactive viewer resolves, frontend adapter requests and health checks always retain at least one free instance.

func ChargeUnits

func ChargeUnits(ctx context.Context, units int64) error

ChargeUnits charges additional governance units to the active guest call, returning a StatusGovernance *Fault when the call's UnitBudget is exceeded. Outside a governed wasm call it is a no-op.

func DataReadBytes

func DataReadBytes(memoryBytes uint64) int64

DataReadBytes computes the per-call byte ceiling for cache.read, archive.open, and archive.read from a plugin's memory grant: clamp(memoryBytes/4, 8 MiB, 512 MiB), treating 0 as the 32 MiB default.

func DeadlineFor

func DeadlineFor(path string) time.Duration

DeadlineFor returns the baseline per-route deadline from SPEC §5.1.2. Routes in GrantedLimitsPath use the plugin's owner-granted deadline instead when dispatched through EffectiveDeadline.

func EffectiveDeadline

func EffectiveDeadline(path string, granted time.Duration, stremio bool) time.Duration

EffectiveDeadline returns the enforced deadline for path given the plugin's granted deadline ceiling and whether it speaks Stremio add-on routes.

func GrantedLimitsPath

func GrantedLimitsPath(path string) bool

GrantedLimitsPath reports whether path runs on the plugin's owner-granted deadline and unit budget rather than a fixed protocol or viewer cap: background kind sync (/browse, /changes, /expand), Live TV lineup and guide sync (/ether/channels.json, /ether/guide/*), jobs, setup, owner actions, agent tools, and event deliveries.

func HostResultBytes

func HostResultBytes(memoryBytes uint64) int64

HostResultBytes returns the maximum JSON host-result envelope for a plugin with the given memory grant, including base64 expansion of DataReadBytes plus 1 MiB for JSON framing.

func HotScale

func HotScale() float64

HotScale is how many times slower than the reference machine this process runs hosted plugin code (1 on the reference machine or faster). The first call measures it; it costs a few milliseconds and runs off the hot path, at plugin load.

func KindSyncPath

func KindSyncPath(path string) bool

KindSyncPath identifies a media kind's background catalogue calls (browse, changes, and expand). Ether makes them in the background to fill a library, never while a viewer waits, so they run on the plugin's granted limits (the manifest's request, capped only by the owner's ceilings) instead of the viewer budget. A big first sync or facet expansion can then take as long as it needs.

func LiveTVSyncPath

func LiveTVSyncPath(path string) bool

LiveTVSyncPath identifies a Live TV source plugin's background lineup and guide refresh calls (/ether/channels.json and /ether/guide/*). Ether makes them in the background to import channels and programmes, never while a viewer waits, so they run on the plugin's granted limits.

func StremioDeadline

func StremioDeadline(path string) time.Duration

StremioDeadline is the per-route deadline for a third-party Stremio add-on or an add-on-backed plugin (ether.stremio: true). Their stream searches routinely take seconds (they search trackers and debrid caches live); the caller's own deadline still bounds every call.

func UnitsForDeadline

func UnitsForDeadline(d time.Duration) int64

UnitsForDeadline converts a call deadline into its matching governance unit budget at one unit per WallUnitInterval (100 µs, or 10,000 units per second), so a viewer route's unit budget matches the deadline it is advertised.

func ViewerRequestPath

func ViewerRequestPath(path string) bool

ViewerRequestPath identifies calls on a viewer's browse/search/playback path. Administrative actions, setup, catalogue sync and background jobs use owner-granted limits.

func WaitBackground

func WaitBackground(ctx context.Context, busy func() bool) error

WaitBackground yields before starting work and at each host-service boundary.

Types

type Candidate

type Candidate struct {
	ReleaseID string
	Facts     json.RawMessage
}

Candidate is one release on the hot path. Facts is the CandidateFacts JSON (SPEC 5.8); ReleaseID keys the memo and must be stable per release.

type Diagnostic

type Diagnostic struct {
	File     string `json:"file,omitempty"`
	Line     int    `json:"line,omitempty"`   // 1-based
	Column   int    `json:"column,omitempty"` // 1-based
	Text     string `json:"text"`
	Severity string `json:"severity"` // "error" or "warning"
}

Diagnostic is one esbuild error or warning, flattened for Studio and install reporting.

func Compile

func Compile(files map[string]string, entry string) (js []byte, sourcemap []byte, diagnostics []Diagnostic)

Compile bundles TypeScript (or JavaScript) plugin sources into one IIFE JavaScript file plus an external source map, using esbuild's Go API. Files maps slash paths ("src/index.ts") to contents; entry is one of those paths. Relative imports resolve inside the map only: a plugin cannot reach the filesystem or node_modules. Exports land on the __plugin global, which the JS runtime normalizes onto globalThis, so `export function handle(...)` and `globalThis.handle = ...` both work. A .js entry that is already a plain script (a prebuilt bundle such as an installed plugin's dist/plugin.js: no import, export or CommonJS module use) is checked and returned unchanged; wrapping it again would hide its globals inside a second IIFE. On failure js is nil and diagnostics carries every error.

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine compiles and caches guest code and builds Plugins. It owns the wazero runtimes used by raw modules; JavaScript instances manage their own (QuickJS) modules. Safe for concurrent use.

func NewEngine

func NewEngine(logger *slog.Logger) *Engine

func (*Engine) Close

func (e *Engine) Close() error

func (*Engine) LoadJS

func (e *Engine) LoadJS(id, version, source string, host Host, lim Limits) (*Plugin, error)

LoadJS loads a JavaScript plugin (TypeScript authors go through Compile first). A probe instance is built eagerly so syntax errors and a missing handle surface at load.

func (*Engine) LoadWASM

func (e *Engine) LoadWASM(id, version string, module []byte, host Host, lim Limits) (*Plugin, error)

LoadWASM compiles (and caches) a raw module and returns the Plugin. A probe instance is built eagerly so load-time failures (bad imports, missing exports) surface here, not on first call.

type Error

type Error struct {
	Status  Status
	Message string
	Trap    bool  // the instance trapped (panic in guest) or was killed by a deadline
	Err     error // optional wrapped sentinel (e.g. ErrPoolBusy)
}

Error is the error type returned by Plugin calls. It carries the Outcome so WS7's health loop can read the status, trap flag and cost from one value.

func (*Error) Error

func (e *Error) Error() string

func (*Error) Unwrap

func (e *Error) Unwrap() error

type Fault

type Fault struct {
	Code    string
	Message string
}

Fault is a classified, non-ambient failure. A Host returns a *Fault to make the call end with that status; Code is one of the Status values other than ok (usually forbidden), or an application code the plugin can catch.

func Forbidden

func Forbidden(message string) *Fault

Forbidden builds a forbidden Fault, e.g. for a revoked permission.

func (*Fault) Error

func (f *Fault) Error() string

type Host

type Host interface {
	CallHost(ctx context.Context, name string, args json.RawMessage) (json.RawMessage, error)
}

Host is the plugin-to-Ether service surface (SPEC 5.1.5). Name is a dotted service method: runtime.limits, kv.get, kv.put, kv.delete, kv.list, kv.usage, badges.set, badges.clear, badges.usage, secret.get, http.fetch, http.fetchToCache, archive.list, archive.open, archive.read, archive.extract, catalog.search, catalog.get, viewers.history, viewers.setWatchState, collections.upsert, kinds.upsert, kinds.delete, kinds.items, sources.resolve, cache.put, cache.open, cache.read, frontend.streamURL, folders.list, folders.stat, notify.owner, log. Args is a JSON array of positional arguments; the result is any JSON value. Return a *Fault to classify the failure (forbidden for a missing or revoked grant).

type HostFunc

type HostFunc func(ctx context.Context, name string, args json.RawMessage) (json.RawMessage, error)

HostFunc adapts a function to Host.

func (HostFunc) CallHost

func (f HostFunc) CallHost(ctx context.Context, name string, args json.RawMessage) (json.RawMessage, error)

type Limits

type Limits struct {
	MemoryBytes uint64        // per instance; default 32 MiB, max wasm32 4 GiB
	Deadline    time.Duration // per handle call; default 30s
	UnitBudget  int64         // per handle call; default UnitsForDeadline(Deadline)
	Stremio     bool          // plugin answers Stremio add-on routes (uses StremioDeadline)
	Busy        func() bool   // background work yields while playback or host load is high
	PoolSize    int           // instances per pool; default DefaultPoolSize (2), max MaxPoolSize (4)
	// Hot-path budgets. A manifest may lower these, never raise them:
	// normalize clamps to the defaults below.
	RankPerCandidate time.Duration // default (and max) 2ms
	RankBatch        time.Duration // default (and max) 50ms
	Costs            map[string]int64
	// contains filtered or unexported fields
}

Limits bound one plugin. Zero fields take the defaults in normalize.

func CallLimits

func CallLimits(ctx context.Context) (Limits, bool)

CallLimits returns the effective per-call limits (after route deadline and viewer unit-budget clamps) when called inside an active hosted plugin call.

type Outcome

type Outcome struct {
	Status         Status
	Units          int64 // host-call units plus wall-time units
	Duration       time.Duration
	Trap           bool
	SlowCandidates int // hot path: candidates that went neutral on budget
	MemoHits       int // hot path: candidates served from the memo
}

Outcome accounts for one finished call.

type Plugin

type Plugin struct {
	// contains filtered or unexported fields
}

Plugin is one loaded hosted plugin. Call serves the request ABI; RankBatch and CondMatch serve the hot path. Safe for concurrent use.

func (*Plugin) Budgets

func (p *Plugin) Budgets() (perCandidate, perBatch time.Duration)

Budgets reports the plugin's effective hot-path budgets: the manifest's (lowered, never raised), scaled by the engine's measured speed.

func (*Plugin) Call

func (p *Plugin) Call(ctx context.Context, req Request) (Response, error)

Call runs the request ABI. The returned error, when non-nil, is a *Error classified for the host's health loop.

func (*Plugin) CallDetailed

func (p *Plugin) CallDetailed(ctx context.Context, req Request) (Response, Outcome, error)

CallDetailed is Call plus the Outcome (units, duration, trap flag).

func (*Plugin) Close

func (p *Plugin) Close() error

func (*Plugin) CondBatch

func (p *Plugin) CondBatch(ctx context.Context, cands []Candidate, operator string, value json.RawMessage) ([]bool, Outcome, error)

CondBatch checks a batch without memoization, suitable for conformance testing. It applies the same per-candidate and batch limits as ranking.

func (*Plugin) CondMatch

func (p *Plugin) CondMatch(ctx context.Context, cand Candidate, operator string, value json.RawMessage) (match bool, out Outcome, err error)

CondMatch evaluates one rule condition. Budget exhaustion and errors read as false with the reason in the Outcome, matching RankBatch's neutral-on-trouble rule.

func (*Plugin) HasCondMatch

func (p *Plugin) HasCondMatch() bool

func (*Plugin) HasHandle

func (p *Plugin) HasHandle() bool

HasHandle, HasRankKey and HasCondMatch report the exported entry points.

func (*Plugin) HasRankKey

func (p *Plugin) HasRankKey() bool

func (*Plugin) ID

func (p *Plugin) ID() string

ID and Version identify the plugin for memo keys and logs.

func (*Plugin) RankBatch

func (p *Plugin) RankBatch(ctx context.Context, cands []Candidate, config json.RawMessage) (scores []int64, out Outcome, err error)

RankBatch scores candidates with the plugin's rank_key. Scores align with cands; a candidate that exceeds its per-candidate budget, or that the batch did not reach before its budget was spent, scores the neutral 0, counts as slow and is not memoized, so a later batch for the same releases picks up where this one stopped. Errors are returned only for instance-level failures (trap, kill at the hard deadline); per-candidate slowness never fails the resolve.

func (*Plugin) Stats

func (p *Plugin) Stats() Stats

func (*Plugin) Version

func (p *Plugin) Version() string

type Request

type Request struct {
	Op      string              `json:"op,omitempty"`
	ID      string              `json:"id,omitempty"`
	Method  string              `json:"method"`
	Path    string              `json:"path"`
	Query   map[string][]string `json:"query,omitempty"`
	Headers map[string]string   `json:"headers,omitempty"`
	Body    json.RawMessage     `json:"body,omitempty"`
}

Request is the HTTP-shaped plugin request (SPEC 5.1.2 / 5.2).

type Response

type Response struct {
	Blob    json.RawMessage   `json:"blob,omitempty"`
	Status  int               `json:"status"`
	Headers map[string]string `json:"headers,omitempty"`
	Body    json.RawMessage   `json:"body,omitempty"`
}

Response is the plugin's answer. Body is the raw JSON the plugin produced as its body field (usually a JSON string), passed through untouched.

type Stats

type Stats struct {
	Calls          int64
	Errors         int64
	Timeouts       int64
	Traps          int64
	Forbidden      int64
	Aborted        int64
	Governance     int64
	SlowCandidates int64
	Units          int64
}

Stats are the health counters WS7's backoff loop consumes.

type Status

type Status string

Status classifies a finished call, following appkit-sandbox.

const (
	StatusOK           Status = "ok"
	StatusInvalidInput Status = "invalid_input"
	StatusError        Status = "error"
	StatusTimeout      Status = "timeout"
	StatusForbidden    Status = "forbidden"
	StatusAborted      Status = "aborted"
	StatusGovernance   Status = "governance"
)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL