README
¶
gpu
GPU access for go-wasmcontainers
WASI containers: wasi-nn inference, wasi:webgpu compute, and
rendering, on WebGPU in the browser (the playground) and on the CPU
natively. Pure Go, CGO_ENABLED=0.
Design and references: docs/DESIGN.md.
Status
| Part | State |
|---|---|
onnx: ONNX model loader and CPU operators |
done (v0.1.0) — see below |
nn: wasi-nn host module (wasi_ephemeral_nn), CPU backend |
done — see below |
wasinn: wasi-nn guest bindings for Go, examples/mnist |
done |
| wasi-nn GPU backend (WebGPU, browser) | next |
| wasi:webgpu compute (browser) | planned |
| rendering to a dashboard pane | planned |
wasi-nn: nn (host) and wasinn (guest)
nn.NewModule() is a host module for the engine (v0.13.0+,
wazero.Options.HostModules / browser.Options.HostModules) serving the
preview 1 wasi-nn interface, wasi_ephemeral_nn (load, load_by_name,
init_execution_context, set_input, get_output, compute), as
Wasmtime and the wasi-nn crate's bindings lower it. A graph goes to the
backend of its encoding (ONNX: the pure-Go CPU backend). The execution
target is checked: gpu needs the container's gpu capability
(wctr run --gpu) and a GPU backend, and otherwise fails with
unsupported_operation — never runs on the CPU in its place. Model bytes,
graphs and contexts per process, tensor rank and data are bounded;
FuzzHostCalls drives arbitrary call sequences over arbitrary memory.
wasinn gives Go guests (GOOS=wasip1) Load, NewContext,
SetInputF32, Compute, OutputF32; elsewhere it builds and returns
ErrUnavailable.
examples/mnist is a container that reads a handwritten
digit: natively it classifies MNIST test pictures 0–3 (labels 7, 2, 1, 0)
correctly, p = 1.000 for the embedded one.
onnx
Loads an ONNX model (onnx.proto, decoded by a bounded protobuf reader)
and runs it on the CPU. Operators: Add, AveragePool, Concat, Constant,
Conv, Div, Dropout (inference), Flatten, Gemm, Identity, MatMul, MaxPool,
Mul, Relu, Reshape, Softmax, Sub, Transpose — default domain, operator
sets 7 and later, float tensors (int64 for shapes).
- Refuses, precisely. A model using anything else fails at load with
every unsupported operator and its node count, e.g.
onnx: unsupported operators: LSTM (2 nodes), com.example:Foo (1 node); supported: Add, …. Unsupported variants (MaxPool indices, Dropout training mode, non-float graph values) are refused at load too, not mid-inference. - Checked op by op against the ONNX node tests (onnx v1.22.0,
onnx/backend/test/data/node): the 122 tests whose models load here are vendored byinternal/cmd/onnxvendor(a derived list, not a hand-written one) and pass with the ONNX backend's tolerances,|out − ref| ≤ 1e-7 + 1e-3·|ref|. MNIST-12 from the ONNX model zoo passes on its own test data. - Untrusted input. The model size, node count, rank, elements per
tensor and in total, string lengths, and the multiply-accumulates of one
inference are bounded (
Limits);FuzzRun(decode, load, run) andFuzzDecodeTensorrun in CI.
Performance has not been measured yet (the machine's load was above 40); convolutions are direct loops over precomputed taps, not im2col + GEMM.
License
Apache-2.0 (LICENSE). Test data: ONNX node tests, Apache-2.0 (onnx/testdata/node/LICENSE); MNIST-12, MIT (onnx/testdata/models).
Directories
¶
| Path | Synopsis |
|---|---|
|
examples
|
|
|
mnist
command
Command mnist is a WASI container that reads a handwritten digit: it embeds MNIST-12 (ONNX model zoo, MIT) and a 28x28 picture, runs the model through wasi-nn, and prints the digit with its probability.
|
Command mnist is a WASI container that reads a handwritten digit: it embeds MNIST-12 (ONNX model zoo, MIT) and a 28x28 picture, runs the model through wasi-nn, and prints the digit with its probability. |
|
internal
|
|
|
cmd/mnistpng
command
Command mnistpng extracts one picture of the MNIST test set (IDX files, gzipped, as published by Y. LeCun, C. Cortes and C. Burges; CC BY-SA 3.0) to a 28x28 grayscale PNG and prints its label, so the mnist example classifies a real handwritten digit whose answer is known.
|
Command mnistpng extracts one picture of the MNIST test set (IDX files, gzipped, as published by Y. LeCun, C. Cortes and C. Burges; CC BY-SA 3.0) to a 28x28 grayscale PNG and prints its label, so the mnist example classifies a real handwritten digit whose answer is known. |
|
cmd/onnxvendor
command
Command onnxvendor copies the ONNX node tests this repository's onnx package can run into onnx/testdata/node: from a checkout of github.com/onnx/onnx (v1.22.0 is the last tag that ships onnx/backend/test/data/node), every test whose model decodes, uses only supported operators and passes onnx.NewSession.
|
Command onnxvendor copies the ONNX node tests this repository's onnx package can run into onnx/testdata/node: from a checkout of github.com/onnx/onnx (v1.22.0 is the last tag that ships onnx/backend/test/data/node), every test whose model decodes, uses only supported operators and passes onnx.NewSession. |
|
reentry
Package reentry measures risk R1 of docs/DESIGN.md: can a host, while a Go wasip1 guest is inside a //go:wasmimport call, call the guest's //go:wasmexport cabi_realloc and write the result into the block it returns? The canonical ABI needs exactly that to return a list or a string from a host function.
|
Package reentry measures risk R1 of docs/DESIGN.md: can a host, while a Go wasip1 guest is inside a //go:wasmimport call, call the guest's //go:wasmexport cabi_realloc and write the result into the block it returns? The canonical ABI needs exactly that to return a list or a string from a host function. |
|
Package nn is the host side of wasi-nn for go-wasmcontainers: the preview 1 (witx) interface, import module "wasi_ephemeral_nn" (github.com/WebAssembly/wasi-nn, wasi-nn.witx), served by the engine's WASI host as a host module, natively and in the browser.
|
Package nn is the host side of wasi-nn for go-wasmcontainers: the preview 1 (witx) interface, import module "wasi_ephemeral_nn" (github.com/WebAssembly/wasi-nn, wasi-nn.witx), served by the engine's WASI host as a host module, natively and in the browser. |
|
Package onnx loads ONNX models (onnx/onnx.proto) and runs a bounded set of operators on the CPU, in pure Go.
|
Package onnx loads ONNX models (onnx/onnx.proto) and runs a bounded set of operators on the CPU, in pure Go. |
|
Package wasinn is the guest side of wasi-nn (preview 1, import module "wasi_ephemeral_nn") for Go programs built with GOOS=wasip1: load a model, set inputs, compute, read outputs.
|
Package wasinn is the guest side of wasi-nn (preview 1, import module "wasi_ephemeral_nn") for Go programs built with GOOS=wasip1: load a model, set inputs, compute, read outputs. |