Documentation
¶
Overview ¶
Package nn is the host side of wasi-nn for go-wasmcontainers: the preview 1 (witx) interface, import module "wasi_ephemeral_nn" (github.com/WebAssembly/wasi-nn, wasi-nn.witx), served by the engine's WASI host as a host module, natively and in the browser.
A graph is loaded by a Backend chosen by encoding, as Wasmtime does; the execution target is checked, not passed through: "gpu" runs on a GPU backend or fails with unsupported_operation, never quietly on the CPU, and needs the container's "gpu" capability (run --gpu).
Everything a guest passes is bounded: model bytes, graphs and contexts per process, tensor dimensions and data.
Index ¶
Constants ¶
const ( TypeF16 = 0 TypeF32 = 1 TypeF64 = 2 TypeU8 = 3 TypeI32 = 4 TypeI64 = 5 )
tensor_type values.
Variables ¶
var DefaultLimits = Limits{ModelBytes: 64 << 20, Graphs: 8, Contexts: 32, Dims: 8, TensorData: 64 << 20}
DefaultLimits suit small models.
Functions ¶
This section is empty.
Types ¶
type AsyncBackend ¶ added in v0.3.0
type AsyncBackend interface {
Async() bool
}
AsyncBackend is a backend whose calls wait on something outside the Go program (WebGPU promises in the browser). Its load and compute run off the host call, which returns a Wait, so the engine never blocks.
type Backend ¶
type Backend interface {
// Encoding is the graph encoding it reads.
Encoding() Encoding
// Targets reports whether it runs on target.
Runs(target Target) bool
// Load builds a graph from the guest's builder buffers.
Load(builders [][]byte, target Target) (Graph, Errno)
}
Backend loads graphs of one encoding.
type CPUBackend ¶
CPUBackend runs ONNX graphs with package onnx, in pure Go. It serves the "cpu" target only.
type Context ¶
type Context interface {
SetInput(index uint32, t *onnx.Tensor) Errno
Compute() Errno
Output(index uint32) (*onnx.Tensor, Errno)
}
Context is one inference's inputs and outputs.
type Limits ¶
type Limits struct {
ModelBytes int // all builders of one load, together
Graphs int // graphs alive per process
Contexts int // contexts alive per process
Dims int // tensor rank
TensorData int // bytes of one input tensor
}
Limits bound what one process may make the host hold.
type Module ¶
type Module struct {
Backends []Backend
Limits Limits
// Named are the graphs load_by_name knows: name to its builders and
// encoding. load_by_name of any other name is not_found.
Named map[string]Named
}
Module is the wasi_ephemeral_nn host module.
func NewModule ¶
func NewModule() *Module
NewModule returns the module with the pure-Go CPU backend.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package webgpu is a wasi-nn backend that runs ONNX graphs on the browser's GPU through WebGPU (WGSL compute kernels), for the "gpu" execution target.
|
Package webgpu is a wasi-nn backend that runs ONNX graphs on the browser's GPU through WebGPU (WGSL compute kernels), for the "gpu" execution target. |