Documentation
¶
Overview ¶
Package gpubatch is the host side of the batched GPU command stream for WASI containers: the import module "wasi_webgpu_batch", served by the engine's host-module mechanism (wasi.HostModule), natively and in the browser. A guest records go-gfx/webgpu work with a remote device (package wasigpu/batchsink does it for Go guests), builds a frame with go-virtio/gpu's wgpubatch, and crosses into the host once per submit.
The host does not interpret the work: it copies the frame out of guest memory, decodes it (wgpubatch.Decode), and hands creations, destructions and streams to a go-gfx/webgpu batch.Context, which remaps the guest's private ids, checks the stream and its buffer ranges (wire.CheckRanges), and replays it on the container's device. The guest is hostile and the replay is the trust boundary; see docs/llm-design.md.
Functions (all i32 parameters, an Errno result):
caps(buf, len) the capset blob (wgpubatch.Caps, version 2)
context_create(out) a context; its handle is stored at out
submit(ctx, frame, frame_len, read, read_len) run a frame, then fill [read, read+read_len)
context_destroy(ctx) release everything the context holds
last_error(ctx, buf, len, out) the message of the context's last failure
(at most 1 KiB, UTF-8): copied to buf, its
full length stored at out; Success with
length 0 when there is none
A process without the "gpu" capability (run --gpu) gets Denied from every function. The container's GPU memory quota is the capability "gpu-memory=BYTES" (run --gpu-memory), 256 MiB if absent, at most 1.5 GiB. Where there is no GPU backend (a Provider of nil) the answer is Unsupported, and a fallback (software) adapter is refused the same way.
Index ¶
Constants ¶
const MaxErrorMessage = 1024
MaxErrorMessage bounds the text last_error returns.
const ModuleName = "wasi_webgpu_batch"
ModuleName is the import module.
Variables ¶
var DefaultLimits = Limits{ Contexts: 4, MaxFrame: 32 << 20, MaxObjects: 4096, MaxRead: 1 << 20, GPUTime: 2 * time.Second, Memory: 256 << 20, MaxMemory: 1610612736, }
DefaultLimits are the quotas of the design (docs/llm-design.md).
var ErrFallback = errors.New("gpubatch: only a fallback adapter")
ErrFallback: the provider refuses a fallback (software) adapter.
Functions ¶
func HostModules ¶
func HostModules(p Provider) []wasi.HostModule
HostModules is NewModule as the engine's host-module list.
Types ¶
type Device ¶
type Device interface {
// Caps describes what the guest may use of it: the adapter's optional
// features granted, and the limits the device has.
Caps() (features uint32, limits wgpubatch.DeviceLimits)
// Backend returns a fresh backend for one context (a batch.Reader if it
// can read back). Its Close releases what the context created.
Backend() batch.Backend
// Destroy loses the device: every context's work fails from now on.
Destroy()
}
Device is one container's GPU device, shared by its contexts.
type Errno ¶
type Errno = uint32
Errno is the result of every function.
const ( Success Errno = 0 InvalidArgument Errno = 1 // a bad handle, or a buffer too small for the capset Denied Errno = 2 // the container has no gpu capability Unsupported Errno = 3 // no GPU backend here, or only a fallback adapter TooLarge Errno = 4 // a frame or a read window above its limit QuotaExceeded Errno = 5 // contexts, GPU memory, pipelines, WGSL size Refused Errno = 6 // a frame or stream the host refuses; the context is lost ContextLost Errno = 7 // the context was lost earlier Timeout Errno = 8 // a submit still running after GPUTime: the device is destroyed Fault Errno = 9 // a guest address range outside its memory Failed Errno = 10 // the backend failed // InvalidShader: the GPU refused an object the guest described: WGSL that // does not compile, or a layout, pipeline or bind group WebGPU rejects. // The context is lost; last_error returns the compiler's message (WGSL // line and column included). InvalidShader Errno = 11 )
Errno values.
type Limits ¶
type Limits struct {
Contexts int // live contexts (default 4)
MaxFrame uint32 // largest frame, bytes (default 32 MiB)
MaxObjects uint32 // guest ids per context (default 4096)
MaxRead uint32 // read window of one submit (default 1 MiB, at most wire.MaxReadWindow)
GPUTime time.Duration // one submit's wait for the GPU (default 2 s)
// Memory is the GPU memory quota when the container made no grant
// (default 256 MiB). MaxMemory is the largest grant honoured (1.5 GiB).
Memory, MaxMemory int64
}
Limits bound one container's use of the module.
type Module ¶
type Module struct {
// Provider opens devices; nil: Unsupported everywhere (native).
Provider Provider
Limits Limits
}
Module is the wasi_webgpu_batch host module.
type Provider ¶
type Provider interface {
// Open makes a device whose GPU memory is bounded by memory bytes.
// ErrFallback or webgpu.ErrFallbackAdapter: only a software adapter;
// any error becomes Unsupported (ErrFallback) or Failed.
Open(memory int64) (Device, error)
}
Provider opens the container's GPU device. Open may wait on the platform; the module calls it off the host call.
Source Files