Documentation
¶
Overview ¶
Package gpubatch is the host side of the batched GPU command stream for WASI containers: the import module "wasi_webgpu_batch", served by the engine's host-module mechanism (wasi.HostModule), natively and in the browser. A guest records go-gfx/webgpu work with a remote device (package wasigpu/batchsink does it for Go guests), builds a frame with go-virtio/gpu's wgpubatch, and crosses into the host once per submit.
The host does not interpret the work: it copies the frame out of guest memory, decodes it (wgpubatch.Decode), and hands creations, destructions and streams to a go-gfx/webgpu batch.Context, which remaps the guest's private ids, checks the stream and its buffer ranges (wire.CheckRanges), and replays it on the container's device. The guest is hostile and the replay is the trust boundary; see docs/llm-design.md.
Functions (all i32 parameters, an Errno result):
caps(buf, len) the capset blob (wgpubatch.Caps, version 2)
context_create(out) a context; its handle is stored at out
submit(ctx, frame, frame_len, read, read_len) run a frame, then fill [read, read+read_len)
context_destroy(ctx) release everything the context holds
last_error(ctx, buf, len, out) the message of the context's last failure
(at most 1 KiB, UTF-8): copied to buf, its
full length stored at out; Success with
length 0 when there is none
A process without the "gpu" capability (run --gpu) gets Denied from every function. The container's GPU memory quota is the capability "gpu-memory=BYTES" (run --gpu-memory), 256 MiB if absent, at most 1.5 GiB. Where there is no GPU backend (a Provider of nil) the answer is Unsupported, and a fallback (software) adapter is refused the same way.
Index ¶
Constants ¶
const CollateralWindow = 10 * time.Second
CollateralWindow is how far before a submit's turn a browser loss of another device still explains that the submit never completed.
const MaxErrorMessage = 1024
MaxErrorMessage bounds the text last_error returns.
const ModuleName = "wasi_webgpu_batch"
ModuleName is the import module.
Variables ¶
var DefaultLimits = Limits{ Contexts: 4, MaxFrame: 32 << 20, MaxObjects: 4096, MaxRead: 1 << 20, GPUTime: 2 * time.Second, Budget: 500 * time.Millisecond, DrainTime: 30 * time.Second, MaxDispatchInvocations: 1 << 26, MaxStreamInvocations: 1 << 30, Memory: 256 << 20, MaxMemory: 1610612736, }
DefaultLimits are the quotas of the design (docs/llm-design.md).
var ErrFallback = errors.New("gpubatch: only a fallback adapter")
ErrFallback: the provider refuses a fallback (software) adapter.
Functions ¶
func HostModules ¶
func HostModules(p Provider) []wasi.HostModule
HostModules is NewModule as the engine's host-module list.
Types ¶
type Device ¶
type Device interface {
// Caps describes what the guest may use of it: the adapter's optional
// features granted, and the limits the device has.
Caps() (features uint32, limits wgpubatch.DeviceLimits)
// Backend returns a fresh backend for one context (a batch.Reader if it
// can read back). Its Close releases what the context created.
Backend() batch.Backend
// Destroy loses the device: every context's work fails from now on.
Destroy()
}
Device is one container's GPU device, shared by its contexts.
type Drainer ¶ added in v0.10.0
Drainer is a Provider that can wait until the GPU has finished the work of a destroyed device (WebGPU's destroy() does not cancel submitted work). Drain returns when a trivial submit on a device the provider owns has completed, or after timeout. A Provider without it gets no drain wait.
type Errno ¶
type Errno = uint32
Errno is the result of every function.
const ( Success Errno = 0 InvalidArgument Errno = 1 // a bad handle, or a buffer too small for the capset Denied Errno = 2 // the container has no gpu capability Unsupported Errno = 3 // no GPU backend here, or only a fallback adapter TooLarge Errno = 4 // a frame or a read window above its limit QuotaExceeded Errno = 5 // contexts, GPU memory, pipelines, WGSL size Refused Errno = 6 // a frame or stream the host refuses; the context is lost ContextLost Errno = 7 // the context was lost earlier Timeout Errno = 8 // a submit over its time budget: the device is destroyed, the grant revoked Fault Errno = 9 // a guest address range outside its memory Failed Errno = 10 // the backend failed // InvalidShader: the GPU refused an object the guest described: WGSL that // does not compile, or a layout, pipeline or bind group WebGPU rejects. // The context is lost; last_error returns the compiler's message (WGSL // line and column included). InvalidShader Errno = 11 // Revoked: the grant was revoked by an earlier Timeout: every call but // context_destroy answers it, for the container's life. Revoked Errno = 12 // Reserved: the guest used the bind group the host reserves for its loop // budget (group 3 on a default device): a pipeline layout with more than // 3 bind group layouts, a WGSL module with a binding in @group(3), or a // SetBindGroup of index 3 or more. The context is lost, the grant is not // revoked; last_error names the slot. Reserved Errno = 13 // Truncated: a shader invocation hit the host's loop budget and was cut // (layer 4): the stream's results are partial and are not returned. The // context is lost and the grant is revoked, as for Timeout; the engine // emits a "sanction" event with kind "truncated". Truncated Errno = 14 // Unbounded: the host cannot bound the module's loops, so it does not run // it: the WGSL does not parse with the host's parser though the browser // accepts it, it has a loop in a module with a vertex or fragment entry // point, or the browser refused the loop-budget rewrite (for example a // shader that already binds as many storage buffers as the device allows: // the capset reports one less). The context is lost, the grant is not // revoked; last_error says which. Unbounded Errno = 15 )
Errno values.
type Limits ¶
type Limits struct {
Contexts int // live contexts (default 4)
MaxFrame uint32 // largest frame, bytes (default 32 MiB)
MaxObjects uint32 // guest ids per context (default 4096)
MaxRead uint32 // read window of one submit (default 1 MiB, at most wire.MaxReadWindow)
// GPUTime is the hard cap of one submit's time on the GPU (default 2 s):
// what a frame that creates objects (shader and pipeline compilation
// included) may take. Budget is the steady-state budget of a frame that
// only streams commands (default 500 ms: the longest legitimate submit
// measured is 226 ms in Firefox (its 200 ms floor), 43 ms in Chromium; docs/gpu-scheduling.md), at most GPUTime. Both run from
// the moment the submit gets its turn, not from when it was queued.
GPUTime, Budget time.Duration
// DrainTime bounds how long the host waits, after a sanction, for the
// destroyed device's remaining work to leave the GPU before it serves
// the next turn (default 30 s).
DrainTime time.Duration
// MaxDispatchInvocations and MaxStreamInvocations are the compute quota
// of one context: no Dispatch, and no stream, may ask for more
// invocations (workgroup counts times the device's largest workgroup;
// wire.CheckInvocations). 0: DefaultLimits'.
MaxDispatchInvocations, MaxStreamInvocations uint64
// Memory is the GPU memory quota when the container made no grant
// (default 256 MiB). MaxMemory is the largest grant honoured (1.5 GiB).
Memory, MaxMemory int64
}
Limits bound one container's use of the module.
type LossWatcher ¶ added in v0.12.0
LossWatcher is a Provider that knows when the browser (not the host) last destroyed one of its devices.
type Module ¶
type Module struct {
// Provider opens devices; nil: Unsupported everywhere (native).
Provider Provider
Limits Limits
// contains filtered or unexported fields
}
Module is the wasi_webgpu_batch host module.
type Provider ¶
type Provider interface {
// Open makes a device whose GPU memory is bounded by memory bytes.
// ErrFallback or webgpu.ErrFallbackAdapter: only a software adapter;
// any error becomes Unsupported (ErrFallback) or Failed.
Open(memory int64) (Device, error)
}
Provider opens the container's GPU device. Open may wait on the platform; the module calls it off the host call.
type Stats ¶ added in v0.10.0
type Stats struct {
Submits int64 `json:"submits"` // submits that ran
Sanctions int64 `json:"sanctions"` // submits over their budget
// MaxSubmit is the longest submit that stayed within its budget, over
// all frames; MaxStream those without a creation (the steady state, held
// to Limits.Budget) and MaxCreate those with one (shader and pipeline
// compilation, held to Limits.GPUTime).
MaxSubmit time.Duration `json:"maxSubmit"`
MaxStream time.Duration `json:"maxStream"`
MaxCreate time.Duration `json:"maxCreate"`
// MaxWait is the longest a submit waited for its turn.
MaxWait time.Duration `json:"maxWait"`
// MaxInvocations is the largest stream, in invocations as the compute
// quota counts them.
MaxInvocations uint64 `json:"maxInvocations"`
// MaxDispatchInvocations is the largest single dispatch, counted alike.
MaxDispatchInvocations uint64 `json:"maxDispatchInvocations"`
// Resets is how many submits did not complete because the browser
// destroyed another device (not an overrun: no sanction, the device is
// reset and the guest answered ContextLost).
Resets int64 `json:"resets"`
// Truncations is how many submits hit the loop budget (layer 4); each is
// also a sanction.
Truncations int64 `json:"truncations"`
// LoopModules is how many guest WGSL modules were parsed for layer 4,
// LoopRewritten how many of them were rewritten, LoopRefused how many
// the browser refused after the rewrite (kept as written, bounded by the
// time budget only), and LoopSpent the time spent parsing, rewriting and
// printing them (the cost of layer 4 at module creation).
LoopModules int64 `json:"loopModules"`
LoopRewritten int64 `json:"loopRewritten"`
LoopRefused int64 `json:"loopRefused"`
LoopSpent time.Duration `json:"loopSpent"`
// Drains is how many times a destroyed device's work was waited out.
Drains int64 `json:"drains"`
}
Stats are what a module measured, for choosing its budget and for tests. Durations are the host's clock from the start of a turn to the end of the submit (decode and queueing excluded): host submit to the GPU's completion, read-back included.
Source Files