batch

package
v0.14.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 4, 2026 License: BSD-3-Clause Imports: 11 Imported by: 0

Documentation

Overview

Package batch is the host side of a batched GPU command stream, without a transport: the table of a guest's object ids, the creation and destruction of its objects, the replay of its streams and, for streams that read back (wire.ReadBuffer), the bounded read window.

A guest records with a remote device (webgpu.NewRemoteDevice) or a webgpu.Recorder and sends three kinds of message: create an object from a wire descriptor, destroy objects, run a stream. A transport — go-virtio's webgpu-batch frames over virtio-gpu, a WebAssembly host module — decodes its own framing and calls Context's three methods; the semantics are here once. The ids a guest uses are private to its Context and are translated (wire.Remap, wire.RemapDescriptor) to the ids the Backend returned before the Backend sees anything, so a stream can only name objects its own context created.

The package is portable: the Backend that runs the work is what needs a GPU (a browser device, a native one, or a test's recording fake).

Index

Constants

View Source 
const DefaultMaxRead = 1 << 20

DefaultMaxRead is the read window a Config without MaxRead grants, 1 MiB.

Variables

View Source 
var (
	// ErrRefused: a message the context's own state refuses (an id in use,
	// an unknown kind, an id not live).
	ErrRefused = errors.New("batch: message refused")
	// ErrLost: the context failed earlier and accepts nothing more.
	ErrLost = errors.New("batch: context lost")
	// ErrRead: a read-back the context refuses: no window, a buffer that is
	// unknown or lacks MAP_READ, a range outside the buffer or the window,
	// a window or reads above the quota.
	ErrRead = errors.New("batch: read-back refused")

	// ErrInvalid: the GPU refused an object the guest described: WGSL that
	// does not compile, a pipeline or bind group that fails WebGPU validation.
	// The error carries the browser's message (the WGSL line and column).
	ErrInvalid = errors.New("batch: the GPU refused the object")
)

Errors. Every refusal wraps one; after any error the Context is lost.

View Source 
var (
	// ErrReserved: the guest used the bind group the host reserved for its
	// loop-budget flag.
	ErrReserved = errors.New("batch: the guest uses the bind group reserved by the host")
	// ErrUnparsable: the host's WGSL parser does not accept the module, so
	// the host cannot bound it.
	ErrUnparsable = errors.New("batch: the host cannot parse the WGSL, so it cannot bound its loops")
	// ErrUnbounded: the host cannot make the module's loops bounded: it does
	// not parse (when the browser accepts it), it has a loop in a vertex or
	// fragment entry point's module, or the browser refused the rewrite of a
	// module it accepts. An unbounded module is never run.
	ErrUnbounded = errors.New("batch: the host cannot bound the WGSL's loops; module refused")
	// ErrTruncated: at least one shader invocation hit the loop budget and
	// was cut; the stream's results are partial and are not returned.
	ErrTruncated = errors.New("batch: a shader exceeded its loop budget and was truncated")
)

Errors of the loop budget.

Functions

func RequiredLimits added in v0.14.0 

func RequiredLimits(ad *webgpu.Adapter) map[string]uint64

RequiredLimits is what a device that uses a Guard should require of its adapter: one more storage buffer per shader stage than WebGPU's default, which is the flag's binding, so that a guest shader that already binds the default 8 can still be rewritten. nil if the adapter offers no more than the default (such a shader is then refused as unboundable). A Quota that admits the limit needs the same map in Quota.Limits.

Types

type Analysis added in v0.14.0 

type Analysis struct {
	Text      string
	Rewritten bool
	Groups    []uint32
}

Analysis is what Analyze found: the text to compile, whether it was rewritten, and the bind groups the module declares (@group of a module-scope variable), sorted.

type Backend 

type Backend interface {
	// Create makes an object of a wire kind from its descriptor (object
	// references already translated) and returns the id the backend's
	// replayer knows it by (nonzero).
	Create(kind uint32, desc []byte) (id uint32, err error)
	// Destroy releases an object Create returned.
	Destroy(id uint32)
	// Replay executes one stream whose ids are all ids Create returned, and
	// returns once the GPU has executed it.
	Replay(words []uint32, data []byte) error
	// Close releases everything the backend holds for the context.
	Close()
}

Backend runs a context's work. A Context has checked and translated everything it passes: ids are the ones Create returned.

type Config 

type Config struct {
	// MaxObjects, if nonzero, bounds the guest ids: a guest id is in
	// [1, MaxObjects).
	MaxObjects uint32
	// MaxRead bounds the read window of one stream: the window a caller
	// passes, and the bytes the stream's reads return together. 0 is
	// DefaultMaxRead; it is capped at wire.MaxReadWindow.
	MaxRead uint32
	// MaxWorkgroups bounds a Dispatch's counts per dimension; 0 is
	// wire.DefaultMaxWorkgroups (WebGPU's default limit).
	MaxWorkgroups uint32
	// WorkgroupInvocations is the most invocations one workgroup can hold,
	// the device's maxComputeInvocationsPerWorkgroup (0: 256, WebGPU's
	// default). A Dispatch is counted at its workgroup counts times this.
	WorkgroupInvocations uint32
	// MaxDispatchInvocations bounds one Dispatch, and MaxStreamInvocations
	// the Dispatches of one stream together, in invocations (0: no bound);
	// see wire.CheckInvocations. A DispatchIndirect is not counted.
	MaxDispatchInvocations, MaxStreamInvocations uint64
}

Config bounds a Context.

type Context 

type Context struct {
	// contains filtered or unexported fields
}

Context is one guest's objects on a Backend. It is not safe for concurrent use.

func New 

func New(be Backend, cfg Config) *Context

New returns a Context on be. If be is a Reader, streams may read back.

func (*Context) Close 

func (c *Context) Close()

Close forgets every object and closes the backend, which releases whatever it holds for the context.

func (*Context) Create 

func (c *Context) Create(id, kind uint32, desc []byte) error

Create makes guest object id of a wire kind from a descriptor (which is translated in place: pass a copy the caller does not keep).

func (*Context) Destroy 

func (c *Context) Destroy(ids ...uint32) error

Destroy releases guest objects; each must be live.

func (*Context) Live 

func (c *Context) Live() int

Live is the number of guest objects alive.

func (*Context) Lookup 

func (c *Context) Lookup(id uint32) (uint32, bool)

Lookup returns the backend's id for guest object id, for a transport that gives an object more than the three messages do (virtio's BIND_RESOURCE).

func (*Context) Lost 

func (c *Context) Lost() error

Lost returns the error that lost the context, or nil.

func (*Context) PeakDispatchInvocations added in v0.11.0 

func (c *Context) PeakDispatchInvocations() uint64

func (*Context) PeakInvocations added in v0.11.0 

func (c *Context) PeakInvocations() uint64

PeakInvocations is the most invocations one stream of this context asked for (wire.CheckInvocations' total): what a stream quota would have to allow. PeakDispatchInvocations is the largest single Dispatch: what a dispatch quota would have to allow.

func (*Context) Stream 

func (c *Context) Stream(words []uint32, data []byte, window []byte) error

Stream runs a guest's stream: its ids are translated in place, so words must be memory the caller owns (a copy of the guest's). Its ReadBuffer commands, checked before anything runs, are served after the replay into window, which the caller owns and the stream writes only inside; a stream that reads needs a window, and a Backend that is a Reader.

type DeviceBackend 

type DeviceBackend struct {
	// contains filtered or unexported fields
}

DeviceBackend is a Reader on a browser device: objects are made from wire descriptors (webgpu.DecodeDescriptor), streams run with Device.Replay and are waited for, and reads are mapAsync, getMappedRange, copy, unmap. It is the one implementation both transports (go-virtio's gpudev, the WebAssembly host module) share. The device's Quota bounds the memory and pipelines its contexts create; create it with the grant of the tenant.

func NewDeviceBackend 

func NewDeviceBackend(dev *webgpu.Device) *DeviceBackend

NewDeviceBackend returns a backend whose objects live on dev. webgpu's replay.js must be installed (webgpu.InstallReplay).

func (*DeviceBackend) Close 

func (b *DeviceBackend) Close()

Close implements Backend: it destroys what is left.

func (*DeviceBackend) Create 

func (b *DeviceBackend) Create(kind uint32, desc []byte) (uint32, error)

Create implements Backend.

func (*DeviceBackend) Destroy 

func (b *DeviceBackend) Destroy(id uint32)

Destroy implements Backend.

func (*DeviceBackend) LoopBudgetStats added in v0.12.0 

func (b *DeviceBackend) LoopBudgetStats() (modules, rewritten, refused int, spent time.Duration)

LoopBudgetStats reports the WGSL modules seen, rewritten and refused as unboundable, and the time spent in the parser and the rewriter.

func (*DeviceBackend) Objects 

func (b *DeviceBackend) Objects() []any

Objects returns the handles the backend holds (for inspection by tests).

func (*DeviceBackend) Read 

func (b *DeviceBackend) Read(id uint32, offset uint64, dst []byte) error

Read implements Reader: GPUBuffer.mapAsync(READ), getMappedRange, copy, unmap, on a buffer of this backend.

func (*DeviceBackend) Replay 

func (b *DeviceBackend) Replay(words []uint32, data []byte) error

Replay implements Backend: the stream may name this backend's objects only, and Replay returns once the GPU has executed it.

func (*DeviceBackend) SetLoopBudget added in v0.12.0 

func (b *DeviceBackend) SetLoopBudget(lb LoopBudget) error

SetLoopBudget turns the loop budget on for the objects this backend creates from now on (call it before the first Create). See LoopBudget and Guard.

type Guard added in v0.14.0 

type Guard struct {
	// contains filtered or unexported fields
}

Guard is the loop budget (LoopBudget) of one browser device: the host's flag, budget and empty bind groups, the table of rewritten modules and pipelines, and the checks every path that reaches createShaderModule must go through. DeviceBackend (the batched stream) uses it by replay id; objects-API hosts (the wasi:webgpu module, wasi-nn) use Shader, Layout, Pipeline, Pass, Finish and Submit. A module the Guard cannot bound is refused with ErrUnbounded: nothing unbounded runs.

func NewGuard added in v0.14.0 

func NewGuard(dev *webgpu.Device, lb LoopBudget) (*Guard, error)

NewGuard makes the guard of a device (see LoopBudget).

func (*Guard) CheckLayout added in v0.14.0 

func (g *Guard) CheckLayout(bgls []*webgpu.BindGroupLayout) error

CheckLayout refuses a guest layout that reaches the reserved group.

func (*Guard) Close added in v0.14.0 

func (g *Guard) Close()

Close destroys the guard's buffers.

func (*Guard) Finish added in v0.14.0 

func (g *Guard) Finish(enc *webgpu.CommandEncoder) (*webgpu.CommandBuffer, error)

Finish is enc.Finish; the command buffer of an armed encoder carries a copy of the flag to a MAP_READ buffer of its own (submits may overlap, so each reads its own copy). The flag is never cleared between submits: once a loop is cut the work is condemned, and Submit re-arms it when it reports it.

func (*Guard) NoteLayout added in v0.14.0 

func (g *Guard) NoteLayout(pl *webgpu.PipelineLayout, bgls []*webgpu.BindGroupLayout)

NoteLayout remembers a guest layout's own groups (after CheckLayout).

func (*Guard) Pass added in v0.14.0 

func (g *Guard) Pass(enc *webgpu.CommandEncoder, pe *webgpu.ComputePassEncoder) *PassGuard

Pass wraps pe, a pass of enc.

func (*Guard) Pipeline added in v0.14.0 

func (g *Guard) Pipeline(d webgpu.ComputePipelineDescriptor) (*webgpu.ComputePipeline, error)

Pipeline makes a compute pipeline (objects API). A pipeline of a rewritten module gets the flag group in its layout; one of an unrewritten module has no loop to bound.

func (*Guard) PipelineAsync added in v0.14.0 

func (g *Guard) PipelineAsync(ctx context.Context, d webgpu.ComputePipelineDescriptor) (*webgpu.ComputePipeline, error)

PipelineAsync is Pipeline with CreateComputePipelineAsync.

func (*Guard) Shader added in v0.14.0 

func (g *Guard) Shader(d webgpu.ShaderModuleDescriptor) (*webgpu.ShaderModule, error)

Shader makes a guest's shader module, rewritten if it has loops. The browser's verdict on the guest's own text comes first, so an invalid module is reported as the browser reports it (the guest's own line and column). A valid module the host cannot bound (it does not parse, it has a loop in a module with a vertex or fragment entry, or the browser refuses the rewrite) is refused with ErrUnbounded: it never runs unbounded.

func (*Guard) ShaderNoWait added in v0.14.0 

func (g *Guard) ShaderNoWait(d webgpu.ShaderModuleDescriptor) (*webgpu.ShaderModule, error)

ShaderNoWait is Shader for a host that cannot wait on the browser (a synchronous call): the same checks and rewrite, but the browser's verdict is not asked for. A module the host cannot bound is refused at once; a module the browser refuses stays invalid (its pipeline fails), and a rewritten text the browser refuses cannot run either.

func (*Guard) Stats added in v0.14.0 

func (g *Guard) Stats() (modules, rewritten, refused int, spent time.Duration)

Stats reports the WGSL modules seen, rewritten and refused as unboundable, and the time spent in the parser and the rewriter.

func (*Guard) Submit added in v0.14.0 

func (g *Guard) Submit(q *webgpu.Queue, cbs ...*webgpu.CommandBuffer) error

Submit is q.Submit; if a command buffer is armed, the flag copies are read after the work (which waits for it): ErrTruncated if a loop was cut.

func (*Guard) SubmitNoWait added in v0.14.0 

func (g *Guard) SubmitNoWait(q *webgpu.Queue, cbs ...*webgpu.CommandBuffer) (func() error, error)

SubmitNoWait is Submit for a host that cannot wait in the call (a synchronous import): it submits and returns a function that waits for the work and reads the flag copies (ErrTruncated if a loop was cut). The function must be called exactly once; it is nil if nothing was armed.

type LoopBudget added in v0.12.0 

type LoopBudget struct {
	Work          uint64 // invocation-iterations per dispatch (> 0)
	MinIterations uint32 // the smallest B (default 1024)
	// MaxIterations is the largest B (default 2^24): Work/invocations alone
	// would give a one-workgroup dispatch billions of iterations, and a loop
	// that serial takes seconds, since an invocation's iterations are not
	// parallel (the first Firefox run of the hostile test found this).
	MaxIterations uint32
	// SmallLoop is the largest constant trip count of a leaf loop that is
	// counted in one addition instead of checked per iteration (default 0: off;
	// transform.Config.SmallLoop). The bound stays exact; measured on the LLM
	// decode it gained nothing (531 against 534 tok/s), so it is off.
	SmallLoop            uint32
	WorkgroupInvocations uint32 // invocations one workgroup counts as (default 256)
	IndirectInvocations  uint64 // a DispatchIndirect counts as this many (default 2^26)
	Group                uint32 // the reserved bind group
}

LoopBudget bounds the GPU time one dispatch can burn inside a loop, which WebGPU cannot cancel once submitted (layer 4 of go-wasmcontainers/gpu's scheduling). A DeviceBackend with a LoopBudget rewrites every guest WGSL module that has a compute entry point (go-gfx/wgsl's transform.LoopBudget): each outermost loop nest of each function activation runs at most B iterations, then the invocation leaves the loop and stores 1 into a host-owned flag. After every stream that ran such a pipeline the backend reads the flag; a set flag fails the stream with ErrTruncated, so a partial result is never mistaken for a complete one.

The budget B of a dispatch is Work divided by its invocations (its workgroup counts times WorkgroupInvocations), rounded down to a power of two and at least MinIterations: a dispatch's loops cost at most about Work invocation-iterations whatever its size, so a big dispatch gets short loops and a small one long loops. A DispatchIndirect, whose counts are in GPU memory, is budgeted as a dispatch of IndirectInvocations.

Reserved slots. In bind group Group, binding 0 is the flag (`var<storage, read_write> atomic<u32>`) and binding 1 the budget (`var<uniform> u32`, bound with a dynamic offset into a host buffer holding 2^0 ... 2^32-1, one per 256 bytes). Group is reserved on the whole device: a guest pipeline layout with more than Group bind group layouts, a WGSL module that declares a binding in @group(Group), and a SetBindGroup of index Group or more are refused with ErrReserved. Group is normally the device's maxBindGroups-1 (3 by default).

func (LoopBudget) Analyze added in v0.14.0 

func (lb LoopBudget) Analyze(code string) (Analysis, error)

Analyze is Instrument with the module's declared groups.

func (LoopBudget) Instrument added in v0.12.0 

func (lb LoopBudget) Instrument(code string) (string, bool, error)

Instrument returns the module text to hand to the browser, and whether it was rewritten. It parses only (never Validate). A module with no loop is returned unchanged: it is bounded already. A module with a loop and only compute entry points is rewritten. It is refused, never passed through, when it cannot be bounded: ErrUnparsable (the host's parser does not accept it), ErrUnbounded (a loop in a module that has a vertex or fragment entry point, whose stages cannot take the storage flag; or the transform failed), and ErrReserved (a binding in the reserved group). The caller decides what the browser says first: an invalid module is reported as the browser reports it.

func (LoopBudget) Iterations added in v0.12.0 

func (lb LoopBudget) Iterations(inv uint64) uint32

Iterations returns the budget B of a dispatch of inv invocations: the per-nest iteration bound its loops get. lb must have its defaults filled.

func (LoopBudget) Validate added in v0.12.0 

func (lb LoopBudget) Validate() error

Validate checks the configuration.

type PassGuard added in v0.14.0 

type PassGuard struct {
	// contains filtered or unexported fields
}

PassGuard wraps a compute pass encoder: before each dispatch of a rewritten pipeline it sets the host's groups (an empty group wherever the layout needs one, the flag group with the dispatch's budget).

func (*PassGuard) DispatchWorkgroups added in v0.14.0 

func (p *PassGuard) DispatchWorkgroups(x, y, z uint32)

DispatchWorkgroups is pe.DispatchWorkgroups.

func (*PassGuard) DispatchWorkgroupsIndirect added in v0.14.0 

func (p *PassGuard) DispatchWorkgroupsIndirect(b *webgpu.Buffer, offset uint64)

DispatchWorkgroupsIndirect is pe.DispatchWorkgroupsIndirect, budgeted as a dispatch of LoopBudget.IndirectInvocations.

func (*PassGuard) End added in v0.14.0 

func (p *PassGuard) End() error

End is pe.End; an encoder whose pass used a rewritten pipeline is armed: Finish adds the flag's copy.

func (*PassGuard) SetBindGroup added in v0.14.0 

func (p *PassGuard) SetBindGroup(index uint32, bg *webgpu.BindGroup, offsets ...uint32) error

SetBindGroup is pe.SetBindGroup; the reserved group is refused.

func (*PassGuard) SetPipeline added in v0.14.0 

func (p *PassGuard) SetPipeline(pl *webgpu.ComputePipeline)

SetPipeline is pe.SetPipeline.

type Reader 

type Reader interface {
	Backend
	Read(id uint32, offset uint64, dst []byte) error
}

Reader is a Backend that can read buffers back. Read copies [offset, offset+len(dst)) of buffer id into dst after a Replay has returned (mapAsync, getMappedRange, copy, unmap); the Context has checked the range, the buffer's MAP_READ usage and the window.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.