Documentation
¶
Overview ¶
Package transport is the pluggable GPU-aware ZAP transport.
Three transports are implemented:
default — standard Go net (CPU parsing). Always available.
uma — Apple Silicon / NVIDIA Grace unified-memory. cgo + darwin.
Network packets land in shared RAM; GPU kernels read them
at GPU memory speed without a copy.
gpudirect — NVIDIA + Mellanox CX-6/7 (GPUDirect RDMA). cgo + linux.
NIC DMA into VRAM, ZAP parsed by GPU kernel, no CPU touch.
dpdk — Linux DPDK + GPU-mapped hugepages. cgo + linux.
Kernel-bypass packet ingestion, GPU reads from mapped huge-
pages. Best for non-Mellanox NICs.
All four implement the same Transport interface. Pick() chooses the best available implementation at runtime, honoring a caller-supplied preference and falling through gracefully when a preferred path is unavailable. There is no `gpu` build tag — cgo is the only gate, and the OS gate selects which native transports can compile in.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrNotAvailable = errors.New("zap/transport: requested transport not available on this host")
ErrNotAvailable is returned by Pick when no transport at the requested preference level is buildable on this host.
Functions ¶
func Registered ¶ added in v0.8.0
func Registered() []string
Registered returns the list of transports that this binary can attempt to instantiate (i.e. the platform-and-build-tag matrix). Used by tools like `zap-info` and the boot-time diagnostic in lqd.
Types ¶
type Buffer ¶ added in v0.8.0
type Buffer interface {
// Bytes returns a Go slice aliasing the buffer. Length is the
// requested size, not the slab class size. Mutating the slice
// mutates GPU-visible memory (when GPU-resident).
Bytes() []byte
// DevicePtr returns the raw device pointer. For UMA this is the
// same address as &Bytes()[0] (unified memory). For non-UMA
// allocators this is nil and the caller must not pass it to CUDA.
DevicePtr() unsafe.Pointer
// Release returns the buffer to its allocator. The slice returned
// by Bytes() must not be used after Release.
Release()
}
Buffer is one slab of managed memory handed out by a BufferAllocator. For the UMA transport on Linux+CUDA, the underlying pointer is cudaMallocManaged memory — the same bytes the GPU sees. Calling Bytes() gives a Go slice that shares the storage; calling DevicePtr() gives the raw device pointer for CUDA kernel launches.
Release returns the buffer to its allocator. Double-Release is a no-op (allocators detect and ignore it).
type BufferAllocator ¶ added in v0.8.0
type BufferAllocator interface {
// AllocBuffer returns a buffer of at least `size` bytes. The
// returned Buffer.Bytes() slice has length `size` even though the
// underlying slab class may be larger.
AllocBuffer(size int) (Buffer, error)
// PoolBytes returns the total bytes currently owned by the pool
// (configurable maximum). Used for metrics.
PoolBytes() int
}
BufferAllocator hands out managed buffers. Transports that can deliver GPU-resident bytes (UMA Linux+CUDA, UMA Darwin+Metal) implement this; transports that can't (default TCP) return ErrNotAvailable from AllocBuffer so the caller knows to fall back to heap allocation.
Capacity in bytes is the operator-configurable pool size (default 4 GiB); allocations from a full pool block until a buffer is Released.
type Capabilities ¶
type Capabilities struct {
// GPUResident is true when message bytes never visit CPU memory.
GPUResident bool
// ZeroCopy is true when no memcpy happens between NIC and the buffer
// returned by Recv. (GPU residency implies zero copy; the reverse
// does not — UMA is GPU-resident-and-zero-copy without DMA.)
ZeroCopy bool
// MinLatencyMicros is a hint for the implementation's floor latency
// on a warm path (informational; not a contract).
MinLatencyMicros float64
}
Capabilities describes what a Transport implementation supports.
type Transport ¶
type Transport interface {
// Name returns the canonical low-cardinality transport identifier
// (matches the env-var spelling): "default", "uma", "gpudirect",
// "dpdk". Used in metrics labels and log lines.
Name() string
// Send delivers msg to peer. Implementations may zero-copy directly
// from GPU memory; the caller MUST NOT reuse the msg buffer until
// the call returns.
Send(ctx context.Context, peer string, msg []byte) error
// Recv blocks until the next message arrives or ctx is cancelled.
// Returned msg may alias a GPU-mapped buffer; the caller MUST copy
// before returning the buffer to the transport.
Recv(ctx context.Context) (peer string, msg []byte, err error)
// Close releases any held resources (sockets, GPU mapped regions,
// DPDK queues, IB QPs).
Close() error
// Caps reports what this transport supports. Callers that need
// GPU-resident bytes (UMA / GPUDirect) gate on Caps().GPUResident
// at construction time, not per-call.
Caps() Capabilities
}
Transport is the wire-level surface multichain gossip uses to ship sealed MultiChainBlocks between validators. The interface is sized for one ZAP message per Send/Recv — batching happens above this layer.
func Pick ¶
Pick returns the requested transport, falling back to the next-best available implementation when the request is unavailable. Empty preference selects automatically: gpudirect > dpdk > uma > default.
The preference string is matched case-insensitively against Name(). Special value "default" returns the standard transport unconditionally.
Pick reads ZAP_TRANSPORT once; subsequent changes to the env var have no effect on already-constructed transports.
First call also emits one log line listing the transports that were probed and which won. Set ZAP_TRANSPORT_QUIET=1 to silence.
Source Files