Documentation
¶
Overview ¶
Package rdma provides Go bindings to libcuobjserver via CGO.
Index ¶
- Constants
- func ConfigureTelemetry(debug bool)
- func DebugTelemetryEnabled() bool
- type Buffer
- type RDMATunables
- type Server
- func (s *Server) AllocHostBuffer(size int) (unsafe.Pointer, error)
- func (s *Server) AllocateChannel() (uint16, error)
- func (s *Server) Close()
- func (s *Server) CloseSession()
- func (s *Server) DeregisterBuffer(buf *Buffer)
- func (s *Server) FreeChannel(id uint16)
- func (s *Server) FreeHostBuffer(ptr unsafe.Pointer)
- func (s *Server) HandleGet(key string, buf *Buffer, remoteStart uint64, size int64, rdmaDescr string, ...) (int64, error)
- func (s *Server) HandlePut(key string, buf *Buffer, remoteStart uint64, size int64, rdmaDescr string, ...) (int64, error)
- func (s *Server) InitRDMAConfig(t RDMATunables) error
- func (s *Server) IsConnected() bool
- func (s *Server) RegisterBuffer(ptr unsafe.Pointer, size int) (*Buffer, error)
- func (s *Server) StartSession() error
Constants ¶
const MaxTransferSize = 1 << 30
MaxTransferSize is the cuObjServer limit per RDMA operation (1 GiB).
Variables ¶
This section is empty.
Functions ¶
func ConfigureTelemetry ¶
func ConfigureTelemetry(debug bool)
ConfigureTelemetry controls cuObjServer telemetry logging. When debug is true, enable info+debug+error logs to stderr/stdout path.
func DebugTelemetryEnabled ¶
func DebugTelemetryEnabled() bool
DebugTelemetryEnabled reports whether verbose RDMA diagnostics are enabled.
Types ¶
type Buffer ¶
type Buffer struct {
// contains filtered or unexported fields
}
Buffer wraps an RDMA-registered memory region.
type RDMATunables ¶
type RDMATunables struct {
// NumDCIs controls the maximum number of concurrent RDMA connections
// (Dynamic Connection Interfaces). Default: 128.
NumDCIs int
// CQDepth is the completion queue depth, limiting outstanding ops.
// Default: 640.
CQDepth uint32
// DCKey is the InfiniBand Dynamic Connection security key. All clients
// must use a matching key. Change this from the default in production.
// Default: 0xffeeddcc.
DCKey uint64
// ServiceLevel sets the IB QoS service level. Default: 0.
ServiceLevel int
// Timeout is the QP ACK timeout exponent: 4.096 * 2^Timeout µs.
// Default: 16 (~268 ms).
Timeout uint8
// HopLimit is the IB packet hop limit (analogous to IP TTL). Default: 4.
HopLimit uint32
// PKeyIndex is the IB partition key index. Default: 0.
PKeyIndex int
// DelayInterval is the polling delay in nanoseconds. Default: 5000.
DelayInterval uint32
// DelayMode selects the polling strategy:
// 0 = none, 1 = batch (default), 2 = per-entry, 3 = adaptive.
DelayMode int
// RetryCount is the QP retry count (0–7). Default: 7.
RetryCount uint8
// QPResetOnFailure controls whether the QP is reset after an RDMA failure.
// Default: true.
QPResetOnFailure bool
// TrafficClass sets the IB traffic class / DSCP bits. Default: 96.
TrafficClass uint32
// MaxRdAtomic is the max outstanding RDMA reads per DCI QP.
// 0 means auto-detect from device capabilities. Default: 0.
MaxRdAtomic int
}
RDMATunables holds RDMA connection tuning parameters that map to cuObjRDMATunableParam. Pass a non-nil pointer in CuServerOpts to apply custom settings before the RDMA session starts; nil means library defaults.
Use DefaultRDMATunables() to start from the library defaults and adjust only the fields you care about.
func DefaultRDMATunables ¶
func DefaultRDMATunables() RDMATunables
DefaultRDMATunables returns a RDMATunables pre-populated with the cuObjServer library defaults. Use this as a starting point and override only the fields you need.
type Server ¶
type Server struct {
// contains filtered or unexported fields
}
Server wraps a cuObjServer instance.
func NewServer ¶
func NewServer(ip string, port uint16, tunables *RDMATunables) (*Server, error)
NewServer creates a cuObjServer bound to the given RDMA IP and port. Uses CUOBJ_PROTO_RDMA_DC_V1 (1001). If tunables is non-nil, the 4-argument constructor is used so the tunable parameters apply to the initial session started by the constructor. Pass nil to use library defaults.
func (*Server) AllocHostBuffer ¶
AllocHostBuffer allocates a 4KB-aligned host buffer of the given size.
func (*Server) AllocateChannel ¶
AllocateChannel obtains a unique channel ID for concurrent RDMA operations.
func (*Server) Close ¶
func (s *Server) Close()
Close destroys the cuObjServer instance. The Server must not be used afterward.
func (*Server) CloseSession ¶
func (s *Server) CloseSession()
CloseSession tears down the RDMA session.
func (*Server) DeregisterBuffer ¶
DeregisterBuffer deregisters a previously registered RDMA buffer.
func (*Server) FreeChannel ¶
FreeChannel releases a previously allocated channel ID.
func (*Server) FreeHostBuffer ¶
FreeHostBuffer releases a buffer previously allocated by AllocHostBuffer.
func (*Server) HandleGet ¶
func (s *Server) HandleGet(key string, buf *Buffer, remoteStart uint64, size int64, rdmaDescr string, channel uint16) (int64, error)
HandleGet performs an RDMA WRITE (server→client) to serve a GET request. The local buffer must already contain the data to send. Returns bytes transferred.
func (*Server) HandlePut ¶
func (s *Server) HandlePut(key string, buf *Buffer, remoteStart uint64, size int64, rdmaDescr string, channel uint16) (int64, error)
HandlePut performs an RDMA READ (client→server) to serve a PUT request. After return, the local buffer contains the data read from the client. Returns bytes transferred.
func (*Server) InitRDMAConfig ¶
func (s *Server) InitRDMAConfig(t RDMATunables) error
InitRDMAConfig applies RDMA tuning parameters. Must be called before StartSession.
func (*Server) IsConnected ¶
IsConnected returns the RDMA connection status.
func (*Server) RegisterBuffer ¶
RegisterBuffer registers a host memory region for RDMA and returns a Buffer handle.
func (*Server) StartSession ¶
StartSession initiates the RDMA listening session. StartSession must not be called concurrently with CloseSession or Close.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package bufferpool provides a pool of pre-registered RDMA buffers to avoid per-request allocation and registration overhead.
|
Package bufferpool provides a pool of pre-registered RDMA buffers to avoid per-request allocation and registration overhead. |
|
Package hostclient implements the client side of the cuObject RDMA DC protocol using libibverbs + mlx5 direct verbs, with no CUDA/GPU dependency.
|
Package hostclient implements the client side of the cuObject RDMA DC protocol using libibverbs + mlx5 direct verbs, with no CUDA/GPU dependency. |
|
Package rcroutes serves the /.hipobj-rc/{prepare,ready,cancel} terminal routes of the hipobj-rc-v2 two-phase transfer protocol.
|
Package rcroutes serves the /.hipobj-rc/{prepare,ready,cancel} terminal routes of the hipobj-rc-v2 two-phase transfer protocol. |
|
Package rcserver binds the RC session server (rdma/librcserver.a) that implements the hipobj-rc-v2 two-phase transfer protocol.
|
Package rcserver binds the RC session server (rdma/librcserver.a) that implements the hipobj-rc-v2 two-phase transfer protocol. |