rdma

package
v0.6.15 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 27, 2026 License: MIT Imports: 5 Imported by: 0

Documentation

Overview

Package rdma provides operational helpers for Apple's generated RDMA bindings.

The package keeps policy that is not part of the C verbs ABI out of github.com/tmc/apple/rdma. It does not open devices, transition queue pairs, or post work requests; callers use it to classify errors, select route GIDs, and decide whether a read-only preflight permits one bounded RTR attempt.

Index

Examples

Constants

View Source 
const (
	// PortActive is the ibv_port_state value for PORT_ACTIVE.
	PortActive int32 = 4

	// LinkLayerThunderbolt is Apple's ibv_port_attr link_layer value for
	// Thunderbolt RDMA.
	LinkLayerThunderbolt uint8 = 100

	// MaxRouteGIDScan bounds automatic route-GID discovery. Apple Thunderbolt
	// devices can report large gid_tbl_len values while only exposing a small
	// useful prefix.
	MaxRouteGIDScan = 8
)
View Source 
const (
	ReasonNoSafeRouteGID     = "active Thunderbolt RDMA device has no automatic-safe route GID; index 0 requires an explicit diagnostic override"
	ReasonRouteGIDIndexZero  = "active Thunderbolt RDMA route GID resolved to index 0"
	ReasonNoActiveTBDevice   = "no PORT_ACTIVE Thunderbolt RDMA device found"
	ReasonNoPeerInterface    = "no AppleThunderboltRDMAPeerInterface entries found"
	ReasonNoXDomainService   = "no IOThunderboltXDomainService entries found"
	ReasonNoRecentLog        = "no recent AppleThunderboltRDMA log lines captured"
	ReasonRecentRTRFailure   = "recent AppleThunderboltRDMA log contains Failed INIT->RTR"
	ReasonReadOnlyPreflight  = "read-only preflight passed; safe_to_attempt_rtr is necessary, not sufficient"
	IORegistryPeerInterface  = "AppleThunderboltRDMAPeerInterface"
	IORegistryXDomainService = "IOThunderboltXDomainService"

	RTRAttemptWarning = "" /* 192-byte string literal not displayed */
)
View Source 
const RTRAttrMask = rdma.IBV_QP_STATE | rdma.IBV_QP_AV | rdma.IBV_QP_PATH_MTU | rdma.IBV_QP_DEST_QPN | rdma.IBV_QP_RQ_PSN

RTRAttrMask is the ibv_modify_qp mask for INIT->RTR.

Variables

View Source 
var (
	ErrRTRUnsafe = errors.New("rdma rtr unsafe")
)

Functions

func DerivePreflightSafety 

func DerivePreflightSafety(report PreflightReport) (bool, []string)

DerivePreflightSafety returns whether report permits one bounded RTR attempt.

A true result is necessary, not sufficient. It is not a production or performance proof.

Example 
package main

import (
	"fmt"

	xrdma "github.com/tmc/apple/x/rdma"
)

func main() {
	routeIndex := 1
	ok, reasons := xrdma.DerivePreflightSafety(xrdma.PreflightReport{
		Devices: []xrdma.PreflightDevice{{
			Name:          "rdma_en1",
			State:         xrdma.PortActive,
			LinkLayer:     xrdma.LinkLayerThunderbolt,
			RouteGIDIndex: &routeIndex,
		}},
		IORegistry: map[string]int{
			xrdma.IORegistryPeerInterface:  1,
			xrdma.IORegistryXDomainService: 1,
		},
		RecentLog: []string{"AppleThunderboltRDMA context allocation/query/free"},
	})

	fmt.Println(ok)
	fmt.Println(reasons[0])
}

Output:
true
read-only preflight passed; safe_to_attempt_rtr is necessary, not sufficient

func FailedRTRLogLine 

func FailedRTRLogLine(line string) bool

FailedRTRLogLine reports whether line contains Apple's INIT-to-RTR failure marker. Apple logs have used both ASCII "->" and Unicode "→".

func IsIPv4MappedGID 

func IsIPv4MappedGID(gid rdma.IbvGID) bool

IsIPv4MappedGID reports whether gid has the IPv4-mapped IPv6 prefix.

func IsZeroGID 

func IsZeroGID(gid rdma.IbvGID) bool

IsZeroGID reports whether gid is all zeros.

func MTUBytes added in v0.6.14 

func MTUBytes(mtu int32) int

MTUBytes returns the byte size for an ibv_mtu value.

func NegotiatedPathMTU added in v0.6.14 

func NegotiatedPathMTU(local, remote int32) int32

NegotiatedPathMTU returns the path MTU both peers can use, i.e. the smaller of the two active MTUs.

func PreflightGIDScanLimit 

func PreflightGIDScanLimit(tableLen int32, requested int) int

PreflightGIDScanLimit returns the bounded diagnostic GID scan length.

func RTRAttr added in v0.6.14 

func RTRAttr(local LocalQP, remote RemoteQP, policy RTRPolicy) (rdma.IbvQPAttr, int, error)

RTRAttr returns the ibv_modify_qp attributes and mask for INIT->RTR.

Example 
package main

import (
	"fmt"

	"github.com/tmc/apple/rdma"
	xrdma "github.com/tmc/apple/x/rdma"
)

func main() {
	remoteGID := rdma.IbvGID{0xfe, 0x80, 0, 0, 0, 0, 0, 0, 0xde, 0xad, 0, 0, 0, 0, 0, 1}

	attr, mask, err := xrdma.RTRAttr(
		xrdma.LocalQP{PortNum: 1, GIDIndex: 0, ActiveMTU: 5},
		xrdma.RemoteQP{LID: 2, QPN: 42, PSN: 7, GID: remoteGID, UseGlobal: true, ActiveMTU: 5},
		xrdma.RTRPolicy{},
	)

	fmt.Println(err)
	fmt.Printf("%#x %d %d %d\n", mask, attr.PathMTU, attr.AHAttr.DLID, attr.AHAttr.GRH.HopLimit)
}

Output:
<nil>
0x101181 5 2 1

func RequireRTRAttemptAllowed 

func RequireRTRAttemptAllowed(allow bool) error

RequireRTRAttemptAllowed returns ErrRTRUnsafe unless allow is true.

Example 
package main

import (
	"errors"
	"fmt"

	xrdma "github.com/tmc/apple/x/rdma"
)

func main() {
	err := xrdma.RequireRTRAttemptAllowed(false)

	fmt.Println(errors.Is(err, xrdma.ErrRTRUnsafe))
	fmt.Println(err)
}

Output:
true
rdma-pingpong drives QP INIT->RTR, which can wedge Apple Thunderbolt RDMA ports; run rdmainfo preflight, run rdma-probe, and read the README first, then pass -allow-rtr for one bounded attempt

func ResourceExhaustionHint added in v0.6.14 

func ResourceExhaustionHint(err error) string

ResourceExhaustionHint returns an operator hint for failures that can indicate the Apple Thunderbolt RDMA per-boot resource exhaustion pattern.

func RouteGIDScanLimit 

func RouteGIDScanLimit(tableLen int32) int

RouteGIDScanLimit returns the bounded automatic GID scan length.

Types

type GIDIndexRangeError added in v0.6.14 

type GIDIndexRangeError int

GIDIndexRangeError reports a source GID index that cannot fit in ibv_global_route.

func (GIDIndexRangeError) Error added in v0.6.14 

func (e GIDIndexRangeError) Error() string

type GIDInfo 

type GIDInfo struct {
	Index      int
	Raw        string
	IPv4Mapped bool
}

GIDInfo is the JSON-friendly form used by diagnostics.

func SelectRouteGIDInfo 

func SelectRouteGIDInfo(gids []GIDInfo, linkLayer uint8) (GIDInfo, bool)

SelectRouteGIDInfo applies the automatic route-GID policy to diagnostic records.

type LocalQP added in v0.6.14 

type LocalQP struct {
	PortNum   uint8
	GIDIndex  int
	ActiveMTU int32
	LinkLayer uint8
}

LocalQP describes the local side of a UC queue-pair connection.

type PreflightDevice 

type PreflightDevice struct {
	Name          string
	State         int32
	LinkLayer     uint8
	RouteGIDIndex *int
}

PreflightDevice is the read-only state for one RDMA device.

type PreflightReport 

type PreflightReport struct {
	Devices    []PreflightDevice
	IORegistry map[string]int
	RecentLog  []string
}

PreflightReport is the read-only evidence needed to decide whether an RDMA RTR attempt is even eligible to run.

type QPChain added in v0.6.14 

type QPChain struct {
	Context rdma.RDMAContext
	PD      rdma.RDMAPD
	MR      rdma.RDMAMR
	CQ      rdma.RDMACQ
	QP      rdma.RDMAQP
}

QPChain owns a simple ibverbs resource chain.

Close tears resources down in reverse dependency order: QP, MR, CQ, PD, then context. Zero handles are ignored, and handles are cleared before provider teardown to make repeated Close calls harmless.

func (*QPChain) Close added in v0.6.14 

func (r *QPChain) Close() error

Close releases the resources in r.

type RTRPolicy added in v0.6.14 

type RTRPolicy struct {
	ZeroDLIDWhenGlobal bool
	HopLimit           uint8
	TrafficClass       uint8
	FlowLabel          uint32
}

RTRPolicy controls provider-specific address-vector choices.

type RemoteQP added in v0.6.14 

type RemoteQP struct {
	Name      string
	LID       uint16
	QPN       uint32
	PSN       uint32
	GIDIndex  int
	GID       rdma.IbvGID
	UseGlobal bool
	ActiveMTU int32
}

RemoteQP describes the remote side of a UC queue-pair connection.

type RouteGID 

type RouteGID struct {
	Index int
	GID   rdma.IbvGID
}

RouteGID is a nonzero GID candidate returned by ibv_query_gid.

func SelectRouteGID 

func SelectRouteGID(gids []RouteGID, preferred int, linkLayer uint8) (RouteGID, bool)

SelectRouteGID selects a route GID for QP RTR setup.

If preferred is non-negative, it is treated as an explicit diagnostic override and must match a nonzero candidate. Automatic selection rejects Thunderbolt index 0, prefers IPv4-mapped GIDs, then accepts index 1. Other link layers may fall back to the first nonzero candidate.

Example 
package main

import (
	"fmt"

	"github.com/tmc/apple/rdma"
	xrdma "github.com/tmc/apple/x/rdma"
)

func main() {
	gid0 := rdma.IbvGID{0xfe, 0x80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1}
	gid1 := rdma.IbvGID{0xfe, 0x80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2}

	route, ok := xrdma.SelectRouteGID([]xrdma.RouteGID{
		{Index: 0, GID: gid0},
		{Index: 1, GID: gid1},
	}, -1, xrdma.LinkLayerThunderbolt)

	fmt.Println(ok, route.Index)
}

Output:
true 1

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.