Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func SanitizeProductName ¶
SanitizeProductName sanitizes the product name as in NVIDIA device plugin.
e.g., "NVIDIA H100 80GB HBM3" becomes "NVIDIA-H100-80GB-HBM3" "NVIDIA H100 PCIe" becomes "NVIDIA-H100-PCIe"
ref. https://github.com/NVIDIA/k8s-device-plugin/blob/f666bc3f836a09ae2fda439f3d7a8d8b06b48ac4/internal/lm/resource.go#L187-L204 ref. https://github.com/NVIDIA/k8s-device-plugin/blob/f666bc3f836a09ae2fda439f3d7a8d8b06b48ac4/internal/lm/resource.go#L314-L322
func SupportFabricStateByGPUProduct ¶
SupportFabricStateByGPUProduct reports whether the GPU surface exposes NVML fabric state telemetry (nvmlDeviceGetGpuFabricInfo*). This is available on Hopper + NVSwitch systems (e.g., H100, H200) where GPUs are registered with NVIDIA Fabric Manager, as well as on newer NVOS-managed systems like GB200.
The nvmlDeviceGetGpuFabricInfo API is specifically designed for Hopper architecture GPUs with NVSwitch, allowing monitoring of GPU registration status with the NVLink fabric. GPU fabric registration status is exposed through the NVML APIs and nvidia-smi.
References:
- NVML API: https://docs.nvidia.com/deploy/nvml-api/group__nvmlDeviceQueries.html "On Hopper + NVSwitch systems, GPU is registered with the NVIDIA Fabric Manager. This API reports the current state of the GPU in the NVLink fabric."
- Fabric Manager Guide: https://docs.nvidia.com/datacenter/tesla/fabric-manager-user-guide/index.html Documents H100/H200 fabric state monitoring and registration process
func SupportedFMByGPUProduct ¶
SupportedFMByGPUProduct returns the GPU fabric manager support status based on the GPU product name. This only reflects the legacy nv-fabricmanager daemon that listens on port 6666 on compute nodes and does not cover NVOS/ NVSM-based fabric state telemetry available on newer systems like GB200.
Types ¶
type MemoryErrorManagementCapabilities ¶
type MemoryErrorManagementCapabilities struct {
// (If supported) GPU can limit the impact of uncorrectable ECC errors to GPU applications.
// Existing/new workloads will run unaffected, both in terms of accuracy and performance.
// Thus, does not require a GPU reset when memory errors occur.
//
// Note thtat there are some rarer cases, where uncorrectable errors are still uncontained
// thus impacting all other workloads being procssed in the GPU.
//
// ref. https://docs.nvidia.com/deploy/a100-gpu-mem-error-mgmt/index.html#error-containments
ErrorContainment bool `json:"error_containment"`
// (If supported) GPU can dynamically mark the page containing uncorrectable errors
// as unusable, and any existing or new workloads will not be allocating this page.
//
// Thus, does not require a GPU reset to recover from most uncorrectable ECC errors.
//
// ref. https://docs.nvidia.com/deploy/a100-gpu-mem-error-mgmt/index.html#dynamic-page-offlining
DynamicPageOfflining bool `json:"dynamic_page_offlining"`
// (If supported) GPU can replace degrading memory cells with spare ones
// to avoid offlining regions of memory. And the row remapping is different
// from dynamic page offlining which is fixed at a hardware level.
//
// The row remapping requires a GPU reset to take effect.
//
// Even for "NVIDIA GeForce RTX 4090", NVML API returns no error on the remapped rows API,
// thus "NVML.Supported" is not a reliable way to check if row remapping is supported.
// We track a separate boolean value based on the GPU product name.
//
// ref. https://docs.nvidia.com/deploy/a100-gpu-mem-error-mgmt/index.html#row-remapping
RowRemapping bool `json:"row_remapping"`
// Message contains the message to the user about the memory error management capabilities.
Message string `json:"message,omitempty"`
}
Contains information about the GPU's memory error management capabilities. ref. https://docs.nvidia.com/deploy/a100-gpu-mem-error-mgmt/index.html#supported-gpus
func SupportedMemoryMgmtCapsByGPUProduct ¶
func SupportedMemoryMgmtCapsByGPUProduct(gpuProductName string) MemoryErrorManagementCapabilities
SupportedMemoryMgmtCapsByGPUProduct returns the GPU memory error management capabilities based on the GPU product name. ref. https://docs.nvidia.com/deploy/a100-gpu-mem-error-mgmt/index.html#supported-gpus