Affected by 
Documentation
¶
Overview ¶
AMD discovery needs a small amount of backend-specific handling beyond the generic llama-server device list. ROCm devices expose their real capability as gfx targets, and the shipped rocBLAS kernels define which of those targets are actually usable. On Linux, KFD topology and DRM sysfs attributes provide the integrated-vs-discrete signal needed for scheduler decisions. On Windows, older HIP driver installs can also leave ROCm libraries present but too old to support GPU inference. These helpers keep that extra validation and warning logic in one place.
Vulkan discovery needs a small amount of normalization around device type. llama-server discovery output does not currently expose a stable structured backend type field, so we use explicit Vulkan UMA metadata when it is present and, on Windows, refine the result with a direct Vulkan API query. The goal is to preserve correct integrated-vs-discrete scheduling decisions without relying on device-name heuristics.
Index ¶
- Variables
- func GPUDevices(ctx context.Context, runners []ml.FilteredRunnerDiscovery) []ml.DeviceInfo
- func GetCPUMem() (memInfo, error)
- func GetSystemInfo() ml.SystemInfo
- func IsNUMA() bool
- func LogDetails(devices []ml.DeviceInfo)
- func RunNativeProbeCommand(ctx context.Context, libDirs []string, out io.Writer) error
Constants ¶
This section is empty.
Variables ¶
var CudaTegra string = os.Getenv("JETSON_JETPACK")
Jetson devices have JETSON_JETPACK="x.y.z" factory set to the Jetpack version installed. Included to drive logic for reducing Ollama-allocated overhead on L4T/Jetson devices.
Functions ¶
func GPUDevices ¶ added in v0.12.4
func GPUDevices(ctx context.Context, runners []ml.FilteredRunnerDiscovery) []ml.DeviceInfo
func GetSystemInfo ¶
func GetSystemInfo() ml.SystemInfo
GetSystemInfo returns host memory information used by scheduling.
func LogDetails ¶ added in v0.12.4
func LogDetails(devices []ml.DeviceInfo)
Types ¶
This section is empty.
Source Files