Documentation
¶
Overview ¶
Package os queries the host OS information (e.g., kernel version).
Index ¶
- Constants
- func DefaultBlockedProcessNameRegexes() []string
- func HasVFSFileMaxLimitReached(line string) bool
- func Match(line string) (eventName string, message string)
- func New(gpudInstance *components.GPUdInstance) (components.Component, error)
- func SetDefaultBlockedProcessThresholds(th BlockedProcessThresholds) error
- func SetStartupBlockedProcessThresholds(th BlockedProcessThresholds)
- type BlockedProcess
- type BlockedProcessThresholds
- type BlockedProcesses
- type FileDescriptors
- type Host
- type Kernel
- type KernelPanicInstance
- type MachineMetadata
- type Platform
- type Uptimes
Constants ¶
const ( // DefaultMaxAllocatedFileHandles is some high number, in case the system is under high file descriptor usage. DefaultMaxAllocatedFileHandles = 10000000 // DefaultMaxRunningPIDs is some high number, in case fd-max is unlimited DefaultMaxRunningPIDs = 900000 )
const ( // EventNameBlockedProcessesPersistent is emitted on the rising edge of a // persistent blocked-process (Linux D-state) episode (LEP-6029). EventNameBlockedProcessesPersistent = "blocked_processes_persistent" // EventNameBlockedProcessesRecovered is emitted when a persistent // blocked-process episode clears. EventNameBlockedProcessesRecovered = "blocked_processes_recovered" )
const ( // DefaultBlockedProcessPersistenceThreshold is the number of consecutive // checks a process must remain blocked (Linux D-state) before it is // considered persistent. The OS component checks once per minute: five // consecutive checks means the process has been blocked for at least four // minutes of wall time (persistenceThreshold check intervals). // // Why five: transient D-state is normal -- most blocking disk I/O // (reads, writes, page faults hitting storage) passes through D for // milliseconds to seconds -- so a single observation must never flag a // node. The 2026-08-12 validation found zero D-state processes on healthy // production GPU nodes (instant scans plus a 30x10s sampling window), // while an induced real D-state process (dd on a suspended device-mapper // device) persisted >=6 minutes. Five one-minute checks sit between the // two: long enough to ignore ordinary I/O waits, short enough to surface // a wedged kernel path within minutes after it stalls. DefaultBlockedProcessPersistenceThreshold = 5 )
const ( // DefaultBlockedProcessRebootThreshold is the number of reboot events (with // no recovery in between) after which a persistent blocked-process // condition escalates from reboot suggestion to hardware inspection. // Kept at parity with the XID component's DefaultRebootThreshold // (components/accelerator/nvidia/xid). DefaultBlockedProcessRebootThreshold = 2 )
const Name = "os"
Name is the ID of the OS component.
Variables ¶
This section is empty.
Functions ¶
func DefaultBlockedProcessNameRegexes ¶ added in v0.13.0
func DefaultBlockedProcessNameRegexes() []string
DefaultBlockedProcessNameRegexes returns the default process-name regexes used to escalate persistent D-state processes on machines with NVIDIA GPUs.
func HasVFSFileMaxLimitReached ¶ added in v0.5.0
HasVFSFileMaxLimitReached returns true if the line indicates that the file-max limit has been reached. ref. https://docs.kernel.org/admin-guide/sysctl/fs.html#file-max-file-nr
func New ¶
func New(gpudInstance *components.GPUdInstance) (components.Component, error)
New creates the OS component.
func SetDefaultBlockedProcessThresholds ¶ added in v0.13.0
func SetDefaultBlockedProcessThresholds(th BlockedProcessThresholds) error
SetDefaultBlockedProcessThresholds validates, compiles, and updates the default thresholds.
func SetStartupBlockedProcessThresholds ¶ added in v0.13.0
func SetStartupBlockedProcessThresholds(th BlockedProcessThresholds)
SetStartupBlockedProcessThresholds records the startup-resolved thresholds as the baseline that session updateConfig falls back to when the control-plane config omits the os component. Called once from the gpud run/scan command wiring after flag resolution.
Types ¶
type BlockedProcess ¶ added in v0.13.0
type BlockedProcess struct {
// PID is the process ID.
PID int32 `json:"pid"`
// Name is the process name (Linux comm, truncated to 15 chars by the kernel).
Name string `json:"name"`
// FirstSeenUnixSeconds is when the process was first observed blocked.
FirstSeenUnixSeconds int64 `json:"first_seen_unix_seconds"`
// LastSeenUnixSeconds is when the process was most recently observed blocked.
LastSeenUnixSeconds int64 `json:"last_seen_unix_seconds"`
// BlockedSeconds is the observed blocked duration (last seen - first seen).
BlockedSeconds int64 `json:"blocked_seconds"`
// ConsecutiveChecks is the number of consecutive checks the process was blocked.
ConsecutiveChecks int `json:"consecutive_checks"`
}
BlockedProcess describes one process observed in uninterruptible sleep.
type BlockedProcessThresholds ¶ added in v0.13.0
type BlockedProcessThresholds struct {
// PersistenceThreshold is the number of consecutive checks a process must
// remain blocked before it is considered persistent. Values <= 0 reset to
// DefaultBlockedProcessPersistenceThreshold.
PersistenceThreshold int `json:"persistence_threshold"`
// NameRegexes gates which persistent D-state processes escalate to
// unhealthy with a repair suggestion. An empty set disables D-state
// process checking entirely.
//
// The gate exists because D-state alone does not identify the cause: a
// dd stuck on a suspended device and an nvidia-smi stuck in a wedged
// driver show the same D state letter in /proc/<pid>/stat (both become
// process.Blocked at the level gpud reads). A persistent D-state process from
// any name degrades the component (something in the kernel is stuck and
// SIGKILL cannot clear it), but a reboot suggestion is only attached when
// the name matches -- i.e., when the stuck process implicates the GPU
// driver path, as in the LEP-6029 incident (nvidia-smi in D during a
// kubelet package upgrade).
NameRegexes []string `json:"name_regexes"`
// contains filtered or unexported fields
}
BlockedProcessThresholds configures the D-state (blocked) process tracking of the os component (LEP-6029), following the same default-override pattern as components/accelerator/nvidia/gpu-counts/threshold.go.
func GetDefaultBlockedProcessThresholds ¶ added in v0.13.0
func GetDefaultBlockedProcessThresholds() BlockedProcessThresholds
GetDefaultBlockedProcessThresholds returns the configured default thresholds.
func GetStartupBlockedProcessThresholds ¶ added in v0.13.0
func GetStartupBlockedProcessThresholds() BlockedProcessThresholds
GetStartupBlockedProcessThresholds returns the thresholds resolved at process startup. If none were recorded (e.g., tests), it returns the current defaults so the updateConfig fallback leaves them unchanged.
func (BlockedProcessThresholds) IsZero ¶ added in v0.13.0
func (t BlockedProcessThresholds) IsZero() bool
IsZero returns true when no process-name regexes are configured, meaning D-state process checking is disabled.
func (BlockedProcessThresholds) MatchesName ¶ added in v0.13.0
func (t BlockedProcessThresholds) MatchesName(name string) bool
MatchesName returns true if the process name matches any configured regex.
type BlockedProcesses ¶ added in v0.13.0
type BlockedProcesses struct {
// CurrentCount is the number of processes blocked at this check.
CurrentCount int `json:"current_count"`
// PersistentCount is the number of processes that met the persistence
// threshold (may exceed len(Persistent) when the output is capped).
PersistentCount int `json:"persistent_count"`
// Persistent is the bounded list of persistent blocked processes.
Persistent []BlockedProcess `json:"persistent,omitempty"`
}
BlockedProcesses summarizes the blocked-process state of one check.
type FileDescriptors ¶ added in v0.5.0
type FileDescriptors struct {
// The number of file descriptors currently allocated on the host (not per process).
AllocatedFileHandles uint64 `json:"allocated_file_handles"`
// The number of running PIDs returned by https://pkg.go.dev/github.com/shirou/gopsutil/v4/process#Pids.
RunningPIDs uint64 `json:"running_pids"`
Usage uint64 `json:"usage"`
Limit uint64 `json:"limit"`
// AllocatedFileHandlesPercent is the percentage of file descriptors that are currently allocated,
// based on the current file descriptor limit and the current number of file descriptors allocated on the host (not per process).
AllocatedFileHandlesPercent string `json:"allocated_file_handles_percent"`
// UsedPercent is the percentage of file descriptors that are currently in use,
// based on the current file descriptor limit on the host (not per process).
UsedPercent string `json:"used_percent"`
ThresholdAllocatedFileHandles uint64 `json:"threshold_allocated_file_handles"`
ThresholdAllocatedFileHandlesPercent string `json:"threshold_allocated_file_handles_percent"`
ThresholdRunningPIDs uint64 `json:"threshold_running_pids"`
ThresholdRunningPIDsPercent string `json:"threshold_running_pids_percent"`
// Set to true if the file handles are supported.
FileHandlesSupported bool `json:"file_handles_supported"`
// Set to true if the file descriptor limit is supported.
FDLimitSupported bool `json:"fd_limit_supported"`
}
FileDescriptors represents the file descriptors of the host.
type KernelPanicInstance ¶ added in v0.6.0
type KernelPanicInstance struct {
// Process ID from the CPU line
PID int
// CPU number where the panic occurred
CPU int
// Process name (e.g., "khungtaskd")
ProcessName string
}
KernelPanicInstance contains information related to a kernel panic event
func (*KernelPanicInstance) Summary ¶ added in v0.6.0
func (k *KernelPanicInstance) Summary() string
Summary returns a human-readable summary of the kernel panic.
type MachineMetadata ¶ added in v0.3.5
type MachineMetadata struct {
BootID string `json:"boot_id"`
DmidecodeUUID string `json:"dmidecode_uuid"`
OSMachineID string `json:"os_machine_id"`
}
MachineMetadata contains stable host identity values gathered from the OS.