Documentation
¶
Overview ¶
Package config provides the gpud configuration data for the server.
Index ¶
- Constants
- Variables
- func DefaultFifoFile() (string, error)
- func DefaultStateFile() (string, error)
- func FifoFilePath(dataDir string) string
- func PackagesDir(dataDir string) string
- func ResolveDataDir(dataDir string) (string, error)
- func StateFilePath(dataDir string) string
- func VersionFilePath(dataDir string) string
- type Config
- type NVSentinelConfig
- type Op
- type OpOption
- func WithBlockdevUsageCommands(commands string) OpOption
- func WithContainerd(c pkgconfigcommon.ContainerdConfig) OpOption
- func WithContainerdServiceActiveCommands(commands string) OpOption
- func WithDBInMemory(b bool) OpOption
- func WithDataDir(dataDir string) OpOption
- func WithExcludedInfinibandDevices(devices []string) OpOption
- func WithFailureInjector(injector *components.FailureInjector) OpOption
- func WithFindmntCommands(commands string) OpOption
- func WithInfinibandClassRootDir(p string) OpOption
- func WithLsblkCommands(commands string) OpOption
- func WithRebootCommands(commands string) OpOption
- func WithSessionEndpoint(endpoint string) OpOption
- func WithSessionMachineID(machineID string) OpOption
- func WithSessionMachineProof(machineProof string) OpOption
- func WithSessionToken(token string) OpOption
Constants ¶
const ( DefaultAPIVersion = "v1" DefaultGPUdPort = 15132 DefaultDataDir = "/var/lib/gpud" DefaultSessionProtocol = "auto" )
Variables ¶
var ( DefaultRefreshPeriod = metav1.Duration{Duration: time.Minute} // keep the metrics only for the last 3 hours DefaultMetricsRetentionPeriod = metav1.Duration{Duration: 3 * time.Hour} // keep component events only for the last 14 days DefaultEventsRetentionPeriod = metav1.Duration{Duration: 14 * 24 * time.Hour} // compact/vacuum is disruptive to existing queries (including reads) // but necessary to keep the state database from growing indefinitely // TODO: disabled for now, until we have a better way to detect the performance issue DefaultCompactPeriod = metav1.Duration{Duration: 0} )
Functions ¶
func DefaultFifoFile ¶
func DefaultStateFile ¶
func FifoFilePath ¶ added in v0.9.0
FifoFilePath returns the FIFO pipe path under the dataDir.
func PackagesDir ¶ added in v0.9.0
PackagesDir returns the packages directory under the dataDir.
func ResolveDataDir ¶ added in v0.9.0
ResolveDataDir resolves and validates a data directory path. If dataDir is empty or matches DefaultDataDir, it uses platform-specific logic:
- For root users (or when /var/lib exists): /var/lib/gpud
- For non-root users: $HOME/.gpud
For non-empty custom paths, it ensures the directory exists and is writable. The directory is created with 0755 permissions if it doesn't exist.
func StateFilePath ¶ added in v0.9.0
StateFilePath returns the state DB file path under the dataDir.
func VersionFilePath ¶ added in v0.9.0
VersionFilePath returns the version file path under the dataDir.
Types ¶
type Config ¶
type Config struct {
APIVersion string `json:"api_version"`
// Address for the server to listen on.
Address string `json:"address"`
// DataDir is the root directory for GPUd state and package artifacts.
DataDir string `json:"data_dir"`
// State file that persists the latest status.
// If empty, the states are not persisted to file.
State string `json:"state"`
// Amount of time to retain states/metrics for.
// Once elapsed, old states/metrics are purged/compacted.
MetricsRetentionPeriod metav1.Duration `json:"metrics_retention_period"`
// Amount of time to retain component events for.
// Once elapsed, old events are purged from the event store.
EventsRetentionPeriod metav1.Duration `json:"events_retention_period"`
// Interval at which to compact the state database.
CompactPeriod metav1.Duration `json:"compact_period"`
// Set true to enable profiler.
Pprof bool `json:"pprof"`
// Set false to disable auto update
EnableAutoUpdate bool `json:"enable_auto_update"`
// Exit code to exit with when auto updating.
// Only valid when the auto update is enabled.
// Set -1 to disable the auto update by exit code.
AutoUpdateExitCode int `json:"auto_update_exit_code"`
// RebootCommands is a bash script to run when the control plane sends a reboot session request.
// Empty preserves the built-in "sudo reboot" path.
RebootCommands string `json:"reboot_commands,omitempty"`
// FindmntCommands overrides how the disk component invokes "findmnt".
// Empty preserves the legacy behavior of locating "findmnt" on PATH and
// running it in the current namespace. When set (e.g.
// "nsenter --target 1 --mount -- findmnt"), it runs in the host mount
// namespace so the disk component reports the host's mounts.
FindmntCommands string `json:"findmnt_commands,omitempty"`
// LsblkCommands overrides how the disk component invokes "lsblk".
// Empty preserves the legacy behavior. When set (e.g.
// "nsenter --target 1 --mount -- lsblk"), it runs in the host mount namespace.
LsblkCommands string `json:"lsblk_commands,omitempty"`
// BlockdevUsageCommands overrides how the disk component collects partition
// usage. Empty preserves the legacy behavior of enumerating mounts via
// gopsutil and measuring usage via the statfs syscall. When set (e.g.
// "nsenter --target 1 --mount -- df"), partitions and usage are read from
// that command's output in the host mount namespace.
BlockdevUsageCommands string `json:"blockdev_usage_commands,omitempty"`
// NFSHostRoot prefixes NFS volume paths for file operations. Empty preserves
// the existing direct-path behavior used by systemd installations. A
// containerized deployment can use /proc/1/root to access the node filesystem
// without mounting every node-group-specific NFS path into the container.
NFSHostRoot string `json:"nfs_host_root,omitempty"`
// ContainerdServiceActiveCommands overrides how the containerd component checks
// whether the containerd service is active. Empty preserves the legacy
// behavior of calling systemd directly (systemd.IsActive), which only sees the
// container's own service manager. When set (e.g.
// "nsenter --target 1 --mount -- systemctl is-active containerd"), the command
// runs against the host's service manager; exit code 0 means active.
ContainerdServiceActiveCommands string `json:"containerd_service_active_commands,omitempty"`
// ContainerdDanglingPodGracePeriod delays counting API-absent READY sandboxes. Zero counts immediately.
ContainerdDanglingPodGracePeriod metav1.Duration `json:"containerd_dangling_pod_grace_period,omitempty"`
Containerd pkgconfigcommon.ContainerdConfig `json:"containerd,omitempty"`
// VersionFile is the file that contains the target version.
// If empty, the version file is not used.
VersionFile string `json:"version_file"`
// A list of nvidia tool command paths to overwrite the default paths.
NvidiaToolOverwrites pkgconfigcommon.ToolOverwrites `json:"nvidia_tool_overwrites"`
// PluginSpecsFile is the file that contains the plugin specs.
PluginSpecsFile string `json:"plugin_specs_file"`
// NVSentinel configures the optional NVSentinel integration.
// When enabled, GPUd serves the NVSentinel PlatformConnector gRPC API on
// a local unix socket. The node's NVSentinel platform-connector forwards
// health events to that socket. Components prefer the NVSentinel data
// point when one exists and fall back to their own detection otherwise.
NVSentinel *NVSentinelConfig `json:"nvsentinel,omitempty"`
// Components specifies the components to enable.
// Leave empty, "*", or "all" to enable all components.
// Or prefix component names with "-" to disable them.
Components []string `json:"components"`
// FailureInjector is the failure injector.
FailureInjector *components.FailureInjector `json:"failure_injector,omitempty"`
// SkipSessionUpdateConfig skips processing of updateConfig session commands. Intended for testing.
SkipSessionUpdateConfig bool `json:"skip_session_update_config"`
// SessionProtocol selects the control-plane session transport: v1, v2, or auto.
SessionProtocol string `json:"session_protocol"`
// DBInMemory enables in-memory SQLite database mode.
// When true, the database is opened as a shared in-memory database (file::memory:?cache=shared)
// instead of using the State file path. Data will not persist across restarts.
// ref. https://github.com/mattn/go-sqlite3?tab=readme-ov-file#faq
DBInMemory bool `json:"db_in_memory"`
// SessionToken is the session token for control plane authentication.
// Used when DBInMemory is true and session credentials are passed via CLI flags.
// This allows gpud up to pass the session token from login to gpud run.
SessionToken string `json:"-"`
// SessionMachineID is the machine ID assigned by the control plane.
// Used when DBInMemory is true and session credentials are passed via CLI flags.
// This allows gpud up to pass the assigned machine ID from login to gpud run.
SessionMachineID string `json:"-"`
// SessionMachineProof is the per-machine proof returned by login.
SessionMachineProof string `json:"-"`
// SessionEndpoint is the control plane endpoint.
// Used when DBInMemory is true and session credentials are passed via CLI flags.
// This allows gpud up to pass the endpoint from login to gpud run.
// The server reads the endpoint from metadata DB, so it must be seeded for in-memory mode.
SessionEndpoint string `json:"-"`
// contains filtered or unexported fields
}
Config provides gpud configuration data for the server
func (*Config) ShouldDisable ¶ added in v0.5.0
ShouldDisable returns true if the component should be disabled. If the component list is not specified, it returns false, meaning no component is disabled by default.
func (*Config) ShouldEnable ¶ added in v0.5.0
ShouldEnable returns true if the component should be enabled. If the component list is not specified, it returns true, meaning every component is enabled by default. A list containing only "-"-prefixed entries enables every component except the denylisted ones. Otherwise the positive entries form an allowlist ("*" and "all" select every component), and "-"-prefixed entries subtract from the result in both modes.
type NVSentinelConfig ¶ added in v0.13.0
type NVSentinelConfig struct {
// Enabled turns the NVSentinel receiver on. Default false.
Enabled bool `json:"enabled"`
// SocketPath is the unix socket GPUd serves for NVSentinel health event
// forwarding. The NVSentinel platform-connector container must reach this
// path. The default works with the stock NVSentinel DaemonSet, which
// mounts the host /var/run/nvsentinel directory at container /var/run.
// Empty means nvsentinel.DefaultSocketPath.
SocketPath string `json:"socket_path,omitempty"`
// EventDedupWindow is how long a received NVSentinel event suppresses
// GPUd's own duplicate detection of the same data point. It needs only
// to cover the delivery skew between the two detectors that watch the
// same incident. Zero means nvsentinel.DefaultEventDedupWindow.
EventDedupWindow metav1.Duration `json:"event_dedup_window,omitempty"`
}
NVSentinelConfig holds the top-level NVSentinel endpoint settings.
type Op ¶
type Op struct {
pkgconfigcommon.ToolOverwrites
FailureInjector *components.FailureInjector
DataDir string
DBInMemory bool
// RebootCommands is an optional bash script used for control-plane reboot requests.
// Empty keeps the default host reboot implementation.
RebootCommands string
// FindmntCommands optionally overrides how the disk component invokes "findmnt".
// Empty keeps the legacy in-namespace behavior.
FindmntCommands string
// LsblkCommands optionally overrides how the disk component invokes "lsblk".
// Empty keeps the legacy in-namespace behavior.
LsblkCommands string
// BlockdevUsageCommands optionally overrides how the disk component collects
// partition usage. Empty keeps the legacy gopsutil + statfs behavior.
BlockdevUsageCommands string
// ContainerdServiceActiveCommands optionally overrides how the containerd
// component checks whether the containerd service is active. Empty keeps the
// legacy in-namespace systemd.IsActive behavior.
ContainerdServiceActiveCommands string
Containerd pkgconfigcommon.ContainerdConfig
// SessionToken is the session token for db-in-memory mode.
// When DBInMemory is true and this is set, the server will seed
// this token into the in-memory database.
SessionToken string
// SessionMachineID is the machine ID for db-in-memory mode.
// When DBInMemory is true and this is set, the server will seed
// this machine ID into the in-memory database.
SessionMachineID string
// SessionMachineProof is the per-machine proof for db-in-memory mode.
SessionMachineProof string
// SessionEndpoint is the control plane endpoint for db-in-memory mode.
// When DBInMemory is true and this is set, the server will seed
// this endpoint into the in-memory database.
// The server reads the endpoint from metadata DB, so it must be seeded for in-memory mode.
SessionEndpoint string
}
type OpOption ¶
type OpOption func(*Op)
func WithBlockdevUsageCommands ¶ added in v0.12.2
WithBlockdevUsageCommands overrides how the disk component collects partition usage. Empty keeps the legacy gopsutil + statfs behavior.
func WithContainerd ¶ added in v0.13.0
func WithContainerd(c pkgconfigcommon.ContainerdConfig) OpOption
WithContainerd selects the runtime installation to monitor.
func WithContainerdServiceActiveCommands ¶ added in v0.12.2
WithContainerdServiceActiveCommands overrides how the containerd component checks whether the containerd service is active. Empty keeps the legacy in-namespace systemd.IsActive behavior.
func WithDBInMemory ¶ added in v0.9.0
WithDBInMemory enables in-memory SQLite database mode. When true, uses file::memory:?cache=shared instead of file-based storage. ref. https://github.com/mattn/go-sqlite3?tab=readme-ov-file#faq
func WithDataDir ¶ added in v0.9.0
WithDataDir overrides the default data directory for GPUd artifacts.
func WithExcludedInfinibandDevices ¶ added in v0.9.0
WithExcludedInfinibandDevices sets the list of InfiniBand device names to exclude from monitoring. Device names should be like "mlx5_0", "mlx5_1", etc. (not full paths).
This is useful for excluding devices that have restricted Physical Functions (PFs) and cause kernel errors (mlx5_cmd_out_err ACCESS_REG) when queried. This is common on NVIDIA DGX, Umbriel, and GB200 systems with ConnectX-7 adapters.
ref. https://github.com/prometheus/node_exporter/issues/3434 https://github.com/leptonai/gpud/issues/1164
func WithFailureInjector ¶ added in v0.6.0
func WithFailureInjector(injector *components.FailureInjector) OpOption
func WithFindmntCommands ¶ added in v0.12.2
WithFindmntCommands overrides how the disk component invokes "findmnt". Empty keeps the legacy in-namespace behavior.
func WithInfinibandClassRootDir ¶ added in v0.5.1
Specifies the root directory of the InfiniBand class.
func WithLsblkCommands ¶ added in v0.12.2
WithLsblkCommands overrides how the disk component invokes "lsblk". Empty keeps the legacy in-namespace behavior.
func WithRebootCommands ¶ added in v0.12.1
WithRebootCommands sets the bash script to run for control-plane reboot requests. Empty keeps the default host reboot implementation.
func WithSessionEndpoint ¶ added in v0.9.0
WithSessionEndpoint sets the control plane endpoint for db-in-memory mode. When DBInMemory is true and this is set, the server will seed this endpoint into the in-memory database. The server reads the endpoint from metadata DB, so it must be seeded for in-memory mode.
func WithSessionMachineID ¶ added in v0.9.0
WithSessionMachineID sets the machine ID for db-in-memory mode. When DBInMemory is true and this is set, the server will seed this machine ID into the in-memory database.
func WithSessionMachineProof ¶ added in v0.12.9
WithSessionMachineProof sets the per-machine proof for db-in-memory mode.
func WithSessionToken ¶ added in v0.9.0
WithSessionToken sets the session token for db-in-memory mode. When DBInMemory is true and this is set, the server will seed this token into the in-memory database for session authentication.