agent

type Agent ¶

type Agent struct {
	// contains filtered or unexported fields
}

Agent is the reconciler and supervisor for all simulators.

func New(cfg Config) *Agent

func (a *Agent) Liveness() health.Probe

func (a *Agent) Readiness() health.Probe

func (a *Agent) Run(ctx context.Context) error

func (a *Agent) Staged() health.Probe

type Applier ¶

type Applier interface {
	// Apply publishes artifacts after all Stage calls have completed (barrier).
	Apply(ctx context.Context, state *State) error
	// Revoke is the inverse of Apply and runs on shutdown before Discard.
	Revoke(ctx context.Context) error
}

Applier is implemented by simulators whose staged artifacts something outside the node acts on (containerd, NFD, the GPU Operator validator). Only gpudriver, pcibus, and cri implement this interface.

type Config ¶

type Config struct {
	Simulators      []Simulator
	Source          StateSource
	Log             *zap.Logger
	ShutdownTimeout time.Duration
	StagingLockPath string
}

Config carries Agent constructor arguments.

type Daemon ¶

type Daemon interface {
	// Run blocks until ctx is cancelled. ctx is the agent's lifetime, so a Run
	// survives every subsequent state update.
	Run(ctx context.Context) error
	// Reload delivers later States to the running daemon, so a profile edit
	// applies without a pod restart. No-op for daemons holding no staged state.
	Reload(ctx context.Context, state *State) error
}

Daemon is implemented by simulators that supervise a long-running background process (fabricmanager marker loop, mock-ib server). The agent launches Run on the Stage barrier of the first successful reconcile, so a daemon may assume its own Stage artifacts exist.

type DeviceSpec ¶

type DeviceSpec struct {
	Index            int
	UUID             string
	MinorNumber      int
	Serial           string
	PCIBusID         string
	Name             string
	Architecture     string
	ComputeCapMajor  int
	ComputeCapMinor  int
	MemoryTotalBytes uint64
	PCIDeviceID      uint32
	PCISubsystemID   uint32
	// VGPUGuest is set when the GPU is a vGPU as a VM sees it, and carries the
	// host driver the hypervisor writes into the vGPU's PCI config space.
	VGPUGuest *VGPUGuestSpec
}

DeviceSpec carries per-GPU identity and static hardware properties.

type FabricState ¶

type FabricState struct {
	// Profile declares NVLink; does not imply fabricmanager runs.
	Enabled bool
	// Fabricmanager readiness marker directory; empty when the daemon is off.
	ManagerStateDir      string
	ClusterUUID          string
	CliqueID             uint32
	LinksPerGPU          int
	BandwidthPerLinkMbps int
}

FabricState describes the NVLink / NVSwitch fabric configuration.

type IMEXState ¶

type IMEXState struct {
	Enabled   bool
	IMEXMajor int
	CapsMajor int
}

IMEXState describes the IMEX capability surface for the DRA compute-domain plugin.

type MIGComputeInstance ¶ added in v0.5.0

type MIGComputeInstance struct {
	ID   uint32
	UUID string
}

MIGComputeInstance is one compute instance, which is one MIG device as a consumer sees it. UUID is what NVML reports for the partition, and the container runtime resolves an allocated partition by that name, so it is carried alongside the ID the capability names are keyed by.

type MIGGPU ¶ added in v0.5.0

type MIGGPU struct {
	Minor        int
	GPUInstances []MIGGPUInstance
}

MIGGPU is one partitioned GPU. Minor is its device-node minor, which is how the capability names identify it.

type MIGGPUInstance ¶ added in v0.5.0

type MIGGPUInstance struct {
	ID               uint32
	ComputeInstances []MIGComputeInstance
}

MIGGPUInstance is one GPU instance and the compute instances inside it.

type MIGState ¶ added in v0.5.0

type MIGState struct {
	// CapsMajor is the char-device major for /dev/nvidia-caps.
	CapsMajor int
	GPUs      []MIGGPU
}

MIGState describes the MIG capability surface: which GPUs boot partitioned and how, so the agent can stage the cap device nodes and the mig-minors table a consumer needs to reach a MIG device it found through NVML.

The instance IDs are compiled from the same profile the mock NVML library loads, via engine.DeclaredMIGLayout, so the two cannot name a partition differently.

func (m MIGState) Caps() []migcaps.Cap

func (m MIGState) Partitioned() bool

type NetworkShape ¶

type NetworkShape struct {
	IBEnabled bool
	// HCACount is already resolved from hca_count or hcas_per_gpu * NumGPUs.
	HCACount         int
	HCAType          string
	FWVersion        string
	HWRev            string
	BoardID          string
	NodeDescTemplate string
	LinkLayer        string
	RateGbps         int
	PortState        string
	PhysState        string
	GUIDPrefix       string
}

NetworkShape describes the simulated InfiniBand HCAs attached to the node. Every field is resolved at compile time so simulators never re-derive defaults. All fields are comparable, so == detects a real IB change.

type NodeMeta ¶

type NodeMeta struct {
	NodeName string
	Hostname string
	HostRoot string
}

NodeMeta carries node identity fields.

type NodeShape ¶

type NodeShape struct {
	NumGPUs  int
	Topology PCIeTopology
	Network  NetworkShape
}

NodeShape describes the simulated node's topology.

type PCIeTopology ¶

type PCIeTopology struct {
	RootComplexes []RootComplex
	CoresPerNUMA  int
}

PCIeTopology describes the PCIe root-complex / NUMA layout.

type Requeuer ¶ added in v0.6.0

type Requeuer interface {
	Requeue() <-chan struct{}
}

Requeuer is implemented by simulators whose Stage publishes work they finish in the background, such as a download. Each value on Requeue reconciles the latest State again, so that work is staged under the same staging lock as the rest of the tree instead of being written while NRI decides.

type RootComplex ¶

type RootComplex struct {
	ID          string
	NUMANode    int
	DeviceBDFs  []string
	CPUAffinity string
}

RootComplex is one PCI host bridge with its attached GPU BDFs and NUMA node.

type Simulator ¶

type Simulator interface {
	Name() string
	// Stage materializes artifacts that are not yet externally visible.
	// Re-runs on every State change and must be idempotent.
	Stage(ctx context.Context, state *State) error
	// Discard is the inverse of Stage and runs only on shutdown.
	Discard(ctx context.Context) error
	// Ready reports whether the simulator is successfully done staging and publishing its artifacts.
	Ready() bool
}

Simulator owns one simulated component's reconciliation lifecycle. Stage and Discard are exact inverses.

type SoftwareVersions ¶

type SoftwareVersions struct {
	DriverVersion string
	NVMLVersion   string
	CUDAVersion   string
}

SoftwareVersions holds driver / NVML / CUDA version strings.

type State ¶

type State struct {
	Generation int64
	Node       NodeMeta
	Software   SoftwareVersions
	NodeShape  NodeShape
	Devices    []DeviceSpec
	Switches   []SwitchSpec
	Fabric     FabricState
	IMEX       IMEXState
	MIG        MIGState
	// ConfigRaw holds the raw YAML profile bytes so gpudriver can write the
	// engine config without re-deriving it from the narrower State fields.
	// TODO(https://github.com/NVIDIA/k8s-test-infra/issues/717): replace with Profile/Runtime *config.YAMLConfig split — Profile carries
	// the static hardware model, Runtime carries operational overrides (GPU_COUNT,
	// faults, fabric) written by GPU_COUNT env, MEP-0001 CP, and nvml-mock-ctl.
	ConfigRaw []byte
	// TopologyRaw holds the cluster ComputeDomain topology document verbatim.
	// It stays raw because the document describes every node and the mock NVML
	// engine picks this node's entry by NODE_NAME at load time, so there is no
	// per-node view for the agent to compile.
	TopologyRaw []byte
	// MIGProfilesRaw holds the board's MIG partition table verbatim, empty on
	// a board that has none. It is carried so gpudriver can stage it beside
	// every config it writes: a consumer process resolves the table relative
	// to the config it was given, and the device plugin is handed no
	// environment that could name it anywhere else.
	MIGProfilesRaw []byte
}

State is the compiled desired simulation state the agent reconciles toward. It maps 1:1 to what MEP-0001's Control Plane will emit.

func (s *State) HasPCITopology() bool

func (s *State) PCITopology() []RootComplex

type StateSource ¶

type StateSource interface {
	Watch(ctx context.Context) <-chan Update
	Close() error
}

StateSource emits State observations. Watch sends the current State immediately on subscribe and pushes on change. A closed channel means the source is terminally done; the agent stops.

type SwitchSpec ¶ added in v0.5.0

type SwitchSpec struct {
	PCIBusID       string
	PCIDeviceID    uint32
	PCISubsystemID uint32
}

SwitchSpec is one NVSwitch that sits on this node's own PCIe bus, as an HGX baseboard's do. It carries only a PCI identity because that is the whole of what the mock can simulate: NVML models an NVSwitch as the far end of a GPU's NVLink and offers no per-switch API, and the nvmlUnit* chassis calls stay stubbed the way they are on real DGX/HGX nodes.

A rack-scale platform has NVSwitches that are not on the node's bus at all — on GB200/GB300 NVL they live in their own switch trays, reachable over NVLink and invisible to the compute tray's lspci. Those profiles declare the switches for NVLink topology and no PCI identity, so nothing is compiled here for them.

type Update ¶

type Update struct {
	State *State
	Err   error
	At    time.Time
}

Update is one observation from a StateSource. Exactly one of State or Err is set. Err leaves the last good State in force; the agent does not reconcile.

type VGPUGuestSpec ¶ added in v0.6.0

type VGPUGuestSpec struct {
	HostDriverVersion string
	HostDriverBranch  string
}

VGPUGuestSpec is the host driver a vGPU guest reads from PCI config space.