Documentation
¶
Overview ¶
Package cgroup is a thin view of a delegated cgroup v2 subtree: create leaves with a memory ceiling and group OOM, open them for clone3, read what they charge, and kill them. Plain file operations; nothing here is clever, which is the point.
Index ¶
- func NVIDIAPresent() bool
- func ScopeOf(procCgroup string) string
- type Dir
- func (d Dir) Ceiling(name string) uint64
- func (d Dir) Child(name string) Dir
- func (d Dir) Children(prefix string) ([]Dir, error)
- func (d Dir) Create(memMax uint64, oomGroup bool) error
- func (d Dir) Ensure(controllers ...string) error
- func (d Dir) Exists() bool
- func (d Dir) Kill() error
- func (d Dir) MemoryCurrent() (uint64, error)
- func (d Dir) Name() string
- func (d Dir) OOMKills() (uint64, error)
- func (d Dir) Open() (*os.File, error)
- func (d Dir) PSI() (PSI, error)
- func (d Dir) PidsMaxHits() (uint64, error)
- func (d Dir) Procs() ([]int, error)
- func (d Dir) Remove() error
- func (d Dir) SetCeiling(high, max uint64) error
- func (d Dir) SetMemoryMin(b uint64) error
- func (d Dir) SetPidsMax(n uint64) error
- func (d Dir) Stat(key string) (uint64, error)
- type GPUGuard
- type PSI
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NVIDIAPresent ¶
func NVIDIAPresent() bool
NVIDIAPresent reports whether the NVIDIA driver holds a character major on this host, as /proc/devices lists it. A host whose /proc/devices cannot be read is taken to have none, like one that never loaded the driver.
Types ¶
type Dir ¶
type Dir struct{ Path string }
Dir is one cgroup directory.
func Delegate ¶
Delegate prepares root, the cgroup fiberd was started in, for the runtime to carve grants under. It needs controllers in cgroup.subtree_control, and a cgroup with processes may not enable any (the no-internal-process rule), so every process there first moves into a `fiberd` leaf. The memory limit whatever started fiberd put on root (a kubelet, Slurm's task/cgroup) stays the ceiling on everything below. Returns the `grants` subtree the runtime owns. Idempotent: an already delegated root is left alone. Controllers the root cannot offer (a Slurm step gets cpuset, cpu and memory, no pids) are skipped; memory is required. The root is live, so a process can enter it after the move and the kernel then refuses the controller. Delegate moves the newcomers and tries again a few times before it gives up.
func Own ¶
Own is the cgroup this process runs in, under mount (normally /sys/fs/cgroup). In a private cgroup namespace that is the mount's root; in the host's (what a privileged Pod gets under containerd, or a Slurm job step under proctrack/cgroup) it is the path /proc/self/cgroup names. Returns the mount root when the named path does not exist under it.
func (Dir) Ceiling ¶
Ceiling is the nearest limit above d: the smallest numeric value of the control file name (memory.max, pids.max) on d's ancestors, read upward until a directory without the file, which is the end of the delegated tree or the mount itself. A private cgroup namespace shows the container's own limit on the mount's root. 0 when every ancestor says max, as a bare host's root does.
func (Dir) Create ¶
Create makes a leaf with a memory ceiling (0 = none) and, when oomGroup is set, makes the kernel kill the whole leaf at once when it is hit.
func (Dir) Ensure ¶
Ensure creates the directory (if needed) and enables controllers for its children, so leaves under it get memory.max and friends.
func (Dir) MemoryCurrent ¶
MemoryCurrent is what the leaf charges now: for a forked fiber, the pages it has made private since the fork, which is W.
func (Dir) PidsMaxHits ¶
PidsMaxHits counts forks refused because this group was at pids.max (pids.events max). It is 0 without a pids controller.
func (Dir) SetCeiling ¶
SetCeiling sets memory.high (throttle: the kernel reclaims and stalls the group, which PSI reports) and memory.max (the hard stop). max of 0 leaves the hard limit unset. Swap is closed for the group: the ceiling is resident memory, and a home that lets jobs swap (Slurm without ConstrainSwapSpace) would otherwise let the zygote's pages slip out under the ceiling instead of stalling, and the ladder would never see the pressure it acts on.
func (Dir) SetMemoryMin ¶
SetMemoryMin protects b bytes of the group from reclaim. The kernel caps it at the parent's effective memory.min, so a protection only holds when every ancestor up to the delegation root carries one too.
func (Dir) SetPidsMax ¶
SetPidsMax caps the tasks in the group and its descendants. 0 leaves it unlimited. A group whose parent offers no pids controller (a Slurm step) has no pids.max and is left alone.
type GPUGuard ¶
type GPUGuard struct {
// contains filtered or unexported fields
}
GPUGuard is one cgroup device program (BPF_PROG_TYPE_CGROUP_DEVICE) attached to a cgroup directory. It refuses every open and mknod of an NVIDIA character device, by major, for every process in that cgroup and below, whatever mount or device node the process sees. Everything else is allowed, so it adds no rule beyond the GPU.
The kernel keeps an attached program alive by the cgroup's own reference, so closing the program descriptor does not detach it. Detaching a program attached with BPF_F_ALLOW_MULTI needs that descriptor back, which is why the guard holds it until Close.
func AttachGPUGuard ¶
AttachGPUGuard loads the device program and attaches it to the cgroup directory at dir with BPF_CGROUP_DEVICE and BPF_F_ALLOW_MULTI, so the programs an ancestor holds (runc's or systemd's device allow list) keep running beside it and a descendant can still attach its own.
Every error names its step and errno. The attach is refused with EPERM when the nearest ancestor with a device program attached it without BPF_F_ALLOW_MULTI, or with exactly one program and no BPF_F_ALLOW_OVERRIDE, and when dir itself already holds a program attached under other flags. That is the attach failing by name the warm reports.
Capabilities, on the 6.17 kernels fiberd targets. The load needs bpf_capable, CAP_BPF or CAP_SYS_ADMIN in the initial user namespace, and so does every bpf command while kernel.unprivileged_bpf_disabled is set, which distributions default to. The attach itself asks for nothing beyond a cgroup directory the caller can open. Covered uses BPF_PROG_QUERY, which needs CAP_NET_ADMIN or CAP_SYS_ADMIN. caps.Proc keeps CAP_SYS_ADMIN and CAP_NET_ADMIN, so nothing new is needed. A fiberd in a child user namespace cannot load the program at all.
func (*GPUGuard) Close ¶
Close detaches the program from its cgroup and closes its descriptor. Fibers already in the cgroup lose the guard at once. Safe to call more than once.
func (*GPUGuard) Covered ¶
Covered reports whether this guard's program runs for the processes of leaf, a cgroup directory at or below the guard's own. It asks the kernel for the device programs in effect there (BPF_PROG_QUERY with BPF_F_QUERY_EFFECTIVE) and looks for the guard's program id. A descendant that attached its own program without BPF_F_ALLOW_MULTI shadows every ancestor's, and the leaf is then not covered.
Source Files