Documentation
¶
Overview ¶
Package slurm handles Slurm version discovery and configuration generation.
Index ¶
- Constants
- func AccountCommands(accounts []config.Account, users []config.User) [][]string
- func AddNodesToConf(existing string, nodes []NodeEntry, conf config.Section) string
- func DefMemPerCPU(workers []NodeEntry) int
- func DiscoverVersion(ctx context.Context, dc *docker.Client, image string) (string, error)
- func EnablesTaskAffinity(conf config.Section) bool
- func GenerateCgroupConf(cgroup config.Section) string
- func GenerateJWTKey() []byte
- func GenerateMungeKey() []byte
- func GenerateNodesConf(workers []NodeEntry, main config.Section) string
- func GeneratePlugstackConf(plugstack config.Section) string
- func GenerateSectionConf(name string, section config.Section) string
- func GenerateSlurmConf(clusterName string, main config.Section, opts ConfOptions) string
- func GenerateSlurmKey() []byte
- func GenerateSlurmdbdConf(slurmdbd config.Section, identity config.IdentityMode, jwt bool) string
- func ParseMemoryMB(s string) (int, error)
- func ReadSlurmConf(read func(path string) (string, error)) (config.Section, error)
- func RemoveNodesFromConf(existing string, names []string) string
- func TaskAffinity(main config.Section) bool
- type ConfOptions
- type NodeEntry
Constants ¶
const ( SlurmKeyFile = "slurm.key" SlurmKeyPath = ConfDir + "/" + SlurmKeyFile )
slurm.key, the key of auth/slurm and cred/slurm with identity clientIds. sind writes it to the config volume, which every node mounts at ConfDir, where the daemons look for it.
const ( JWTKeyFile = "jwt_hs256.key" JWTKeyPath = ConfDir + "/" + JWTKeyFile )
jwt_hs256.key, the HS256 key auth/jwt signs and checks tokens with, for slurmrestd on a managed api node. sind writes it to the config volume, where slurmctld and slurmdbd find it by AuthAltParameters' jwt_key.
const ( ConfDir = "/etc/slurm" SlurmConfFile = "slurm.conf" SlurmConfPath = ConfDir + "/" + SlurmConfFile NodesConfFile = "sind-nodes.conf" NodesConfPath = ConfDir + "/" + NodesConfFile SlurmdbdConfFile = "slurmdbd.conf" SlurmdbdConfPath = ConfDir + "/" + SlurmdbdConfFile )
Container paths for Slurm configuration files.
const ( DBHost = "db" AccountingDB = "slurm_acct_db" StorageUser = "slurm" )
Accounting: the db node's hostname and the MariaDB database and user slurmdbd stores accounting data in.
const ( ControllerHost = "controller" BackupControllerHost = "controller-backup" StateSaveLocation = "/var/spool/slurmctld" )
Controller hostnames and the slurmctld state directory.
const ( MungeDir = "/etc/munge" MungeKeyFile = "munge.key" MungeKeyPath = MungeDir + "/" + MungeKeyFile )
Container paths for munge configuration.
const DefaultSlurmctldTimeout = 20
DefaultSlurmctldTimeout is the SlurmctldTimeout sind writes for a primary/backup controller pair, in seconds. Slurm's default of 120s makes failover tests needlessly slow; users can override it in the main section.
const DefaultTaskPlugin = "task/cgroup"
DefaultTaskPlugin is the TaskPlugin sind sets unless the main section sets one. It leaves out task/affinity: sind limits a node's CPUs with a CPU quota, not a cpuset, so every worker sees all host CPUs, and task/affinity would bind the jobs of every worker to the same first host CPUs.
const JWTKeySize = 32
JWTKeySize is the size of jwt_hs256.key in bytes, as Slurm's JWT guide creates it (dd if=/dev/random bs=32 count=1).
const MungeKeySize = 128
MungeKeySize is the default munge key size in bytes (1024 bits), matching the mungekey --create default.
const SlurmKeySize = 1024
SlurmKeySize is the size of slurm.key in bytes, as Slurm's documentation creates it (dd if=/dev/random bs=1024 count=1).
Variables ¶
This section is empty.
Functions ¶
func AccountCommands ¶ added in v0.11.0
AccountCommands returns the sacctmgr commands, without "sacctmgr -i", that create the cluster's Slurm accounts and the users' associations and roles, in the order they must run: the accounts, parents first, as declared; then each user with accounts, default account and admin level; then the coordinators, who must exist as Slurm users.
func AddNodesToConf ¶
AddNodesToConf adds node definitions to existing sind-nodes.conf content. A node that already has a NodeName line, such as one a failed `sind create worker` left behind, gets its line replaced, as Slurm refuses to start with a node defined twice; the other nodes get new lines. The PartitionName Nodes= list is updated to include both existing and new nodes, each once; the rest of the partition line stays (see partitionLine). Content without a partition line, as RemoveNodesFromConf leaves it without managed nodes, gets a new one, the default partition unless conf, the cluster's slurm.conf as ReadSlurmConf returns it, declares another one (see GenerateNodesConf).
func DefMemPerCPU ¶ added in v0.11.0
DefMemPerCPU returns the default memory per allocated CPU, in MB, that lets every managed worker run a job on each of its CPUs: the smallest RealMemory/CPUs over the workers, rounded down. Slurm's default select/cons_tres treats memory as a consumable resource, and without a default a job that does not ask for memory gets all of a node's memory, so a worker would run one such job at a time whatever its CPUs. It returns 0, for none, without workers or with a worker that has less than 1 MB per CPU.
func DiscoverVersion ¶
DiscoverVersion runs slurmctld -V in an ephemeral container and returns the Slurm version string (e.g. "25.11.0").
func EnablesTaskAffinity ¶ added in v0.11.0
EnablesTaskAffinity reports whether a cluster's slurm.conf, as ReadSlurmConf returns it, enables task/affinity, as slurmd on a worker added now would read it: whether its TaskPlugin lists it. Without a TaskPlugin, Slurm runs no task plugin.
func GenerateCgroupConf ¶
GenerateCgroupConf generates the cgroup.conf content for cgroupv2 support. The cgroup section can append content (string form) or include a .conf.d directory (map form).
func GenerateJWTKey ¶ added in v0.11.0
func GenerateJWTKey() []byte
GenerateJWTKey generates a random jwt_hs256.key.
func GenerateMungeKey ¶
func GenerateMungeKey() []byte
GenerateMungeKey generates a random munge authentication key.
func GenerateNodesConf ¶
GenerateNodesConf generates the sind-nodes.conf content with a NodeName line for each managed worker (see ManagedWorkers) and the partition all with every one of them. Partition all is the default partition unless the main section declares another one (see declaresDefaultPartition): slurm.conf includes sind-nodes.conf after the main section, and of several default partitions slurmctld takes the last. AddNodesToConf decides the same way when it writes the partition line again.
func GeneratePlugstackConf ¶ added in v0.7.0
GeneratePlugstackConf generates the plugstack.conf content. Always includes the .conf.d directory for fragment support.
func GenerateSectionConf ¶ added in v0.7.0
GenerateSectionConf generates the content for a standalone config file (gres.conf, topology.conf). String form returns content directly; map form returns include lines for each fragment file.
func GenerateSlurmConf ¶
func GenerateSlurmConf(clusterName string, main config.Section, opts ConfOptions) string
GenerateSlurmConf generates the main slurm.conf content for a cluster. Node definitions are not included here; they go in sind-nodes.conf. The main section can append content (string form) or include a .conf.d directory (map form).
The cluster name, the controller, SlurmUser, the spool directories and PlugStackConfig are fixed: sind's volumes and images depend on them. The parameters in baseDefaults follow, each unless the main section sets it, and opts.DefMemPerCPU unless it sets DefMemPerCPU or DefMemPerNode, which Slurm does not take together. With opts.BackupController, controller-backup is configured as the backup controller and SlurmctldTimeout defaults to DefaultSlurmctldTimeout unless the main section sets it. With opts.Accounting, the accounting parameters in accountingDefaults are added, with opts.Identity those of the identity mode, and with opts.JWT those of jwtDefaults, each unless the main section sets it.
func GenerateSlurmKey ¶ added in v0.11.0
func GenerateSlurmKey() []byte
GenerateSlurmKey generates a random slurm.key.
func GenerateSlurmdbdConf ¶ added in v0.11.0
GenerateSlurmdbdConf generates slurmdbd.conf for the db node: slurmdbd runs as SlurmUser and stores accounting data in the local MariaDB as StorageUser, which MariaDB authenticates by unix_socket as the OS user slurmdbd runs as, or by the StoragePass the slurmdbd section sets. It authenticates with munge, or with identity clientIds with auth/slurm and the users' client IDs, as slurmctld does, and with jwt also with auth/jwt (see jwtDefaults), each JWT parameter unless the slurmdbd section sets it: slurmdbd has no default key path. The slurmdbd section can append content (string form) or include .conf.d fragments (map form).
The pid file lives in /run/slurmdbd, the RuntimeDirectory slurmdbd.service creates for the slurm user: /run itself is not writable for it. sind writes no StoragePass line, since slurmdbd rejects an empty value.
func ParseMemoryMB ¶
ParseMemoryMB parses a memory limit in Docker's size syntax (e.g. "2g", "512m", "1.5GiB") and returns it in megabytes (see config.MemoryMB).
func ReadSlurmConf ¶ added in v0.11.0
ReadSlurmConf returns a cluster's slurm.conf as slurmctld and slurmd read it, as a section whose content holds its lines with each include directive replaced by the lines of the file it names. read returns a file of the config volume by path. sind-nodes.conf holds only sind's nodes and partition and is left out, as are include paths with a "*", which Slurm skips.
func RemoveNodesFromConf ¶
RemoveNodesFromConf removes the named nodes from existing sind-nodes.conf content. NodeName lines for the removed nodes are deleted and the PartitionName Nodes= list is updated; the rest of the partition line stays (see partitionLine). The partition line goes with the last node, and content without one gets none.
func TaskAffinity ¶ added in v0.11.0
TaskAffinity reports whether the slurm.conf sind generates with a main section enables task/affinity: whether the main section's TaskPlugin, or else DefaultTaskPlugin, lists it. slurmstepd then sets each task's CPU affinity after the task has become the job's user, which needs CAP_SYS_NICE for users other than root.
Types ¶
type ConfOptions ¶ added in v0.11.0
type ConfOptions struct {
// BackupController configures controller-backup as the backup
// controller of a primary/backup pair.
BackupController bool
// Accounting sends accounting data to the slurmdbd on a managed db node
// and gathers per-job resource usage.
Accounting bool
// Identity adds the settings of the identity modes other than local:
// nss_slurm for both, and auth/slurm with client IDs for clientIds.
Identity config.IdentityMode
// JWT adds auth/jwt as an alternative authentication type, for
// slurmrestd on a managed api node (see jwtDefaults).
JWT bool
// DefMemPerCPU is the default memory of a job per allocated CPU, in
// MB, 0 for none (see DefMemPerCPU).
DefMemPerCPU int
}
ConfOptions selects the optional parts of the generated slurm.conf.
type NodeEntry ¶
NodeEntry represents a single node to add to sind-nodes.conf.
func ManagedWorkers ¶ added in v0.11.0
ManagedWorkers returns the sind-nodes.conf entries of a cluster's managed workers, in config order. Workers are named worker-0, worker-1, ... across all worker groups, unmanaged ones included, which get no entry. The Slurm CPUs and RealMemory of a worker are its container's CPU and memory limits; a memory limit that config.MemoryMB rejects is an error.
Source Files