Documentation
¶
Overview ¶
Package host performs node prerequisites: directories, kernel modules, sysctls, swapoff, and node-IP/hostname detection.
Index ¶
- Variables
- func AllowRestart() error
- func CheckFloor() error
- func CheckNodeIP(ip string, dualStack bool) error
- func CreateFileExclusively(path string, data []byte, perm os.FileMode) (os.FileInfo, error)
- func DetectNodeIPs(dualStack bool) (string, error)
- func Dirs() error
- func DisableUnit() error
- func EnableUnit() error
- func EnsureBPFFS()
- func EnsureRootShared()
- func Footprint() []string
- func Hostname() (string, error)
- func InstallUnit(argv []string) error
- func NodeIP() (string, error)
- func NodeIPs(dualStack bool) (string, error)
- func OwnedFirewalldInterfaces() ([]string, error)
- func Prereqs(o PrereqOptions) error
- func Primary(nodeIP string) string
- func ProfileSysctlFiles() []string
- func ReloadSystemd() error
- func RestartUnit() error
- func StopUnit() error
- func SuppressRestart() error
- func SyncDir(dir string) error
- func SystemdAvailable() bool
- func TrustPodCIDRs(_ string)
- func TrustedPodCIDRs() ([]string, error)
- func UnitArgv() ([]string, error)
- func UnitEnabled() (bool, error)
- func UnitExecPath() (string, error)
- func UnitMainPID() (int, error)
- func UnreachableKind(a netip.Addr) string
- func UntrustPodCIDRs() error
- func UntrustPodCIDRsWithWriter(w io.Writer) error
- func WriteFileAtomically(path string, data []byte, perm os.FileMode) error
- type NodeIPChoice
- type PrereqOptions
Constants ¶
This section is empty.
Variables ¶
var ( // ModulesLoadFile, SysctlConf and SysctlCISConf alias // version.ModulesLoadFile, version.SysctlConf and version.SysctlCISConf. ModulesLoadFile = version.ModulesLoadFile SysctlConf = version.SysctlConf SysctlCISConf = version.SysctlCISConf // CNIDir is where the CNI plugin's network configuration lives. CNIDir = "/etc/cni/net.d" // KubeletDir is the kubelet's default root directory. Stratum's kubelet // keeps its own state under version.DataDir, but its fixed-path serving // cert directory (internal/pki) stays here, and Dirs creates it. KubeletDir = "/var/lib/kubelet" )
The host paths bring-up writes outside stratum's three roots (version.DataDir, version.ConfDir, version.RunDir) are declared beside version.SystemdUnitPath — the one home, which the owning-package record keys and which this package reads: Prereqs writes, applySysctls loads and Footprint returns them through the version variables. ModulesLoadFile, SysctlConf and SysctlCISConf stay here as read-through aliases because internal/uninstall's delete list names them here; the literals live once, in version, so no second spelling of the paths exists for the writer, the sysctl loader and uninstall's delete list to drift between.
var ErrNotDurable = atomicfile.ErrNotDurable
ErrNotDurable is atomicfile.ErrNotDurable: the file was replaced, but the replacement could not be made durable. Existing callers match it by this name.
var ErrStrayCopy = atomicfile.ErrStrayCopy
ErrStrayCopy is atomicfile.ErrStrayCopy: the temporary name could not be removed, and still holds the file's contents. Existing callers match it by this name.
Functions ¶
func AllowRestart ¶
func AllowRestart() error
AllowRestart removes the runtime override SuppressRestart wrote, restoring Restart=always for the unit. A missing override file is not an error: rollback calls this unconditionally, whether or not SuppressRestart ever ran. But an absent file must not skip the reload: a prior AllowRestart call can have removed the file and then failed the reload that follows (systemctl unreachable, a wedged dbus) — the caller's retry finds no file to remove, and skipping the reload too would leave systemd's live config still carrying the old Restart=no it was never told to forget.
func CheckFloor ¶
func CheckFloor() error
CheckFloor enforces the platform floor before anything starts: cgroup v2 (the runtime is cgroup-v2-only) and a kernel new enough for Cilium's kube-proxy replacement (>=4.19.57 hard floor, >=5.10 target). Failing early with the requirement beats a kubelet and a CNI failing later in ways that do not name it.
func CheckNodeIP ¶
CheckNodeIP refuses a node IP kubelet --node-ip could not register for a node of the given stack, in the form every start path saves: an IPv4 address on a single-stack node, because a cluster's primary family is IPv4; on a dual-stack node its pair, the IPv4 first and then the IPv6. It also refuses a loopback, all-zeros, multicast or link-local address, which the kubelet never registers as a node IP on any host, and the broadcast address, which names every host on the link rather than one node (UnreachableKind judges an IPv4-mapped IPv6 as the IPv4 it names). Whether the address is on one of this host's interfaces is left to the kubelet. ResolveNodeIP and ResolveNodeIPWithoutConfig return an explicit source verbatim, so a worker's join and its restart check here the address they resolved before they save it, or a value that is not an IP address, or one that drops or adds an address family, outlives the start that was given it. Controller starts do not call it.
func CreateFileExclusively ¶
CreateFileExclusively is WriteFileAtomically for a file that must never replace one already at path. The at-rest key is minted on an observation, made earlier, that no key exists; a rename would silently replace a key that appeared since -- one an operator put there, whose Secrets would then be unreadable. The create-only write itself lives in internal/atomicfile: it fails with an error matching fs.ErrExist when a file is already at path, and returns the created file's FileInfo once it exists -- even when the directory sync then fails -- so a caller that has to take the file back off can remove exactly the inode this call created, and nothing that replaced it. If the temp name outlives that link, the error matches ErrStrayCopy and names it.
func DetectNodeIPs ¶
DetectNodeIPs is NodeIPs without the $NODE_IP override: the default-route IPv4, plus a node IPv6 under dual-stack. ResolveNodeIP uses it as the last resort, after every explicit source has been considered.
func Dirs ¶
func Dirs() error
Dirs creates the on-disk layout used by the control plane. Note MkdirAll does not tighten a dir that already exists, so private modes are enforced with an explicit Chmod.
func DisableUnit ¶
func DisableUnit() error
DisableUnit disables the reboot-persistence unit from starting at the NEXT boot — the first step of `stratum uninstall`, before it touches any running process or on-disk state. Boot enablement is not the same thing as the ACTIVE unit's restart behavior: disabling alone does nothing to stop Restart=always from relaunching the unit right now if it happens to exit for some unrelated reason while uninstall is still tearing the node down — SuppressRestart is what covers that. What DisableUnit alone protects against is a reboot that interrupts uninstall (or a failed rollback): without it, that reboot would bring the whole node back up over whatever state uninstall had already deleted.
A missing unit (never installed, or a non-systemd host) is not an error: there is nothing to disable. A real systemctl failure is returned so uninstall does not proceed to stop and delete state while the unit may still be armed to restart it.
func EnableUnit ¶
func EnableUnit() error
EnableUnit (re-)enables the reboot-persistence unit for the next boot, without touching its ExecStart, restarting it, or requiring SystemdAvailable() to have been true when the unit was first written — only that it is now. Used by `stratum uninstall`'s rollback to restore enablement when DisableUnit succeeded but a later teardown step failed, so a failed uninstall never leaves a still-running node stripped of the reboot persistence it had before.
A missing unit is not an error, matching DisableUnit/StopUnit: rollback runs unconditionally, and a node that was never enabled (or never systemd) has nothing to restore.
func EnsureBPFFS ¶
func EnsureBPFFS()
EnsureBPFFS mounts the BPF filesystem at /sys/fs/bpf with shared propagation when it is not already there. Cilium's mount-bpf-fs init container requires it (systemd hosts mount it at boot; containers and minimal hosts do not).
func EnsureRootShared ¶
func EnsureRootShared()
EnsureRootShared makes / a recursively shared mount when it is private. systemd makes / shared at boot; other inits (BusyBox, OpenRC) do not, and on those hosts every mount stays private, so containerd refuses the host-to-container bind mounts Cilium's pods make ("path ... is mounted on ... but it is not a shared or slave mount") and the node never becomes Ready. containerd accepts a shared or a slave mount for those, so a root that is already either is left alone; making a slave root shared would also cut it off from the mounts it follows. Cilium's one bidirectional mount, on /sys/fs/bpf, needs a shared mount, which EnsureBPFFS provides whatever / is. The change is recursive, as systemd's is: every mount under / becomes shared too, which on these hosts are all private. Nothing here is fatal: if the mount fails, Cilium's own error will say why.
func Footprint ¶
func Footprint() []string
Footprint returns the paths bring-up writes outside stratum's three roots: the modules-load file, both sysctl files (the CIS one under --cis) and the drop-in of every other registered profile that declares sysctls, the CNI dir, the kubelet dir, the audit log dir, the systemd unit, and containerd's certs.d and registry conf.d file. The paths version declares — the sysctl and modules-load files among them — are named through their variables and the /etc and /var drop-ins through the declarations above, so each path has one home. Uninstall covers every path of this set — the delete targets named from it, except the kubelet dir, which the teardown's removeLegacyKubeletDir removes under its keep-foreign-state contract instead of the plain delete loop — which is why the set is exactly bring-up's: a missing entry leaves a written path undeclared (and unremoved); an extra one makes uninstall delete a path bring-up does not own. /opt/local-path-provisioner is deliberately absent: SELinux only relabels that directory (spec.storage.localPathProvisioner), stratum never writes it.
func InstallUnit ¶
InstallUnit renders and enables the systemd unit that restarts argv — the exact steady-state command line this node was brought up with (never a one-shot join/token flag; see runUp's ExecStart builder) — after a reboot or crash.
The in-process supervisor (internal/supervisor) still parents containerd, kubelet and the operator; kubelet in turn reconciles the control-plane static pods. None of that is systemd's job — this unit wraps only the single top-level `stratum up` process, the one layer nothing else restarts after the node itself reboots. A separate unit per containerd/kubelet/operator would supervise those three twice over, once from systemd and once from the supervisor already running inside `stratum up`.
func NodeIP ¶
NodeIP returns $NODE_IP if set, else the IP of the default outbound route. A detection failure names $NODE_IP, the one source it reads.
It knows nothing about the address a node was actually brought up with. Code acting on a node that already runs wants nodestate.NodeIP, which prefers the recorded address; this is the bare fallback beneath it.
func NodeIPs ¶
NodeIPs returns the value for kubelet --node-ip: the IPv4 from NodeIP, plus a node IPv6 when dualStack is requested (kubelet wants "<v4>,<v6>"). If $NODE_IP is set it is honored verbatim (it may already carry both, comma-separated). dualStack with no usable node IPv6 is a hard error rather than a half-configured node that would fail to register its IPv6. A detection failure names $NODE_IP, the one source it reads.
func OwnedFirewalldInterfaces ¶
OwnedFirewalldInterfaces lists interface bindings Stratum will remove, with their zones. This includes the non-trusted LXC bridge exclusion.
func Prereqs ¶
func Prereqs(o PrereqOptions) error
Prereqs registers Stratum's SELinux file contexts and relabels its files, loads kernel modules, applies sysctls, and disables swap. It also writes the node profile's sysctl set; a profile that declares one depends on it (the cis profile's kubelet protectKernelDefaults is a hard boot gate), so a sysctl-apply failure is then FATAL rather than an opaque kubelet crash-loop later.
func Primary ¶
Primary returns the first (IPv4-primary) address of a possibly comma-joined node IP ("v4,v6" -> "v4"). Use it where a single IP is required — etcd peer/ client URLs, the Cilium k8sServiceHost — while kubelet/PKI take the full pair.
func ProfileSysctlFiles ¶
func ProfileSysctlFiles() []string
ProfileSysctlFiles are the sysctl drop-ins Prereqs writes for the binary's registered profiles that declare sysctls, other than the cis profile's, which is version.SysctlCISConf. A node runs only a profile its binary registers, so these cover every profile drop-in that binary's nodes carry.
func ReloadSystemd ¶
func ReloadSystemd() error
ReloadSystemd re-reads unit files. `stratum uninstall` calls this once, after it has deleted the (now stopped, disabled) unit file on disk, folding a failure into its own firstErr: systemd otherwise keeps serving the just-deleted unit's last-loaded definition.
func RestartUnit ¶
func RestartUnit() error
RestartUnit restarts the reboot-persistence unit. With KillMode=process systemd signals only the supervisor, which stops the components it runs and leaves the containers to their shims; systemd then starts ExecStart again.
func StopUnit ¶
func StopUnit() error
StopUnit stops the reboot-persistence unit. `stratum uninstall` calls this only after the static pods are torn down and every stratum host process is confirmed stopped, right before it deletes on-disk state (which includes the unit file itself) — stopping the unit any earlier sends the top-level `stratum up` process a SIGTERM that cascades into stopping kubelet through its own shutdown path, before kubelet has had the chance to reap the static pods removal asks it to.
func SuppressRestart ¶
func SuppressRestart() error
SuppressRestart drops in a runtime override that disarms the ACTIVE unit's Restart= without touching the persisted unit file under /etc or its boot enablement (DisableUnit). `systemctl disable` only stops the unit from starting at the NEXT boot — the unit stays active, with Restart=always still in force, until something changes ITS live configuration. Without this, an unrelated exit of `stratum up` during uninstall's teardown (a panic, a signal, anything short of StopUnit) would have systemd relaunch it in the middle of removing static pods and stopping host processes, racing the very teardown uninstall is mid-way through.
The override lives under /run, so a real reboot clears it on its own with no cleanup needed; AllowRestart removes it explicitly for a node that keeps running — an aborted uninstall, or once StopUnit is about to take the unit down for good and the override no longer means anything.
A missing unit is not an error, matching DisableUnit/StopUnit/EnableUnit: there is nothing active to protect from restarting, so writing a drop-in for a unit that does not exist would only leave an orphaned directory under /run — reachable on a real systemd host where `stratum up` never installed a unit (never systemd-managed, or uninstalled already).
func SystemdAvailable ¶
func SystemdAvailable() bool
SystemdAvailable reports whether this host's PID 1 is systemd — the only init system the reboot-persistence unit supports. Non-systemd hosts are out of scope: InstallUnit/DisableUnit/StopUnit are no-ops there rather than errors.
func TrustPodCIDRs ¶
func TrustPodCIDRs(_ string)
TrustPodCIDRs trusts Cilium's host-side pod interfaces, not the pod source range. A source binding also accepts spoofed pod addresses arriving on an external NIC. The argument is retained for existing callers; migration uses the ownership record. It never reloads firewalld: under the iptables backend a reload flushes Cilium's rules with it.
Idempotent, and called on every start of every role. It warns rather than fails: a host whose firewall cannot be changed still gets a cluster, and the warning carries the exact command, where failing would take down a node that may be working.
func TrustedPodCIDRs ¶
TrustedPodCIDRs returns the sources stratum added to firewalld, for uninstall's dry-run disclosure. An unreadable record is an error, not an empty list: the real run would fail on it, and the preview must say so.
func UnitArgv ¶
UnitArgv is the command line the reboot-persistence unit's ExecStart runs: the argv every restart of the unit starts. It is read from the unit file rather than from `systemctl show`, whose argv[] rendering joins the arguments with spaces unquoted, so a --config path with a space in it could not be told apart from two arguments. A unit that does not exist is an fs.ErrNotExist error, and so is a masked one (a symlink to /dev/null, as unitMasked reads it): systemd never starts it, and InstallUnit leaves the mask in place. A drop-in that sets ExecStart= is an error: systemd would run the drop-in's command, and this reads only the unit file stratum writes, so it does not guess which one wins. A drop-in that cannot be opened because it is gone (a dangling symlink, or one removed meanwhile) is skipped, as systemd skips it; only the unit file's own absence is fs.ErrNotExist.
func UnitEnabled ¶
UnitEnabled reports whether the reboot-persistence unit is currently enabled to start at boot — the same check InstallUnit uses to decide whether an earlier enable actually succeeded. `stratum uninstall` calls this before DisableUnit, so a teardown step that fails afterward can restore exactly the enablement it found (never force one) — see EnableUnit.
func UnitExecPath ¶
UnitExecPath is the program the reboot-persistence unit's ExecStart runs: the binary a restart of the unit starts.
func UnitMainPID ¶
UnitMainPID is the pid systemd runs the reboot-persistence unit as, 0 when systemd is not running it: no systemd, or the unit is absent or inactive.
func UnreachableKind ¶
UnreachableKind names the kind of address a is when, on any network, it cannot name one node or one apiserver that every other host can reach, and is "" for any other address. It is the one list of such kinds: CheckNodeIP refuses them as a worker's node IP, and a join URL must not name them. An all-zeros or loopback address makes the host that dials it reach itself, a multicast or broadcast one names many hosts rather than one, and a link-local one is not forwarded past a single network link. An IPv4-mapped IPv6 address is unmapped first, so it is judged as the IPv4 it names. Only the limited broadcast, 255.255.255.255, is a kind: a subnet's own broadcast (10.0.0.255 in 10.0.0.0/24) depends on a prefix length that a bare address does not carry. A zone is not judged; a caller that cannot carry one refuses it itself. The address a must be valid: the zero Addr is none of these kinds and answers "", so a caller refuses an address that does not parse before it calls this. The name reads after "is", as in `"127.0.0.1" is a loopback address`.
func UntrustPodCIDRs ¶
func UntrustPodCIDRs() error
UntrustPodCIDRs removes source and interface bindings Stratum added, from the configurations it added them to and no others. With firewalld stopped, runtime configuration is already gone and permanent configuration is edited offline, so the binding does not come back at the next firewalld start. Self-logging; returns an error carrying the exact commands for whatever it could not remove.
func UntrustPodCIDRsWithWriter ¶
UntrustPodCIDRsWithWriter removes owned firewall bindings and reports progress to w.
func WriteFileAtomically ¶
WriteFileAtomically writes data to path by creating a temp file in the SAME directory, fsyncing it, then renaming it into place. It writes the systemd unit, the firewalld record and the shared certs a control plane downloads; a CIS at-rest key minted on this node, which must never replace a file already at path, goes through CreateFileExclusively instead. The writer itself — unique hidden same-dir temp name, SELinux label, mode write before fsync, file fsync, rename, dir fsync, ErrNotDurable/ErrStrayCopy — lives in internal/atomicfile and is shared with every other package that writes a start-critical file; this wrapper only routes this package's test seams into it, so its behavior is exactly atomicfile.WriteFile's.
Types ¶
type NodeIPChoice ¶
type NodeIPChoice struct {
IP string // "v4" or "v4,v6", as kubelet --node-ip takes it
Source string // human-readable origin, for the startup log
}
NodeIPChoice is the node IP a start path settled on, and where it came from.
func ResolveNodeIP ¶
func ResolveNodeIP(flag, config, state string, detect func() (string, error)) (NodeIPChoice, error)
ResolveNodeIP is the one precedence every start path uses to decide this node's IP, except that a start that does not read spec.network.nodeIP skips step 3 through ResolveNodeIPWithoutConfig:
- flag — --node-ip on this invocation
- state — the address this node was brought up with (node.yaml)
- config — spec.network.nodeIP
- $NODE_IP
- detect — the default-route source address, else the single global IPv4
state outranks config and $NODE_IP: the reboot-persistence unit's ExecStart never carries --node-ip, so a restart's only explicit source is whatever spec.network.nodeIP/$NODE_IP happen to say NOW — which can legitimately differ from what the node was actually brought up with (an operator editing the config for the NEXT node, not this one). Config outranking state would let a restart resolve a different address than the one this node's etcd membership and certificates name, which nodeIPMoveRefusal (internal/cluster/prepare.go) then correctly refuses to start on — turning a config edit for some other purpose into a node that cannot reboot. state is empty on a true first boot, so config decides that case; --node-ip still outranks state for the deliberate-move workflow nodeIPMoveRefusal documents.
Every start path must use this one precedence: a path that picks its own sources can re-issue a node's apiserver cert for whichever NIC holds the default route on a host with a management NIC and a fast fabric.
Explicit sources are used verbatim (they may carry "v4,v6"). detect is only called when every other source is empty, so a node on an air-gapped network with a pinned address never needs a route to 8.8.8.8.
func ResolveNodeIPWithoutConfig ¶
func ResolveNodeIPWithoutConfig(flag, state string, detect func() (string, error)) (NodeIPChoice, error)
ResolveNodeIPWithoutConfig is ResolveNodeIP without its config source, for a start that does not read spec.network.nodeIP: flag, then state, then $NODE_IP, then detect. Such a start uses it so that the hint it adds to a detection failure names only sources the start reads.
type PrereqOptions ¶
type PrereqOptions struct {
// Profile is the node's profile: its sysctl set is written and applied,
// and its SELinux requirement holds.
Profile profile.Profile
// SELinuxOptOut is spec.selinux: false.
SELinuxOptOut bool
// Existing is a node that was brought up before, which a missing SELinux
// policy does not stop (selinux.Options.Existing).
Existing bool
}
PrereqOptions are the node settings host preparation depends on.
Source Files