Documentation
¶
Overview ¶
Package shape implements tc HTB hierarchy persistence and the `network shape` CLI verb. It renders the boot-replay script (solo-provisioner-tc-egress.sh) with bandwidth-parameterised class configurations, manages the solo-provisioner-tc-egress.service oneshot unit, and applies live tc class changes to the $EGRESS physical NIC.
The $VETH HTB is deliberately NOT persisted here: the veth interface does not survive reboot (Cilium recreates it on pod start), so persisting its qdisc would be meaningless. The daemon pod-lifecycle watcher reinstalls the $VETH HTB on the next pod create event by reading the class configs stored under ClassConfigDir.
Index ¶
- Constants
- func ApplyTcEgressScript(ctx context.Context) error
- func DetectEgressInterface() (string, error)
- func EnsureTcEgressUnit(ctx context.Context) error
- func FormatSpeedHint(mbit int) string
- func ParseSpeedMbit(s string) (int, bool)
- func ProvisionDefaultEgressShape(ctx context.Context, nicName, trunkRate string) error
- func ReadLinkSpeedMbit(nic string) (int, bool)
- func RenderAndApplyDefaultEgress(ctx context.Context, nic string) error
- type ClassConfig
- type Config
- type DeviceConfig
- type Manager
- func (m *Manager) CreateClass(ctx context.Context, cls *ClassConfig, force bool) (bool, error)
- func (m *Manager) CreateDevice(ctx context.Context, dev *DeviceConfig, force bool) (bool, error)
- func (m *Manager) DeleteClass(ctx context.Context, name string) error
- func (m *Manager) DeleteDevice(ctx context.Context, dir string) error
- func (m *Manager) ProvisionDefaultEgress(ctx context.Context, nicName, trunkRate string) error
- func (m *Manager) RenderAndApplyEgress(ctx context.Context, nic string) error
- func (m *Manager) SetClass(ctx context.Context, name string, rate, ceil *string, prio *int) error
- func (m *Manager) ShowAll() (string, error)
- func (m *Manager) ShowClass(name string) (string, error)
- func (m *Manager) ShowDevice(dir string) (string, error)
- type TCRunner
Constants ¶
const ( DirIngress = "ingress" // $VETH (pod traffic, applied by the daemon pod-lifecycle watcher) DirEgress = "egress" // $EGRESS physical NIC )
Direction constants for tc device/class mapping.
const ( // TcEgressScriptPath is the shell script that replays the $EGRESS HTB // hierarchy at boot. Lives under /usr/local/sbin (root-executable tools). TcEgressScriptPath = "/usr/local/sbin/solo-provisioner-tc-egress.sh" // TcEgressService is the systemd oneshot unit that executes TcEgressScriptPath // at boot, before solo-provisioner-daemon.service starts. TcEgressService = "solo-provisioner-tc-egress.service" // TcEgressServiceUnitPath is the absolute path where the unit file is // installed so systemd can discover it. TcEgressServiceUnitPath = "/usr/lib/systemd/system/" + TcEgressService // ShapeConfigDir is the root of the shape configuration tree persisted by // the `network shape` CLI verb. ShapeConfigDir = "/etc/solo-provisioner/network/shape" // DeviceConfigDir holds one JSON file per configured tc device ("ingress" // or "egress"), describing the root qdisc rate and default class. DeviceConfigDir = ShapeConfigDir + "/devices" // ClassConfigDir holds one JSON file per configured tc class, describing its // bandwidth parameters (rate, ceil, prio). The daemon pod-lifecycle watcher // reads this directory to reinstall $VETH classes on each pod create event. ClassConfigDir = ShapeConfigDir + "/classes" // ShapeLockDir is the directory containing the tc apply lock (on tmpfs so // it is auto-cleared on reboot). ShapeLockDir = "/run/solo-provisioner/network" // ShapeLockPath is the flock acquired (LOCK_EX) for the duration of any // tc mutating verb, preventing concurrent modifications. ShapeLockPath = ShapeLockDir + "/.tc-applying" )
const DefaultLinkSpeedMbit = 1000
DefaultLinkSpeedMbit is the link speed assumed when sysfs cannot report one (common on virtual NICs and at early boot, where /sys/class/net/<nic>/speed reads -1). It mirrors the fallback baked into the tc-egress boot script's sysfs-detection block; keep the two in sync.
Variables ¶
This section is empty.
Functions ¶
func ApplyTcEgressScript ¶
ApplyTcEgressScript ensures the tc-egress systemd unit is installed, then restarts it so the kernel picks up the HTB hierarchy immediately without waiting for a reboot. EnsureTcEgressUnit is idempotent (SHA-256 gated) so repeated calls are cheap. RestartService resets a previously-failed unit before running it, so a corrected script takes effect without a manual reset-failed. Using restart (rather than executing the script directly) keeps unit state visible: on success the unit shows active (exited), on failure failed — matching what operators see after a reboot.
func DetectEgressInterface ¶
DetectEgressInterface returns the name of the interface that carries the default route — the physical $EGRESS NIC the HTB hierarchy should be attached to. On multi-NIC hosts the desired interface must be specified explicitly via --egress-interface.
Fails with an actionable error when no default route is found (e.g. the routing table is not yet populated or the host has no default gateway).
func EnsureTcEgressUnit ¶
EnsureTcEgressUnit installs (or updates) the solo-provisioner-tc-egress.service unit file, daemon-reloads systemd, and enables the unit for boot. SHA-256 comparison is used so the write, reload, and enable are skipped when the on-disk content already matches the embedded template — making repeated installs cheap while ensuring a template change (e.g. ordering fix) is applied automatically on the next `block node install`.
func FormatSpeedHint ¶ added in v0.23.0
FormatSpeedHint converts a Mbit/s value into a tc-style bandwidth string suitable for operator-facing prompts (e.g. 1000 → "1gbit", 10000 → "10gbit", 100 → "100mbit").
func ParseSpeedMbit ¶ added in v0.23.0
ParseSpeedMbit parses a tc-style bandwidth string (e.g. "1gbit", "100mbit") into Mbit/s. Returns (0, false) for empty input or unrecognised formats.
func ProvisionDefaultEgressShape ¶ added in v0.23.0
ProvisionDefaultEgressShape configures the egress shape registry with the three default HTB classes at proportions derived from trunkRate, then renders and applies the boot script. Convenience wrapper over NewManager().ProvisionDefaultEgress.
func ReadLinkSpeedMbit ¶ added in v0.23.0
ReadLinkSpeedMbit reads the link speed of nic from the kernel sysfs entry /sys/class/net/<nic>/speed and returns it in Mbit/s.
Returns (0, false) when the speed is unavailable: the file is missing (virtual NIC, tunnel), the value is non-positive (kernel reports -1 for unknown/down links), or the content is non-numeric. Callers treat false as "no hint available" and must never block an install on this.
func RenderAndApplyDefaultEgress ¶ added in v0.23.0
RenderAndApplyDefaultEgress renders the tc-egress script for nic from the shape registry (or sysfs fallback when no config exists) and applies it. Used when no trunk rate is supplied (e.g. block node reconfigure without --link-rate).
Types ¶
type ClassConfig ¶ added in v0.23.0
type ClassConfig struct {
Name string `json:"name"`
Rate string `json:"rate"`
Ceil string `json:"ceil,omitempty"` // defaults to Rate when empty
Prio int `json:"prio"`
CreatedAt time.Time `json:"created_at"`
}
ClassConfig is the persisted bandwidth configuration for one named tc class. One JSON file per class under ClassConfigDir.
type Config ¶ added in v0.23.0
type Config struct {
ScriptPath string
LockPath string
NICDetect func() (string, error)
SpeedDetect func(nic string) (int, bool)
ApplyEgress func(ctx context.Context) error
TCRunner TCRunner
}
Config customises a Manager. The zero value is not useful; prefer NewManager.
type DeviceConfig ¶ added in v0.23.0
type DeviceConfig struct {
Dir string `json:"dir"` // "ingress" or "egress"
Rate string `json:"rate"` // root HTB trunk class rate
DefaultClass string `json:"default_class"` // class name; unmatched traffic falls here
CreatedAt time.Time `json:"created_at"`
}
DeviceConfig is the persisted root-level tc configuration for one traffic direction. One JSON file per device under DeviceConfigDir (named by Dir).
Dir "egress" targets the $EGRESS physical NIC and drives the re-rendered tc-egress.sh boot script. Dir "ingress" targets $VETH and is consumed by the daemon pod-lifecycle watcher; no script is rendered for it.
type Manager ¶ added in v0.23.0
type Manager struct {
// contains filtered or unexported fields
}
Manager implements the `network shape` verb: create, set, show, and delete operations for tc HTB device roots and per-class bandwidth configurations.
Egress mutations (Dir "egress") re-render TcEgressScriptPath and restart the tc-egress.service oneshot so the kernel picks up changes immediately. Ingress mutations (Dir "ingress") only write config; the daemon pod-lifecycle watcher reads them and applies them to each new veth interface.
func NewManager ¶ added in v0.23.0
func NewManager() *Manager
NewManager returns a Manager wired to the live kernel and production paths.
func NewManagerWithConfig ¶ added in v0.23.0
NewManagerWithConfig returns a Manager, filling unset Config fields with their production defaults.
func (*Manager) CreateClass ¶ added in v0.23.0
CreateClass creates (or replaces with --force) a per-class bandwidth configuration. The device for the class direction must exist before adding classes. For egress classes: re-renders TcEgressScriptPath and restarts tc-egress.service. For ingress classes: writes config only (daemon pod-lifecycle watcher handles VETH apply).
Returns true if the config was created or replaced, false if it already existed and force was not set.
func (*Manager) CreateDevice ¶ added in v0.23.0
CreateDevice creates (or replaces with --force) the root device configuration. For "egress": re-renders TcEgressScriptPath and restarts tc-egress.service. For "ingress": writes config only (daemon pod-lifecycle watcher handles VETH apply).
Returns true if the config was created or replaced, false if it already existed and force was not set.
func (*Manager) DeleteClass ¶ added in v0.23.0
DeleteClass removes a class configuration. Fails if the class is referenced as the device's default class or by any policy's --stamp/--reply-stamp. For egress classes: re-renders TcEgressScriptPath and restarts tc-egress.service.
func (*Manager) DeleteDevice ¶ added in v0.23.0
DeleteDevice removes a device configuration. Fails if any classes are still configured for this device (delete classes first).
func (*Manager) ProvisionDefaultEgress ¶ added in v0.23.0
ProvisionDefaultEgress configures the egress device root and three default HTB classes at proportions derived from trunkRate (partner 40%/70%, public 30%/70%, reserve-egress 30%/100%), then renders and applies the boot script. trunkRate may be "auto", which is resolved to the detected link speed at create time (see resolveAutoRateString). Existing configs are always replaced. Called by block node install so the shape registry is the single source of truth from first install.
func (*Manager) RenderAndApplyEgress ¶ added in v0.23.0
RenderAndApplyEgress renders the tc-egress boot script for nic from stored shape config (if available) or the sysfs auto-detect default, then installs and restarts tc-egress.service. Idempotent.
func (*Manager) SetClass ¶ added in v0.23.0
SetClass atomically updates one or more bandwidth parameters of an existing class. Only non-nil pointer fields are changed; nil means "keep current value". For egress classes: runs `tc class change` on the live kernel and re-renders the boot script for reboot persistence. For ingress classes: updates config only.
func (*Manager) ShowAll ¶ added in v0.23.0
ShowAll returns a human-readable summary of all configured devices and classes.
type TCRunner ¶ added in v0.23.0
type TCRunner interface {
// ClassChange runs `tc class change dev <nic> parent 1:1 classid 1:<minor>
// htb rate <rate> ceil <ceil> prio <prio>` on the live kernel.
ClassChange(ctx context.Context, nic, minor, rate, ceil string, prio int) error
}
TCRunner abstracts live kernel tc class operations for testability.
Source Files