nodeagent

package
v1.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 1, 2026 License: Apache-2.0 Imports: 82 Imported by: 0

Documentation

Index

Constants

View Source
const (
	// CNIConfDir is the standard directory kubelet scans for CNI configs.
	CNIConfDir = "/etc/cni/net.d"

	// CNIConfFile is our CNI conflist filename. The "00-" prefix guarantees
	// lexicographic priority over all other CNI configs (Multus uses "00-multus.conf",
	// but "00-cybericebox" < "00-multus" alphabetically).
	CNIConfFile = "00-cybericebox.conflist"

	// CNIBinDir is where CNI plugin binaries are installed.
	CNIBinDir = "/opt/cni/bin"
)
View Source
const (
	// ConditionWired is the type of the node-agent's condition on a Connection.
	ConditionWired = "Wired"

	WaitEndpointNode      = "WaitingForEndpointNode"  // an endpoint device has no node yet (its pod is not placed or not started)
	WaitVNI               = "WaitingForVNI"           // the connection or the switch has no VNI yet
	WaitNetworkObject     = "WaitingForNetworkObject" // the LabVPN or LabGateway of the lab is not there yet
	WaitInvalidNetworkIdx = "InvalidNetworkIndex"     // the lab network index is 0 or out of 1..254: the pod has no interface for it
	WaitDevicePod         = "WaitingForDevicePod"     // the device's pod is not published yet
	WaitOVSPort           = "WaitingForOVSPort"       // the port of an endpoint does not exist on this node yet
	WaitPortOwner         = "WaitingForPortOwner"     // the port of an endpoint still belongs to the pod that was replaced (its new pod is being wired)
	WaitVNIRelease        = "WaitingForVNIRelease"    // deleting: the nodes have not acknowledged the release of the lab's VNI yet
	WaitCleanupBlocked    = "CleanupBlocked"          // deleting: a step of the cleanup cannot finish yet (the message says which error)
	ReasonProgrammed      = "Programmed"              // the ports are programmed

)

A Connection that the node-agent cannot program yet says why, in the status, instead of requeueing in silence: the condition Wired is False with a fixed reason (a code, no paths) and a short message, True once the ports are programmed. One log line is written when the reason changes, not at every retry. (The condition Ready is the operator's: the ports are connected.)

View Source
const (
	AnnotationNetworks       = names.AnnotationNetworks
	AnnotationDefaultNetwork = names.AnnotationDefaultNetwork
)
View Source
const (
	// PreflightReady, PreflightDisabled and the prefix of an unavailable result are the values of the node annotations.
	PreflightReady    = "ready"
	PreflightDisabled = "disabled"
)
View Source
const (
	CodeContainerdRootMissing = pubnames.ReasonContainerdRootMissing
	CodeSnapshotterMissing    = pubnames.ReasonSnapshotterMissing
	CodeRuntimeSocket         = pubnames.ReasonRuntimeSocket
	CodeSnapshotRegistry      = pubnames.ReasonSnapshotRegistry
	CodeCgroupV2Missing       = pubnames.ReasonCgroupV2Missing
	CodePersistenceDisabled   = pubnames.ReasonPersistenceDisabled
	CodeObservationDisabled   = pubnames.ReasonObservationDisabled
	CodeNodeUnconfirmed       = pubnames.ReasonNodeUnconfirmed
)

The fixed reason codes of a failed check. They are what leaves the cluster (GetFeatures): no node names, paths or error text. The detail stays in the Node annotation ("unavailable: CODE: detail"), the Event and the log, for the operator of the cluster.

View Source
const FailModeSecure = "secure"

FailModeSecure is the only fail mode of br-ovs: no flow, no forwarding.

View Source
const GenevePort = "ovsgnv0"

GenevePort is the single per-node Geneve VTEP (spec §7: "One Geneve port per node: options:remote_ip=flow, options:key=flow. The remote VTEP address and VNI are set per-flow"). All cross-node tunnels share this port; flow rules set tun_dst (NXM_NX_TUN_IPV4_DST) and tun_id per packet.

View Source
const StartingTaintKey = "cybericebox.com/node-starting"

StartingTaintKey is the startup taint of a new lab worker. Infra puts it on the node when it is created (and tells the cluster autoscaler it is a startup taint, so the node counts as not usable yet instead of as a real node that fits nothing); the node-agent removes it once the node can take lab pods.

Variables

AllCodes is the list of codes (the allow-list of a caller; public as pkg/names.UnavailableReasonCodes).

View Source
var ErrPortOwnerChanged = errors.New("port belongs to another Pod incarnation")
View Source
var ErrPortOwnerUnknown = errors.New("port ownership unknown")

Functions

func BringUpInNetNS

func BringUpInNetNS(netnsPath, ifaceName string) error

BringUpInNetNS brings an interface UP inside the target netns.

func CheckInNetNS

func CheckInNetNS(netnsPath, ifaceName string) error

CheckInNetNS returns nil if the named interface exists inside the target netns.

func CodeOf added in v1.8.0

func CodeOf(v string) (string, bool)

CodeOf is the reason code of an annotation value that is a failed check; a value that carries none (an older node-agent) is NODE_UNCONFIRMED.

func ConntrackAccountingOn

func ConntrackAccountingOn(netnsPath string) bool

ConntrackAccountingOn reports whether both switches are already on in the target netns. Reading works with the read-only /proc/sys of the node-agent container; the node prep (the chart's host-prep init container, or the node image) sets them on the host and through the nf_conntrack module parameters, so a new pod namespace normally starts with them on.

func DeleteInNetNS

func DeleteInNetNS(netnsPath, ifaceName string) error

DeleteInNetNS deletes an interface inside the target netns.

func EnableConntrackAccounting

func EnableConntrackAccounting(netnsPath string) error

EnableConntrackAccounting makes sure conntrack byte accounting and flow timestamps are on in the target netns: nothing to do when they already are, otherwise it tries to write them. The write only works when /proc/sys is writable; in the unprivileged node-agent container it is not, and then the error says so (the node prep is the place to set them).

func GroupComponent

func GroupComponent(pod *corev1.Pod) string

GroupComponent says which of a group's own pods this is: names.ComponentVPN, names.ComponentGateway, or "" for any other pod (a device pod). A pod made before the component label is told by `app`.

func GroupPodAttachments

func GroupPodAttachments(ctx context.Context, c client.Reader, namespace, component string) ([]netattach.Attachment, error)

GroupPodAttachments is what the VPN or gateway pod of a group needs wired: one interface lab{N} per lab network, taken from the group's LabVPN or LabGateway objects (the operator makes one per lab with a VPN or an internet leg, and the object lives until the VPN or gateway process has cleaned up after the lab). The pod template carries no list of them, so adding or removing a lab never changes the Deployment and never restarts the pod; the node-agent attaches and detaches the interfaces in the running pod.

func GroupPortsPresent

func GroupPortsPresent(namespace, component string, present map[string]bool) []string

GroupPortsPresent returns the ports among present that belong to the group's pod of this component (any lab index).

func GroupPortsPresentOwned added in v1.1.0

func GroupPortsPresentOwned(namespace, component string, podUID types.UID, owners map[string]types.UID) []string

GroupPortsPresentOwned selects only this exact Pod incarnation's legs. Empty owners are legacy/unknown and require a separate live-replacement check.

func InstallCNIConf

func InstallCNIConf(confDir, agentSocket string, fallbackTimeout time.Duration) error

InstallCNIConf writes CNIConfFile into confDir by wrapping the first existing CNI config it finds with cni-gate. Blocks until a base config appears (retry every 5 s): the real CNI (Cilium) can take minutes on a fresh node (image pull, agent start), and until it is there no pod may be given a network. There is no automatic fallback: only when fallbackTimeout is positive (an explicit opt-in, nodeAgent.cniFallbackTimeout) is a built-in ptp delegate written after that long without a base config.

func IsUnavailable added in v1.8.0

func IsUnavailable(v string) (string, bool)

IsUnavailable says the value is a failed check, and gives its detail (for the operator of the cluster).

func MoveToNetNS

func MoveToNetNS(ifaceName, netnsPath string) error

MoveToNetNS moves an interface from the host netns to the target netns by path. Uses netlink directly to avoid iproute2 versions that reject absolute paths in "ip link set netns <path>".

func NodeAddresses

func NodeAddresses(nodes []corev1.Node) []net.IP

NodeAddresses are the internal IPv4 addresses of the nodes, sorted.

func PodNetNSFromCRI

func PodNetNSFromCRI(ctx context.Context, criSock, podUID string) (string, error)

PodNetNSFromCRI queries the container runtime via CRI to get the network namespace path for a pod sandbox identified by podUID.

func RenameInNetNS

func RenameInNetNS(netnsPath, oldName, newName string) error

RenameInNetNS renames an interface inside a target netns. Does not change UP/DOWN state — interface stays UP if it was UP before the move.

func SetMACInNetNS

func SetMACInNetNS(netnsPath, ifaceName, mac string) error

SetMACInNetNS sets the hardware address of an interface inside the target netns.

func SetupDeviceState

func SetupDeviceState(mgr ctrl.Manager, cfg *Config) error

SetupDeviceState adds the platform registry's node side to the manager: the localhost registry forwarder (snapshots and the image cache are both pulled through it) and, with state persistence on, the snapshot engine that follows the snapshot-backed device containers of this node. It does nothing when no registry is configured.

func SetupNodeReadiness added in v1.7.0

func SetupNodeReadiness(mgr ctrl.Manager, cfg *Config, labeler *NodeLabeler) error

SetupNodeReadiness adds the two things that make this node usable for labs, in order: the label that lab pods require, then the removal of the startup taint, which waits for the label and for the tun resource.

func SetupRuntimeObservation added in v1.1.0

func SetupRuntimeObservation(mgr ctrl.Manager, cfg *Config, network *NetworkAttachReconciler) error

SetupRuntimeObservation only starts an explicitly enabled native producer. It never turns the advertised support flags on, and bounds each scan deadline.

func SetupStateEngine added in v1.8.0

func SetupStateEngine(mgr ctrl.Manager, cfg *Config) error

SetupStateEngine adds the snapshot engine that follows the snapshot-backed device containers of this node. SetupDeviceState calls it when persistence is on; PreflightRetry calls it later when the start-up check passes after it failed (the registry came up).

func StaleGroupPorts

func StaleGroupPorts(namespace, component string, wanted []netattach.Attachment, present map[string]bool) []string

StaleGroupPorts returns the ports among present that belong to the group's VPN or gateway pod (any lab index) and are not in wanted: the legs of labs that are gone, to be detached from the running pod.

func StartGRPCServer

func StartGRPCServer(sockPath string, srv *NodeAgentServer) (*grpc.Server, error)

StartGRPCServer starts the NodeAgent gRPC server on a Unix socket.

func Unavailable added in v1.8.0

func Unavailable(code, detail string) string

Unavailable is the annotation value for a check that failed: "unavailable: CODE: detail".

func ValidPortKey

func ValidPortKey(key string) bool

ValidPortKey says whether key may be used as the name of a veth the node-agent creates or deletes.

func VethPeerName

func VethPeerName(stableKey string) string

VethPeerName returns the pod-side name of the veth pair for a given host-side (stableKey). Always ≤15 chars (Linux IFNAMSIZ limit). Falls back to "v"+sha256[:6] when the naive "v"+stableKey would exceed the limit.

func WaitForLink(name string, timeout time.Duration) error

WaitForLink polls until the named link appears in the current netns or the deadline passes.

func Watchdog

func Watchdog(ctx context.Context, interval time.Duration, failures int, ping func(context.Context) error, onLost func(error))

Watchdog calls ping every interval and calls onLost after failures misses in a row. The node-agent's two channels to OVS (the database and OpenFlow) do not reconnect: after an OVS crash they stay dead and every lab of the node stays dark until the pod restarts, so the node-agent exits (onLost) and the kubelet starts it again, which programs the bridge from scratch. It returns when ctx ends or after onLost.

Types

type AccessPortChecker

type AccessPortChecker struct {
	Client   client.Client
	NodeName string
	CRISock  string
	Recorder record.EventRecorder
	// Interval is the period of the check; zero means accessPortInterval.
	Interval time.Duration
	// contains filtered or unexported fields
}

AccessPortChecker keeps the source routing of the access port (see package accessroute) in place in the pods whose device may change it from inside: a pod with NET_ADMIN (the extended profile, or the capabilities of an in-image DHCP client) can flush the table, delete the rule or the port. A pod without NET_ADMIN cannot, and is left alone: cni-gate set it up once and for all. A lost rule or table is restored; a lost port cannot be (the device is fixed with "Reset device"): it is logged once and put on the pod as a Warning event, and the device status stays the operator's.

func (*AccessPortChecker) Check

func (c *AccessPortChecker) Check(ctx context.Context)

Check runs one round over the pods of the node.

func (*AccessPortChecker) NeedLeaderElection

func (c *AccessPortChecker) NeedLeaderElection() bool

NeedLeaderElection: every node-agent looks after the pods of its own node.

func (*AccessPortChecker) Start

func (c *AccessPortChecker) Start(ctx context.Context) error

Start blocks until ctx ends.

type BandwidthWatcher added in v1.8.0

type BandwidthWatcher struct {
	NodeName string
	// Dir is where the ConfigMap is mounted (bandwidth.ConfigMapDir).
	Dir   string
	Flows GeneveLimiter
}

BandwidthWatcher gives the flow manager the Geneve ceilings of this node. The operator works them out (it knows the node's label, the reserves of the platform pods and the labs) and writes them, with the limits of the other components, under the name of each node in the ConfigMap laboratory-bandwidth; the DaemonSet mounts it, and this reads the key of its own node. Nothing here watches the API, computes a bandwidth or polls: the kubelet refreshes the file when the operator changes it, and the flow manager touches an OVS meter only when a rate changes. A node-agent made without the volume has no limits.

func (*BandwidthWatcher) NeedLeaderElection added in v1.8.0

func (w *BandwidthWatcher) NeedLeaderElection() bool

NeedLeaderElection: every node-agent watches its own node's key.

func (*BandwidthWatcher) Start added in v1.8.0

func (w *BandwidthWatcher) Start(ctx context.Context) error

Start applies the limits at the start and at each change, until ctx ends.

type CPUThrottleMonitor added in v1.7.0

type CPUThrottleMonitor struct {
	Client     client.Client
	NodeName   string
	CgroupRoot string
	Interval   time.Duration
	// contains filtered or unexported fields
}

CPUThrottleMonitor reports how much the CPU limit holds a device back: every cpuThrottleInterval it reads the CFS counters (cpu.stat) of the device containers of its node, keeps the last few minutes of them, and writes the share of throttled periods to Device.status.cpuThrottle when it changes by a step. The lab controller copies it to the lab status, the agent sends it on (LabDeviceStatus.cpu_throttled_percent), and the platform alerts on it.

func (*CPUThrottleMonitor) Check added in v1.7.0

func (m *CPUThrottleMonitor) Check(ctx context.Context)

Check runs one round over the devices of the node.

func (*CPUThrottleMonitor) NeedLeaderElection added in v1.7.0

func (m *CPUThrottleMonitor) NeedLeaderElection() bool

NeedLeaderElection: every node-agent looks after the pods of its own node.

func (*CPUThrottleMonitor) Start added in v1.7.0

func (m *CPUThrottleMonitor) Start(ctx context.Context) error

Start blocks until ctx ends.

type Config

type Config struct {
	// Bandwidth are the chart's bandwidth values (BANDWIDTH_*): with it on, the Geneve traffic of the node is held to its class by
	// OVS meters (see package bandwidth).
	Bandwidth                  bandwidth.Settings
	RuntimeObservation         bool          `env:"RUNTIME_OBSERVATION_ENABLED" envDefault:"false"`
	RuntimeObservationInterval time.Duration `env:"RUNTIME_OBSERVATION_INTERVAL" envDefault:"5s"`
	NodeName                   string        `env:"NODE_NAME,required"`
	OVSSock                    string        `env:"OVS_SOCK"   envDefault:"/run/openvswitch/db.sock"`
	GRPCSock                   string        `env:"GRPC_SOCK"  envDefault:"/run/cybericebox/node-agent.sock"`
	Bridge                     string        `env:"OVS_BRIDGE" envDefault:"br-ovs"`
	// HealthAddr is where the readiness and liveness probes are served. The node-agent is on the host network, so it is the loopback
	// only (the probes name host 127.0.0.1): the port is not open on the node's address.
	HealthAddr string `env:"HEALTH_ADDR" envDefault:"127.0.0.1:9440"`
	CRISock    string `env:"CRI_SOCK"   envDefault:"/run/k0s/containerd.sock"`
	// DevicePluginDir is the kubelet's device-plugins directory (the kubelet hardcodes /var/lib/kubelet/device-plugins, also on k0s);
	// the node-agent registers the extended resource cybericebox.com/tun there. TunSlots is how many devices of that
	// resource a node advertises (each is the same /dev/net/tun, the count only bounds the extended devices per node).
	DevicePluginDir string `env:"DEVICE_PLUGIN_DIR" envDefault:"/var/lib/kubelet/device-plugins"`
	// TunCheckPath is what the plugin looks at to see that the tun device exists on the HOST: the sysfs entry of the tun misc device
	// (the container's own /dev has no tun even when the host does).
	TunCheckPath string `env:"TUN_CHECK_PATH"   envDefault:"/sys/class/misc/tun/dev"`
	TunSlots     int    `env:"TUN_SLOTS"         envDefault:"1000"`
	// PortPolicingKbps is the rate (kbit/s) a device's veth port may send into the bridge: the storm control of the shared switch
	// (0 = off). OVSWatchInterval is how often the node-agent asks OVS (database and OpenFlow) for a sign of life; after
	// OVSWatchFailures misses in a row it exits so that it is restarted and the bridge is programmed again.
	PortPolicingKbps int           `env:"PORT_POLICING_KBPS" envDefault:"500000"`
	OVSWatchInterval time.Duration `env:"OVS_WATCH_INTERVAL" envDefault:"15s"`
	OVSWatchFailures int           `env:"OVS_WATCH_FAILURES" envDefault:"3"`
	// ImagePullConcurrency is how many images of a prepull request the node pulls at once;
	// ImagePullTimeout bounds one image.
	ImagePullConcurrency int           `env:"IMAGE_PULL_CONCURRENCY" envDefault:"2"`
	ImagePullTimeout     time.Duration `env:"IMAGE_PULL_TIMEOUT"     envDefault:"5m"`

	// The platform registry (zot): the snapshots of device state and the image cache.
	//
	// StateRegistryAddr is host:port of the registry Service (empty = names.RegistryServiceAddr: the chart always installs it there).
	// The node-agent relays it to 127.0.0.1:StateForwardPort (names.RegistryForwardPort, a constant), so snapshot and cached images are
	// referenced as localhost:<port>/... and pulled by the node's containerd over plain HTTP without any host configuration.
	StateRegistryAddr     string `env:"STATE_REGISTRY_ADDR"`
	StateRegistryUser     string `env:"STATE_REGISTRY_USER"`
	StateRegistryPassword string `env:"STATE_REGISTRY_PASSWORD"`
	StateForwardPort      int
	// The reader account of the registry (it may read the snapshots and the base repository; the forwarder adds it to the
	// node runtime's pulls of those). Empty: the forwarder is a plain relay.
	StateRegistryReaderUser     string `env:"STATE_REGISTRY_READER_USER"`
	StateRegistryReaderPassword string `env:"STATE_REGISTRY_READER_PASSWORD"`
	// StatePersistence turns on the snapshot engine (the forwarder alone serves
	// the image cache).
	StatePersistence bool `env:"STATE_PERSISTENCE_ENABLED" envDefault:"false"`
	// ContainerdRoot is the containerd root directory of the host as mounted in the pod (the same path): the snapshots of a device's
	// writable layer are read from its snapshotter. The preflight checks it.
	ContainerdRoot string `env:"CONTAINERD_ROOT"`
	// ContainerdNamespace is the containerd namespace of the kubelet's CRI.
	ContainerdNamespace string `env:"CONTAINERD_NAMESPACE" envDefault:"k8s.io"`
	// CgroupRoot is where the host's cgroup v2 tree is mounted in the pod.
	CgroupRoot string `env:"CGROUP_ROOT" envDefault:"/host/sys/fs/cgroup"`
	// StateMaxWatchDirs is how many directories of one device's writable layer are watched with inotify; past it the layer is
	// only polled.
	StateMaxWatchDirs int `env:"STATE_MAX_WATCH_DIRS" envDefault:"2000"`
	// What one device may push (R-5): the least time between two pushes, the most state in StatePushBudgetWindow (a Kubernetes
	// quantity; "0" = no budget), and how long the replaced manifest of a device is kept before it is deleted.
	StateMinPushInterval  time.Duration `env:"STATE_MIN_PUSH_INTERVAL" envDefault:"60s"`
	StatePushBudget       string        `env:"STATE_PUSH_BUDGET" envDefault:"2Gi"`
	StatePushBudgetWindow time.Duration `env:"STATE_PUSH_BUDGET_WINDOW" envDefault:"1h"`
	StateSupersededGrace  time.Duration `env:"STATE_SUPERSEDED_GRACE" envDefault:"2m"`
	// StateRegistryCapacity is the size of the registry volume (a quantity; empty = unknown, no refusal) and
	// StateRegistryReserve the fraction of it that is kept free: a push that would pass that line is refused.
	StateRegistryCapacity string  `env:"STATE_REGISTRY_CAPACITY"`
	StateRegistryReserve  float64 `env:"STATE_REGISTRY_RESERVE" envDefault:"0.1"`
	// StateWorkDir holds the temporary layer files of a snapshot.
	StateWorkDir string `env:"STATE_WORK_DIR" envDefault:"/var/cache/cybericebox/state"`
}

func LoadConfig

func LoadConfig() (*Config, error)

func (*Config) Apply added in v1.8.0

func (c *Config) Apply(r PreflightResult)

Apply switches off in the configuration what the node cannot do, so the node-agent starts without it instead of failing.

func (*Config) PreflightInput added in v1.8.0

func (c *Config) PreflightInput() PreflightInput

PreflightInput is the part of the configuration the start-up check reads.

type ConnectionReconciler

type ConnectionReconciler struct {
	client.Client
	Reader      client.Reader
	NodeName    string
	NodeAddress string
	OVS         *OVSManager
	Flows       *FlowManager
	Recorder    record.EventRecorder
}

ConnectionReconciler programs br-ovs based on Connection CRDs.

Three connection scenarios (spec §8):

  1. Device↔Device — both ports in connVNI; t6 floods within connVNI.
  2. Device↔Switch — device port enters SW_VNI; t6 is rebuilt by aggregating ALL connections to that switch (devices on any node).
  3. Switch↔Switch — local patch-pair; each end registers in its switch's VNI; t6 for each switch VNI is rebuilt to include the patch port.

Pod migration is handled by the DevicePortReconciler stamping a new NodeAddress on the Device, which triggers connectionsForDevice → requeue this reconciler. reconcileCreate is fully idempotent: it removes and re-adds t0 before setting a new value, and atomically replaces the t6 flood entry for the affected VNI.

func (*ConnectionReconciler) Reconcile

func (r *ConnectionReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error)

func (*ConnectionReconciler) SetupWithManager

func (r *ConnectionReconciler) SetupWithManager(mgr ctrl.Manager) error

type DevicePortReconciler

type DevicePortReconciler struct {
	client.Client
	NodeName    string
	NodeAddress string
}

DevicePortReconciler manages the ovs-cleanup finalizer on Device CRDs and stamps each local device with the node's Geneve VTEP address so that ConnectionReconciler on peer nodes can determine the tunnel destination. OVS port creation/deletion for container devices is handled by ConnectionReconciler. Switch/hub devices have no physical OVS resource; their finalizer is removed immediately on delete.

func (*DevicePortReconciler) Reconcile

func (r *DevicePortReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error)

func (*DevicePortReconciler) SetupWithManager

func (r *DevicePortReconciler) SetupWithManager(mgr ctrl.Manager) error

type FlowManager

type FlowManager struct {

	// Ledger, when set, records the VNIs this node has flows for (internal/vniledger): written before the first flow of a VNI is
	// installed, taken out after RetireVNI confirms the flows gone. The operator does not return a VNI to the pool while it is there.
	Ledger VNILedger
	// contains filtered or unexported fields
}

FlowManager programs tables 0 and 6 of the OVS pipeline on br-ovs. The bridge is fail-secure (see OVSManager.ensureBridge): table 0 ends in a drop, so only a port with its own t0 flow forwards anything.

v1 pipeline (MAC learning deferred to a future version):

t0 — normalise in_port → set metadata=VNI, reg0=origin, resubmit(,6)
  local port:  reg0=0  (local origin)
  Geneve port: reg0=1  (remote origin), move tun_id→metadata
t6 — flood by (metadata=VNI, reg0):
  reg0=0 (local):  all local ports + Geneve to each remote VTEP
  reg0=1 (remote): all local ports only  — prevents Geneve re-flood loops

All t6 entries for a given VNI are rebuilt atomically (delete-all-for-VNI then re-add), so reconcileCreate can be called idempotently.

func NewFlowManager

func NewFlowManager(ovsRunDir, bridge string) (*FlowManager, error)

func (*FlowManager) AddEgressFlow

func (f *FlowManager) AddEgressFlow(localPort, _ string, vni uint, _ string) error

AddEgressFlow replaces the old per-remote-node egress rule. Now just binds the local port to its VNI via t0; Geneve flood is handled by RebuildT6Flood called from the connection reconciler.

func (*FlowManager) AddIngressFlow

func (f *FlowManager) AddIngressFlow(_, _ string, _ uint) error

AddIngressFlow is a no-op in the new pipeline. The single Geneve t0 entry installed by InitGeneveIngress handles all ingress.

func (*FlowManager) AddLocalSwitchFlow

func (f *FlowManager) AddLocalSwitchFlow(localPort string, vni uint) error

AddLocalSwitchFlow binds a local port to its VNI via t0 (same as AddT0Port).

func (*FlowManager) AddT0Port

func (f *FlowManager) AddT0Port(portName string, vni uint) error

AddT0Port installs a t0 entry for a local device or patch port:

t0, priority=90, in_port=PORT → set reg0=0, load VNI→metadata, resubmit(,6)

Idempotent and atomic: OFPFC_ADD with the same match and priority replaces the flow, so the port is never without its t0 flow while it is rebound (a delete first would leave a gap).

func (*FlowManager) Close

func (f *FlowManager) Close() error

Close closes the underlying OpenFlow connection.

func (*FlowManager) DelFlowsByPort

func (f *FlowManager) DelFlowsByPort(portName string) error

DelFlowsByPort removes the t0 entry for portName.

func (*FlowManager) DelFlowsByVNI

func (f *FlowManager) DelFlowsByVNI(vni uint, _ string) error

DelFlowsByVNI removes all t6 flood entries for vni.

func (*FlowManager) DelT0Port

func (f *FlowManager) DelT0Port(portName string) error

DelT0Port removes the t0 entry for portName. No-op if the port is already gone from the port map.

func (*FlowManager) DelT6Flood

func (f *FlowManager) DelT6Flood(vni uint) error

DelT6Flood removes all flood entries matching metadata=VNI (both reg0 values, and the meter tables) and the meter of the VNI.

func (*FlowManager) InitGeneveIngress

func (f *FlowManager) InitGeneveIngress() error

InitGeneveIngress installs the t0 entries of the shared Geneve port, one per node the cluster has:

t0, priority=100, in_port=GENEVE, tun_src=NODE → set reg0=1, move tun_id→metadata, resubmit(,6)

Geneve from any other source (UDP 6081 is open to whoever can reach the node) matches no flow and is dropped by the table-0 default, so a host that is not a node cannot inject frames into a VNI. The catch-all entry of earlier versions (in_port only) is removed. Safe to call multiple times; OFPFC_ADD replaces an existing entry. Must be called after AddGenevePort so the port is visible to OVS. The sources are set by SetGeneveSources; until the first call nothing is accepted.

func (*FlowManager) Ping

func (f *FlowManager) Ping() error

Ping asks OVS for a sign of life over the OpenFlow channel (the port description is the one request that needs an answer).

func (*FlowManager) RebuildT6Flood

func (f *FlowManager) RebuildT6Flood(vni uint, localPorts, remoteVTEPs []string) error

RebuildT6Flood atomically replaces the t6 flood entries for vni:

priority=110, metadata=VNI, reg0=0 → output all localPorts + Geneve to each remoteVTEP
priority=100, metadata=VNI, reg0=1 → output all localPorts only

Deletes any previous t6 entries for this VNI first. No-op (deletes only) when localPorts is empty.

With the Geneve limits on and a remote VTEP, the Geneve outputs move out of t6 into two more tables, each behind a meter (see geneve_meters.go): local delivery stays unmetered, only what leaves the node is counted.

func (*FlowManager) RetireVNI added in v1.1.0

func (f *FlowManager) RetireVNI(vni uint) error

RetireVNI removes every flow of the VNI in every table, confirmed, and then the Geneve meter of the lab (the flows that used it are gone by then).

func (*FlowManager) SetGeneveLimits added in v1.8.0

func (f *FlowManager) SetGeneveLimits(l bandwidth.Limits) error

SetGeneveLimits gives the node its Geneve ceilings (from the operator's annotation on the node): zero limits turn the meters off. A change of the rates is a mod-meter on the meters that changed; turning on or off builds the floods of the split labs again, with or without the meters.

func (*FlowManager) SetGeneveSources

func (f *FlowManager) SetGeneveSources(ips []net.IP) error

SetGeneveSources makes these node addresses the only ones whose Geneve traffic is accepted: a flow is added for each new address and removed for each that is gone. Called with the same set it does nothing. It needs the Geneve port to exist (a call before that is remembered and applied by InitGeneveIngress).

type GeneveLimiter added in v1.8.0

type GeneveLimiter interface {
	SetGeneveLimits(bandwidth.Limits) error
}

GeneveLimiter is what the watcher drives: the flow manager.

type GeneveSourceSync

type GeneveSourceSync struct {
	Reader   client.Reader
	Flows    *FlowManager
	Interval time.Duration
}

GeneveSourceSync keeps the Geneve ingress of the bridge limited to the nodes of the cluster (R-12): every Interval it reads the nodes' internal addresses and gives them to FlowManager.SetGeneveSources. A node that joins is accepted within one interval; one that leaves stops being accepted. The cluster's own address is included (harmless), and a read that fails keeps the last set.

func (*GeneveSourceSync) NeedLeaderElection

func (g *GeneveSourceSync) NeedLeaderElection() bool

NeedLeaderElection: every node-agent runs it for itself.

func (*GeneveSourceSync) Start

func (g *GeneveSourceSync) Start(ctx context.Context) error

Start runs until ctx ends.

func (*GeneveSourceSync) Sync

func (g *GeneveSourceSync) Sync(ctx context.Context) error

Sync does one pass.

type ImagePullReconciler

type ImagePullReconciler struct {
	client.Client
	// Reader reads Secrets straight from the API server: the node-agent has no watch on them.
	Reader          client.Reader
	NodeName        string
	ImagesNamespace string
	Puller          imagepull.Puller
	// Concurrency is how many images are pulled at once; Timeout bounds one image.
	Concurrency int
	Timeout     time.Duration
}

ImagePullReconciler answers the scheduler's image prepull requests (ImagePull) by pulling the images onto this node through the container runtime. Nothing from an image runs. Each node-agent writes only its own entry of the status.

func (*ImagePullReconciler) Reconcile

func (r *ImagePullReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error)

func (*ImagePullReconciler) SetupWithManager

func (r *ImagePullReconciler) SetupWithManager(mgr ctrl.Manager) error

type LifecycleReporter added in v1.1.0

type LifecycleReporter struct {
	Owner    *os.File
	Client   client.Client
	Reader   client.Reader
	Observer *NativeRuntimeObserver
	Interval time.Duration
	// contains filtered or unexported fields
}

LifecycleReporter polls direct API identities and direct native state. Every mutation is an optimistic status merge limited to this node's report rows.

func (*LifecycleReporter) Start added in v1.1.0

func (r *LifecycleReporter) Start(ctx context.Context) error

type NativeRuntimeObserver added in v1.1.0

type NativeRuntimeObserver struct {
	Runtime                                             *containerd.Client
	Namespace, NodeName, BootID, CgroupRoot, JournalDir string
	Network                                             *NetworkAttachReconciler
	Reader                                              runtimeclient.Reader
	// contains filtered or unexported fields
}

NativeRuntimeObserver uses direct containerd task and cgroup views. API absence is deliberately irrelevant. It never fills missing ownership from absence.

func (*NativeRuntimeObserver) ObserveOwnedRuntime added in v1.1.0

func (o *NativeRuntimeObserver) ObserveOwnedRuntime(ctx context.Context, id lab.OwnedRuntimeIdentity, fresh ...bool) lab.OwnedRuntimeReport

func (*NativeRuntimeObserver) ObserveScope added in v1.1.0

type NetAttachment

type NetAttachment = netattach.Attachment

NetAttachment is one parsed entry from the networks annotation.

func ParseNetworkAnnotation

func ParseNetworkAnnotation(annotation string) []NetAttachment

ParseNetworkAnnotation parses the network.cybericebox.com/networks annotation (see netattach.Parse). Entries whose interface name or MAC is not valid are dropped: the operator and the API validate both, and a value that got past them must not rename or re-address anything in the pod netns.

type NetworkAttachReconciler

type NetworkAttachReconciler struct {
	client.Client
	Reader   client.Reader
	NodeName string
	OVS      *OVSManager
	Flows    *FlowManager
	CRISock  string
	// contains filtered or unexported fields
}

NetworkAttachReconciler attaches veth ports to pods based on AnnotationNetworks. The host-side of each veth stays in root netns and is added to the OVS bridge; the pod-side is moved into the pod netns and renamed to the desired interface name.

func (*NetworkAttachReconciler) DelVethWithFlowsExpected added in v1.1.0

func (r *NetworkAttachReconciler) DelVethWithFlowsExpected(key string, uid types.UID, row string) error

func (*NetworkAttachReconciler) DelVethWithFlowsOwned added in v1.1.0

func (r *NetworkAttachReconciler) DelVethWithFlowsOwned(key string, ownerUID types.UID) error

DelVethWithFlowsOwned returns an error rather than a cleanup ACK when the stable key belongs to another incarnation or its ownership is unknown.

func (*NetworkAttachReconciler) DelVethWithFlowsOwnedJournaled added in v1.1.0

func (r *NetworkAttachReconciler) DelVethWithFlowsOwnedJournaled(key string, uid types.UID, prepared func(string) error, expectedRow ...string) error

DelVethWithFlowsOwnedJournaled fsyncs the physical cleanup witness while the exact owner row still exists. Only its guarded row retirement follows it.

func (*NetworkAttachReconciler) Reconcile

func (r *NetworkAttachReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error)

func (*NetworkAttachReconciler) SetupWithManager

func (r *NetworkAttachReconciler) SetupWithManager(mgr ctrl.Manager) error

type NodeAgentServer

type NodeAgentServer struct {
	nodev1.UnimplementedNodeAgentServer
	// contains filtered or unexported fields
}

NodeAgentServer implements the NodeAgent gRPC service for CNI delegation.

func NewNodeAgentServer

func NewNodeAgentServer(ovs *OVSManager, flows *FlowManager) *NodeAgentServer

func (*NodeAgentServer) GetPodAnnotation

GetPodAnnotation reads a single pod annotation on behalf of the CNI plugin, which has no direct access to the Kubernetes API. Retries until the pod appears in the controller-runtime cache to handle the CNI/cache timing race where the pod object is not yet visible when CNI runs.

func (*NodeAgentServer) SetK8sClient

func (s *NodeAgentServer) SetK8sClient(c client.Client)

SetK8sClient provides the Kubernetes API client. Called from main after the manager is created.

func (*NodeAgentServer) SetK8sReader added in v1.1.0

func (s *NodeAgentServer) SetK8sReader(r client.Reader)

SetK8sReader supplies the uncached lifecycle input for native CNI guards.

func (*NodeAgentServer) SetupNetworks

SetupNetworks is called by cni-gate during CNI ADD. It waits for the pod to appear in the controller-runtime cache, creates OVS veth pairs for all interfaces listed in AnnotationNetworks, moves each pod-side veth into the provided netns, and returns how cni-gate should configure the default interface.

type NodeLabeler

type NodeLabeler struct {
	Client   client.Client
	NodeName string
	// Interval is how often the label is asserted again (someone may remove it); zero means 1 minute.
	Interval time.Duration
	// Annotations are put on the node in the same patch as the label, so a node that is ready for labs always has them: the result of
	// the start-up check (Preflight). Recorder, when set, tells a failed check as a Warning Event on the node, once.
	Annotations map[string]string
	Recorder    record.EventRecorder
	// contains filtered or unexported fields
}

NodeLabeler marks the node of this node-agent as ready for lab pods with the label names.LabelNodeAgentReady, for as long as it runs, and removes the label when it stops. Every lab pod requires the label, so nothing is placed on a node whose node-agent is not there: a device pod on such a node would have no cni-gate and no OVS wiring. It is a manager Runnable, started after the controllers' caches, i.e. once the node-agent can serve pods (OVS is programmed, the gRPC socket listens).

func NewNodeLabeler added in v1.8.0

func NewNodeLabeler(mgr ctrl.Manager, cfg *Config, pre PreflightResult) *NodeLabeler

NewNodeLabeler is the labeler of this node with the result of the start-up check as the annotations it keeps asserting. The retry of a failed check shares it (SetAnnotation), so a later result is not put back to the start-up one.

func (*NodeLabeler) NeedLeaderElection

func (l *NodeLabeler) NeedLeaderElection() bool

NeedLeaderElection: every node-agent labels its own node.

func (*NodeLabeler) SetAnnotation added in v1.8.0

func (l *NodeLabeler) SetAnnotation(key, value string)

SetAnnotation changes one of the annotations the labeler keeps asserting. Whoever learns a newer result of a check puts it here (and on the node); a value patched on the node alone would be put back to the start-up result at the next assertion.

func (*NodeLabeler) Start

func (l *NodeLabeler) Start(ctx context.Context) error

Start blocks until ctx ends, then removes the label.

type OVSBridge

type OVSBridge struct {
	UUID  string   `ovsdb:"_uuid"`
	Name  string   `ovsdb:"name"`
	Ports []string `ovsdb:"ports"`
	// FailMode is "secure" on our bridge: without a flow a frame is dropped, never switched (the standalone
	// default installs a NORMAL flow, which would bridge the ports of different teams).
	FailMode *string `ovsdb:"fail_mode"`
}

type OVSInterface

type OVSInterface struct {
	UUID    string            `ovsdb:"_uuid"`
	Name    string            `ovsdb:"name"`
	Type    string            `ovsdb:"type"`
	Options map[string]string `ovsdb:"options"`
	Ifindex *int              `ovsdb:"ifindex"`
	// IngressPolicingRate (kbps) and IngressPolicingBurst (kb) police what a port may send into the bridge: the storm control of a
	// shared switch. 0 = not policed.
	IngressPolicingRate  int `ovsdb:"ingress_policing_rate"`
	IngressPolicingBurst int `ovsdb:"ingress_policing_burst"`
}

type OVSManager

type OVSManager struct {
	// RuntimeRetirement persists first exact physical cleanup before row loss.
	RuntimeRetirement       func(string, types.UID, string) error
	RuntimePrepare          func(string, types.UID, string) error
	VNIRetirement           func(context.Context, lab.OwnedVNI, *FlowManager) error
	FabricPrepare           func(string, types.UID, string) error
	FabricRetirement        func(string, types.UID, string) error
	FabricRetirementAbsent  func(string, types.UID) error
	RuntimeRetirementAbsent func(string, types.UID) error
	// contains filtered or unexported fields
}

genevePortName is retained for transitional call sites and returns the shared port name regardless of the remote address argument. OVSManager programs the single br-ovs bridge via libovsdb (OVSDB JSON-RPC over Unix socket).

func NewOVSManager

func NewOVSManager(bridge, sockPath string) (*OVSManager, error)

func (*OVSManager) AddGenevePort

func (m *OVSManager) AddGenevePort(_, _ string) error

AddGenevePort creates the single shared Geneve VTEP. The arguments are retained for source compatibility with older call sites and ignored; the concrete VTEP target is driven from flow rules (NXM_NX_TUN_IPV4_DST) so only one port is needed per node.

func (*OVSManager) AddInternalPort

func (m *OVSManager) AddInternalPort(stableKey string) (string, error)

AddInternalPort creates an OVS internal port in br-ovs with a random kernel interface name. The stableKey (used for reconcile idempotency and cleanup) is stored in external_ids["port-key"]. Returns the random kernel interface name that will appear in the host netns.

func (*OVSManager) AddPatchPair

func (m *OVSManager) AddPatchPair(nameA, nameB string) error

AddPatchPair creates two paired patch ports — spec §7-§8: switch↔switch connections materialise as a patch-pair (each end is a port in its own switch's VNI; no third VNI is introduced). Patch is always intra-node; cross-node hops travel via the partner switch's Geneve mesh.

Idempotent: re-creating an existing pair is a no-op.

func (*OVSManager) AddPatchPairOwned added in v1.1.0

func (m *OVSManager) AddPatchPairOwned(a, b string, owner types.UID) error

func (*OVSManager) AddVethPort

func (m *OVSManager) AddVethPort(stableKey string) error

AddVethPort creates a veth pair where stableKey is the host-side name (stays in OVS/root netns) and VethPeerName(stableKey) is the pod-side (moved to pod netns by the caller). Idempotent: EEXIST on kernel side is ignored; port already in OVS is a no-op. Stores stableKey in external_ids so FindPortByKey can locate it.

func (*OVSManager) AddVethPortOwned added in v1.1.0

func (m *OVSManager) AddVethPortOwned(stableKey string, ownerUID types.UID) error

AddVethPortOwned never overwrites another Pod's stable port ownership.

func (*OVSManager) Close

func (m *OVSManager) Close()

Close disconnects from the OVSDB server.

func (*OVSManager) DelFabricPortOwned added in v1.1.0

func (m *OVSManager) DelFabricPortOwned(key string, owner types.UID, flows *FlowManager, expectedRow ...string) error

func (*OVSManager) DelPort

func (m *OVSManager) DelPort(name string) error

DelPort removes a port from br-ovs by name (idempotent).

func (*OVSManager) DelPortByKey

func (m *OVSManager) DelPortByKey(stableKey string) error

DelPortByKey removes the OVS port whose external_ids["port-key"] matches stableKey. Idempotent: no-op if not found.

func (*OVSManager) DelVethPort

func (m *OVSManager) DelVethPort(stableKey string) error

DelVethPort removes the OVS port and deletes the kernel veth pair for stableKey. Deleting the host-side also removes the pod-side (veth pair invariant). Idempotent: no-op if already gone.

func (*OVSManager) EnsurePolicing

func (m *OVSManager) EnsurePolicing(portName string) error

EnsurePolicing applies the policing rate to an existing veth port (one made before the setting, or under another value).

func (*OVSManager) FindPortByKey

func (m *OVSManager) FindPortByKey(stableKey string) (string, bool, error)

FindPortByKey returns the OVS port name whose external_ids["port-key"] matches stableKey. Returns ("", false, nil) if not found.

func (*OVSManager) Ping

func (m *OVSManager) Ping(ctx context.Context) error

Ping asks ovsdb-server for an answer: the node-agent has lost it when this fails.

func (*OVSManager) PortExists

func (m *OVSManager) PortExists(name string) (bool, error)

PortExists checks whether a named port exists on br-ovs.

func (*OVSManager) PortKeys

func (m *OVSManager) PortKeys() (map[string]bool, error)

PortKeys returns the stable keys of the OVS ports the platform made (external_ids["port-key"]), as a set.

func (*OVSManager) PortOwners added in v1.1.0

func (m *OVSManager) PortOwners() (map[string]types.UID, error)

func (*OVSManager) RetireVNIOwned added in v1.1.0

func (m *OVSManager) RetireVNIOwned(ctx context.Context, binding lab.OwnedVNI, flows *FlowManager, authority ...func(context.Context, lab.OwnedVNI) error) error

RetireVNIOwned serializes the immutable lease check, correlated flow barrier and durable ACK with attachment creation. Numeric VNI alone is never an owner.

func (*OVSManager) SetPolicing

func (m *OVSManager) SetPolicing(kbps int)

SetPolicing sets the storm control of the veth ports: each may send at most kbps into the bridge (burst a tenth of it). A pure flood or a loop through a Linux bridge inside one lab is held to that rate instead of the whole datapath of the node. 0 switches it off.

func (*OVSManager) WaitForPortSetup

func (m *OVSManager) WaitForPortSetup(portName string, timeout time.Duration) error

WaitForPortSetup polls netlink until the kernel interface portName has type "openvswitch" with a stable ifindex for at least 300ms.

Why ifindex stability matters: vswitchd may delete+recreate the vport during its initial dpif_port_add setup when it detects the interface left root netns (OVS_VPORT_CMD_DEL + OVS_VPORT_CMD_NEW in rapid succession, logged as "deleted→added" in vswitchd). The recreated interface has a different ifindex. Waiting for ifindex stability ensures we move the interface only after vswitchd has entered its "Phase 2" (established) state, where a netns move causes it to set ifindex=0 rather than trigger another delete+recreate.

type OVSOpen_vSwitch

type OVSOpen_vSwitch struct {
	UUID    string   `ovsdb:"_uuid"`
	Bridges []string `ovsdb:"bridges"`
}

type OVSPort

type OVSPort struct {
	UUID        string            `ovsdb:"_uuid"`
	Name        string            `ovsdb:"name"`
	Interfaces  []string          `ovsdb:"interfaces"`
	ExternalIDs map[string]string `ovsdb:"external_ids"`
}

type PreflightInput added in v1.8.0

type PreflightInput struct {
	// Persistence and Observation say what is asked for.
	Persistence, Observation bool
	ContainerdRoot, CRISock  string
	CgroupRoot               string
	RegistryAddr             string
}

PreflightInput is what the check needs of the node-agent's configuration.

type PreflightResult added in v1.8.0

type PreflightResult struct {
	// StatePersistence and RuntimeObservation are annotation values.
	StatePersistence, RuntimeObservation string
}

PreflightResult is what the node supports.

func Preflight added in v1.8.0

func Preflight(ctx context.Context, cfg PreflightInput) PreflightResult

Preflight checks, for what the config asks for, that the node has it: persistence needs the containerd root with a snapshotter, the CRI socket and the snapshot registry; native runtime observation needs the cgroup v2 tree and the CRI socket. Nothing here blocks for long (the registry is asked with a short timeout) and nothing is changed on the node.

func (PreflightResult) Annotations added in v1.8.0

func (r PreflightResult) Annotations() map[string]string

Annotations are the Node annotations of the result.

func (PreflightResult) Failures added in v1.8.0

func (r PreflightResult) Failures() []string

Failures are the unavailable results, one line each, for the Event and the log.

type PreflightRetry added in v1.8.0

type PreflightRetry struct {
	Log logr.Logger
	// Input is the check; only the persistence part is repeated.
	Input PreflightInput
	// First is the annotation value the first check gave (unavailable: ...).
	First string
	// Enable starts the engine; Annotate puts the annotation on the node; Told tells the recovery (an Event). Told may be nil.
	Enable   func() error
	Annotate func(ctx context.Context, value string) error
	Told     func(message string)
	// Interval is the time between checks (30 s by default); Check is the check itself (Preflight by default).
	Interval time.Duration
	Check    func(ctx context.Context, in PreflightInput) PreflightResult
}

PreflightRetry repeats the start-up check of state persistence while it fails. A check that fails at the start of a node-agent is often early, not wrong: the registry of a fresh cluster is not up yet, the container runtime is still starting. The node-agent starts without the snapshot engine (and says so on the Node); this brings the engine up in the running process as soon as the check passes, and keeps the Node annotation current while it fails. It runs only when persistence was asked for and the first check failed.

func (*PreflightRetry) NeedLeaderElection added in v1.8.0

func (r *PreflightRetry) NeedLeaderElection() bool

NeedLeaderElection: every node-agent looks after its own node.

func (*PreflightRetry) Start added in v1.8.0

func (r *PreflightRetry) Start(ctx context.Context) error

Start blocks until the engine is up (or ctx ends).

type StartupTaintRemover added in v1.7.0

type StartupTaintRemover struct {
	Client   client.Client
	NodeName string
	// TunSlots is how many tun devices the node-agent offers; 0 means the plugin is off and the resource is not waited for.
	TunSlots int
	// Interval is how often the node is looked at while the taint is on; zero means 2 seconds.
	Interval time.Duration
}

StartupTaintRemover removes StartingTaintKey from the node of this node-agent once the node can really take a lab pod: the node-agent-ready label is set and the extended resource cybericebox.com/tun is in the node's allocatable (the kubelet has the device plugin registered and its devices listed). A node without the taint is left alone, so it is safe to ship before the taint exists. It is a manager Runnable.

func (*StartupTaintRemover) NeedLeaderElection added in v1.7.0

func (r *StartupTaintRemover) NeedLeaderElection() bool

NeedLeaderElection: every node-agent looks after its own node.

func (*StartupTaintRemover) Reconcile added in v1.7.0

func (r *StartupTaintRemover) Reconcile(ctx context.Context) (bool, error)

Reconcile does one step: true when there is nothing left to do (the taint is not on the node, or was just removed).

func (*StartupTaintRemover) Start added in v1.7.0

func (r *StartupTaintRemover) Start(ctx context.Context) error

Start blocks until the taint is gone (or was never there for a minute of looking) or ctx ends.

type VNILedger added in v1.8.0

type VNILedger interface {
	Note(vni uint) error
	Clear(vni uint) error
}

VNILedger is the node's record of the VNIs it carries flows of. Note stores before it returns (an error means: install nothing).

type VNIOrphanReconciler added in v1.8.0

type VNIOrphanReconciler struct {
	Reader   client.Reader
	OVS      *OVSManager
	Flows    *FlowManager
	Ledger   *vniledger.Ledger
	Recorder record.EventRecorder
	NodeName string
	Interval time.Duration
}

VNIOrphanReconciler compares, at start and then every Interval, the VNIs this node carries flows for (OpenFlow and the node's ledger) with the leases of the pool, and removes the flows of a VNI that no lease holds: the number is retired (its owner let it go) or back in the pool. This is the cleanup for an owner that is gone for good (a Lab deleted with the node away, a release that never ran).

It touches nothing it is not sure about. A number whose slot is held (a lease record, or a legacy hold) belongs to someone and is left alone. A number the pool cannot judge (unreadable pool, a Device or Connection still naming a slot the pool says is free) is left alone and counted as undecided. The removal happens under the same lock as the wiring of the pods, with the slot read again under it, and the ledger entry goes only after the flows are confirmed gone.

func (*VNIOrphanReconciler) NeedLeaderElection added in v1.8.0

func (r *VNIOrphanReconciler) NeedLeaderElection() bool

NeedLeaderElection: every node-agent runs it for its own node.

func (*VNIOrphanReconciler) Start added in v1.8.0

func (r *VNIOrphanReconciler) Start(ctx context.Context) error

Start runs a pass now (the start of the node-agent) and then every Interval until ctx ends.

func (*VNIOrphanReconciler) Sweep added in v1.8.0

func (r *VNIOrphanReconciler) Sweep(ctx context.Context) (dropped, undecided int, err error)

Sweep does one pass; it returns how many VNIs it removed the flows of and how many it could not judge.

Directories

Path Synopsis
Package ofclient implements a minimal OpenFlow 1.3 client over a Unix socket.
Package ofclient implements a minimal OpenFlow 1.3 client over a Unix socket.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL