nodeagent

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 6, 2026 License: Apache-2.0 Imports: 54 Imported by: 0

Documentation

Index

Constants

View Source
const (
	// CNIConfDir is the standard directory kubelet scans for CNI configs.
	CNIConfDir = "/etc/cni/net.d"

	// CNIConfFile is our CNI conflist filename. The "00-" prefix guarantees
	// lexicographic priority over all other CNI configs (Multus uses "00-multus.conf",
	// but "00-cybericebox" < "00-multus" alphabetically).
	CNIConfFile = "00-cybericebox.conflist"

	// CNIBinDir is where CNI plugin binaries are installed.
	CNIBinDir = "/opt/cni/bin"
)
View Source
const (
	AnnotationNetworks       = names.AnnotationNetworks
	AnnotationDefaultNetwork = names.AnnotationDefaultNetwork
)
View Source
const FailModeSecure = "secure"

FailModeSecure is the only fail mode of br-ovs: no flow, no forwarding.

View Source
const GenevePort = "ovsgnv0"

GenevePort is the single per-node Geneve VTEP (spec §7: "One Geneve port per node: options:remote_ip=flow, options:key=flow. The remote VTEP address and VNI are set per-flow"). All cross-node tunnels share this port; flow rules set tun_dst (NXM_NX_TUN_IPV4_DST) and tun_id per packet.

Variables

This section is empty.

Functions

func BringUpInNetNS

func BringUpInNetNS(netnsPath, ifaceName string) error

BringUpInNetNS brings an interface UP inside the target netns.

func CheckInNetNS

func CheckInNetNS(netnsPath, ifaceName string) error

CheckInNetNS returns nil if the named interface exists inside the target netns.

func ConntrackAccountingOn

func ConntrackAccountingOn(netnsPath string) bool

ConntrackAccountingOn reports whether both switches are already on in the target netns. Reading works with the read-only /proc/sys of the node-agent container; the node prep (the chart's host-prep init container, or the node image) sets them on the host and through the nf_conntrack module parameters, so a new pod namespace normally starts with them on.

func DeleteInNetNS

func DeleteInNetNS(netnsPath, ifaceName string) error

DeleteInNetNS deletes an interface inside the target netns.

func EnableConntrackAccounting

func EnableConntrackAccounting(netnsPath string) error

EnableConntrackAccounting makes sure conntrack byte accounting and flow timestamps are on in the target netns: nothing to do when they already are, otherwise it tries to write them. The write only works when /proc/sys is writable; in the unprivileged node-agent container it is not, and then the error says so (the node prep is the place to set them).

func GroupComponent

func GroupComponent(pod *corev1.Pod) string

GroupComponent says which of a group's own pods this is: names.ComponentVPN, names.ComponentGateway, or "" for any other pod (a device pod). A pod made before the component label is told by `app`.

func GroupPodAttachments

func GroupPodAttachments(ctx context.Context, c client.Reader, namespace, component string) ([]netattach.Attachment, error)

GroupPodAttachments is what the VPN or gateway pod of a group needs wired: one interface lab{N} per lab network, taken from the group's LabVPN or LabGateway objects (the operator makes one per lab with a VPN or an internet leg, and the object lives until the VPN or gateway process has cleaned up after the lab). The pod template carries no list of them, so adding or removing a lab never changes the Deployment and never restarts the pod; the node-agent attaches and detaches the interfaces in the running pod.

func GroupPortsPresent

func GroupPortsPresent(namespace, component string, present map[string]bool) []string

GroupPortsPresent returns the ports among present that belong to the group's pod of this component (any lab index).

func InstallCNIConf

func InstallCNIConf(confDir, agentSocket string, fallbackTimeout time.Duration) error

InstallCNIConf writes CNIConfFile into confDir by wrapping the first existing CNI config it finds with cni-gate. Blocks until a base config appears (retry every 5 s): the real CNI (Cilium) can take minutes on a fresh node (image pull, agent start), and until it is there no pod may be given a network. There is no automatic fallback: only when fallbackTimeout is positive (an explicit opt-in, nodeAgent.cniFallbackTimeout) is a built-in ptp delegate written after that long without a base config.

func MoveToNetNS

func MoveToNetNS(ifaceName, netnsPath string) error

MoveToNetNS moves an interface from the host netns to the target netns by path. Uses netlink directly to avoid iproute2 versions that reject absolute paths in "ip link set netns <path>".

func NodeAddresses

func NodeAddresses(nodes []corev1.Node) []net.IP

NodeAddresses are the internal IPv4 addresses of the nodes, sorted.

func PodNetNSFromCRI

func PodNetNSFromCRI(ctx context.Context, criSock, podUID string) (string, error)

PodNetNSFromCRI queries the container runtime via CRI to get the network namespace path for a pod sandbox identified by podUID.

func RenameInNetNS

func RenameInNetNS(netnsPath, oldName, newName string) error

RenameInNetNS renames an interface inside a target netns. Does not change UP/DOWN state — interface stays UP if it was UP before the move.

func SetMACInNetNS

func SetMACInNetNS(netnsPath, ifaceName, mac string) error

SetMACInNetNS sets the hardware address of an interface inside the target netns.

func SetupDeviceState

func SetupDeviceState(mgr ctrl.Manager, cfg *Config) error

SetupDeviceState adds the platform registry's node side to the manager: the localhost registry forwarder (snapshots and the image cache are both pulled through it) and, with state persistence on, the snapshot engine that follows the snapshot-backed device containers of this node. It does nothing when no registry is configured.

func StaleGroupPorts

func StaleGroupPorts(namespace, component string, wanted []netattach.Attachment, present map[string]bool) []string

StaleGroupPorts returns the ports among present that belong to the group's VPN or gateway pod (any lab index) and are not in wanted: the legs of labs that are gone, to be detached from the running pod.

func StartGRPCServer

func StartGRPCServer(sockPath string, srv *NodeAgentServer) (*grpc.Server, error)

StartGRPCServer starts the NodeAgent gRPC server on a Unix socket.

func ValidPortKey

func ValidPortKey(key string) bool

ValidPortKey says whether key may be used as the name of a veth the node-agent creates or deletes.

func VethPeerName

func VethPeerName(stableKey string) string

VethPeerName returns the pod-side name of the veth pair for a given host-side (stableKey). Always ≤15 chars (Linux IFNAMSIZ limit). Falls back to "v"+sha256[:6] when the naive "v"+stableKey would exceed the limit.

func WaitForLink(name string, timeout time.Duration) error

WaitForLink polls until the named link appears in the current netns or the deadline passes.

func Watchdog

func Watchdog(ctx context.Context, interval time.Duration, failures int, ping func(context.Context) error, onLost func(error))

Watchdog calls ping every interval and calls onLost after failures misses in a row. The node-agent's two channels to OVS (the database and OpenFlow) do not reconnect: after an OVS crash they stay dead and every lab of the node stays dark until the pod restarts, so the node-agent exits (onLost) and the kubelet starts it again, which programs the bridge from scratch. It returns when ctx ends or after onLost.

Types

type AccessPortChecker

type AccessPortChecker struct {
	Client   client.Client
	NodeName string
	CRISock  string
	Recorder record.EventRecorder
	// Interval is the period of the check; zero means accessPortInterval.
	Interval time.Duration
	// contains filtered or unexported fields
}

AccessPortChecker keeps the source routing of the access port (see package accessroute) in place in the pods whose device may change it from inside: a pod with NET_ADMIN (the extended profile, or the capabilities of an in-image DHCP client) can flush the table, delete the rule or the port. A pod without NET_ADMIN cannot, and is left alone: cni-gate set it up once and for all. A lost rule or table is restored; a lost port cannot be (the device is fixed with "Reset device"): it is logged once and put on the pod as a Warning event, and the device status stays the operator's.

func (*AccessPortChecker) Check

func (c *AccessPortChecker) Check(ctx context.Context)

Check runs one round over the pods of the node.

func (*AccessPortChecker) NeedLeaderElection

func (c *AccessPortChecker) NeedLeaderElection() bool

NeedLeaderElection: every node-agent looks after the pods of its own node.

func (*AccessPortChecker) Start

func (c *AccessPortChecker) Start(ctx context.Context) error

Start blocks until ctx ends.

type Config

type Config struct {
	NodeName string `env:"NODE_NAME,required"`
	OVSSock  string `env:"OVS_SOCK"   envDefault:"/run/openvswitch/db.sock"`
	GRPCSock string `env:"GRPC_SOCK"  envDefault:"/run/cybericebox/node-agent.sock"`
	Bridge   string `env:"OVS_BRIDGE" envDefault:"br-ovs"`
	// HealthAddr is where the readiness and liveness probes are served. The node-agent is on the host network, so it is the loopback
	// only (the probes name host 127.0.0.1): the port is not open on the node's address.
	HealthAddr string `env:"HEALTH_ADDR" envDefault:"127.0.0.1:9440"`
	CRISock    string `env:"CRI_SOCK"   envDefault:"/run/k0s/containerd.sock"`
	// DevicePluginDir is the kubelet's device-plugins directory (the kubelet hardcodes /var/lib/kubelet/device-plugins, also on k0s);
	// the node-agent registers the extended resource cybericebox.com/tun there. TunSlots is how many devices of that
	// resource a node advertises (each is the same /dev/net/tun, the count only bounds the extended devices per node).
	DevicePluginDir string `env:"DEVICE_PLUGIN_DIR" envDefault:"/var/lib/kubelet/device-plugins"`
	// TunCheckPath is what the plugin looks at to see that the tun device exists on the HOST: the sysfs entry of the tun misc device
	// (the container's own /dev has no tun even when the host does).
	TunCheckPath string `env:"TUN_CHECK_PATH"   envDefault:"/sys/class/misc/tun/dev"`
	TunSlots     int    `env:"TUN_SLOTS"         envDefault:"1000"`
	// PortPolicingKbps is the rate (kbit/s) a device's veth port may send into the bridge: the storm control of the shared switch
	// (0 = off). OVSWatchInterval is how often the node-agent asks OVS (database and OpenFlow) for a sign of life; after
	// OVSWatchFailures misses in a row it exits so that it is restarted and the bridge is programmed again.
	PortPolicingKbps int           `env:"PORT_POLICING_KBPS" envDefault:"500000"`
	OVSWatchInterval time.Duration `env:"OVS_WATCH_INTERVAL" envDefault:"15s"`
	OVSWatchFailures int           `env:"OVS_WATCH_FAILURES" envDefault:"3"`
	// ImagePullConcurrency is how many images of a prepull request the node pulls at once;
	// ImagePullTimeout bounds one image.
	ImagePullConcurrency int           `env:"IMAGE_PULL_CONCURRENCY" envDefault:"2"`
	ImagePullTimeout     time.Duration `env:"IMAGE_PULL_TIMEOUT"     envDefault:"5m"`

	// The platform registry (zot): the snapshots of device state and the image cache.
	//
	// StateRegistryAddr is host:port of the registry Service (empty = names.RegistryServiceAddr: the chart always installs it there).
	// The node-agent relays it to 127.0.0.1:StateForwardPort (names.RegistryForwardPort, a constant), so snapshot and cached images are
	// referenced as localhost:<port>/... and pulled by the node's containerd over plain HTTP without any host configuration.
	StateRegistryAddr     string `env:"STATE_REGISTRY_ADDR"`
	StateRegistryUser     string `env:"STATE_REGISTRY_USER"`
	StateRegistryPassword string `env:"STATE_REGISTRY_PASSWORD"`
	StateForwardPort      int
	// The reader account of the registry (it may read the snapshots and the base repository; the forwarder adds it to the
	// node runtime's pulls of those). Empty: the forwarder is a plain relay.
	StateRegistryReaderUser     string `env:"STATE_REGISTRY_READER_USER"`
	StateRegistryReaderPassword string `env:"STATE_REGISTRY_READER_PASSWORD"`
	// StatePersistence turns on the snapshot engine (the forwarder alone serves
	// the image cache).
	StatePersistence bool `env:"STATE_PERSISTENCE_ENABLED" envDefault:"false"`
	// ContainerdNamespace is the containerd namespace of the kubelet's CRI.
	ContainerdNamespace string `env:"CONTAINERD_NAMESPACE" envDefault:"k8s.io"`
	// CgroupRoot is where the host's cgroup v2 tree is mounted in the pod.
	CgroupRoot string `env:"CGROUP_ROOT" envDefault:"/host/sys/fs/cgroup"`
	// StateMaxWatchDirs is how many directories of one device's writable layer are watched with inotify; past it the layer is
	// only polled.
	StateMaxWatchDirs int `env:"STATE_MAX_WATCH_DIRS" envDefault:"2000"`
	// What one device may push (R-5): the least time between two pushes, the most state in StatePushBudgetWindow (a Kubernetes
	// quantity; "0" = no budget), and how long the replaced manifest of a device is kept before it is deleted.
	StateMinPushInterval  time.Duration `env:"STATE_MIN_PUSH_INTERVAL" envDefault:"60s"`
	StatePushBudget       string        `env:"STATE_PUSH_BUDGET" envDefault:"2Gi"`
	StatePushBudgetWindow time.Duration `env:"STATE_PUSH_BUDGET_WINDOW" envDefault:"1h"`
	StateSupersededGrace  time.Duration `env:"STATE_SUPERSEDED_GRACE" envDefault:"2m"`
	// StateRegistryCapacity is the size of the registry volume (a quantity; empty = unknown, no refusal) and
	// StateRegistryReserve the fraction of it that is kept free: a push that would pass that line is refused.
	StateRegistryCapacity string  `env:"STATE_REGISTRY_CAPACITY"`
	StateRegistryReserve  float64 `env:"STATE_REGISTRY_RESERVE" envDefault:"0.1"`
	// StateWorkDir holds the temporary layer files of a snapshot.
	StateWorkDir string `env:"STATE_WORK_DIR" envDefault:"/var/cache/cybericebox/state"`
}

func LoadConfig

func LoadConfig() (*Config, error)

type ConnectionReconciler

type ConnectionReconciler struct {
	client.Client
	NodeName    string
	NodeAddress string
	OVS         *OVSManager
	Flows       *FlowManager
	Recorder    record.EventRecorder
}

ConnectionReconciler programs br-ovs based on Connection CRDs.

Three connection scenarios (spec §8):

  1. Device↔Device — both ports in connVNI; t6 floods within connVNI.
  2. Device↔Switch — device port enters SW_VNI; t6 is rebuilt by aggregating ALL connections to that switch (devices on any node).
  3. Switch↔Switch — local patch-pair; each end registers in its switch's VNI; t6 for each switch VNI is rebuilt to include the patch port.

Pod migration is handled by the DevicePortReconciler stamping a new NodeAddress on the Device, which triggers connectionsForDevice → requeue this reconciler. reconcileCreate is fully idempotent: it removes and re-adds t0 before setting a new value, and atomically replaces the t6 flood entry for the affected VNI.

func (*ConnectionReconciler) Reconcile

func (r *ConnectionReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error)

func (*ConnectionReconciler) SetupWithManager

func (r *ConnectionReconciler) SetupWithManager(mgr ctrl.Manager) error

type DevicePortReconciler

type DevicePortReconciler struct {
	client.Client
	NodeName    string
	NodeAddress string
}

DevicePortReconciler manages the ovs-cleanup finalizer on Device CRDs and stamps each local device with the node's Geneve VTEP address so that ConnectionReconciler on peer nodes can determine the tunnel destination. OVS port creation/deletion for container devices is handled by ConnectionReconciler. Switch/hub devices have no physical OVS resource; their finalizer is removed immediately on delete.

func (*DevicePortReconciler) Reconcile

func (r *DevicePortReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error)

func (*DevicePortReconciler) SetupWithManager

func (r *DevicePortReconciler) SetupWithManager(mgr ctrl.Manager) error

type FlowManager

type FlowManager struct {
	// contains filtered or unexported fields
}

FlowManager programs tables 0 and 6 of the OVS pipeline on br-ovs. The bridge is fail-secure (see OVSManager.ensureBridge): table 0 ends in a drop, so only a port with its own t0 flow forwards anything.

v1 pipeline (MAC learning deferred to a future version):

t0 — normalise in_port → set metadata=VNI, reg0=origin, resubmit(,6)
  local port:  reg0=0  (local origin)
  Geneve port: reg0=1  (remote origin), move tun_id→metadata
t6 — flood by (metadata=VNI, reg0):
  reg0=0 (local):  all local ports + Geneve to each remote VTEP
  reg0=1 (remote): all local ports only  — prevents Geneve re-flood loops

All t6 entries for a given VNI are rebuilt atomically (delete-all-for-VNI then re-add), so reconcileCreate can be called idempotently.

func NewFlowManager

func NewFlowManager(ovsRunDir, bridge string) (*FlowManager, error)

func (*FlowManager) AddEgressFlow

func (f *FlowManager) AddEgressFlow(localPort, _ string, vni uint, _ string) error

AddEgressFlow replaces the old per-remote-node egress rule. Now just binds the local port to its VNI via t0; Geneve flood is handled by RebuildT6Flood called from the connection reconciler.

func (*FlowManager) AddIngressFlow

func (f *FlowManager) AddIngressFlow(_, _ string, _ uint) error

AddIngressFlow is a no-op in the new pipeline. The single Geneve t0 entry installed by InitGeneveIngress handles all ingress.

func (*FlowManager) AddLocalSwitchFlow

func (f *FlowManager) AddLocalSwitchFlow(localPort string, vni uint) error

AddLocalSwitchFlow binds a local port to its VNI via t0 (same as AddT0Port).

func (*FlowManager) AddT0Port

func (f *FlowManager) AddT0Port(portName string, vni uint) error

AddT0Port installs a t0 entry for a local device or patch port:

t0, priority=90, in_port=PORT → set reg0=0, load VNI→metadata, resubmit(,6)

Idempotent and atomic: OFPFC_ADD with the same match and priority replaces the flow, so the port is never without its t0 flow while it is rebound (a delete first would leave a gap).

func (*FlowManager) Close

func (f *FlowManager) Close() error

Close closes the underlying OpenFlow connection.

func (*FlowManager) DelFlowsByPort

func (f *FlowManager) DelFlowsByPort(portName string) error

DelFlowsByPort removes the t0 entry for portName.

func (*FlowManager) DelFlowsByVNI

func (f *FlowManager) DelFlowsByVNI(vni uint, _ string) error

DelFlowsByVNI removes all t6 flood entries for vni.

func (*FlowManager) DelT0Port

func (f *FlowManager) DelT0Port(portName string) error

DelT0Port removes the t0 entry for portName. No-op if the port is already gone from the port map.

func (*FlowManager) DelT6Flood

func (f *FlowManager) DelT6Flood(vni uint) error

DelT6Flood removes all t6 entries matching metadata=VNI (both reg0=0 and reg0=1).

func (*FlowManager) InitGeneveIngress

func (f *FlowManager) InitGeneveIngress() error

InitGeneveIngress installs the t0 entries of the shared Geneve port, one per node the cluster has:

t0, priority=100, in_port=GENEVE, tun_src=NODE → set reg0=1, move tun_id→metadata, resubmit(,6)

Geneve from any other source (UDP 6081 is open to whoever can reach the node) matches no flow and is dropped by the table-0 default, so a host that is not a node cannot inject frames into a VNI. The catch-all entry of earlier versions (in_port only) is removed. Safe to call multiple times; OFPFC_ADD replaces an existing entry. Must be called after AddGenevePort so the port is visible to OVS. The sources are set by SetGeneveSources; until the first call nothing is accepted.

func (*FlowManager) Ping

func (f *FlowManager) Ping() error

Ping asks OVS for a sign of life over the OpenFlow channel (the port description is the one request that needs an answer).

func (*FlowManager) RebuildT6Flood

func (f *FlowManager) RebuildT6Flood(vni uint, localPorts, remoteVTEPs []string) error

RebuildT6Flood atomically replaces the t6 flood entries for vni:

priority=110, metadata=VNI, reg0=0 → output all localPorts + Geneve to each remoteVTEP
priority=100, metadata=VNI, reg0=1 → output all localPorts only

Deletes any previous t6 entries for this VNI first. No-op (deletes only) when localPorts is empty.

func (*FlowManager) SetGeneveSources

func (f *FlowManager) SetGeneveSources(ips []net.IP) error

SetGeneveSources makes these node addresses the only ones whose Geneve traffic is accepted: a flow is added for each new address and removed for each that is gone. Called with the same set it does nothing. It needs the Geneve port to exist (a call before that is remembered and applied by InitGeneveIngress).

type GeneveSourceSync

type GeneveSourceSync struct {
	Reader   client.Reader
	Flows    *FlowManager
	Interval time.Duration
}

GeneveSourceSync keeps the Geneve ingress of the bridge limited to the nodes of the cluster (R-12): every Interval it reads the nodes' internal addresses and gives them to FlowManager.SetGeneveSources. A node that joins is accepted within one interval; one that leaves stops being accepted. The cluster's own address is included (harmless), and a read that fails keeps the last set.

func (*GeneveSourceSync) NeedLeaderElection

func (g *GeneveSourceSync) NeedLeaderElection() bool

NeedLeaderElection: every node-agent runs it for itself.

func (*GeneveSourceSync) Start

func (g *GeneveSourceSync) Start(ctx context.Context) error

Start runs until ctx ends.

func (*GeneveSourceSync) Sync

func (g *GeneveSourceSync) Sync(ctx context.Context) error

Sync does one pass.

type ImagePullReconciler

type ImagePullReconciler struct {
	client.Client
	// Reader reads Secrets straight from the API server: the node-agent has no watch on them.
	Reader          client.Reader
	NodeName        string
	ImagesNamespace string
	Puller          imagepull.Puller
	// Concurrency is how many images are pulled at once; Timeout bounds one image.
	Concurrency int
	Timeout     time.Duration
}

ImagePullReconciler answers the scheduler's image prepull requests (ImagePull) by pulling the images onto this node through the container runtime. Nothing from an image runs. Each node-agent writes only its own entry of the status.

func (*ImagePullReconciler) Reconcile

func (r *ImagePullReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error)

func (*ImagePullReconciler) SetupWithManager

func (r *ImagePullReconciler) SetupWithManager(mgr ctrl.Manager) error

type NetAttachment

type NetAttachment = netattach.Attachment

NetAttachment is one parsed entry from the networks annotation.

func ParseNetworkAnnotation

func ParseNetworkAnnotation(annotation string) []NetAttachment

ParseNetworkAnnotation parses the network.cybericebox.com/networks annotation (see netattach.Parse). Entries whose interface name or MAC is not valid are dropped: the operator and the API validate both, and a value that got past them must not rename or re-address anything in the pod netns.

type NetworkAttachReconciler

type NetworkAttachReconciler struct {
	client.Client
	NodeName string
	OVS      *OVSManager
	Flows    *FlowManager
	CRISock  string
	// contains filtered or unexported fields
}

NetworkAttachReconciler attaches veth ports to pods based on AnnotationNetworks. The host-side of each veth stays in root netns and is added to the OVS bridge; the pod-side is moved into the pod netns and renamed to the desired interface name.

func (*NetworkAttachReconciler) Reconcile

func (r *NetworkAttachReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error)

func (*NetworkAttachReconciler) SetupWithManager

func (r *NetworkAttachReconciler) SetupWithManager(mgr ctrl.Manager) error

type NodeAgentServer

type NodeAgentServer struct {
	nodev1.UnimplementedNodeAgentServer
	// contains filtered or unexported fields
}

NodeAgentServer implements the NodeAgent gRPC service for CNI delegation.

func NewNodeAgentServer

func NewNodeAgentServer(ovs *OVSManager, flows *FlowManager) *NodeAgentServer

func (*NodeAgentServer) GetPodAnnotation

GetPodAnnotation reads a single pod annotation on behalf of the CNI plugin, which has no direct access to the Kubernetes API. Retries until the pod appears in the controller-runtime cache to handle the CNI/cache timing race where the pod object is not yet visible when CNI runs.

func (*NodeAgentServer) SetK8sClient

func (s *NodeAgentServer) SetK8sClient(c client.Client)

SetK8sClient provides the Kubernetes API client. Called from main after the manager is created.

func (*NodeAgentServer) SetupNetworks

SetupNetworks is called by cni-gate during CNI ADD. It waits for the pod to appear in the controller-runtime cache, creates OVS veth pairs for all interfaces listed in AnnotationNetworks, moves each pod-side veth into the provided netns, and returns how cni-gate should configure the default interface.

type NodeLabeler

type NodeLabeler struct {
	Client   client.Client
	NodeName string
	// Interval is how often the label is asserted again (someone may remove it); zero means 1 minute.
	Interval time.Duration
}

NodeLabeler marks the node of this node-agent as ready for lab pods with the label names.LabelNodeAgentReady, for as long as it runs, and removes the label when it stops. Every lab pod requires the label, so nothing is placed on a node whose node-agent is not there: a device pod on such a node would have no cni-gate and no OVS wiring. It is a manager Runnable, started after the controllers' caches, i.e. once the node-agent can serve pods (OVS is programmed, the gRPC socket listens).

func (*NodeLabeler) NeedLeaderElection

func (l *NodeLabeler) NeedLeaderElection() bool

NeedLeaderElection: every node-agent labels its own node.

func (*NodeLabeler) Start

func (l *NodeLabeler) Start(ctx context.Context) error

Start blocks until ctx ends, then removes the label.

type OVSBridge

type OVSBridge struct {
	UUID  string   `ovsdb:"_uuid"`
	Name  string   `ovsdb:"name"`
	Ports []string `ovsdb:"ports"`
	// FailMode is "secure" on our bridge: without a flow a frame is dropped, never switched (the standalone
	// default installs a NORMAL flow, which would bridge the ports of different teams).
	FailMode *string `ovsdb:"fail_mode"`
}

type OVSInterface

type OVSInterface struct {
	UUID    string            `ovsdb:"_uuid"`
	Name    string            `ovsdb:"name"`
	Type    string            `ovsdb:"type"`
	Options map[string]string `ovsdb:"options"`
	Ifindex *int              `ovsdb:"ifindex"`
	// IngressPolicingRate (kbps) and IngressPolicingBurst (kb) police what a port may send into the bridge: the storm control of a
	// shared switch. 0 = not policed.
	IngressPolicingRate  int `ovsdb:"ingress_policing_rate"`
	IngressPolicingBurst int `ovsdb:"ingress_policing_burst"`
}

type OVSManager

type OVSManager struct {
	// contains filtered or unexported fields
}

genevePortName is retained for transitional call sites and returns the shared port name regardless of the remote address argument. OVSManager programs the single br-ovs bridge via libovsdb (OVSDB JSON-RPC over Unix socket).

func NewOVSManager

func NewOVSManager(bridge, sockPath string) (*OVSManager, error)

func (*OVSManager) AddGenevePort

func (m *OVSManager) AddGenevePort(_, _ string) error

AddGenevePort creates the single shared Geneve VTEP. The arguments are retained for source compatibility with older call sites and ignored; the concrete VTEP target is driven from flow rules (NXM_NX_TUN_IPV4_DST) so only one port is needed per node.

func (*OVSManager) AddInternalPort

func (m *OVSManager) AddInternalPort(stableKey string) (string, error)

AddInternalPort creates an OVS internal port in br-ovs with a random kernel interface name. The stableKey (used for reconcile idempotency and cleanup) is stored in external_ids["port-key"]. Returns the random kernel interface name that will appear in the host netns.

func (*OVSManager) AddPatchPair

func (m *OVSManager) AddPatchPair(nameA, nameB string) error

AddPatchPair creates two paired patch ports — spec §7-§8: switch↔switch connections materialise as a patch-pair (each end is a port in its own switch's VNI; no third VNI is introduced). Patch is always intra-node; cross-node hops travel via the partner switch's Geneve mesh.

Idempotent: re-creating an existing pair is a no-op.

func (*OVSManager) AddVethPort

func (m *OVSManager) AddVethPort(stableKey string) error

AddVethPort creates a veth pair where stableKey is the host-side name (stays in OVS/root netns) and VethPeerName(stableKey) is the pod-side (moved to pod netns by the caller). Idempotent: EEXIST on kernel side is ignored; port already in OVS is a no-op. Stores stableKey in external_ids so FindPortByKey can locate it.

func (*OVSManager) Close

func (m *OVSManager) Close()

Close disconnects from the OVSDB server.

func (*OVSManager) DelPort

func (m *OVSManager) DelPort(name string) error

DelPort removes a port from br-ovs by name (idempotent).

func (*OVSManager) DelPortByKey

func (m *OVSManager) DelPortByKey(stableKey string) error

DelPortByKey removes the OVS port whose external_ids["port-key"] matches stableKey. Idempotent: no-op if not found.

func (*OVSManager) DelVethPort

func (m *OVSManager) DelVethPort(stableKey string) error

DelVethPort removes the OVS port and deletes the kernel veth pair for stableKey. Deleting the host-side also removes the pod-side (veth pair invariant). Idempotent: no-op if already gone.

func (*OVSManager) EnsurePolicing

func (m *OVSManager) EnsurePolicing(portName string) error

EnsurePolicing applies the policing rate to an existing veth port (one made before the setting, or under another value).

func (*OVSManager) FindPortByKey

func (m *OVSManager) FindPortByKey(stableKey string) (string, bool, error)

FindPortByKey returns the OVS port name whose external_ids["port-key"] matches stableKey. Returns ("", false, nil) if not found.

func (*OVSManager) Ping

func (m *OVSManager) Ping(ctx context.Context) error

Ping asks ovsdb-server for an answer: the node-agent has lost it when this fails.

func (*OVSManager) PortExists

func (m *OVSManager) PortExists(name string) (bool, error)

PortExists checks whether a named port exists on br-ovs.

func (*OVSManager) PortKeys

func (m *OVSManager) PortKeys() (map[string]bool, error)

PortKeys returns the stable keys of the OVS ports the platform made (external_ids["port-key"]), as a set.

func (*OVSManager) SetPolicing

func (m *OVSManager) SetPolicing(kbps int)

SetPolicing sets the storm control of the veth ports: each may send at most kbps into the bridge (burst a tenth of it). A pure flood or a loop through a Linux bridge inside one lab is held to that rate instead of the whole datapath of the node. 0 switches it off.

func (*OVSManager) WaitForPortSetup

func (m *OVSManager) WaitForPortSetup(portName string, timeout time.Duration) error

WaitForPortSetup polls netlink until the kernel interface portName has type "openvswitch" with a stable ifindex for at least 300ms.

Why ifindex stability matters: vswitchd may delete+recreate the vport during its initial dpif_port_add setup when it detects the interface left root netns (OVS_VPORT_CMD_DEL + OVS_VPORT_CMD_NEW in rapid succession, logged as "deleted→added" in vswitchd). The recreated interface has a different ifindex. Waiting for ifindex stability ensures we move the interface only after vswitchd has entered its "Phase 2" (established) state, where a netns move causes it to set ifindex=0 rather than trigger another delete+recreate.

type OVSOpen_vSwitch

type OVSOpen_vSwitch struct {
	UUID    string   `ovsdb:"_uuid"`
	Bridges []string `ovsdb:"bridges"`
}

type OVSPort

type OVSPort struct {
	UUID        string            `ovsdb:"_uuid"`
	Name        string            `ovsdb:"name"`
	Interfaces  []string          `ovsdb:"interfaces"`
	ExternalIDs map[string]string `ovsdb:"external_ids"`
}

Directories

Path Synopsis
Package ofclient implements a minimal OpenFlow 1.3 client over a Unix socket.
Package ofclient implements a minimal OpenFlow 1.3 client over a Unix socket.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL