demux

package
v1.7.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 20, 2026 License: Apache-2.0 Imports: 34 Imported by: 0

Documentation

Index

Constants

View Source
const SessionTTL = conntrackTTL

SessionTTL is how long a completed session lives after its last packet; the fast path leaves it to the slow path after that too.

Variables

View Source
var FastPathResults = []string{
	"forwarded_to_server", "forwarded_to_client", "passed_miss", "passed_source", "passed_stale", "dropped_rate",
	"passed_route", "passed_meta",
}

FastPathResults are the counters of the in-kernel fast path, in the order of its enum wg_stat.

Functions

func CacheOptions added in v1.4.0

func CacheOptions(namespace, serviceName, nodeName string) cache.Options

CacheOptions restricts a manager's cache to the one Service and the one Node, so the demux needs get/list/watch only on those (services in its namespace, and nodes) and does not hold the cluster's objects in memory.

func EntryAddrs added in v1.4.0

func EntryAddrs(svc *corev1.Service, node *corev1.Node, hostIP string) []netip.AddrPort

EntryAddrs computes the address:port pairs the fast path matches on this node: the node addresses with each UDP nodePort, and the load balancer and external IPs with each UDP port. Only IPv4 is returned (the fast path is IPv4, IPv6 stays on the slow path). The result is sorted and has no duplicates. Either object may be nil.

func MetricsHandler added in v1.4.0

func MetricsHandler(mode Mode, d *Demux, fast func() map[string]uint64) http.Handler

MetricsHandler serves the demux's metrics in the Prometheus format. fast reads the fast path's counters by result (nil in userspace mode).

Types

type ConnEntry

type ConnEntry struct {
	PeerIndex      uint32
	SenderSocket   Socket
	ReceiverSocket Socket
	LastSeen       time.Time
	// contains filtered or unexported fields
}

type ConnTrack

type ConnTrack struct {
	// contains filtered or unexported fields
}

func NewConnTrack

func NewConnTrack() *ConnTrack

func NewConnTrackWithLimits

func NewConnTrackWithLimits(l Limits) *ConnTrack

func (*ConnTrack) AddPartial

func (c *ConnTrack) AddPartial(ci uint32, clientSocket, serverSocket Socket, groups ...string) bool

AddPartial creates the Ci entry after type 1 forward (Si unknown yet). Returns false (and changes nothing) when the index is already taken by a different live session (spec §4: the handshake is dropped so upstream WireGuard retries with a new index, instead of overwriting a peer), or when the table is full.

func (*ConnTrack) Cleanup

func (c *ConnTrack) Cleanup() int

Cleanup removes the stale entries now and says how many it removed.

func (*ConnTrack) Complete

func (c *ConnTrack) Complete(si, ci uint32, clientSocket, serverSocket Socket) bool

Complete fills in Si after type 2 response, creating mirror entry. Returns false when Si collides with a live session that is not the matching peer, or the table is full: the response is dropped and the handshake fails; upstream WG retries.

func (*ConnTrack) CompleteFrom added in v1.4.0

func (c *ConnTrack) CompleteFrom(si, ci uint32, clientSocket, serverSocket Socket, replyIfindex int) bool

CompleteFrom is Complete with the device the answer arrived on, where the fast path takes the server's replies from.

func (*ConnTrack) DeleteUnowned added in v1.4.0

func (c *ConnTrack) DeleteUnowned(index uint32, del func(uint32) error) bool

DeleteUnowned calls del for an index no live entry owns, under the table's lock, so a handshake cannot take the index between the check and the delete. It says whether del was called.

func (*ConnTrack) Expected

func (c *ConnTrack) Expected(index uint32) (Socket, bool)

Expected is the source the session of an index expects its packets from.

func (*ConnTrack) Len

func (c *ConnTrack) Len() int

Len is the number of entries.

func (*ConnTrack) Lookup

func (c *ConnTrack) Lookup(receiverIndex uint32) (dst Socket, found bool)

Lookup finds the entry, updates LastSeen, and returns ReceiverSocket. Used for type-4 userspace fallback: receiverIndex → forward to ReceiverSocket.

func (*ConnTrack) LookupForward

func (c *ConnTrack) LookupForward(receiverIndex uint32, src Socket) (dst Socket, found bool)

LookupForward returns the forwarding destination for a type-4 transport packet. receiver_index identifies the session:

  • entries[Si].SenderSocket = serverSocket → forward client→server traffic
  • entries[Ci].SenderSocket = clientSocket → forward server→client traffic

ReceiverSocket is the expected source of packets carrying this index. A packet from another source is the session roaming, which is allowed, but at most once per RoamInterval for a session: a packet from a third address within the interval is not forwarded and moves nothing (so a spoofed packet cannot keep dragging the session around).

func (*ConnTrack) LookupPartial

func (c *ConnTrack) LookupPartial(index uint32) (client, backend Socket, found bool)

LookupPartial returns the sockets of a handshake in progress: the client that sent the init (SenderSocket) and the backend it was forwarded to (ReceiverSocket, the only address its type-2 answer may come from).

func (*ConnTrack) LookupSender

func (c *ConnTrack) LookupSender(index uint32) (Socket, bool)

LookupSender returns SenderSocket for the given index without updating LastSeen.

func (*ConnTrack) LookupStrict

func (c *ConnTrack) LookupStrict(receiverIndex uint32, src Socket) (dst Socket, found bool)

LookupStrict is LookupForward without roaming: the packet is forwarded only from the exact source the session expects (cookie replies).

func (*ConnTrack) RemoveGroup added in v1.1.0

func (c *ConnTrack) RemoveGroup(group string)

RemoveGroup retires only sessions belonging to the changed/deleted group.

func (*ConnTrack) Restore added in v1.4.0

func (c *ConnTrack) Restore(clientIdx, serverIdx uint32, client, server Socket, group string, seen time.Time, replyIfindex int, entry Socket) bool

Restore re-creates a completed session from the data plane after a restart (both indices, group, reply device and entry address given; an unset entry is learnt again from the data plane). The data plane already holds the session, so nothing is pushed back. It refuses an index that a live different session owns, a full table, and a session that is already past its TTL.

func (*ConnTrack) RunTTLCleanup

func (c *ConnTrack) RunTTLCleanup(stop <-chan struct{})

RunTTLCleanup removes stale entries in a background goroutine.

func (*ConnTrack) Seen added in v1.4.0

func (c *ConnTrack) Seen(at map[uint32]time.Time)

Seen applies the data plane's last-seen times (newer only) before a cleanup.

func (*ConnTrack) SessionCount added in v1.4.0

func (c *ConnTrack) SessionCount() int

SessionCount is the number of completed live sessions.

func (*ConnTrack) Sessions added in v1.4.0

func (c *ConnTrack) Sessions() []Forward

Sessions lists the completed sessions as the two directions each, ordered by index, for restore checks and metrics.

func (*ConnTrack) SetSink added in v1.4.0

func (c *ConnTrack) SetSink(s Sink, src SourceFor)

SetSink sets the data plane the sessions are mirrored into. A nil sink keeps the pure userspace behaviour.

func (*ConnTrack) SinkErrors added in v1.4.0

func (c *ConnTrack) SinkErrors() uint64

SinkErrors is how many sink calls (and source lookups) failed; the slow path keeps forwarding for those sessions.

func (*ConnTrack) UpdateRoaming

func (c *ConnTrack) UpdateRoaming(receiverIndex uint32, newSender Socket)

UpdateRoaming updates sender socket when client IP changes.

type Demux

type Demux struct {
	// contains filtered or unexported fields
}

Demux handles incoming WireGuard UDP packets on the one public port every team shares.

type 1 (handshake init)   forwarded to the group's VPN pod found by mac1; rate-limited (all sources together)
type 2 (handshake answer) accepted only from the backend the init was forwarded to
type 3 (cookie reply)     forwarded like a transport packet, but only from the exact expected source
type 4 (transport data)   forwarded by receiver index; roaming allowed at most once per interval

Nothing here is authenticated (the demux does not hold the keys: the VPN pods do, and WireGuard's own handshake is the boundary), so everything a stranger can make it hold or spend is capped (see Limits) and nothing is logged per packet.

func New

func New(listenAddr string, table *Table, ct *ConnTrack) (*Demux, error)

func NewWithLimits

func NewWithLimits(listenAddr string, table *Table, ct *ConnTrack, l Limits) (*Demux, error)

func (*Demux) Close

func (d *Demux) Close()

func (*Demux) LocalAddr

func (d *Demux) LocalAddr() net.Addr

LocalAddr is the address the demux listens on.

func (*Demux) Received added in v1.4.0

func (d *Demux) Received() [5]uint64

Received is how many datagrams of each WireGuard message type (index 1..4) the demux has taken.

func (*Demux) Run

func (d *Demux) Run(stop <-chan struct{})

Run processes incoming WireGuard packets until stop is closed. Several goroutines read the socket (a UDP socket is safe to read from many), so the cost of one packet does not hold the others up.

type EntryWatcher added in v1.4.0

type EntryWatcher struct {
	client.Client
	NodeName         string
	HostIP           string
	ServiceName      string
	ServiceNamespace string
	OnChange         func([]netip.AddrPort)
	// contains filtered or unexported fields
}

EntryWatcher keeps the entry addresses current. It reads the one Service and the one Node of this demux; the manager cache is restricted to exactly those two objects (see CacheOptions), so any event in it is an event of one of them. OnChange gets the full sorted set whenever it differs from the last one.

func (*EntryWatcher) Reconcile added in v1.4.0

func (w *EntryWatcher) Reconcile(ctx context.Context, _ ctrl.Request) (ctrl.Result, error)

func (*EntryWatcher) SetupWithManager added in v1.4.0

func (w *EntryWatcher) SetupWithManager(mgr ctrl.Manager) error

SetupWithManager reconciles one fixed key on any Service or Node event: the result depends on both objects, not on which one moved.

type Forward added in v1.4.0

type Forward struct {
	Index     uint32 // receiver index carried by the packets
	Session   uint32 // the client -> server direction's index in both directions: that entry owns the session's bucket and last-seen
	Expected  Socket // the only source accepted (else the packet goes to the slow path)
	NewSource Socket // source written into the packet (entry addr towards the client; probe source towards the server)
	Dest      Socket // destination written into the packet
	ToClient  bool
	// Towards the client only: Via is the address the server sends to (the client -> server direction's NewSource), and
	// ReplyIfindex the device the server's answer arrived on (0 = unknown, the replies stay on the slow path).
	Via          Socket
	ReplyIfindex int
}

Forward is one direction of a session as the fast path needs it.

type LabGroupWatcher

type LabGroupWatcher struct {
	client.Client
	Table          *Table
	VPNServicePort int
	// contains filtered or unexported fields
}

LabGroupWatcher is a controller-runtime reconciler keeping Table in sync with LabGroups.

func (*LabGroupWatcher) Reconcile

func (w *LabGroupWatcher) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error)

func (*LabGroupWatcher) SetupWithManager

func (w *LabGroupWatcher) SetupWithManager(mgr ctrl.Manager) error

func (*LabGroupWatcher) Synced

func (w *LabGroupWatcher) Synced(ctx context.Context) bool

Synced says whether every LabGroup the cache holds has been reconciled into the table at least once: only then does the demux know the groups' keys, and a handshake of a group that is not in the table is dropped. It is the readiness of the wg-demux container.

type Limits

type Limits struct {
	// MaxEntries is the most conntrack entries (two per session) in the whole table. A handshake that would pass it is
	// dropped (the client retries).
	MaxEntries int
	// PartialTTL is how long a handshake in progress (the init was forwarded, no answer yet) keeps its entry. It is short:
	// the entry is made before anything is authenticated, so a flood of inits must not be able to fill the table for the
	// full conntrack lifetime. A completed session keeps its entry for conntrackTTL after its last packet.
	PartialTTL time.Duration
	// GlobalHandshakeRate and GlobalHandshakeBurst limit handshake initiations of all sources together: each costs a scan of every
	// group's key, so the sources cannot add up to more than the demux can pay.
	GlobalHandshakeRate  float64
	GlobalHandshakeBurst int
	// MissRate and MissBurst limit, for all sources together, the packets that match no session or come from an unexpected
	// address (a guess of the 32-bit index, a spoof, a roam): guessing an index costs one of them, so guessing is rate-limited.
	MissRate  float64
	MissBurst int
	// RoamInterval is the least time between two address changes of one session.
	RoamInterval time.Duration
	// SessionRate and SessionBurst limit the transport packets of one session per second: the demux is shared by everyone,
	// and one participant with a valid session must not be able to use its CPU up. 0 = unlimited.
	SessionRate  float64
	SessionBurst int
	// Readers is how many goroutines read the socket (1 when zero).
	Readers int
}

Limits bound what an unauthenticated sender can make the demux hold or do. The demux is shared by every team and reads a public UDP port, so every table and every cost it spends on a packet from a stranger has a cap.

There is no limit per source address: one NAT (a venue, a campus) hides a whole event behind one address, and behind a load balancer in SNAT mode every client has a node's address. What is capped is the whole table, the handshakes and the unmatched packets of all sources together, and the packets of one session.

func DefaultLimits

func DefaultLimits() Limits

DefaultLimits mirror the chart (proxy.wg.limits).

type Mac1Key

type Mac1Key [32]byte

Mac1Key is the 32-byte key used to compute and verify mac1 in WireGuard handshake init.

type Mode added in v1.4.0

type Mode string

Mode is how the demux moves transport packets: in userspace, or in the kernel through a TC or XDP program.

const (
	ModeUserspace Mode = "userspace"
	ModeTC        Mode = "tc"
	ModeXDP       Mode = "xdp"
)

type Sink added in v1.4.0

type Sink interface {
	Upsert(f Forward) error
	Delete(index uint32) error
}

Sink mirrors the sessions of a ConnTrack into a data plane. ConnTrack calls it with its lock held, so an implementation must be quick and must not call back into the ConnTrack.

type Socket

type Socket struct {
	IP   net.IP
	Port uint16
}

type SourceFor added in v1.4.0

type SourceFor interface {
	TowardsServer(server Socket) (Socket, bool)
	TowardsClient(client Socket) (Socket, bool) // the entry address the client's handshake arrived at
}

SourceFor gives NewSource for each direction; nil in userspace mode.

type Table

type Table struct {
	OnChange func(string) // invoked under table lock; callback must not enter Table
	// contains filtered or unexported fields
}

Table is an in-memory mac1_key→UID/backend mapping, rebuilt from LabGroup watches. Brute-force scan is O(n) but handshake init is rare (~once per 2 min per peer).

func NewTable

func NewTable() *Table

func (*Table) Delete

func (t *Table) Delete(uid string)

Delete removes the entry for uid.

func (*Table) Entries added in v1.4.0

func (t *Table) Entries() []TableEntry

Entries is a copy of the table.

func (*Table) FindByMac1

func (t *Table) FindByMac1(packet []byte) (uid string, backend *net.UDPAddr, found bool)

FindByMac1 brute-forces mac1 verification. packet is the raw WireGuard type-1 packet (UDP payload). mac1 occupies bytes [len-32 : len-16]; the message body for MAC is packet[:len-32]. Returns the matched group UID and its VPN Service backend (host:port).

func (*Table) RunResolver

func (t *Table) RunResolver(stop <-chan struct{}, every time.Duration)

RunResolver keeps the resolved address of every backend fresh, off the read loop, until stop is closed. A backend that cannot be resolved keeps its last good address.

func (*Table) RunResolverContext added in v1.1.0

func (t *Table) RunResolverContext(ctx context.Context, every time.Duration)

func (*Table) Update

func (t *Table) Update(uid, pubKey, backend string) error

type TableEntry

type TableEntry struct {
	UID       string
	PublicKey string
	Mac1Key   Mac1Key
	// Backend is the in-cluster Service address (host:port) for the VPN server of this group; Resolved is it resolved, kept up to
	// date by Table.RunResolver (the read loop never does a DNS lookup).
	Backend  string
	Resolved *net.UDPAddr
}

Directories

Path Synopsis
Package fastpath is the in-kernel fast path of the WireGuard demux (modes tc and xdp): the eBPF programs of wg_fastpath.bpf.c, which forward transport packets of known sessions, and the pinned maps the demux keeps them in.
Package fastpath is the in-kernel fast path of the WireGuard demux (modes tc and xdp): the eBPF programs of wg_fastpath.bpf.c, which forward transport packets of known sessions, and the pinned maps the demux keeps them in.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL