network

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 9, 2026 License: Apache-2.0 Imports: 22 Imported by: 0

Documentation

Overview

Package network is ADR 006's WireGuard mesh: the abstraction that ADR 006 says "has to be designed in Phase 1 as an interface, not a concrete WireGuard dependency" and that, until TASKS.md 3.4, did not exist at all (the directory the project's repo layout reserves for it was empty).

The split this package draws, and why:

  • Everything that decides *what the mesh should look like* (Plan, AllocateAddress, BuildRecords, Coordinator) is pure Go over plain values. It needs no root, no kernel module, no TUN device, and no network. It is therefore table-driven testable, which matters because it is where the actual product decisions live: full mesh rather than hub-and-spoke, which node an app's DNS name points at, what happens when a node reports a key that collides with another.
  • Everything that *makes the kernel agree with that decision* (Device, uapi.go, detect.go) is deliberately thin, and is the only part that touches wireguard-go, privileged syscalls, or /sys. It cannot be unit tested without root; what it can do is be small enough that a human reviewing it can see it is a faithful translation of a DeviceConfig, and no more.

That boundary is the same one internal/docker already draws between Runtime (the narrow surface controllers depend on) and Client (the part that actually talks to a daemon), and it exists here for the same reason: the interesting failure modes are in the decisions, not in the syscalls.

This package deliberately does not import internal/brand. The DNS zone is derived from a short name passed in by the caller (see Zone), so nothing here embeds a product name, matching the rule that no product name string appears anywhere in source, and the package stays testable without loading brand.yaml.

Index

Constants

View Source
const DefaultKeepalive = 25 * time.Second

DefaultKeepalive is the persistent-keepalive interval set on every peer. Non-zero by default on purpose: ADR 003 already commits to nodes that may sit behind NAT ("works behind NAT and residential connections"), and a NAT mapping with no traffic through it expires in well under a minute on consumer hardware. 25s is WireGuard's own documented recommendation for exactly this case.

View Source
const DefaultListenPort = 51820

DefaultListenPort is the UDP port a node's WireGuard device listens on unless an operator overrides it. 51820 is WireGuard's own conventional port; using the convention means an operator's existing firewall muscle memory applies unchanged.

View Source
const DefaultZoneSuffix = "internal"

DefaultZoneSuffix is the parent of every internal name. ".internal" is reserved by RFC 6762/8375 convention for exactly this (names that are only meaningful inside one network and must never resolve publicly), which matters more than it sounds: a suffix that could be a real TLD means a lookup that misses the mesh resolver silently escapes to the public internet instead of failing.

View Source
const KeyLen = 32

KeyLen is the length in bytes of a Curve25519 key, public or private.

Variables

View Source
var (
	// ErrUnknownNode is returned by Plan when the node it was asked to
	// plan for is not in the inventory it was given.
	ErrUnknownNode = errors.New("network: node not present in the mesh inventory")

	// ErrZeroKey is returned when a node's public key is the all-zero
	// value. That is never a legitimate key; it is what an unset field
	// looks like, and treating it as a real key would configure a peer
	// that can never handshake while looking fine in the config.
	ErrZeroKey = errors.New("network: node has no public key")

	// ErrDuplicatePublicKey is returned when two nodes report the same
	// public key. WireGuard identifies peers solely by public key, so
	// two nodes sharing one is not a duplicate row to be tolerated, it
	// makes the mesh silently misroute: whichever peer entry was written
	// last wins for both nodes.
	ErrDuplicatePublicKey = errors.New("network: two nodes share a public key")

	// ErrDuplicateAddress is returned when two nodes claim the same mesh
	// address. Same class of failure as a duplicate key, one layer up.
	ErrDuplicateAddress = errors.New("network: two nodes share a mesh address")

	// ErrAddressOutsideMesh is returned when a node's mesh address is
	// not inside the configured mesh CIDR, which would produce an
	// AllowedIPs entry routing traffic somewhere the operator never
	// delegated to this mesh.
	ErrAddressOutsideMesh = errors.New("network: node address is outside the mesh CIDR")

	// ErrNoAddressAvailable is returned by AllocateAddress when the mesh
	// CIDR is fully allocated.
	ErrNoAddressAvailable = errors.New("network: no free address left in the mesh CIDR")

	// ErrInvalidMeshCIDR is returned when the configured mesh CIDR is
	// unset, not canonical, or not IPv4.
	ErrInvalidMeshCIDR = errors.New("network: invalid mesh CIDR")

	// ErrMeshClosed is returned by a Mesh whose Close has already been
	// called.
	ErrMeshClosed = errors.New("network: mesh is closed")
)

The errors this package returns. Every one of them is a real, reachable state rather than a defensive check: a control plane assembling a mesh from node-reported values is assembling it from data it did not generate itself, so every one of these is something a misbehaving or misconfigured node can actually cause.

View Source
var DefaultMeshCIDR = netip.MustParsePrefix("10.181.0.0/16")

DefaultMeshCIDR is the private range the mesh allocates from unless an operator overrides it.

10.181.0.0/16 rather than something more memorable: the whole point of picking an obscure corner of RFC 1918 space is that it is unlikely to collide with whatever the operator's existing LAN, VPN, or cloud VPC already uses. 10.0.0.0/24 and 192.168.1.0/24 are the two ranges most likely to already be in use on a machine someone is about to add to a fleet, and a mesh address that collides with the host's own LAN route is a routing failure that looks like a mesh failure.

A /16 holds 65534 hosts, far beyond the 1-to-10 machines this project targets. Sized for headroom rather than fit because narrowing a CIDR after nodes hold addresses from the wider one is the migration ValidateInventory's ErrAddressOutsideMesh exists to make loud, and nobody should have to perform it.

Functions

func AllocateAddress

func AllocateAddress(cidr netip.Prefix, taken []netip.Addr) (netip.Addr, error)

AllocateAddress returns the lowest address in cidr that is not already in taken.

Lowest-free rather than next-sequential or random: it is deterministic (the same inputs always produce the same output, which is what makes this table-testable), and it reuses the address of a removed node, which matters because a mesh that only ever allocates upward would eventually exhaust even a /16 in a fleet that churns nodes.

The network address itself (cidr's own base) is skipped: it is not a usable host address. The broadcast address is not skipped, because a WireGuard interface is point-to-point and has no broadcast domain; excluding it would be borrowing an Ethernet convention that does not apply here.

func EncodeUAPI

func EncodeUAPI(cfg DeviceConfig, current []Key) string

EncodeUAPI renders cfg as a UAPI "set" payload, given the set of peer public keys the device currently has.

current is what makes this a converging apply rather than a destructive one. The obvious encoding is to emit replace_peers=true and let the device discard everything it had, which is correct in the level- triggered sense and one line shorter. It is rejected here because wireguard-go's replace_peers destroys and rebuilds every Peer object, discarding the established session keys with them, so every reconcile tick would force a fresh handshake with every node in the fleet. On a 15-second resync that is a mesh that never finishes handshaking.

Instead: peers in current but not in cfg are explicitly removed, and peers in cfg are upserted. An upsert of an unchanged peer leaves its session untouched, so a reconcile pass that changes nothing costs nothing, which is the property level-triggered reconcile design depends on to be run repeatedly and safely.

The private key is written first and only when set. A zero PrivateKey is skipped rather than written as 32 zero bytes: writing zeros would tell the device to discard its identity, which is the opposite of "this field is not being changed."

func MeshAddresses

func MeshAddresses(nodes []NodeInfo) []netip.Addr

MeshAddresses extracts the assigned mesh addresses from an inventory, for a caller that needs the set of addresses currently in use (the store, persisting them; a status endpoint, showing them).

func ParseEndpoint

func ParseEndpoint(s string) (string, error)

ParseEndpoint validates a "host:port" endpoint string and returns it normalized. Returns an empty string with no error for an empty input: "no endpoint yet" is a legitimate state (see NodeInfo.Endpoint), not a parse failure.

func PlanAll

func PlanAll(nodes []NodeInfo, opts PlanOptions) (map[string]DeviceConfig, error)

PlanAll computes every node's DeviceConfig in one pass, validating the inventory once instead of once per node. This is what the Coordinator calls: distributing configs one node at a time through repeated Plan calls would revalidate the same inventory N times and, worse, could distribute a partially-planned mesh if validation failed partway through.

func ValidateInventory

func ValidateInventory(nodes []NodeInfo, meshCIDR netip.Prefix) error

ValidateInventory checks a whole node inventory for the collisions that would make a mesh silently misroute rather than loudly fail, and returns the first problem it finds.

Only nodes that are Ready are checked: a node that has enrolled but not yet reported a key or been assigned an address is a normal in-progress state (it simply is not a peer yet), and failing the whole inventory for it would mean one half-enrolled node stops the entire fleet from meshing.

meshCIDR bounds every node address. It is checked rather than assumed because addresses are persisted values that outlive any single run: an operator who narrows APP_MESH_CIDR after nodes were allocated addresses from a wider one would otherwise get AllowedIPs entries pointing outside the range they meant to delegate, and would find out from a routing symptom rather than an error.

func Zone

func Zone(shortName string) string

Zone returns the internal DNS zone for a brand short name, e.g. a short name of "Acme" gives "acme.internal".

Derived from a caller-supplied short name rather than a constant because the product name string must never appear in source, and because the zone is genuinely user-visible: it appears in every connection string a user writes, so it has to follow a rebrand rather than being frozen at whatever the name was when this was written. Callers pass brand.Brand.ShortName.

A short name with nothing usable in it (empty, or entirely punctuation) falls back to "mesh", which is descriptive rather than branded and so is safe to hardcode.

Types

type Backend

type Backend string

Backend names which WireGuard implementation a Mesh is running on. Not a bool "kernel or not": Disabled is a real, expected state (a single-node deployment never brings a mesh up at all, ADR 006's "single-node deployments do not pay any WireGuard cost"), and the value is surfaced in Status so an operator can tell the fast path from the portable one without guessing.

const (
	// BackendKernel is the in-kernel WireGuard module: the fast path,
	// used when the module is present and this process can configure it.
	BackendKernel Backend = "kernel"

	// BackendUserspace is wireguard-go: ADR 006's portability path, for
	// hosts where loading a kernel module is not an option.
	BackendUserspace Backend = "userspace"

	// BackendDisabled is no mesh at all. Everything still resolves and
	// reconciles; nothing is encrypted or routed between machines,
	// because there is only one machine.
	BackendDisabled Backend = "disabled"
)

The three states a Mesh's backend can be in.

type ConfigSink

type ConfigSink interface {
	// ApplyMesh sends cfg to nodeID and returns that node's reported
	// identity. An error means this one node did not get its config;
	// Distribute treats that as this node's problem, never the fleet's.
	ApplyMesh(ctx context.Context, nodeID string, cfg DeviceConfig) (NodeIdentity, error)
}

ConfigSink delivers one node's DeviceConfig to that node and returns what the node reports back.

An interface, and the only thing this package knows about transport, because the real implementation belongs to the agent Session stream and that stream's wire contract (proto/agent/v1/agent.proto) needs one coherent mental model and a single reviewer end to end, not several people each touching an uncoordinated slice of it. Extending it is a small and entirely mechanical change: one new arm on AgentRequest.op and one on AgentResponse.result, carrying the fields of DeviceConfig (minus PrivateKey, which never crosses) and NodeIdentity respectively, plus a case in internal/agent.Execute that calls Mesh.Apply. It is left out of this change so that it lands as its own reviewed diff against the transport, rather than as a wire-contract change buried inside a networking one.

What ships here instead: this interface, an in-process implementation (LocalSink) that is exactly what a single-node control plane needs and what the reconcile path exercises today, and the full distribution logic above it, all of it testable without a network.

type Coordinator

type Coordinator struct {
	// contains filtered or unexported fields
}

Coordinator distributes mesh configuration across a fleet.

Holds no state about the fleet between passes, on purpose: every Distribute call is given the complete inventory and recomputes everything from it. Caching "what I sent last time" would make this edge-triggered, and a control plane restart would then leave the fleet in whatever state it was in with nothing to reconcile it back.

func NewCoordinator

func NewCoordinator(sink ConfigSink, opts PlanOptions, options ...CoordinatorOption) *Coordinator

NewCoordinator builds a Coordinator that distributes through sink using opts. A zero PlanOptions.MeshCIDR is filled in with DefaultMeshCIDR rather than rejected, so a caller that has no opinion about addressing gets a working mesh instead of an error about a field they did not know existed.

func (*Coordinator) Distribute

func (c *Coordinator) Distribute(ctx context.Context, nodes []NodeInfo) ([]NodeInfo, DistributeResult, error)

Distribute runs one full pass: allocate any missing addresses, plan every node's config, send each one, and collect what came back.

Returns an error only for a problem with the inventory itself (a duplicate key, an address outside the mesh, an exhausted CIDR), because those make the whole plan wrong and distributing a wrong plan is worse than distributing nothing. A node that simply could not be reached is recorded in its NodeResult.Err and does not fail the pass.

The returned inventory is the input with any newly allocated addresses and newly reported identities folded in. The caller persists it; this function deliberately has no store, so that the allocation decision and the persistence of it are separable and the decision stays testable.

func (*Coordinator) SetObservedEndpoint

func (c *Coordinator) SetObservedEndpoint(nodeID, endpoint string) error

SetObservedEndpoint records where the control plane saw nodeID connect from. Passing "" forgets a previously observed endpoint, which is what should happen when a node disconnects: a stale endpoint for a node on a dynamic address is worse than none, since WireGuard will keep sending handshake initiations to whoever holds that address now.

type CoordinatorOption

type CoordinatorOption func(*Coordinator)

CoordinatorOption configures NewCoordinator.

func WithCoordinatorLogger

func WithCoordinatorLogger(l *slog.Logger) CoordinatorOption

WithCoordinatorLogger sets the structured logger.

type DNSServer

type DNSServer struct {
	// contains filtered or unexported fields
}

DNSServer serves A records for a Resolver's zone over UDP and TCP.

func NewDNSServer

func NewDNSServer(resolver *Resolver, logger *slog.Logger) *DNSServer

NewDNSServer builds a server for resolver. It does not listen until Start is called.

func (*DNSServer) Addr

func (s *DNSServer) Addr() netip.AddrPort

Addr reports the address the server actually bound, which is how a caller that asked for port 0 finds out what it got.

func (*DNSServer) Close

func (s *DNSServer) Close() error

Close shuts both listeners down. Idempotent.

func (*DNSServer) Start

func (s *DNSServer) Start(ctx context.Context, addr string) error

Start binds addr and begins serving on both UDP and TCP.

Both, not just UDP: a response larger than 512 bytes forces a client to retry over TCP, and a UDP-only server turns that into a hang rather than a retry. Answers here are small enough today that it should never happen, which is exactly why it would be an unpleasant surprise later.

addr of the form "host:0" picks a free port, which is what the tests use; Addr reports the port actually bound.

type DetectResult

type DetectResult struct {
	Backend Backend
	Reason  string
}

DetectResult is Detect's full answer: which backend to use, and why. The reason is not decoration. "Userspace" alone is indistinguishable from a misconfiguration; "userspace because the kernel module is not loaded" is an operator's next action.

func Detect

func Detect(p Probe) DetectResult

Detect chooses the WireGuard backend for this node.

The order of the checks is the decision:

  1. Not privileged: nothing works, not even userspace, because wireguard-go still needs to create a TUN device. This is reported as Disabled rather than as an error so a control plane that cannot mesh still starts and still reconciles containers, degraded but running. A platform that refuses to boot because it cannot bring up a VPN it may not even need on a single-node install would be a worse failure than the one it is reporting.
  2. Kernel module loaded, on Linux: the fast path.
  3. Everything else: wireguard-go. ADR 006's whole point is that this is a supported outcome, not a fallback to apologize for.

type Device

type Device struct {
	// contains filtered or unexported fields
}

Device is a Mesh backed by a real WireGuard device.

func (*Device) Apply

func (d *Device) Apply(ctx context.Context, cfg DeviceConfig) error

Apply converges the device on cfg. See EncodeUAPI for why this reads the device's current peers first instead of replacing them wholesale.

func (*Device) Close

func (d *Device) Close() error

Close tears the device down. Idempotent.

func (*Device) Status

func (d *Device) Status(ctx context.Context) (Status, error)

Status reads the device's live state.

type DeviceConfig

type DeviceConfig struct {
	NodeID     string
	PrivateKey Key

	// Address is this node's own mesh address with the mesh CIDR's
	// prefix length (e.g. 10.181.0.3/16, not /32), because that prefix
	// is what tells the host routing table to send the whole mesh range
	// out the WireGuard interface.
	Address    netip.Prefix
	ListenPort int
	Peers      []PeerConfig
}

DeviceConfig is the complete desired WireGuard state for exactly one node. Complete, not incremental: see Mesh.Apply on why there is no delta form.

PrivateKey is the local node's own, and is the one field in this package that must never be logged, serialized to the control plane, or included in any status response. It is only ever populated on the node the config is for, by that node, immediately before Apply; a DeviceConfig that crossed the wire from the control plane always has a zero PrivateKey and the receiving node fills it in from its own local key file.

func Plan

func Plan(selfID string, nodes []NodeInfo, opts PlanOptions) (DeviceConfig, error)

Plan computes the DeviceConfig for selfID from the full node inventory.

The returned config's PrivateKey is deliberately zero: this runs on the control plane, which does not have and must never have any node's private key (see DeviceConfig's own doc comment). The node fills it in from its own local key file before calling Apply.

A node in the inventory that is not Ready (no public key yet, no address yet) is skipped as a peer rather than treated as an error. Node enrollment (ADR 003) and mesh readiness are separate events, and a fleet where one machine enrolled thirty seconds ago must still be able to mesh the machines that are ready. The consequence is that the mesh converges over successive passes rather than atomically, which is the same level-triggered convergence every other controller in this codebase already has, not a weaker guarantee.

selfID itself does not need to be Ready. A node that has not yet reported a key still gets a valid plan naming its peers, which is exactly what it needs on its very first pass: it is the act of applying that plan that produces the key it will report back.

func (DeviceConfig) LogValue

func (c DeviceConfig) LogValue() slog.Value

LogValue implements slog.LogValuer so a DeviceConfig can be logged without leaking PrivateKey. Without this, any structured log line that passed a DeviceConfig would print 32 bytes of private key material into the log file; with it, the private key is structurally unreachable from the logging path rather than merely omitted by convention at each call site. Same "make the safe thing the only thing" reasoning internal/secrets applies to env values.

func (DeviceConfig) PeerByNodeID

func (c DeviceConfig) PeerByNodeID(nodeID string) (PeerConfig, bool)

PeerByNodeID returns the peer entry for nodeID, if this config has one.

type DeviceOption

type DeviceOption func(*deviceOptions)

DeviceOption configures NewDevice.

func WithLinkConfigurator

func WithLinkConfigurator(lc LinkConfigurator) DeviceOption

WithLinkConfigurator supplies the interface addressing step this package deliberately does not implement. See this file's header.

func WithLogger

func WithLogger(l *slog.Logger) DeviceOption

WithLogger sets the structured logger. Defaults to slog.Default().

func WithMTU

func WithMTU(mtu int) DeviceOption

WithMTU overrides the TUN device MTU. Defaults to wireguard-go's own DefaultMTU (1420), which is 1500 minus WireGuard's worst-case encapsulation overhead; lowering it is a real operator need on links that are themselves tunnelled (PPPoE, some VPS providers), where the default produces silent fragmentation-related stalls rather than an error.

func WithShortName

func WithShortName(s string) DeviceOption

WithShortName sets the brand short name the interface name is derived from (the product name never appears in source, so this is passed in, not looked up). Callers pass brand.Brand.ShortName.

type Disabled

type Disabled struct {
	// contains filtered or unexported fields
}

Disabled is a Mesh that accepts configuration and does nothing with it.

Apply is not an error: a single-node control plane still runs the same coordinator code path as a ten-node one, and making the zero-peer case an error would mean the mesh coordinator needs a "is the mesh on" branch that the multi-node path does not exercise, which is precisely the kind of untested-in-the-common-case branch that fails when it finally runs.

Applying a config with peers *is* recorded in Status, so an operator looking at a node whose services cannot reach another node sees "mesh backend: disabled, 3 peers configured" rather than an empty status indistinguishable from a working one.

func NewDisabled

func NewDisabled(reason string) *Disabled

NewDisabled returns a Mesh that does nothing, recording reason so Status can explain why. reason comes from DetectResult.Reason when the mesh is disabled by detection, or from the caller when it is disabled by configuration (a single-node deployment).

func (*Disabled) Apply

func (d *Disabled) Apply(ctx context.Context, cfg DeviceConfig) error

Apply records cfg without configuring anything.

func (*Disabled) Close

func (d *Disabled) Close() error

Close marks this mesh closed. Idempotent.

func (*Disabled) LocalAddress

func (d *Disabled) LocalAddress() netip.Prefix

LocalAddress reports the mesh address this node was last configured with, or the zero Prefix if none. Used by the DNS layer, which needs to answer for locally-placed services with an address that is reachable even when the mesh itself is disabled.

func (*Disabled) Reason

func (d *Disabled) Reason() string

Reason reports why this mesh is disabled.

func (*Disabled) Status

func (d *Disabled) Status(ctx context.Context) (Status, error)

Status reports the disabled backend and whatever config was last applied, with no peer state (there are no peers, only peer intentions).

type DistributeResult

type DistributeResult struct {
	// Nodes is one entry per node in the inventory, in node ID order.
	Nodes []NodeResult

	// Updated lists the nodes whose reported identity differed from what
	// the inventory held going in. A non-empty Updated means the caller
	// must persist those identities and that the next pass will produce a
	// different plan; an empty one means the mesh has converged.
	Updated []string
}

DistributeResult is one full distribution pass.

func (DistributeResult) Failed

func (r DistributeResult) Failed() []NodeResult

Failed reports whether any node failed. Distribute itself returns a nil error in that case (see NodeResult.Err), so this is how a caller asks.

type Key

type Key [KeyLen]byte

Key is one Curve25519 key. The same type covers public and private keys because they are the same shape and WireGuard itself does not distinguish them at the wire level; which one a given value is comes from the field it sits in (DeviceConfig.PrivateKey versus PeerConfig.PublicKey), not from its type.

An array, not a slice, on purpose: it is comparable with ==, usable as a map key (which the duplicate-key detection in Validate relies on), and cannot be nil or the wrong length, so no length check is needed at any use site.

func GeneratePrivateKey

func GeneratePrivateKey() (Key, error)

GeneratePrivateKey returns a new random Curve25519 private key, clamped per RFC 7748 the way WireGuard requires.

This runs on the node that will use the key, never on the control plane: a node's private key never leaves the machine it belongs to, and the control plane only ever sees public keys (see Coordinator's doc comment for why the protocol is shaped to make that the only possible flow, rather than a convention that could be broken by a later change).

func ParseKey

func ParseKey(s string) (Key, error)

ParseKey decodes a standard-base64 key, the form wg(8) emits and the form the control plane stores and ships over the wire.

func (Key) Hex

func (k Key) Hex() string

Hex returns the lowercase hex encoding, the form wireguard-go's UAPI protocol uses (unlike wg(8)'s config files, which use base64). Kept distinct from String rather than making one of them the default, so neither the human-facing form nor the wire form is a silent choice at the call site.

func (Key) IsZero

func (k Key) IsZero() bool

IsZero reports whether this is the all-zero key, which is what an unset field looks like and never a legitimate key. Callers use this rather than comparing against Key{} directly so the intent ("this node has not reported a key yet") reads at the call site.

func (Key) PublicKey

func (k Key) PublicKey() (Key, error)

PublicKey derives the public key for this private key.

func (Key) String

func (k Key) String() string

String returns the standard base64 encoding, the same textual form wg(8) prints and accepts. Note this is the *only* String on this type: a private key formats identically to a public one, so nothing here can distinguish them for redaction purposes. Callers must not log a DeviceConfig.PrivateKey; see DeviceConfig's own doc comment.

type LinkConfigurator

type LinkConfigurator interface {
	// SetAddress assigns addr to the interface named iface and brings the
	// interface up.
	SetAddress(ctx context.Context, iface string, addr netip.Prefix) error
}

LinkConfigurator assigns a mesh address to the network interface the WireGuard device was created on, and brings it up. See this file's header for why it is a seam rather than an implementation.

Implementations must be idempotent: Apply calls SetAddress on every reconcile pass with the same address, the same level-triggered contract Mesh.Apply itself has.

type LocalSink

type LocalSink struct {
	// contains filtered or unexported fields
}

LocalSink is the ConfigSink for a node whose Mesh this process holds directly: the control plane's own node in every deployment, and every node in a single-node one.

This is the in-process counterpart to internal/agent.Local, and exists for the same reason that type does: single-node mode runs in the same process as the control plane, communicating over an in-memory transport that implements the same interface as the real one. The code path above it is identical whether there is one node or ten.

func NewLocalSink

func NewLocalSink(nodeID string, mesh Mesh, privateKey Key) (*LocalSink, error)

NewLocalSink builds a sink that applies configs to mesh as nodeID, using privateKey as this node's identity.

func (*LocalSink) ApplyMesh

func (s *LocalSink) ApplyMesh(ctx context.Context, nodeID string, cfg DeviceConfig) (NodeIdentity, error)

ApplyMesh applies cfg to the local mesh and reports this node's identity back.

A config addressed to a different node is an error rather than being silently applied: a sink that accepted anyone's config would apply another node's peer list, and every peer entry in it would be wrong (including, worst of all, one for this node itself).

func (*LocalSink) PublicKey

func (s *LocalSink) PublicKey() Key

PublicKey reports this node's public key.

type Mesh

type Mesh interface {
	// Apply converges this node's WireGuard device on cfg. Peers present
	// on the device but absent from cfg are removed; peers in cfg that
	// the device does not have are added; a peer whose endpoint or
	// allowed IPs changed is updated in place, not torn down and
	// recreated (tearing down would drop the session key and force a new
	// handshake for no reason).
	Apply(ctx context.Context, cfg DeviceConfig) error

	// Status reports what the device currently looks like, including
	// per-peer last-handshake times. This is what makes "peer
	// unreachable" observable rather than silent: a peer configured
	// minutes ago with a zero LastHandshake has never completed a
	// handshake, which is the mesh's equivalent of a failing health
	// check.
	Status(ctx context.Context) (Status, error)

	// Close tears the device down. Safe to call more than once.
	Close() error
}

Mesh is the abstraction ADR 006 requires: everything above it knows "make this node's networking match this desired state" and nothing about WireGuard, kernel modules, or TUN devices.

Apply is level-triggered and idempotent, the same contract every reconciler in this codebase already has: reconcilers are idempotent, level-triggered, and safe to interrupt, never edge-triggered. Callers hand it the complete desired peer set every time, never a delta: there is deliberately no AddPeer/RemovePeer pair, because a delta API makes the caller responsible for tracking what it already sent, which is precisely the edge-triggered design that contract rules out.

func NewDevice

func NewDevice(ctx context.Context, probe Probe, opts ...DeviceOption) (Mesh, error)

NewDevice creates a WireGuard device for this node.

The backend is chosen by Detect, and a BackendDisabled result is returned as a *Disabled rather than an error: a node that cannot mesh must still run (see Detect's doc comment for why).

A BackendKernel result currently still runs wireguard-go. That is a real, deliberate gap and not an oversight: driving the in-kernel module means creating the interface over netlink (RTM_NEWLINK with a "wireguard" link kind) and configuring it over the wireguard generic netlink family, which is a second, Linux-only implementation of everything in this file plus its own dependency. Detection is built now, per ADR 006 and TASKS.md 3.4, because the decision point and its reasoning are what a later change needs in place; the second backend itself is worth landing on its own, against a real Linux node, rather than written blind here. Status reports the backend actually in use, so this never claims otherwise.

type NodeIdentity

type NodeIdentity struct {
	// PublicKey is derived from the private key the node generated and
	// keeps locally. This is the only key material that ever moves.
	PublicKey Key

	// ListenPort is the UDP port the node's device actually bound, which
	// can differ from the one it was asked for if that port was taken.
	ListenPort int

	// Endpoint is the node's own view of how it can be reached
	// ("host:port"), which may be empty. It is a hint, not an authority:
	// a node behind NAT genuinely does not know its own public address,
	// and the control plane's observation of where the node's gRPC
	// session dialed from is a better source. Coordinator prefers the
	// observed value when it has one; see Distribute.
	Endpoint string
}

NodeIdentity is what a node reports back about itself when it applies a mesh config: the half of its configuration only it can know.

type NodeInfo

type NodeInfo struct {
	// ID is store.Node.ID. The empty string is not valid here, unlike in
	// desired_services.node_id where "" means "the control plane's own
	// local node" (migrations/0009_node_placement.sql): a mesh has no
	// implicit node, every participant including the control plane's own
	// has a real row and a real key.
	ID   string
	Name string

	// PublicKey is this node's WireGuard public key. Zero until the node
	// has reported one, which is a normal transient state for a node
	// that enrolled but has not yet come up on the mesh, not an error:
	// Plan skips such a node as a peer rather than failing the whole
	// plan, so one not-yet-ready node cannot stop the rest of the fleet
	// from meshing.
	PublicKey Key

	// Endpoint is "host:port", the UDP address other nodes dial to reach
	// this one. May be empty for a node behind NAT that has never been
	// observed dialing out: WireGuard learns a peer's endpoint from its
	// first inbound handshake, so a NATted node is reachable as soon as
	// it initiates, which (ADR 003's reverse-dial design) it always
	// does. An empty endpoint therefore means "wait for it to talk to
	// us," not "unreachable."
	Endpoint string

	// Address is this node's mesh IP, assigned by AllocateAddress and
	// persisted by the control plane. Invalid (the zero netip.Addr)
	// until assigned, same "not ready yet" meaning as a zero PublicKey.
	Address netip.Addr
}

NodeInfo is everything the control plane knows about one node for mesh purposes: enough to make every *other* node able to reach it.

PublicKey and Endpoint are reported by the node itself (only the node can know its own key, and only the node's own view plus the control plane's observation of where it dialed from can establish a usable endpoint). Address is assigned by the control plane, because address allocation is the one part that cannot be decided locally without collisions. That split is the whole reason distribution is a two-way exchange rather than a one-way push; see Coordinator.

func AllocateAddresses

func AllocateAddresses(nodes []NodeInfo, cidr netip.Prefix) ([]NodeInfo, error)

AllocateAddresses assigns an address to every node in nodes that does not already have a valid one, leaving existing assignments untouched, and returns the updated inventory. Existing assignments are never revoked or renumbered: a node's mesh address appearing in another node's AllowedIPs and in live connection state means renumbering it is a disconnection, so allocation is strictly additive.

Returns a new slice rather than mutating in place so a caller that fails to persist the result has not already half-applied it to the inventory it is still holding.

func (NodeInfo) Ready

func (n NodeInfo) Ready() bool

Ready reports whether this node has everything Plan needs to make it a reachable peer for other nodes.

type NodeResult

type NodeResult struct {
	NodeID   string
	Config   DeviceConfig
	Identity NodeIdentity

	// Err is non-nil when this node could not be reached or refused the
	// config. Recorded per node rather than aborting the pass: this is
	// the same principle dynamicSource already applies when a node's
	// transport is unavailable ("one broken resource must never block the
	// rest"), and it matters more here, because the node most likely to
	// be unreachable is the one that just went down, and that is exactly
	// when the *other* nodes most need an updated peer list.
	Err error
}

NodeResult is the outcome of distributing to one node.

type PeerConfig

type PeerConfig struct {
	// NodeID is not part of WireGuard's own model at all. It is carried
	// so every log line about a peer can name the resource by the ID the
	// rest of this codebase uses, matching the convention that every log
	// line describing a resource includes its ID, rather than by a
	// base64 key that matches nothing else in the database.
	NodeID    string
	PublicKey Key
	Endpoint  string

	// AllowedIPs is what traffic this peer is permitted to send and
	// what destinations route to it. For a full mesh of single-address
	// nodes this is exactly one /32 per peer, never 0.0.0.0/0: a
	// default route through a peer would make every node a potential
	// exit node for every other, which is a materially different
	// security posture than "these machines can reach each other" and
	// not something ADR 006 asks for.
	AllowedIPs []netip.Prefix

	PersistentKeepalive time.Duration
}

PeerConfig is one entry in a node's WireGuard device configuration: another node, as seen from this one.

type PeerStatus

type PeerStatus struct {
	// NodeID is carried through from the PeerConfig that created this
	// peer so a caller can log a peer by the ID everything else in this
	// codebase logs resources by, rather than by a base64 public key.
	// WireGuard itself has no concept of a node ID, so this is empty for
	// any peer the device has that no current DeviceConfig named.
	NodeID    string
	PublicKey Key
	Endpoint  string

	// LastHandshake is zero when no handshake has ever completed with
	// this peer, which is exactly the "configured but unreachable" case:
	// a peer entry can exist indefinitely without the other end ever
	// answering.
	LastHandshake time.Time
	TransferRx    int64
	TransferTx    int64
}

PeerStatus is one peer as the local device currently sees it.

func UnhealthyPeers

func UnhealthyPeers(st Status, now time.Time) []PeerStatus

UnhealthyPeers returns the peers in st that have not handshaken recently enough, for a caller building a node-health view (TASKS.md 3.7) or logging why a cross-node call is failing.

func (PeerStatus) Healthy

func (p PeerStatus) Healthy(now time.Time, staleAfter time.Duration) bool

Healthy reports whether this peer has completed a handshake recently enough to be considered reachable. WireGuard rekeys roughly every two minutes on an active session, so a handshake older than staleAfter means traffic has not flowed; a zero LastHandshake means it never has.

type Placement

type Placement struct {
	// Service is the name the service is addressed by. This is the
	// user-facing name from app.yaml, which is what makes it usable in a
	// connection string.
	Service string

	// NodeID is the node the service currently runs on. The empty string
	// means the control plane's own node, exactly as 0009's schema
	// defines it, which is why BuildRecords needs to be told which node
	// ID that is rather than being able to infer it.
	NodeID string
}

Placement is one service's current node assignment, as desired_services.node_id / desired_databases.node_id record it (migrations/0009_node_placement.sql).

type PlanOptions

type PlanOptions struct {
	// MeshCIDR is the private range every node's address comes from. It
	// determines both the prefix length on DeviceConfig.Address (so the
	// host routes the whole mesh out the WireGuard interface) and the
	// bound ValidateInventory checks every node address against.
	MeshCIDR netip.Prefix

	// ListenPort is the UDP port each node's device listens on. Zero
	// means DefaultListenPort.
	ListenPort int

	// Keepalive is the persistent-keepalive interval set on every peer.
	// Zero means DefaultKeepalive; explicitly disabling it requires a
	// negative value, so "I did not set this field" and "I want no
	// keepalive" are distinguishable. See DefaultKeepalive for why the
	// default is non-zero.
	Keepalive time.Duration
}

PlanOptions are the fleet-wide settings a plan is computed against, as opposed to the per-node facts in the inventory.

type Probe

type Probe interface {
	// OS is the GOOS this node is running. WireGuard's kernel module is
	// a Linux thing; nothing else has one to detect.
	OS() string

	// KernelModuleLoaded reports whether the wireguard kernel module is
	// present and loaded. On Linux this is the existence of
	// /sys/module/wireguard.
	KernelModuleLoaded() bool

	// Privileged reports whether this process can create and configure a
	// network interface at all. A loaded kernel module is useless to an
	// unprivileged process, which is exactly the "restricted kernels,
	// some container-based VPS hosts" case ADR 006 cites as the reason
	// the userspace path exists.
	Privileged() bool
}

Probe supplies the evidence Detect reasons over. The real implementation (SystemProbe) reads the local filesystem; tests supply values directly.

An interface rather than three function fields because the three signals are only meaningful together, and a caller that supplied two of three would get a nonsense answer from a zero-valued third.

type Record

type Record struct {
	// Name is the fully qualified name, lowercase, with no trailing dot.
	Name    string
	Address netip.Addr

	// Service and NodeID are carried for logging and for the API surface
	// that will show an operator where a name currently points. They are
	// not part of resolution.
	Service string
	NodeID  string
}

Record is one resolvable name.

type RecordSet

type RecordSet struct {
	Zone       string
	Records    []Record
	Unresolved []Unresolved
}

RecordSet is one complete, level-triggered snapshot of what every internal name resolves to.

func BuildRecords

func BuildRecords(zone, selfID string, placements []Placement, nodes []NodeInfo) (RecordSet, error)

BuildRecords turns placements plus the node inventory into the complete record set for zone.

selfID is the control plane's own node ID, needed to resolve the empty NodeID that 0009's schema uses for "local". Passing it explicitly rather than having this function guess (say, "the first node in the list") keeps the one piece of ambient context this needs visible at the call site.

Two name families are produced:

  • <service>.<zone> for every placement. This is the one that matters: it is what goes in a connection string.
  • <node-name>.node.<zone> for every ready node. Under a .node label so a node called "db" can never collide with a service called "db", which is otherwise a genuinely likely name for both.

A duplicate service name is an error rather than a skipped record: two services answering to one name means whichever record won would silently take the other's traffic, and unlike an unresolved placement there is no correct partial answer to give.

type Resolver

type Resolver struct {
	// contains filtered or unexported fields
}

Resolver answers name lookups from a record set. Safe for concurrent use: SetRecords swaps the whole table under a write lock while lookups take a read lock, so a lookup never observes a half-updated table (an in-place update would let a service briefly resolve to nothing during an ordinary reconcile pass).

func NewResolver

func NewResolver(zone string) *Resolver

NewResolver builds an empty Resolver for zone. An empty Resolver answers every lookup with "not found", which is the correct behavior before the first reconcile pass has run: NXDOMAIN is a clear failure, where a stale or guessed answer would be a silent misroute.

func (*Resolver) Authoritative

func (r *Resolver) Authoritative(name string) bool

Authoritative reports whether name falls inside this resolver's zone. The DNS server uses it to decide between answering NXDOMAIN (a name in our zone that does not exist) and REFUSED (a name we have no business answering for at all), a distinction that matters to a resolver deciding whether to try another upstream.

func (*Resolver) Lookup

func (r *Resolver) Lookup(name string) (netip.Addr, bool)

Lookup resolves name to a mesh address. Accepts the name with or without a trailing dot and in any case, since a DNS query arrives fully qualified with a trailing dot while a human types neither.

func (*Resolver) Records

func (r *Resolver) Records() []Record

Records returns a copy of the current table, for the status API and for tests. A copy, not the live slice: handing out the internal slice would let a caller mutate resolution state without the lock.

func (*Resolver) SetRecords

func (r *Resolver) SetRecords(set RecordSet)

SetRecords replaces the entire table. Level-triggered by construction: there is no AddRecord/RemoveRecord pair, for the same reason Mesh.Apply has no delta form.

func (*Resolver) Zone

func (r *Resolver) Zone() string

Zone reports the zone this resolver is authoritative for.

type Status

type Status struct {
	Backend    Backend
	Interface  string
	PublicKey  Key
	ListenPort int
	Address    netip.Prefix
	Peers      []PeerStatus
}

Status is one node's observed mesh state, the "observed" half of the reconcile pair whose "desired" half is DeviceConfig.

func ParseUAPIStatus

func ParseUAPIStatus(raw string, nodeIDs map[Key]string) (Status, error)

ParseUAPIStatus parses a UAPI "get" response into a Status.

The peer entries in a UAPI response are positional, not nested: a public_key= line begins a new peer and every subsequent key belongs to it until the next public_key=. Getting that wrong attributes one peer's handshake time to another, which is why this is a real parser with a test rather than a regexp over the blob.

nodeIDs maps public keys back to node IDs so the returned PeerStatus values can be logged by node ID, matching this codebase's structured logging convention of every resource carrying its ID. WireGuard has no idea what a node ID is, so a peer whose key is not in the map gets an empty NodeID, which is itself informative: it is a peer on the device that no current plan accounts for.

type SystemProbe

type SystemProbe struct{}

SystemProbe is the real Probe, reading this machine's own state.

func (SystemProbe) KernelModuleLoaded

func (SystemProbe) KernelModuleLoaded() bool

KernelModuleLoaded reports whether /sys/module/wireguard exists, which is how the kernel exposes a loaded module.

Deliberately does not attempt to load the module (no modprobe, no shelling out, matching this codebase's standing "never shell out" rule, which applies here for the same reason it applies to Docker). Loading a kernel module is a host-configuration decision an operator makes, not something a deploy platform should do behind their back on a machine they may not fully control.

func (SystemProbe) OS

func (SystemProbe) OS() string

OS reports the compiled-in GOOS.

func (SystemProbe) Privileged

func (SystemProbe) Privileged() bool

Privileged reports whether this process is likely able to create a network interface.

Euid 0 is the check, with a deliberate caveat: on Linux, CAP_NET_ADMIN without full root is also sufficient, and this returns false for that case. That is a conservative wrong answer (a capable process is told it is not), chosen over the alternative because reading capabilities portably means parsing /proc/self/status and getting the bit position right, and being wrong in the other direction means the mesh reports itself up and then fails at interface creation. Detect's Reason string names privileges explicitly so an operator running with capabilities rather than root gets a message they can act on rather than silence.

type Unresolved

type Unresolved struct {
	Service string
	NodeID  string
	Reason  string
}

Unresolved is one placement that could not be turned into a record, and why.

Returned rather than silently dropped because a missing record is not a missing row in a list, it is a connection string that will fail at runtime with NXDOMAIN, and the operator needs to be able to see that before a container does. This mirrors dynamicSource's own "log and skip the broken resource, do not fail the whole pass" handling of an unreachable node (cmd/levelrail/main.go), one layer up: the pass still produces every record it can.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL