Documentation
¶
Overview ¶
Package ipalloc holds the relay-side, in-process connection address allocators for the vpc.apoxy.dev relay (APO-825 §2.8, where a slot is called a "block").
A relay serving a network holds one or more slots in it. A slot is a 16-bit endpoint identifier — the same identifier the infrastructure endpoint allocator hands out — and it owns the addresses formed by varying byte 11 of the overlay ULA:
fd61:706f:7879:nnnn:nnss:sscc::/96
^^^^^^^^ ^^^^ ^^
network | connection index, 1-255
slot (endpoint id)
The slot sits ABOVE the connection index, so a slot is a single /88 and every address it owns shares 88 bits. Both properties are load-bearing and neither survives the two fields being swapped:
- A slot is advertisable as one route. Relays do not federate, so an agent must route each relay's addresses over that relay's own session; with the connection index above the slot a slot is a strided set, not a prefix, and cannot be expressed as a route at all.
- Source address selection lands on the right relay. An agent connected to several relays carries one address per relay on one device, so the kernel picks the source by longest matching prefix (RFC 6724 rule 8). That has to discriminate on the slot; with the connection index above it, it discriminates on a per-connection counter instead and sources traffic from an address the destination's relay never leased.
Byte 11 == 0 is the slot's own endpoint /96 — the address the infrastructure allocator assigned — and is never handed to a connection.
Slot ids below MinSlotID are reserved: see its doc comment.
Conflict-freedom is structural, not lock-based: a slot is held by exactly one relay, so within its own slots a relay is the sole allocator and needs no apiserver round-trip on the connect path. The single coordination point is the SlotLeaser, and the property it must provide is narrow — two relays serving one network never hold the same slot. Slot count is the scarce dimension (65536 per network, bounding relays times held slots); connection count is elastic, because a relay that fills a slot simply leases another.
Callers must pass the infra-assigned NetworkID; its uniqueness (and thus the disjointness of every network's address space) is the network provisioner's contract (§2.8), not something this package establishes or checks.
Index ¶
Constants ¶
const ConnsPerSlot = 255
ConnsPerSlot is how many connections one slot carries: byte 11 takes the values 1-255, since 0 is the slot's own endpoint /96.
const MinSlotID = 1 << 8
MinSlotID is the lowest slot id a leaser may hand out. Ids below it have a zero high byte, which puts a zero in ULA byte 9 — and byte 9 is where the pre-slot addressing scheme carried its own fields, so such a connection address is indistinguishable from an address minted under the old scheme. Reserving the low 256 ids keeps the two disjoint for as long as both are on the wire; it costs 0.4% of the slot space. Drop the reservation once the old scheme is gone.
Variables ¶
var ( // ErrNoSlots is returned when a network's slots are all leased. ErrNoSlots = errors.New("no available slots in network") // ErrSlotExhausted is returned when a slot's connection addresses are all // in use; the caller should lease another slot. ErrSlotExhausted = errors.New("connection slot exhausted") )
Functions ¶
func ConnPrefix ¶ added in v0.22.0
ConnPrefix returns the /96 for connection index i within a slot. Index 0 is the slot's endpoint address, so connections are numbered from 1.
func EndpointPrefix ¶ added in v0.22.0
EndpointPrefix returns connection index 0 of the slot, which is reserved and never handed to a connection.
It is NOT the address the infrastructure allocator assigned the backing Endpoint: infra places an endpoint identifier in ULA bytes 10-11, while a slot places it in bytes 9-10. The identifier is borrowed as an opaque uniqueness token — that is the only property the leaser has to provide — and the address space it names here is derived independently.
func SlotPrefix ¶ added in v0.22.0
SlotPrefix returns the slot's /88 — the whole range it owns, its endpoint address and all 255 connection addresses. This is the unit a relay advertises to its agents: one route per slot it holds, so an agent routes each relay's addresses over that relay's own session and no more.
Types ¶
type ConnAllocator ¶
type ConnAllocator struct {
// contains filtered or unexported fields
}
ConnAllocator sub-allocates per-connection addresses from a single leased slot, entirely in-process. It is the sole allocator within its slot, so no cross-process coordination is needed; a mutex guards the two bitmaps against concurrent connect/disconnect.
Each connection gets a /96 (IPv6, always) and a /32 (IPv4, best-effort). The v4 side is the weaker of the two by design: v4 is egress-only, so v4 exhaustion — or a slot leased without a /24 — degrades a connection to v6-only rather than failing it. The /24 is the slot's own (Slot.V4); its lifetime is the lease's, so there is nothing to return here when the slot goes away.
func NewConnAllocator ¶
func NewConnAllocator(s Slot) *ConnAllocator
NewConnAllocator returns an allocator over a leased slot.
func (*ConnAllocator) Allocate ¶
Allocate returns a connection's /96 and, best-effort, its /32. A zero-value (invalid) v4 prefix means this slot has no v4 space left (or never had any); the caller should run the connection v6-only. ErrSlotExhausted means the v6 pool is full; the caller should lease another slot (see Full).
func (*ConnAllocator) Full ¶
func (a *ConnAllocator) Full() bool
Full reports whether the v6 pool is exhausted, i.e. the caller must lease another slot to accept more connections.
func (*ConnAllocator) Release ¶
func (a *ConnAllocator) Release(v6, v4 netip.Prefix)
Release returns a connection's addresses to the slot's pools. A zero-value prefix for either family is ignored (e.g. a v6-only connection, or a connection whose /32 was dropped because it could not be programmed).
type LocalSlotLeaser ¶ added in v0.22.0
type LocalSlotLeaser struct {
// contains filtered or unexported fields
}
LocalSlotLeaser is the OSS/single-tenant SlotLeaser. It tracks each network's slot ids entirely in process memory — there is no infra tier, so leases have no TTL (Renew is a no-op) and nothing survives a restart. The API is identical to the cloud infra-backed implementation so the relay wiring is the same in both modes.
It is correct only while a single process allocates for a network. Two processes each start from an empty bitmap and both hand out the lowest free id, so their connections collide on identical /96s. Any deployment running more than one relay per network must use an infra-backed leaser.
func NewLocalSlotLeaser ¶ added in v0.22.0
func NewLocalSlotLeaser() *LocalSlotLeaser
NewLocalSlotLeaser returns a LocalSlotLeaser.
type Slot ¶ added in v0.22.0
type Slot struct {
// Network is the infra-assigned 24-bit network identifier.
Network tunnet.NetworkID
// ID is the 16-bit endpoint identifier, unique within Network.
ID tunnet.EndpointID
// V4 is the /24 backing the slot's best-effort IPv4, assigned by the
// leaser. Its uniqueness domain is wider than the slot id's: the id is
// unique per network, but an agent connected to several relays demuxes v4
// by bare source address, and a relay routes every network through one
// route table — so the /24 must not repeat across relays or networks. The
// infra leaser allocates it globally (from 240.0.0.0/4); the local leaser
// from its process-wide pool, which is the whole world in OSS. A zero
// value runs the slot v6-only (§2.4).
V4 netip.Prefix
}
Slot is an endpoint identifier held by exactly one relay within one network, together with the 255 connection addresses hanging off it.
func SlotOf ¶ added in v0.22.0
SlotOf decomposes an overlay address into the slot that owns it and the connection index within that slot. ok is false when the address is an infrastructure endpoint address rather than a connection one (byte 11 == 0), or when it predates slot addressing (slot id below MinSlotID) — together those are the test that keeps relay space and endpoint space apart.
type SlotLeaser ¶ added in v0.22.0
type SlotLeaser interface {
// Lease reserves and returns a slot nobody else holds in the network, or
// ErrNoSlots if the network's identifier space is exhausted.
Lease(ctx context.Context, network tunnet.NetworkID) (Slot, error)
// Renew extends a held lease. OSS has no lease TTL, so this is a no-op
// there; cloud refreshes the backing Endpoint's heartbeat.
Renew(ctx context.Context, s Slot) error
// Release returns a slot to the pool.
Release(ctx context.Context, s Slot) error
}
SlotLeaser hands out endpoint slots within a network. A relay holds at least one lease per (relay × network) and takes more as connections fill them, renewed on the relay's heartbeat cadence and released at drain (§5). OSS satisfies this from the local process's own view of the network (LocalSlotLeaser); cloud satisfies it from infra-apiz Endpoints, whose allocator is already the single writer of endpoint identifiers per shard.
type V4SlicePool ¶ added in v0.22.0
type V4SlicePool struct {
// contains filtered or unexported fields
}
V4SlicePool hands out the /24s of 100.64.0.0/10, one per slot. It backs LocalSlotLeaser, where one process is the whole deployment; cloud slots get their /24 from the infra endpoint allocator instead, which allocates globally so the same /32 can never reach one agent from two relays.
func NewV4SlicePool ¶ added in v0.22.0
func NewV4SlicePool() *V4SlicePool
NewV4SlicePool returns an empty pool over 100.64.0.0/10.
Source Files