Documentation
¶
Overview ¶
Package arfeed pkg/deployment/ar/arfeed/arfeed.go c4-net-discovery
Wire shape of the address-resolver's CXO bindings feed — the tree layout, the record type and the codec, shared by the publisher (pkg/deployment/ar/api) and any reader. It depends on nothing but the VisorData type and the CXO frame helpers, so a visor can import it without dragging in the AR service's HTTP stack.
Why this tree is shaped the way it is ¶
The feed exists to be read with CXO's Preview (pkg/cxo/node.Conn.Preview): fetch the publisher's latest Root over an already-open connection, walk ONLY the branches the lookup needs, and hold nothing afterwards. That pays off only if the tree is keyed by the thing being looked up. The dmsg-discovery clients-by-server feed is the counter-example — keyed by SERVER, so resolving one client means pulling that server's whole batch.
Three properties of treestore and registry.Refs decide the layout, and each one was measured rather than assumed:
- A level's children are individual objects, one network fetch each. So SCANNING a level of N children costs N round-trips.
- treestore encodes each level's children in sorted Name order (publisher.go sortedNames -> encodeNode), so a reader can address a child by INDEX and skip the scan entirely.
- Refs is a Merkle tree of degree 16. Addressing element i inside a level of more than 16 children walks the branch nodes preceding it, so a 256-wide level costs between 1 and ~16 EXTRA fetches depending on where the key lands. A level of at most 16 children is a single Refs node: one fetch, whatever the index.
Hence two levels of one hex character each, 16 wide apiece:
<pk-hex[2]>/<pk-hex[3]> // FrameGzip(v1, JSON map pk-hex -> PeerBindings)
256 buckets, every one of them always published (empty ones included) so both levels stay dense and index-addressable. A lookup is then a FIXED handful of object fetches — schema registry, root TreeNode, root Refs node, level-1 entry, level-2 TreeNode, its Refs node, the bucket entry — at any network size and for any key. Widening to 4096 buckets later is one more level and three more fetches.
The offset matters ¶
The bucket is named from hex[2:4], not hex[0:2]. A skywire public key is a COMPRESSED secp256k1 point, so its leading byte is always 0x02 or 0x03: bucketing on the first two hex characters puts the entire network into two buckets and silently turns each "one indexed fetch" into a whole-network download. Measured on the 1711 peers the production AR held on 2026-09-06: hex[0:2] gives 2 buckets of 886 and 825; hex[2:4] gives 255 occupied buckets with a maximum occupancy of 17. The offset skips the parity byte and starts at the first byte of the X coordinate, which is uniform.
Bucket occupancy is the tuning knob. At ~1.7k bound peers a bucket holds around seven records, a few hundred bytes gzipped, so a lookup transfers the target plus a handful of incidental neighbors. An order of magnitude more peers wants a third level — a wire-format change, which is what the leading version byte is for: an old reader rejects the frame cleanly and falls back to HTTP rather than misparsing it.
Package arfeed pkg/deployment/ar/arfeed/reach.go c4-net-discovery
Reachability: which visors can actually be transported, per transport type.
A binding at the address resolver says a visor LISTENS for a type. It does not say anyone can get through to it: a public visor can accept stcpr while its router drops inbound UDP, and a sudph binding outlives the UDP control connection the AR needs to make both sides punch at once. Autoconnect needs the difference, and it cannot learn it by waiting for a peer to succeed first — nothing would be tried.
The chicken-and-egg is broken per type by whatever can make the first attempt without a peer:
- stcpr, squicr, swtr — a listener behind a router either admits inbound traffic or it does not, whoever sends it. The AR probes the bound address itself: a TCP connect for stcpr, and for the QUIC-based types one packet with an unknown QUIC version, which a QUIC listener always answers with a version negotiation. The result is the AR's, recorded as Open / Closed.
- sudph — success is pairwise: it depends on the NAT behavior of BOTH sides, and it needs the receiver's live UDP control connection at the AR (Live) for the simultaneous open. Each visor measures its own NAT class with STUN (NAT); a pair is tried only when the two classes can punch.
- webrtc — ICE, also pairwise; the same NAT-class rule applies.
Confirmations from real use sit on top: each visor states when it last ACCEPTED an inbound transport of each type (Inbound). A confirmation ranks a peer first; it never gates one, so nothing depends on a peer having succeeded before.
Index ¶
- Constants
- Variables
- func BucketPath(pk cipher.PubKey) string
- func BucketPathAt(i int) string
- func DecodeBucket(blob []byte) (map[string]*PeerBindings, error)
- func DecodeReachBucket(blob []byte) (map[cipher.PubKey]*Reach, error)
- func EncodeBucket(peers map[cipher.PubKey]*PeerBindings) ([]byte, error)
- func EncodeReachBucket(peers map[cipher.PubKey]*Reach) ([]byte, error)
- func Indices(pk cipher.PubKey) []int
- func PunchCompatible(a, b string) bool
- func ReachBucket(pk cipher.PubKey) int
- func ReachPath(bucket int) string
- func Segments(pk cipher.PubKey) []string
- type PeerBindings
- type Reach
- type ReachDecl
- type Verdict
Constants ¶
const ( TypeSTCPR = "stcpr" TypeSUDPH = "sudph" TypeQUIC = "squicr" TypeWT = "swtr" TypeWEBRTC = "webrtc" )
Transport type names as they appear in a Reach record. They are the canonical wire strings of the types, so a record can be read without the transport package.
const ( NATOpen = "open" // no NAT: public address on the interface NATFullCone = "full-cone" // any host may send once a mapping exists NATRestricted = "restricted" // only hosts the visor sent to (any port) NATPortRestricted = "port-restricted" // only the exact host:port the visor sent to NATSymmetric = "symmetric" // a new mapping per destination NATUDPFirewall = "udp-firewall" // public address, but filtered like port-restricted NATBlocked = "blocked" // no UDP at all )
NAT classes, as measured by the visor's own STUN probe.
const AcceptsNone = "none"
AcceptsNone is the whole Accepts list of a visor taking no new automatic transports. Every type is then Unreachable to autoconnect, also on visors that predate it, so the visor also leaves Inbound out.
const BucketCount = 1 << (4 * Levels)
BucketCount is the number of buckets the publisher always materializes.
const Fanout = 16
Fanout is the number of children at each level — one hex character. It is deliberately not larger: registry.Refs has degree 16, so a level of at most 16 children is a single Refs node and an indexed fetch there costs one round-trip regardless of the index.
const InboundGranularity = time.Hour
InboundGranularity is the resolution Inbound times are published at. Coarse on purpose: a visor accepting transports all day must not rewrite its record every time it does.
const InboundRecent = 24 * time.Hour
InboundRecent is how long an accepted inbound transport counts as a confirmation.
const KeyOffset = 2
KeyOffset is the index of the first hex character of a peer's public key used to name its path. It is one byte in, not zero, because a compressed secp256k1 key's leading byte is always 0x02 or 0x03 — see the package doc.
const Levels = 2
Levels is the depth of the bucket tree. Each level is named by one hex character of the peer's public key.
const ReachBuckets = 16
ReachBuckets is the number of reach leaves: one per hex digit.
const ReachPathPrefix = "reach/"
ReachPathPrefix is the tree prefix of the reach feed. One leaf per bucket, reach/<h>, where <h> is the key's first hex digit after the 02/03 prefix.
const ReachVersion = 1
ReachVersion is the frame version of a reach bucket leaf.
const Version = 1
Version is the wire-format version byte of a bucket leaf body. Bump it on any breaking change to the bucket encoding OR to the tree layout; readers reject other values rather than misparsing them.
Variables ¶
var ErrBadVersion = errors.New("arfeed: unsupported bucket frame version")
ErrBadVersion is returned by DecodeBucket for a frame whose version byte is not Version. Callers treat it as "I cannot read this", not as an error worth retrying.
Functions ¶
func BucketPath ¶
BucketPath returns the full tree path of pk's bucket, e.g. "a/3".
func BucketPathAt ¶
BucketPathAt returns the path of the i'th bucket in the dense enumeration the publisher materializes, for 0 <= i < BucketCount.
func DecodeBucket ¶
func DecodeBucket(blob []byte) (map[string]*PeerBindings, error)
DecodeBucket parses a bucket leaf body. An unknown version byte yields ErrBadVersion so a reader degrades to its fallback path instead of misparsing a newer wire shape.
func DecodeReachBucket ¶ added in v1.3.98
DecodeReachBucket decodes one reach bucket leaf.
func EncodeBucket ¶
func EncodeBucket(peers map[cipher.PubKey]*PeerBindings) ([]byte, error)
EncodeBucket serializes one bucket's peer records into a leaf body: a JSON object keyed by peer public key in sorted order (so unchanged content re-encodes to identical bytes), version-framed and gzipped.
func EncodeReachBucket ¶ added in v1.3.98
EncodeReachBucket encodes one reach bucket leaf. Keys are sorted so equal content encodes to equal bytes — the feed is content-addressed, and an unchanged bucket must cost nothing to republish.
func Indices ¶
Indices returns the per-level child POSITIONS of pk's bucket — what a reader hands to an indexed child fetch instead of scanning the level for a name. Valid only because the publisher keeps every level dense and treestore writes children in sorted name order. A malformed segment yields -1, which an indexed fetch treats as a miss.
func PunchCompatible ¶ added in v1.3.98
PunchCompatible reports whether two NAT classes can hole-punch to each other with a simultaneous open. An empty class is unknown and assumed compatible; blocked is compatible with nothing.
A symmetric NAT picks a new external port per destination, so its peer cannot know where to send. That is fatal only when the peer also filters by port: another symmetric NAT, or a port-restricted one (a UDP firewall filters the same way). Open, full-cone and address-restricted peers admit it.
func ReachBucket ¶ added in v1.3.98
ReachBucket returns the bucket index of a key: its first hex digit after the 02/03 compression prefix, which is uniformly distributed.
Types ¶
type PeerBindings ¶
type PeerBindings struct {
STCPR *addrresolver.VisorData `json:"stcpr,omitempty"`
SUDPH *addrresolver.VisorData `json:"sudph,omitempty"`
QUIC *addrresolver.VisorData `json:"squicr,omitempty"`
WT *addrresolver.VisorData `json:"swtr,omitempty"`
}
PeerBindings is one peer's address-resolver record: whichever of the four bindable transport types the AR currently holds for it. Fields are pointers so an absent binding is absent from the JSON rather than an empty object, and the field order is fixed by the struct so a re-encode of unchanged content produces identical bytes — a wire no-op on CXO's content-addressed store.
func (*PeerBindings) Empty ¶
func (b *PeerBindings) Empty() bool
Empty reports whether the record carries no binding at all, in which case the publisher drops the peer from its bucket.
func (*PeerBindings) Get ¶
func (b *PeerBindings) Get(t types.Type) *addrresolver.VisorData
Get returns the binding for one transport type, or nil.
func (*PeerBindings) Set ¶
func (b *PeerBindings) Set(t types.Type, v *addrresolver.VisorData) bool
Set installs the binding for one transport type; a nil value clears it. Reports whether the type was a bindable one.
type Reach ¶ added in v1.3.98
type Reach struct {
ReachDecl
// Open lists the types whose bound address answered the AR's probe.
Open []string `json:"open,omitempty"`
// Closed lists the types whose bound address did not.
Closed []string `json:"closed,omitempty"`
// Live is true while the visor holds a sudph UDP control connection at
// the AR — without it the AR cannot ask it to punch.
Live bool `json:"udp,omitempty"`
// Bound lists the types the visor has a binding for at the AR.
Bound []string `json:"bound,omitempty"`
}
Reach is one peer's record in the reach feed: the visor's own declaration plus what the AR itself observed.
func (*Reach) InboundWithin ¶ added in v1.3.98
InboundWithin reports whether the peer accepted an inbound transport of the type within InboundRecent of now.
type ReachDecl ¶ added in v1.3.98
type ReachDecl struct {
// NAT is the visor's STUN-measured NAT class; empty when not measured.
NAT string `json:"nat,omitempty"`
// Accepts lists the types the visor serves inbound.
Accepts []string `json:"acc,omitempty"`
// Inbound maps a type to the time (unix seconds, rounded down to
// InboundGranularity) the visor last accepted an inbound transport of it.
Inbound map[string]int64 `json:"in,omitempty"`
}
ReachDecl is what a visor states about itself, published as the "reach" leaf of its AR-bind feed.
type Verdict ¶ added in v1.3.98
type Verdict int
Verdict is how likely a transport of one type to a peer is to work. Ordered: a higher verdict is tried first.
const ( // Unreachable means the attempt is known to fail; it is not tried. Unreachable Verdict = iota // Untested means nothing is known either way — an old visor that states // nothing, or a probe that has not run yet. Tried within a small budget. Untested // Capable means everything measurable says it should work. Capable // Confirmed means it has worked: the AR's probe got through, or the peer // recently accepted an inbound transport of this type. Confirmed )
Source Files