api

Overview ¶

Package api pkg/deployment/sd/api/api.go c4-net-discovery

Package api pkg/deployment/sd/api/charts.go

Package api pkg/deployment/sd/api/cxo_publisher.go c4-net-discovery

CXO publisher for service-discovery's services tree.

Subscribers (the hypervisor's network visualizer + tab-specific consumers) read the live set of registered services from a single TreeStore feed instead of polling /api/services?type=... over HTTP.

Wire shape: ONE batched leaf per service type ¶

Each service type gets exactly one leaf carrying that type's whole live service set:

services/<type>/all        // FrameGzip(v1, JSON []servicedisc.Service)

Older builds published one leaf PER service at services/<type>/<pk>/entry (+ an optional /tombstone) — O(#services), one tiny object per registered service. A large deployment's Root then could not finish filling over a subscriber's short-lived delivering dmsg conn. Sharding by type collapses the object count to O(#types) (~10). The batch leaf lives UNDER the type dir (…/all, not a bare services/<type>) so every existing consumer prefix — services/, services/visor/, services/<type>/ — still walks straight onto it; a service PK is 66 hex chars and never collides with the "all" segment.

Encoding is JSON+gzip, not fixed-layout binary: servicedisc.Service carries several optional, variable-length nested structs (Geo, VPNInfo, CoinInfo, LocalIPs), so it is the "awkward for binary" case — unlike telemetrywire's fixed rows. Services are sorted by PK so an unchanged set re-encodes to identical bytes (a CXO content-addressed wire no-op), then version-framed + gzipped (cxoutils.FrameGzip). The version byte gates the format: a bumped version is rejected cleanly by an old reader rather than misparsed.

Interop / deploy order ¶

Deploy is SERVICE-FIRST. A not-yet-upgraded reader walking a type prefix looks for the OLD .../<pk>/entry leaves, finds only the batched …/all leaf, reports an empty snapshot for that type, and falls back to HTTP — safe degradation. Upgraded readers prefer the batched leaf and still parse OLD per-service leaves, so either publisher shape resolves.

A deregistration / expiry is the ABSENCE of a service from its type's batched leaf — there is no tombstone leaf class anymore (the old per-service tombstones were write-only across the codebase).

Heartbeat short-circuit ¶

SD's register/heartbeat rate is high. The worker holds the live per-type service set and, on a re-register, compares the new marshaled Service against the stored one; when the materially-visible content is unchanged it skips the re-encode entirely, so a steady heartbeat stream does not churn the tree.

HTTP-path decoupling: PutEntry / DelEntry are non-blocking. The HTTP register / deregister handlers call them inline, but the underlying treestore.Publisher mutex is contended by subscriber I/O — under load (many concurrent subscribers) a synchronous Put can block the HTTP goroutine long enough to pile up thousands of pending registrations and stall the entire visor refresh loop. Instead we route every publish through a single buffered-channel worker and drop on overflow. The per-type service state map is owned by that single worker goroutine, so its mutations need no lock. CXO data quality (last-write-wins, up to publishQueueDepth events in flight) degrades gracefully; the HTTP path stays fast.

Package api pkg/deployment/sd/api/service_ingest.go c4-net-discovery

SD-registration-over-CXO ingest: the server side of the fan-in path where visors publish their live service-entry set as a CXO feed instead of re-POSTing every entry over a fresh dmsg stream every 90s (each with a full Noise handshake — the secp256k1 handshakeResponder that dominates discovery -service CPU). The CXO aggregator (pkg/deployment/sd/regcxo) calls IngestServiceFromCXO once per entry in each batch it replicates.

This is purely ADDITIVE and dual-write: the HTTP POST /api/services path remains authoritative and keeps writing the same store on the same 90s schedule. The CXO ingest therefore never clobbers a fresh HTTP register:

• When a record already exists (the common case — HTTP registered first), the CXO ingest is a keepalive: it re-writes the STORED entry to refresh its TTL, off a warm CXO connection instead of a fresh handshake. It writes back exactly what is there, so it can never overwrite a newer HTTP registration (mirrors the AR's bind keepalive, which likewise refreshes off the stored record rather than the incoming one).
• When no record exists yet, only the non-visor types take a fresh insert. A type=visor registration is gated on ipIsPublic(entry, observed-remote- addr), and the CXO path has no observed remote address to check against — so a fresh visor entry is left to HTTP, exactly as a fresh SUDPH bind is left to the AR's UDP path.

Deregistration is ABSENCE, not a tombstone: a service the visor stops publishing simply stops being refreshed here and expires on the store's existing entry TTL (Config.EntryTimeout). There is deliberately no CXO delete path.