api

Overview ¶

Package api pkg/dmsg/discovery/api/api.go c1-net-dmsg

Package api pkg/dmsg/discovery/api/charts.go c4-net-discovery

Package api pkg/dmsg/discovery/api/cxo_publisher.go c1-net-dmsg

CXO publisher for dmsg-discovery's "clients by server" view.

The hypervisor's network visualizer needs to know "which clients are currently delegated to each dmsg server" — the same view AllClientsByServer / ClientsByServer expose over HTTP. This publisher mirrors that as a TreeStore feed.

Wire shape: ONE batched leaf per server ¶

Each dmsg server gets exactly one leaf carrying that server's whole delegated-client set:

clients-by-server/<server-pk>        // FrameGzip(v1, JSON []disc.Entry)

Older builds published one leaf PER (server, client) PAIR at clients-by-server/<server>/<client>/entry — O(#pairs), thousands of tiny objects in one Root. A large deployment's Root then could not finish filling over a subscriber's short-lived delivering dmsg conn (only the top slice landed). Batching to one leaf per server drops the object count to O(#servers) (~tens–120), which is what lets the Root fill complete. The only consumers (the visor dmsg-entry lookup resolver and tpviz's network visualizer) want server-grouped data, so per-server batching loses no used addressability.

Encoding is JSON+gzip, not fixed-layout binary: a client's disc.Entry carries a signature and a variable-length delegated-server list, so it is the "awkward for binary" case — unlike telemetrywire's fixed 53-byte rows. The per-server array is JSON-marshaled (entries sorted by client PK so unchanged content re-encodes to identical bytes → a wire no-op on CXO's content-addressed store), then framed with a leading version byte and gzipped (cxoutils.FrameGzip). The version byte gates the format: a bumped version is rejected cleanly by an old reader rather than misparsed.

Interop / deploy order ¶

Deploy is SERVICE-FIRST. A not-yet-upgraded reader walking the feed looks for the OLD .../<client>/entry leaves, finds none under the new batched shape, reports an empty snapshot, and falls back to HTTP — safe degradation, never a wrong answer. Upgraded readers prefer the batched leaf and still parse an OLD per-item leaf as a fallback, so either publisher shape resolves.

A deregistration is the ABSENCE of a client from its server's batched leaf — there is no tombstone leaf class (the old per-(server,client) tombstones were write-only across the codebase and leaked RAM in the publisher's CXDS; observed 327 MB → 921 MB over 12h). Re-encoding a server's single leaf on a membership change is far cheaper than the old whole-tree churn: the tree is now O(#servers) leaves, so each publish clones + encodes a tiny tree.

Server entries are ignored by this publisher — a server PK is the path-prefix bucket, not a member of the view.

HTTP-path decoupling: the HTTP register / deregister handlers call PublishSetEntry / PublishDelEntry inline. Internally those route every mutation through a single buffered-channel worker (mirrors SD's pattern) so the HTTP goroutine never blocks on the treestore.Publisher mutex — which under load is contended by subscriber I/O and can stall register throughput long enough to time out visor heartbeats. The per-server client state map is owned by that single worker goroutine, so its mutations need no lock. Overflow drops are counted; CXO data quality degrades gracefully, HTTP stays fast.

Package api pkg/dmsg/discovery/api/cxo_servers.go

Registered dmsg servers on the clients-by-server feed, so visors keep their server cache from the snapshot they already hold instead of asking for all_servers over HTTP every few minutes.

Each server is one leaf at clients-by-server/servers/<server-pk>: its signed discovery entry, version-framed and gzipped. The path sits under the feed's prefix because subscribers fetch only that prefix. Older readers ignore it: they take clients-by-server/<pk> as a batch and a deeper path only when it ends in /entry.

A server re-signs its entry on every refresh and its free session count moves with every client, so a leaf is rewritten only when something a client dials by changes: its addresses, type, version or protocol. The signed body stays valid; its timestamp and session count are as of that change.

Package api pkg/dmsg/discovery/api/error_handler.go c1-net-dmsg

Package api pkg/dmsg/discovery/api/ratelimit.go c1-net-dmsg

Per-client request rate limiting for the discovery HTTP API. This is the server-side counterpart to the client-side serve-loop floor: it protects the discovery service even from clients that DON'T have that fix, by capping how many requests any single dmsg client can issue per second. The limits are set far above any legitimate client's discovery usage; the only thing they stop is a misbehaving client busy-looping its discover/dial loop and pinning the service at 100% CPU with tens of thousands of requests a second.

Keyed by the dmsg public key — the noise-authenticated transport identity in RemoteAddr — NOT a request header, so a client can't evade the limit by opening fresh streams (the PK is constant) or by spoofing X-Real-IP. The middleware is registered BEFORE chi's RealIP middleware for exactly that reason. Fail-open: anything that isn't a dmsg PK (clearnet IPs, which may sit behind a shared proxy) passes through untouched.

Package api pkg/dmsg/discovery/api/registration_ingest.go c1-net-dmsg

Registration-over-CXO ingest: the server side of the fan-in path where visors publish their own signed discovery entry as a CXO feed instead of re-PUTting it over HTTP on a timer (each PUT a fresh Noise + PQ handshake — the load that dominates dmsg-discovery CPU). The CXO aggregator (pkg/dmsg/discovery/regcxo) calls IngestEntryFromCXO for each entry it replicates.

Package api pkg/dmsg/discovery/api/server_health.go c2-dmsg-core