Help us shape the future of Go! Take the 2026 Go Developer Survey

telemetrywire

package
v1.3.102 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 23, 2026 License: AGPL-3.0 Imports: 6 Imported by: 0

Documentation

Overview

Package telemetrywire pkg/telemetrywire/checkpoints.go c3-vis-core

Checkpoint shards carry each transport's stream digest checkpoints (see pkg/transport/stream_digest.go) beside the telemetry shards, at transports/checkpoints/<sh>, so a TPD that does not read them ignores them.

Wire layout (little-endian): version (1) = 0x02, shard (1), count (2), then count entries of 16 + 2*184 bytes: the transport uuid, then for sent and then received the epoch (8), then CheckpointRecent finest and CheckpointLevels-1 coarse checkpoints, each an offset (8) and a digest (8).

Package telemetrywire pkg/telemetrywire/telemetrywire.go c3-vis-core

telemetrywire is the SHARED, compact, sharded binary codec for the visor→TPD CXO telemetry feed. It is imported by BOTH the publishing side (pkg/visor/stats) and the consuming side (pkg/deployment/tpd/cxoaggregator) so the byte layout cannot drift between them — the two halves share one encoder/decoder rather than re-declaring a wire struct on each side.

Why sharded binary

The telemetry feed formerly published ONE JSON leaf per live transport at transports/<uuid>/current. A busy hub (~851 transports) produced ~851 leaves → ~1700 CXO objects in one Root, which could not finish filling over TPD's short-lived announce conn (only the top slice landed → busy-hub telemetry undercount). Object count was the fill bottleneck.

This codec replaces those N per-transport leaves with 16 FIXED sharded binary leaves: a transport is assigned to shard uuid[0]>>4 (0..15), and every transport in a shard is packed into one binary blob published at transports/telemetry/<sh> (<sh> = lowercase 2-hex). A busy hub's telemetry subtree is then 16 leaves regardless of transport count, so the whole-Root fill completes.

Wire layout (little-endian) 

off size field
0   1    version  = 0x02
1   1    shard    = 0x00..0x0f
2   2    count    uint16
4   ...  entries[count], 53 bytes each:
       +0   16  transport uuid (raw 16 bytes)
       +16  8   sent_bytes      uint64
       +24  8   recv_bytes      uint64
       +32  4   throughput_bps  float32 (peak goodput EWMA)
       +36  4   latency_min_ms  float32
       +40  4   latency_max_ms  float32
       +44  4   latency_avg_ms  float32
       +48  4   sampled_at      uint32 (unix SECONDS)
       +52  1   type            uint8 enum

The version byte gates the format: a future breaking change bumps it (0x03, …) and both sides branch on the value. DecodeShard rejects any other version, so an old reader fails a new blob cleanly rather than misparsing it. This is why the deploy is TPD-FIRST: TPD (which reads shards AND the legacy current leaves as a fallback) must ship before visors switch to shard-only publishing, or an old TPD silently loses a new visor's telemetry during the rollout window.

Index

Constants

View Source 
const (
	TypeUnknown uint8 = 0
	TypeSTCPR   uint8 = 1
	TypeSUDPH   uint8 = 2
	TypeSTCP    uint8 = 3
	TypeDMSG    uint8 = 4
	TypeSQUICR  uint8 = 5 // tptypes.QUIC, wire name "squicr"
	TypeWEBRTC  uint8 = 6
	TypeSWTR    uint8 = 7 // tptypes.WT, wire name "swtr"
	TypeSWSR    uint8 = 8 // tptypes.WS, wire name "swsr"
)

Transport-type enum codes. 0..7 match the original spec; 8 (swsr / WS) is the deterministic extension for the WS transport that the visor fleet also runs — tptypes.Known() carries it, so both sides map it rather than collapsing it to unknown. Unknown/empty types encode as 0, which TPD treats as "skip uptime" exactly like the legacy empty-type path (RecordTransportHeartbeat is not called for a zero-code entry).

View Source 
const CheckpointLevels = 4

CheckpointLevels is how many checkpoint levels each direction carries.

View Source 
const CheckpointRecent = 8

CheckpointRecent is how many finest checkpoints each direction carries.

View Source 
const CheckpointVersion = 0x02

CheckpointVersion is the checkpoint shard format version.

View Source 
const ShardCount = 16

ShardCount is the fixed number of shards a transport set is spread across — the low nibble of the uuid's first byte (uuid[0]>>4).

View Source 
const TypeFlagSameNetwork uint8 = 0x80

TypeFlagSameNetwork is set on an entry's Type when the transport's peer is on the reporting visor's own network: reached over a private address, or at the visor's own public IP. Such a transport moves bytes between two visors on one IP, which the reward system does not pay for; the flag rides the type byte so the wire size is unchanged, and older readers see an unknown type rather than a misread one.

View Source 
const Version = 0x02

Version is the current wire-format version byte. Bump on any breaking change to the header or entry layout; readers reject other values.

Variables

View Source 
var (
	ErrShort          = errors.New("telemetrywire: blob shorter than header")
	ErrVersion        = errors.New("telemetrywire: unsupported version byte")
	ErrShardRange     = errors.New("telemetrywire: shard byte out of range")
	ErrLengthMismatch = errors.New("telemetrywire: blob length does not match count")
)

Errors returned by DecodeShard. All indicate a malformed or wrong-version blob; the caller drops it rather than misparsing.

Functions

func CheckpointLeafPath added in v1.3.101 

func CheckpointLeafPath(shard uint8) string

CheckpointLeafPath is the CXO leaf path of a checkpoint shard.

func CodeToType 

func CodeToType(c uint8) string

CodeToType maps an enum code back to its canonical transport-type wire string. TypeUnknown (0) and any unrecognized code yield "" — the same value the legacy empty-type path used to signal "skip uptime".

func EncodeCheckpointShard added in v1.3.101 

func EncodeCheckpointShard(shard uint8, entries []CheckpointEntry) []byte

EncodeCheckpointShard encodes one shard's checkpoint entries.

func EncodeShard 

func EncodeShard(shard uint8, entries []Entry) []byte

EncodeShard packs entries into one shard blob. The caller is responsible for passing only entries that belong to shard (ShardOf == shard); the shard byte is written to the header verbatim. Entry order is preserved, so a caller that wants a byte-stable blob for change detection should sort entries by ID first.

func LeafPath 

func LeafPath(shard uint8) string

LeafPath is the stable CXO leaf path for a shard's telemetry blob: "transports/telemetry/<sh>" with <sh> the lowercase 2-hex shard.

func ParseCheckpointLeafPath added in v1.3.101 

func ParseCheckpointLeafPath(path string) (uint8, bool)

ParseCheckpointLeafPath returns the shard of a checkpoint leaf path.

func SameNetwork added in v1.3.99 

func SameNetwork(code uint8) bool

SameNetwork reports whether code carries TypeFlagSameNetwork.

func ShardOf 

func ShardOf(id uuid.UUID) uint8

ShardOf returns the shard a transport ID belongs to: the high nibble of the uuid's first byte (uuid[0]>>4), 0..15.

func TypeToCode 

func TypeToCode(s string) uint8

TypeToCode maps a transport-type wire string to its enum code, normalizing legacy aliases (e.g. "quic"/"squic" → squicr, "wt" → swtr, "ws" → swsr) first. An empty or unrecognized type yields TypeUnknown (0) — TPD treats that as "skip uptime".

Types

type Checkpoint added in v1.3.101 

type Checkpoint struct {
	Offset uint64
	Digest uint64
}

Checkpoint is a stream digest at a byte offset of a transport's conn.

type CheckpointEntry added in v1.3.101 

type CheckpointEntry struct {
	ID         uuid.UUID
	Sent, Recv DirCheckpoints
}

CheckpointEntry is one transport's checkpoints.

func DecodeCheckpointShard added in v1.3.101 

func DecodeCheckpointShard(b []byte) (shard uint8, entries []CheckpointEntry, err error)

DecodeCheckpointShard decodes a checkpoint shard.

func (CheckpointEntry) HasAny added in v1.3.101 

func (e CheckpointEntry) HasAny() bool

HasAny reports whether either direction has reached a checkpoint.

type DirCheckpoints added in v1.3.101 

type DirCheckpoints struct {
	Epoch  uint64
	Recent [CheckpointRecent]Checkpoint
	Coarse [CheckpointLevels - 1]Checkpoint
}

DirCheckpoints is one direction of a transport's current conn: the conn's epoch, its latest finest checkpoints, oldest first, and the latest coarse ones.

func (DirCheckpoints) Points added in v1.3.101 

func (d DirCheckpoints) Points() []Checkpoint

Points returns the checkpoints the direction holds, finest first.

type Entry 

type Entry struct {
	ID            uuid.UUID
	SentBytes     uint64
	RecvBytes     uint64
	ThroughputBps float32
	LatMin        float32
	LatMax        float32
	LatAvg        float32
	SampledAtUnix uint32
	Type          uint8
}

Entry is one transport's telemetry row — the decoded form of a 53-byte wire record. Both sides operate on Entry; only this package knows the byte layout.

func DecodeShard 

func DecodeShard(b []byte) (shard uint8, entries []Entry, err error)

DecodeShard parses a shard blob produced by EncodeShard. It validates the version byte, the shard range, and that the total length matches the declared count exactly — a truncated or over-long blob is rejected rather than partially parsed.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.