Help us shape the future of Go! Take the 2026 Go Developer Survey

telemetrywire

package
v1.3.104-alpha1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 30, 2026 License: AGPL-3.0 Imports: 6 Imported by: 0

Documentation

Overview

Package telemetrywire pkg/telemetrywire/checkpoints.go c3-vis-core

Checkpoint shards carry each transport's stream digest checkpoints (see pkg/transport/stream_digest.go) beside the telemetry shards, at transports/checkpoints/<sh>, so a TPD that does not read them ignores them.

Wire layout (little-endian): version (1) = 0x02, shard (1), count (2), then count entries of 16 + 2*184 bytes: the transport uuid, then for sent and then received the epoch (8), then CheckpointRecent finest and CheckpointLevels-1 coarse checkpoints, each an offset (8) and a digest (8).

Package telemetrywire pkg/telemetrywire/telemetrywire.go c3-vis-core

telemetrywire is the SHARED, compact, sharded binary codec for the visor→TPD CXO telemetry feed. It is imported by BOTH the publishing side (pkg/visor/stats) and the consuming side (pkg/deployment/tpd/cxoaggregator) so the byte layout cannot drift between them — the two halves share one encoder/decoder rather than re-declaring a wire struct on each side.

Why sharded binary

The telemetry feed formerly published ONE JSON leaf per live transport at transports/<uuid>/current. A busy hub (~851 transports) produced ~851 leaves → ~1700 CXO objects in one Root, which could not finish filling over TPD's short-lived announce conn (only the top slice landed → busy-hub telemetry undercount). Object count was the fill bottleneck.

This codec replaces those N per-transport leaves with 16 FIXED sharded binary leaves: a transport is assigned to shard uuid[0]>>4 (0..15), and every transport in a shard is packed into one binary blob published at transports/telemetry/<sh> (<sh> = lowercase 2-hex). A busy hub's telemetry subtree is then 16 leaves regardless of transport count, so the whole-Root fill completes.

Wire layout (little-endian) 

off size field
0   1    version  = 0x02
1   1    shard    = 0x00..0x0f
2   2    count    uint16
4   ...  entries[count], 53 bytes each:
       +0   16  transport uuid (raw 16 bytes)
       +16  8   sent_bytes      uint64
       +24  8   recv_bytes      uint64
       +32  4   throughput_bps  float32 (peak goodput EWMA)
       +36  4   latency_min_ms  float32
       +40  4   latency_max_ms  float32
       +44  4   latency_avg_ms  float32
       +48  4   sampled_at      uint32 (unix SECONDS)
       +52  1   type            uint8 enum

The version byte gates the format: a future breaking change bumps it (0x03, …) and both sides branch on the value. DecodeShard rejects any other version, so an old reader fails a new blob cleanly rather than misparsing it. This is why the deploy is TPD-FIRST: TPD (which reads shards AND the legacy current leaves as a fallback) must ship before visors switch to shard-only publishing, or an old TPD silently loses a new visor's telemetry during the rollout window.

Index

Constants

View Source 
const (
	TypeUnknown uint8 = 0
	TypeSTCPR   uint8 = 1
	TypeSUDPH   uint8 = 2
	TypeSTCP    uint8 = 3
	TypeDMSG    uint8 = 4
	TypeSQUICR  uint8 = 5 // tptypes.QUIC, wire name "squicr"
	TypeWEBRTC  uint8 = 6
	TypeSWTR    uint8 = 7 // tptypes.WT, wire name "swtr"
	TypeSWSR    uint8 = 8 // tptypes.WS, wire name "swsr"
)

Transport-type enum codes. 0..7 match the original spec; 8 (swsr / WS) is the deterministic extension for the WS transport that the visor fleet also runs — tptypes.Known() carries it, so both sides map it rather than collapsing it to unknown. Unknown/empty types encode as 0, which TPD treats as "skip uptime" exactly like the legacy empty-type path (RecordTransportHeartbeat is not called for a zero-code entry).

View Source 
const CheckpointLevels = 4

CheckpointLevels is how many checkpoint levels each direction carries.

View Source 
const CheckpointRecent = 8

CheckpointRecent is how many finest checkpoints each direction carries.

View Source 
const CheckpointVersion = 0x02

CheckpointVersion is the checkpoint shard format version.

View Source 
const ShardCount = 16

ShardCount is the fixed number of shards a transport set is spread across — the low nibble of the uuid's first byte (uuid[0]>>4).

View Source 
const TypeFlagSameNetwork uint8 = 0x80

TypeFlagSameNetwork is set on an entry's Type when the transport's peer is on the reporting visor's own network: reached over a private address, or at the visor's own public IP. Such a transport moves bytes between two visors on one IP, which the reward system does not pay for; the flag rides the type byte so the wire size is unchanged, and older readers see an unknown type rather than a misread one.

View Source 
const Version = 0x02

Version is the current wire-format version byte. Bump on any breaking change to the header or entry layout; readers reject other values.

Variables

View Source 
var (
	ErrShort          = errors.New("telemetrywire: blob shorter than header")
	ErrVersion        = errors.New("telemetrywire: unsupported version byte")
	ErrShardRange     = errors.New("telemetrywire: shard byte out of range")
	ErrLengthMismatch = errors.New("telemetrywire: blob length does not match count")
)

Errors returned by DecodeShard. All indicate a malformed or wrong-version blob; the caller drops it rather than misparsing.

Functions

func CheckpointLeafPath added in v1.3.101 

func CheckpointLeafPath(shard uint8) string

CheckpointLeafPath is the CXO leaf path of a checkpoint shard.

func CodeToType 

func CodeToType(c uint8) string

CodeToType maps an enum code back to its canonical transport-type wire string. TypeUnknown (0) and any unrecognized code yield "" — the same value the legacy empty-type path used to signal "skip uptime".

func EncodeCheckpointShard added in v1.3.101 

func EncodeCheckpointShard(shard uint8, entries []CheckpointEntry) []byte

EncodeCheckpointShard encodes one shard's checkpoint entries.

func EncodeShard 

func EncodeShard(shard uint8, entries []Entry) []byte

EncodeShard packs entries into one shard blob. The caller is responsible for passing only entries that belong to shard (ShardOf == shard); the shard byte is written to the header verbatim. Entry order is preserved, so a caller that wants a byte-stable blob for change detection should sort entries by ID first.

func LeafPath 

func LeafPath(shard uint8) string

LeafPath is the stable CXO leaf path for a shard's telemetry blob: "transports/telemetry/<sh>" with <sh> the lowercase 2-hex shard.

func ParseCheckpointLeafPath added in v1.3.101 

func ParseCheckpointLeafPath(path string) (uint8, bool)

ParseCheckpointLeafPath returns the shard of a checkpoint leaf path.

func SameNetwork added in v1.3.99 

func SameNetwork(code uint8) bool

SameNetwork reports whether code carries TypeFlagSameNetwork.

func ShardOf 

func ShardOf(id uuid.UUID) uint8

ShardOf returns the shard a transport ID belongs to: the high nibble of the uuid's first byte (uuid[0]>>4), 0..15.

func TypeToCode 

func TypeToCode(s string) uint8

TypeToCode maps a transport-type wire string to its enum code, normalizing legacy aliases (e.g. "quic"/"squic" → squicr, "wt" → swtr, "ws" → swsr) first. An empty or unrecognized type yields TypeUnknown (0) — TPD treats that as "skip uptime".

Types

type Checkpoint added in v1.3.101 

type Checkpoint struct {
	Offset uint64
	Digest uint64
}

Checkpoint is a stream digest at a byte offset of a transport's conn.

type CheckpointEntry added in v1.3.101 

type CheckpointEntry struct {
	ID         uuid.UUID
	Sent, Recv DirCheckpoints
}

CheckpointEntry is one transport's checkpoints.

func DecodeCheckpointShard added in v1.3.101 

func DecodeCheckpointShard(b []byte) (shard uint8, entries []CheckpointEntry, err error)

DecodeCheckpointShard decodes a checkpoint shard.

func (CheckpointEntry) HasAny added in v1.3.101 

func (e CheckpointEntry) HasAny() bool

HasAny reports whether either direction has reached a checkpoint.

type DirCheckpoints added in v1.3.101 

type DirCheckpoints struct {
	Epoch  uint64
	Recent [CheckpointRecent]Checkpoint
	Coarse [CheckpointLevels - 1]Checkpoint
}

DirCheckpoints is one direction of a transport's current conn: the conn's epoch, its latest finest checkpoints, oldest first, and the latest coarse ones.

func (DirCheckpoints) Points added in v1.3.101 

func (d DirCheckpoints) Points() []Checkpoint

Points returns the checkpoints the direction holds, finest first.

type Entry 

type Entry struct {
	ID            uuid.UUID
	SentBytes     uint64
	RecvBytes     uint64
	ThroughputBps float32
	LatMin        float32
	LatMax        float32
	LatAvg        float32
	SampledAtUnix uint32
	Type          uint8
}

Entry is one transport's telemetry row — the decoded form of a 53-byte wire record. Both sides operate on Entry; only this package knows the byte layout.

func DecodeShard 

func DecodeShard(b []byte) (shard uint8, entries []Entry, err error)

DecodeShard parses a shard blob produced by EncodeShard. It validates the version byte, the shard range, and that the total length matches the declared count exactly — a truncated or over-long blob is rejected rather than partially parsed.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.