Help us shape the future of Go! Take the 2026 Go Developer Survey

store

package
v1.3.99 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 4, 2026 License: AGPL-3.0 Imports: 19 Imported by: 0

Documentation

Overview

Package store pkg/deployment/tpd/store/all_transports_cache.go c4-net-discovery

Package store pkg/deployment/tpd/store/bandwidth_day_index.go c4-net-discovery

Package store pkg/deployment/tpd/store/batch_ops.go c4-net-discovery

Batched forms of the per-transport writes a CXO snapshot drives, so one snapshot costs a handful of pipelines rather than a round trip (or a pipeline) per transport.

Package store pkg/deployment/tpd/store/cxo_metrics_layout.go c4-net-discovery

Layout of TPD's transport-metrics CXO feed, plus the pivot/merge pair that converts between it and the []TransportMetric window shape every caller already expects.

The feed publishes ONE LEAF PER CALENDAR DAY at

metrics/day/<YYYY-MM-DD>

rather than one leaf per day-window. A window (1, 7, 30 days) is assembled reader-side from the N newest day leaves. The point is that a past day cannot change, so its leaf hashes the same on every republish and CXO ships nothing for it — whereas one leaf per window meant a 30-day body was a new object every time any single byte moved, and the whole multi-megabyte window went back over the wire even though 29 of its 30 days were settled.

Two fields are current-state, not day-scoped: Live (a transport's registration can expire at any moment) and Latency (a rolling min/max/avg that moves constantly). Carrying them in every day leaf would make every past day mutable again and defeat the whole exercise — across tens of thousands of transports, at least one liveness flip between cycles is a certainty, so every leaf would be a new object every time. They therefore live ONLY in the newest day's leaf, which is rewritten every cycle anyway. Older leaves carry the day-scoped facts and the transport's stable identity (ID, Type, Edges). MergeDailyMetrics walks the days newest-first, so the newest leaf's values win and the merged record has the shape it always had.

The rule this implies, stated plainly: the current day's leaf holds exactly what a 1-day query returns — every transport with a latency measurement or bandwidth today — and a transport that appears in NO current-day record reads back Live=false with no latency. That is the one behavioral difference from the old per-window bodies, where a 30-day window reported current liveness for a transport last seen three weeks ago. In practice a registered transport has a latency record (lat:<id> outlives registration by 35 days), so it is in the current leaf regardless of whether it moved bytes today.

These live in the store package rather than next to the publisher because they are the contract BETWEEN the publisher and its readers (pkg/visor, pkg/tpviz), and TransportMetric — the thing being pivoted — is defined right here.

Package store pkg/deployment/tpd/store/edge_entries_cache.go c4-net-discovery

Package store pkg/deployment/tpd/store/expired_entries_cache.go c4-net-discovery

Package store pkg/deployment/tpd/store/live_set.go c4-net-discovery

The live transport set: TPD's transports held in memory by the store that writes them.

Every transport TPD knows arrives through this store (the CXO snapshot reconcile, the HTTP register), so the writing process can keep the whole set — entries, latency, throughput — as it writes it, and answer the whole-set reads from memory. Those reads were an index SMEMBERS plus an MGET of every transport, latency and throughput key: the routing, metrics and all-transports publishers each minute, and the route finder every 15 s, ~80k transports each time on prod01 (2026-09-30).

Only the writer can hold it: a process that does not write (a route finder on its own) would see nothing change, so it keeps reading redis. A route finder in the same process as TPD reads TPD's set through SharedLiveStore.

Entries lapse after the same TTL as their redis keys, refreshed by the same writes (register, touch). The redis index sets used to be pruned of lapsed IDs by the scans this replaces, so the set's sweep prunes them.

Package store pkg/deployment/tpd/store/memory_store.go c4-net-discovery

Package store pkg/deployment/tpd/store/pubkey_cache.go c4-net-discovery

Package store pkg/deployment/tpd/store/redis_bandwidth.go c4-net-discovery

Package store pkg/deployment/tpd/store/redis_daily_totals.go c4-net-discovery

Network-wide daily totals, kept as reports arrive so nothing has to scan every transport to answer "how much crossed the network on day D".

A transport's daily bandwidth is the MAX over its reporters of (sent+recv) — both edges see the same bytes (transportDailyBandwidth). The bandwidth script keeps that max per transport per day in the transport's daily hash ("max") and adds only its growth to the day's network hash, so the network total is exactly the sum of the per- transport maxima without ever reading them back. Latency is the mean, over the transports that reported one that day, of each one's latest average.

The totals start on the day the first report of this code lands (netDailySince). That day and earlier ones are read the old way, scanning every transport's daily hash; each settled day is read once (see store.OpenMetricsDays), so the fallback costs nothing after it.

Package store pkg/deployment/tpd/store/redis_metrics.go c4-net-discovery

Package store pkg/deployment/tpd/store/redis_store.go c4-net-discovery

Package store pkg/deployment/tpd/store/redis_transport.go c4-net-discovery

Package store pkg/deployment/tpd/store/redis_uptime.go c4-net-discovery

Package store pkg/deployment/tpd/store/store.go c4-net-discovery

Package store pkg/deployment/tpd/store/telemetry_batch.go c4-net-discovery

Telemetry applied in batches: a telemetry shard (or a flush of paced snapshots) carries many transports, and applying them one at a time was a round trip per script per transport. ApplyTelemetry does the same writes as UpdateBandwidth, UpdateThroughput and UpdateLatency for a whole batch in two pipelines: one MGET of the current throughput peaks, then everything else.

Package store pkg/deployment/tpd/store/today_metrics.go c4-net-discovery

Today's per-transport metrics, kept current by what changed.

The CXO metrics publisher rewrites today's leaf every minute. Reading it whole meant one HGETALL (today's bandwidth hash) and one GET (latency) per transport per minute — ~107k of each on the live deployment (2026-09-30), nearly all for transports whose figures had not moved. Bandwidth and latency are written through this store (UpdateBandwidth, UpdateLatency), so it knows exactly which transports moved: those are re-read, and every other row is carried over.

Settled days are handled the other way round: a day that can no longer change is saved, as the gzipped leaf bytes the publisher produced, next to the data it summarizes. A restart then reads 29 small values instead of re-aggregating 30 days (~4.9M HGETALLs per restart, measured).

Package store pkg/deployment/tpd/store/transport_beat_memo.go c4-net-discovery

Package store pkg/deployment/tpd/store/visorbw.go c4-net-discovery

Per-visor daily bandwidth: what the reward system pays pool 2 from.

Pool 2 pays each visor for the bytes it sent, but not for traffic between visors on one IP: an operator could otherwise pay themselves for moving bytes between their own visors. Each visor marks such transports itself — a peer it reaches at a private address or at its own public IP is on its own network (transport.ManagedTransport.SameNetwork) — and TPD keeps that mark on the transport's day (DailyEdgeBandwidth.SameNetwork). When a day settles, TPD reduces it to per-visor totals without the marked transports: a ~50 KB day, where shipping every transport to the reward host to judge there cost ~9 MB.

Index

Constants

View Source
const MetricsDateFormat = "2006-01-02"

MetricsDateFormat is the calendar-date layout used in a day leaf's path. It matches the format the bw:daily:<id>:<date> Redis keys already use, so a leaf's name and its source rows agree.

View Source
const MetricsDayPrefix = "metrics/day/"

MetricsDayPrefix is the sub-tree the per-day leaves live under. It is also what a subscriber's path prefix must cover.

View Source
const MetricsLateWindow = 15 * time.Minute

MetricsLateWindow is how long after UTC midnight yesterday stays open: bandwidth reports for it keep arriving for a few minutes after the rollover.

View Source
const VisorBWDayPrefix = "visorbw/day/"

VisorBWDayPrefix is the per-visor bandwidth feed's sub-tree; a settled day is at VisorBWDayPath(date).

View Source
const VisorBWVersion = 3

VisorBWVersion is the current VisorBWDay version. It continues the reward system's bandwidth-file versions (1: a {pk: bytes} map; 2: per-transport sent bytes, from bw-collect), so a leaf body IS a hist/<date>_bandwidth.json.

Variables

View Source
var (
	// ErrNotEnoughACKs means that we're still waiting for a visor to confirm the transport registration
	ErrNotEnoughACKs = errors.New("not enough ACKs")

	// ErrAlreadyRegistered indicates that transport ID is already in use.
	ErrAlreadyRegistered = errors.New("ID already registered")

	// ErrTransportNotFound indicates that requested transport is not registered.
	ErrTransportNotFound = errors.New("transport not found")

	// ErrBadEntry is returned when entry is malformed.
	ErrBadEntry = errors.New("bad entry format")

	// ErrUnknownStoreType means that store type is unknown.
	ErrUnknownStoreType = errors.New("unknown store type")
)

Functions

func MetricsDayDate

func MetricsDayDate(path string) (string, bool)

MetricsDayDate extracts the calendar date from a day-leaf path or from one of its part paths. ok is false for any path that is not under MetricsDayPrefix — including the legacy metrics/days/<n> window leaves, which a reader must not mistake for a day.

func MetricsDayPartPath

func MetricsDayPartPath(date string, part int) string

MetricsDayPartPath returns the path of one part of a day leaf that did not fit in a single CXO object. Zero-padded so a lexical sort of the paths is the publication order.

func MetricsDayPath

func MetricsDayPath(date string) string

MetricsDayPath returns the leaf path for one calendar day. Dates in this format sort lexically into chronological order, so a reader recovers "the N newest days" from a plain string sort.

func MetricsWindowDates

func MetricsWindowDates(now time.Time, days int) []string

MetricsWindowDates returns the calendar dates of a `days`-long window ending at now, NEWEST FIRST. The stepping matches buildTransportMetrics exactly (UTC, one AddDate per day back) so a pivoted row always finds its date in the window.

func OpenMetricsDays added in v1.3.98

func OpenMetricsDays(now time.Time) int

OpenMetricsDays is how many of the newest days can still change at now: today, plus yesterday inside MetricsLateWindow. Every older day is settled — its figures are final, so a publisher computes it once and never reads it from the store again.

func PivotDailyMetrics

func PivotDailyMetrics(metrics []TransportMetric, dates []string) map[string][]TransportMetric

PivotDailyMetrics splits a window's metrics into one slice per calendar day. dates is the window newest-first (as returned by MetricsWindowDates); every date gets an entry, empty days included, so the published path set is a deterministic function of the window rather than of the data.

dates[0] — the current day — is the only leaf that carries Live and Latency, and it carries a record for every transport that has latency at all, not just those with bandwidth today. That keeps pkg/tpviz's latency graph (which reads a single day) seeing exactly the transports the old 1-day window showed it.

A record with neither latency nor a row on the current day is omitted from the current leaf, mirroring the store's own rule that a transport with no metrics data is not reported.

func SenderBytes added in v1.3.99

func SenderBytes(d DailyEdgeBandwidth) (sentA, sentB uint64)

SenderBytes is what each edge of a transport sent on one day. Each edge reports its own counters on its own schedule, so usually only one has a fresh record: that side is trusted for both directions (its recv is what the other sent). An edge whose record is all zeros has not reported yet. These are the rules the reward system has paid pool 2 by since 2026-06-03.

func VisorBWDayPath added in v1.3.99

func VisorBWDayPath(date string) string

VisorBWDayPath is the leaf for one settled day.

Types

type BandwidthAggregation

type BandwidthAggregation struct {
	TransportID string `json:"transport_id"`
	Period      string `json:"period"`     // "daily", "weekly", "monthly"
	PeriodKey   string `json:"period_key"` // e.g., "2026-02-10", "2026-W06", "2026-02"
	Bandwidth   uint64 `json:"bandwidth"`  // total bytes (sent + recv)
	UpdatedAt   int64  `json:"updated_at"`
}

BandwidthAggregation stores aggregated bandwidth for a time period.

type BatchStore added in v1.3.99

type BatchStore interface {
	// TouchTransports extends the lifetime of registered transports that are
	// still listed: the entry keys, and the reporter's edge index. Nothing is
	// rewritten.
	TouchTransports(ctx context.Context, reporter cipher.PubKey, ids []uuid.UUID) error
	// DeregisterTransports removes transports and returns the ones that
	// existed.
	DeregisterTransports(ctx context.Context, ids []uuid.UUID) ([]*transport.Entry, error)
	// RecordTransportHeartbeats records an uptime heartbeat for each entry.
	RecordTransportHeartbeats(ctx context.Context, entries []*transport.Entry, at time.Time) error
}

BatchStore is implemented by stores that can apply a snapshot's writes in batches. The API uses it when present.

type CumulativeAggregate

type CumulativeAggregate struct {
	Bandwidth uint64                          `json:"bandwidth,omitempty"`
	Latency   float64                         `json:"latency,omitempty"`
	ByType    map[string]*TypeMetricAggregate `json:"by_type,omitempty"`
}

CumulativeAggregate contains cumulative totals across all days.

type DailyAggregate

type DailyAggregate struct {
	Date      string                          `json:"date"`
	Bandwidth uint64                          `json:"bandwidth,omitempty"`
	Latency   float64                         `json:"latency,omitempty"`
	ByType    map[string]*TypeMetricAggregate `json:"by_type,omitempty"`
}

DailyAggregate contains network-wide aggregate for a single day.

type DailyBandwidthEntry

type DailyBandwidthEntry struct {
	Date      string `json:"date"` // "2006-01-02"
	Bandwidth uint64 `json:"bw"`   // total bytes (sent + recv)
}

DailyBandwidthEntry stores bandwidth total for a single day.

type DailyEdgeBandwidth

type DailyEdgeBandwidth struct {
	Date string         `json:"date"`
	A    *EdgeBandwidth `json:"a,omitempty"`
	B    *EdgeBandwidth `json:"b,omitempty"`
	// SameNetwork: an edge reported that day that the other is on its own
	// network (one IP). The bytes are real, but the reward system does not
	// pay for them.
	SameNetwork bool `json:"same_network,omitempty"`
}

DailyEdgeBandwidth contains per-day bandwidth with edge A and B data.

type DailyVisorAggregate

type DailyVisorAggregate struct {
	Date      string                   `json:"date"`
	Bandwidth *VisorBandwidthAggregate `json:"bandwidth,omitempty"`
	Latency   float64                  `json:"latency,omitempty"`
}

DailyVisorAggregate contains per-day visor aggregate data.

type EdgeBandwidth

type EdgeBandwidth struct {
	Sent uint64 `json:"sent"` // Cumulative bytes sent
	Recv uint64 `json:"recv"` // Cumulative bytes received
}

EdgeBandwidth contains bandwidth data from one edge's perspective.

type LatencyRecord

type LatencyRecord struct {
	Min       int64 `json:"min"` // microseconds
	Max       int64 `json:"max"` // microseconds
	Avg       int64 `json:"avg"` // microseconds
	UpdatedAt int64 `json:"updated_at"`
}

LatencyRecord is the durable per-transport latency snapshot persisted at transport-discovery:lat:<id>. It lives independently of the registration blob (TransportData) so a 5-min entry-timeout cycle doesn't erase latency the way it did when the values were stored only inside the tp: key. Bandwidth gets the same independence via bw:daily:* keys; this gives latency the equivalent durability so /metrics doesn't show a transport with bandwidth-today but no latency just because the visor's re-registration window briefly lapsed.

Last-writer-wins on the per-transport key: latency is round-trip and both edges observe the same RTT modulo measurement noise, so no merge or per-edge tracking. Min/Max/Avg are stored in microseconds to match the existing TransportData layout. UpdatedAt is informational (lets readers gauge staleness) and not currently surfaced to the API.

type MetricsQuery

type MetricsQuery struct {
	Days      int    // 0 = all available (max 35)
	Type      string // Filter by transport type (stcpr, sudph, dmsg, stcp)
	Live      string // Filter by liveness: "true", "false", "all" (default: "all")
	Edges     bool   // Include actual public keys in response
	Bandwidth bool   // Include bandwidth data (default: true)
	Latency   bool   // Include latency data (default: true)
}

MetricsQuery contains query parameters for metrics endpoints.

type NetworkMetricResponse

type NetworkMetricResponse struct {
	Daily      []DailyAggregate     `json:"daily"`
	Cumulative *CumulativeAggregate `json:"cumulative"`
}

NetworkMetricResponse is the response for GET /metric.

type Store

type Store interface {
	TransportStore
}

Store stores Transport metadata and generated nonce values.

func SharedLiveStore added in v1.3.99

func SharedLiveStore(url string) (Store, bool)

SharedLiveStore returns the store in this process that holds the live transport set for the redis at url, if any.

type TelemetryBatchStore added in v1.3.99

type TelemetryBatchStore interface {
	ApplyTelemetry(ctx context.Context, updates []TelemetryUpdate) error
}

TelemetryBatchStore is implemented by stores that apply telemetry in batches.

type TelemetryUpdate added in v1.3.99

type TelemetryUpdate struct {
	ID                     uuid.UUID
	Reporter               cipher.PubKey
	Sent, Recv             uint64
	ThroughputBps          float64
	LatMin, LatMax, LatAvg float64
	Type                   string
	// SameNetwork: the reporter marked the peer as on its own network, so
	// the transport's bytes that day are not paid for.
	SameNetwork bool
}

TelemetryUpdate is one edge's report for one transport.

type ThroughputRecord

type ThroughputRecord struct {
	Bps       float64 `json:"bps"`
	UpdatedAt int64   `json:"updated_at"`
}

ThroughputRecord is the durable per-transport PEAK-goodput snapshot persisted at transport-discovery:tput:<id>. Bps is the passively observed peak (an EWMA tracked on the visor and pushed on the sharded telemetry feed); UpdateThroughput keeps the MAX across the two edges' reports so a momentarily-idle edge can't erase the other's peak. Shares latency's retention window and last-writer-wins independence from the registration blob. UpdatedAt is informational (staleness gauge).

type TransportData

type TransportData struct {
	ID         string `json:"id"`
	EdgeA      string `json:"edge_a"`
	EdgeB      string `json:"edge_b"`
	Type       string `json:"type"`
	Label      string `json:"label"`
	LatencyMin int64  `json:"lat_min"`     // Minimum latency in microseconds
	LatencyMax int64  `json:"lat_max"`     // Maximum latency in microseconds
	LatencyAvg int64  `json:"lat_avg"`     // Average latency in microseconds
	Bandwidth  uint64 `json:"bandwidth"`   // Total bytes (sent + recv)
	LastUpdate int64  `json:"last_update"` // Unix timestamp of last update
}

type TransportLatency

type TransportLatency struct {
	Min int64 `json:"min"` // Minimum observed latency in microseconds
	Max int64 `json:"max"` // Maximum observed latency in microseconds
	Avg int64 `json:"avg"` // Average latency in microseconds
}

TransportLatency contains latency statistics for a transport in microseconds. Latency is at the transport level (not per-edge) since it's round-trip.

type TransportMetric

type TransportMetric struct {
	ID      string               `json:"id"`
	Type    string               `json:"type"`
	Live    bool                 `json:"live"`
	Edges   []string             `json:"edges,omitempty"`   // Only included if query.Edges=true
	Latency *TransportLatency    `json:"latency,omitempty"` // Transport-level latency stats
	Daily   []DailyEdgeBandwidth `json:"daily"`             // Per-day bandwidth by edge
}

TransportMetric contains per-transport metrics with latency at transport level and daily bandwidth breakdowns per edge.

func MergeDailyMetrics

func MergeDailyMetrics(days [][]TransportMetric) []TransportMetric

MergeDailyMetrics rebuilds the []TransportMetric window shape from day slices given NEWEST FIRST. Records are joined on transport ID; daily rows accumulate in the order the days were supplied (newest first, matching buildTransportMetrics), and the first day that mentions a transport supplies its Live/Latency/Type/Edges.

The result is sorted by ID so a window assembled from the same leaves is byte-identical every time.

type TransportStore

type TransportStore interface {
	// RegisterTransport registers/refreshes a transport. reporter is the
	// authenticated registering edge: only that edge's per-edge index TTL is
	// refreshed, so an offline edge's index expires even while its live peer
	// keeps re-registering the shared transport. A zero reporter (or one that
	// is not an edge) falls back to refreshing both edges (legacy behavior).
	RegisterTransport(context.Context, cipher.PubKey, *transport.SignedEntry) error
	RegisterTransportsBatch(context.Context, cipher.PubKey, []*transport.SignedEntry) error
	DeregisterTransport(context.Context, uuid.UUID) error
	GetTransportByID(context.Context, uuid.UUID) (*transport.Entry, error)
	GetTransportsByEdge(context.Context, cipher.PubKey) ([]*transport.Entry, error)
	// GetTransportsByEdgeNoLatency is the cheap variant for callers
	// (DHT mirror, transport-count stats) that don't need the
	// durable latency overlay. Skips the per-call MGET on lat:<id>
	// keys and the JSON decode that follows.
	GetTransportsByEdgeNoLatency(context.Context, cipher.PubKey) ([]*transport.Entry, error)
	GetNumberOfTransports(context.Context) (map[types.Type]int, error)
	// GetTransportSummary returns network-wide aggregate counts (total,
	// per-type, unique visors) computed in a single index+MGET pass that
	// never materializes []*transport.Entry — the cheap path behind GET
	// /all-transports/stats, so a client counting transports doesn't have
	// to fetch every entry. selfTransports=false excludes loopback
	// self-transports (Edges[0]==Edges[1]), matching GetAllTransports.
	GetTransportSummary(ctx context.Context, selfTransports bool) (*TransportSummary, error)
	GetAllTransports(context.Context, bool) ([]*transport.Entry, error)
	// Bandwidth ingest (called by the CXO aggregator with the
	// reporter's cumulative counters; deltas are computed
	// internally against the per-reporter previous snapshot).
	UpdateBandwidth(ctx context.Context, transportID string, reporterPK cipher.PubKey, sent, recv uint64, tpType string) error
	// Latency ingest (called by the CXO aggregator with the
	// reporter's window min/max/avg in milliseconds). RTT-symmetric,
	// so last-writer-wins across the two edges is acceptable.
	UpdateLatency(ctx context.Context, transportID string, minMS, maxMS, avgMS float64, tpType string) error
	// Throughput ingest (called by the CXO aggregator with the
	// reporter's passively-observed PEAK goodput estimate in bytes/sec,
	// carried on the sharded telemetry feed). Recorded per-transport with
	// a peak-preserving max so the higher of the two edges' observations
	// survives; overlaid onto Entry.ThroughputBps on read.
	UpdateThroughput(ctx context.Context, transportID string, reporterPK cipher.PubKey, bps float64) error
	// Bandwidth query methods (legacy)
	GetTransportBandwidth(ctx context.Context, tpID uuid.UUID, period string, limit int) ([]BandwidthAggregation, error)
	GetVisorBandwidth(ctx context.Context, pk cipher.PubKey, period string, limit int) ([]BandwidthAggregation, error)
	GetAllVisorSummaries(ctx context.Context, v2 bool, timeline bool) ([]VisorSummary, error)
	RecordHeartbeat(ctx context.Context, pk cipher.PubKey, version string) error
	GetDailyTimeline(ctx context.Context, pkHex string, now time.Time) map[string]string
	// Transport uptime tracking (stcpr/sudph only).
	// at is the time the heartbeat was *observed* on the visor side
	// (snap.SampledAt for the CXO path, time.Now() for the HTTP
	// register path); the slot bit and date bucket derive from it,
	// so leaf-arrival skew that crosses a 5-minute boundary still
	// credits the correct slot. Zero time falls back to time.Now().
	RecordTransportHeartbeat(ctx context.Context, tpID uuid.UUID, tpType string, at time.Time) error
	// IngestTransportTimeline OR-merges a visor-supplied per-transport
	// uptime bitmap into the persistent timeline. Used by the CXO
	// aggregator to absorb the visor's locally-tracked bitmap, which
	// captures slots TPD's heartbeat path never observed (e.g.
	// during TPD downtime). bitmap must be 36 bytes; date is the
	// UTC YYYY-MM-DD the bitmap covers.
	IngestTransportTimeline(ctx context.Context, tpID uuid.UUID, date string, bitmap []byte) error
	GetTransportUptimeSummaries(ctx context.Context, tpIDs []uuid.UUID, v2 bool, timeline bool) ([]TransportUptimeSummary, error)
	GetTransportUptimeByVisor(ctx context.Context, pk cipher.PubKey, v2 bool, timeline bool) ([]TransportUptimeSummary, error)
	GetTransportDailyTimeline(ctx context.Context, tpID string, now time.Time) map[string]string
	BackupAndCleanOldBandwidth(ctx context.Context, backupPath string) error
	// New metrics methods
	GetNetworkMetrics(ctx context.Context, query MetricsQuery) (*NetworkMetricResponse, error)
	GetVisorAggregateMetrics(ctx context.Context, pks []cipher.PubKey, query MetricsQuery) (map[string]*VisorMetricResponse, error)
	GetAllTransportMetrics(ctx context.Context, query MetricsQuery) ([]TransportMetric, error)
	GetTransportMetricsByIDs(ctx context.Context, ids []uuid.UUID, query MetricsQuery) ([]TransportMetric, error)
	GetTransportMetricsByVisors(ctx context.Context, pks []cipher.PubKey, query MetricsQuery) ([]TransportMetric, error)
	Close()
}

TransportStore stores Transport metadata.

func New

func New(ctx context.Context, config storeconfig.Config, ttl time.Duration, logger *logging.Logger) (TransportStore, error)

New constructs a new Store of requested type.

type TransportSummary

type TransportSummary struct {
	Total        int            `json:"total_transports"`
	ByType       map[string]int `json:"by_type"`
	UniqueVisors int            `json:"unique_visors"`

	// Partial reports that at least one batch of the underlying read failed,
	// so Total and ByType undercount by an unknown amount.
	//
	// This exists because the counts are assembled from batched MGETs over the
	// transport index, and a failed batch used to be skipped with a bare
	// `continue` — silently removing up to mgetBatch transports from the
	// answer while still returning a plausible-looking number. A consumer had
	// no way to tell "there are N transports" from "N is what I managed to
	// read", which is indistinguishable from the network shrinking.
	//
	// Callers that publish or chart these numbers must treat Partial as
	// "do not trust this sample" rather than as data.
	Partial bool `json:"partial,omitempty"`
	// MissingBatches counts the failed batches, for logging and for judging
	// how far off Total might be.
	MissingBatches int `json:"missing_batches,omitempty"`
}

TransportSummary is a network-wide aggregate of registered transports, computed server-side so a client can obtain the counts without fetching (and counting) the full transport list. Marshals to the same JSON shape as GET /all-transports/stats.

type TransportUptimeSummary

type TransportUptimeSummary struct {
	ID       uuid.UUID         `json:"id"`
	Online   bool              `json:"on"`
	Type     string            `json:"type,omitempty"`
	EdgeA    string            `json:"edge_a,omitempty"`
	EdgeB    string            `json:"edge_b,omitempty"`
	Daily    map[string]string `json:"daily,omitempty"`
	Timeline map[string]string `json:"timeline,omitempty"`
}

TransportUptimeSummary holds uptime data for a single transport. Edges are omitted by default — include via ?edges=true query param.

type TypeMetricAggregate

type TypeMetricAggregate struct {
	Bandwidth uint64  `json:"bandwidth,omitempty"`
	Latency   float64 `json:"latency,omitempty"`
}

TypeMetricAggregate contains aggregate metrics for a transport type.

type VisorBWDay added in v1.3.99

type VisorBWDay struct {
	Version int    `json:"version"`
	Date    string `json:"date"`
	// Transports is how many transports moved bytes that day, and
	// SameNetworkExcluded how many of them were left out because an edge
	// marked the other as on its own network.
	Transports          int `json:"transports"`
	SameNetworkExcluded int `json:"same_network_excluded"`
	// Visors maps a visor's public key (hex) to the bytes it sent that day,
	// by transport type.
	Visors map[string]map[string]uint64 `json:"visors"`
}

VisorBWDay is one settled day of per-visor bandwidth.

func ComputeVisorBW added in v1.3.99

func ComputeVisorBW(records []TransportMetric, date string) VisorBWDay

ComputeVisorBW reduces one day's per-transport records to per-visor sent bytes by type, leaving out transports marked same-network that day.

type VisorBandwidthAggregate

type VisorBandwidthAggregate struct {
	Sent  uint64 `json:"sent"`
	Recv  uint64 `json:"recv"`
	Total uint64 `json:"total"`
}

VisorBandwidthAggregate contains bandwidth totals for a visor.

type VisorCumulativeAggregate

type VisorCumulativeAggregate struct {
	Bandwidth *VisorBandwidthAggregate `json:"bandwidth,omitempty"`
	Latency   float64                  `json:"latency,omitempty"`
}

VisorCumulativeAggregate contains cumulative visor totals.

type VisorMetricResponse

type VisorMetricResponse struct {
	Daily      []DailyVisorAggregate     `json:"daily"`
	Cumulative *VisorCumulativeAggregate `json:"cumulative"`
}

VisorMetricResponse is the response for a single visor in GET /metric/visor/{pks}.

type VisorSummary

type VisorSummary struct {
	PK       cipher.PubKey     `json:"pk"`
	Online   bool              `json:"on"`
	Version  string            `json:"version,omitempty"`
	Daily    map[string]string `json:"daily,omitempty"`
	Timeline map[string]string `json:"timeline,omitempty"`
}

VisorSummary holds a visor's uptime data. This format matches the Uptime Tracker's response format. v3 adds Timeline — a per-day string of 288 chars (one per 5-minute slot) where '.' = heartbeat received and ' ' = missed.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL