metrics

package
v0.38.21 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 22, 2026 License: MIT Imports: 8 Imported by: 0

Documentation

Overview

Package metrics owns the application's Prometheus instrumentation.

Rationale: kube-prometheus-stack already runs in the cluster and scrapes node/k8s/cnpg/argocd metrics out of the box. Adding a /metrics endpoint to rivolt with an explicit ServiceMonitor lights up per-handler latency, Rivian-result-class counters, and the other app-shaped signals the cluster-level scrapers can't see.

All metrics live on a private *prometheus.Registry rather than the global default. Reasons:

  • Tests can spin up a fresh registry per test.
  • We don't accidentally collide with go-collector / process-collector registrations from libraries that touch promauto.
  • The handler is exported scoped to this registry, so /metrics output is exactly what we asked for.

Naming follows Prometheus conventions: snake_case, base unit suffix (_seconds, _bytes, _total).

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Metrics

type Metrics struct {

	// HTTP server side.
	HTTPRequestsTotal   *prometheus.CounterVec
	HTTPRequestDuration *prometheus.HistogramVec

	// Rivian upstream classification (transient/outage/rate-limited/
	// user-action/unknown — see internal/rivian/errclass.go). Counts
	// outcomes per call class, not per error string, so cardinality
	// stays bounded.
	RivianResultsTotal *prometheus.CounterVec

	// Subscription leases owned by this pod. The lease layer
	// increments/decrements as it claims and releases.
	SubscriptionLeases prometheus.Gauge
	// LeaseUnstreamed counts episodes of a lease this pod holds without
	// a subscription behind it (the coordinator re-subscribes; alert on
	// any increase).
	LeaseUnstreamed prometheus.Counter

	// Circuit breaker telemetry. State is a 0/1/2 gauge
	// (closed/half_open/open) so Grafana can alert on
	// max_over_time != 0. Trips is a counter labelled by reason
	// ("rate_limited" | "outage") so we can answer "what's making
	// it open" in one PromQL.
	RivianBreakerState prometheus.Gauge
	RivianBreakerTrips *prometheus.CounterVec
	// Global token bucket gating. Counter increments every time
	// the bucket says no, partitioned by class so we can answer
	// "is priority being starved".
	RivianRateLimitBlocked *prometheus.CounterVec
	// AI provider spend per user is intentionally NOT exposed here.
	// User-id labels would blow up cardinality at 1000 vehicles. The
	// existing internal/ai/usage.go writes per-user totals to
	// Postgres; surface those via a Grafana SQL data source if/when
	// we want to chart them.
	AIRequestsTotal *prometheus.CounterVec

	// Transactional email outcomes. `provider` is the wired email
	// backend (today: "resend"), `status` is "ok" / "error" /
	// "rate_limited" / "not_configured". An alert rule watches the
	// monthly increase against Resend's free-tier cap (3000) so we
	// see the wall coming a few days out, not the morning after.
	EmailSendTotal *prometheus.CounterVec

	// Signup-to-first-vehicle funnel. One increment per step a user
	// reaches (signup_created, rivian_login_attempt, rivian_connected,
	// rivian_no_vehicles, ...; see internal/api/funnel.go for the
	// full set). `event` is a fixed vocabulary, never a user id, so
	// cardinality stays bounded. The per-user trail lives in the
	// matching "funnel" log line instead.
	FunnelEventsTotal *prometheus.CounterVec

	// Authenticated requests by how they were authenticated: cookie, kratos,
	// header (trusted-proxy SSO), bypass. A fixed vocabulary; no user data.
	// (Impersonation is logged but not counted separately: it rides on a
	// cookie-authenticated admin.)
	AuthRequestsTotal *prometheus.CounterVec

	// Server-side funnel events by outcome: recorded, skipped_dnt (the request
	// sent Do Not Track / GPC), skipped_disabled, skipped_other, failed. Event
	// name and outcome only; no user data.
	ServerEventsTotal *prometheus.CounterVec

	// Completed vehicle sleeps by outcome: "failed" when the car woke
	// within a minute of falling asleep, "lasted" otherwise. A rising
	// failed share is a car that can't stay asleep (vampire drain);
	// the per-vehicle detail is in the "vehicle sleep ended" log line.
	VehicleSleepsTotal *prometheus.CounterVec

	// Parallax drive-dynamics values ignored instead of applied, by topic:
	// cloud replays of a topic's last snapshot (every resubscribe re-sends
	// it with its original timestamp) and frames older than a minute. A
	// replayed gear D once flapped a charging truck's sessions.
	ParallaxReplaysIgnoredTotal *prometheus.CounterVec

	// Demo rate-limit bucket keys by source: "cf" / "lb" when the direct
	// peer is a Cloudflare edge / our private load-balancer hop and
	// CF-Connecting-IP was used; "xff" when the key is the rightmost
	// X-Forwarded-For entry itself (through the Cloudflare tunnel that is
	// the visitor, appended by Cloudflare: a per-visitor key); "peer" when
	// there was no forwarded address and the connection address is the key
	// (a shared bucket behind an unforwarded proxy). Only a high "peer"
	// share on a proxied install signals visitors sharing buckets.
	DemoClientKeyTotal *prometheus.CounterVec

	// Demo requests rejected by a rate limiter (HTTP 429), by route class
	// (general, samples, heavy, tiles) and by which budget bound (client
	// or global). A sustained global count is the signal to look at; no
	// addresses are recorded.
	DemoRateLimitedTotal *prometheus.CounterVec

	// Failed live recorder writes by kind (drive_upsert, charge_upsert,
	// stale_charge). Nonzero means drives or charges are being lost or left
	// stale, typically a database outage.
	RecorderWriteFailuresTotal *prometheus.CounterVec
	// Sign-in lockouts: an email address reached the failed-login limit and is
	// refused for the rest of the window. No labels, so no address is recorded.
	// A burst is a guessing or targeted-lockout campaign.
	LoginLockoutsTotal prometheus.Counter
	// WebSocket "next" frames dropped because a subscription's 16-slot channel
	// was full (a consumer stuck behind a slow database). A missed gear frame
	// can merge or lose a drive.
	WSFramesDroppedTotal prometheus.Counter

	// Non-transactional messages (internal/engage), by kind and outcome (sent,
	// failed, already_sent, skipped_*). No user ids in labels.
	EngagementSendsTotal *prometheus.CounterVec
	// Claims left in 'sending' past 15 minutes: a crash between claim and
	// delivery. Never resent, so a nonzero value is how a lost message shows.
	EngagementStuckSending prometheus.Gauge
	// vehicle_state partitions dropped by sample retention.
	SamplePartitionsDropped prometheus.Counter
	// Archive objects that exist but fail to read back (corrupt, truncated):
	// for a thinned month, full-resolution data lost. Alert on any.
	ArchiveReadCorruptTotal prometheus.Counter
	// Archive objects flagged for re-archiving because their month's rows no
	// longer matched them (a write during a month's first archive).
	ArchiveRepairsTotal prometheus.Counter
	// Telemetry archive objects (one per user, vehicle and month) by outcome
	// (verified, failed), and the bytes of the verified ones.
	ArchiveObjectsTotal *prometheus.CounterVec
	ArchiveBytesTotal   prometheus.Counter
	// Archive deletions (account deletion, history reset): objects deleted,
	// and requests pending longer than archive.DeleteWithin (alert on > 0).
	ArchiveObjectsDeletedTotal prometheus.Counter
	ArchiveDeletionsOverdue    prometheus.Gauge
	// Unix time of the deletion job's last completed run, set to the start
	// time when the job starts: an alert on its age catches a pod whose runs
	// never complete (the overdue gauge is only set by a completed run).
	// 0 while the archive is off.
	ArchiveDeletionsLastRun prometheus.Gauge
	// contains filtered or unexported fields
}

Metrics is the rivolt-specific instrumentation surface. One instance is built at startup and shared with anything that emits a metric. Keep it small — adding a metric here means it shows up in Grafana, so each one should answer a real question you'd want to ask at 3am.

func New

func New() *Metrics

New constructs a Metrics with all collectors registered against a fresh registry. Pass the returned *Metrics into any package that needs to record a sample. Call Handler() to mount /metrics.

func (*Metrics) HTTPMiddleware

func (m *Metrics) HTTPMiddleware(next http.Handler) http.Handler

HTTPMiddleware records request count + latency for every handler. Uses chi's RoutePattern (e.g. "/api/state/{vehicleID}") rather than the raw URL so cardinality stays bounded as vehicles scale.

Place AFTER chi route mounting (chi resolves the pattern when the handler runs, not before) — i.e. add it inside Router.Group / on individual subrouters that have routes mounted, OR rely on chi's behaviour of populating RouteContext as routing happens. We mount it at the root using a deferred capture: the recorder reads chi.RouteContext(r.Context()).RoutePattern() AFTER ServeHTTP, by which point chi has filled it in.

func (*Metrics) Handler

func (m *Metrics) Handler() http.Handler

Handler returns the http.Handler for /metrics scoped to this Metrics' private registry. Mount under /metrics with no auth — the Prometheus scraper inside the cluster reaches it via the pod IP, and the Ingress doesn't expose it externally (see deploy/helm/rivolt/templates/ingress.yaml).

func (*Metrics) Registry

func (m *Metrics) Registry() *prometheus.Registry

Registry returns the underlying registry. Tests use this to gather expected samples without going through the HTTP handler.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL