Documentation
¶
Overview ¶
Package metrics owns the Prometheus registry and the /metrics HTTP handler. Consumers declare their own metrics (typically in their package's metrics.go), register them via Register, and use prometheus.Counter/Gauge/Histogram normally.
Conventions ¶
Every metric must set Namespace to metrics.Namespace ("relay") and pick a Subsystem matching the package name (e.g. "eventlog", "auth", "usage").
Naming:
- Counters end in _total (prometheus convention; client_golang enforces this)
- Gauges describe a current value (e.g. queue_depth)
- Histograms end in _seconds or _bytes
Labels: use sparingly. High-cardinality labels (e.g. per-request IDs) explode memory. Prefer separate counters over a label with O(N) values.
Index ¶
- Constants
- Variables
- func Handler() http.Handler
- func ProviderKeyDown(reason string)
- func RecordLost(kind string)
- func RecordServed(source string, status int, total, upstream time.Duration)
- func Register(collectors ...prometheus.Collector)
- func SafeLabel(s string) string
- func StatusClass(code int) string
Constants ¶
const Namespace = "relay"
Namespace is the shared Prometheus namespace for every Relay metric. Concrete metrics SHOULD set this as their Namespace and pick a Subsystem matching their package (e.g. "eventlog", "auth", "usage").
Variables ¶
var ( // RecordsLost counts background records dropped because a bounded // emitter queue was full (usage rows, payload captures). The async // contract mandates this counter: a drop you can't see is a billing // or audit hole. `kind` is the emitter: "usage" | "payload". RecordsLost = prometheus.NewCounterVec( prometheus.CounterOpts{ Namespace: Namespace, Name: "records_lost_total", Help: "Background records dropped due to a full emitter queue, by kind (usage/payload).", }, []string{"kind"}, ) // ProviderKeysDown counts the times a pooled provider key was placed // into cooldown by a failure — the leading indicator of heading // toward "no healthy keys in pool". `reason` is the failure class // (auth/rate_limit/server_error/network/local_rl). // // A counter, not a gauge: breaker state lives in shared kv, so a // per-pod gauge of "keys down right now" would be inconsistent across // the fleet. Trip counts sum cleanly. A faithful global gauge is // deferred (would need a kv scan — not lightweight). ProviderKeysDown = prometheus.NewCounterVec( prometheus.CounterOpts{ Namespace: Namespace, Name: "provider_keys_down_total", Help: "Times a pooled provider key was put into cooldown by a failure, by reason.", }, []string{"reason"}, ) )
Health signals: the two "am I about to have a problem" metrics from docs/metrics.md — silent data loss (Q3) and provider keys dying (Q4). Both are one-liner emitters the owning packages call at the relevant moment; nothing here reaches back into them.
var ( // RequestsTotal answers "how much traffic, how many errors" (Q1/Q2). // `status` is a bounded class (2xx/4xx/5xx), never the raw code. RequestsTotal = prometheus.NewCounterVec( prometheus.CounterOpts{ Namespace: Namespace, Name: "requests_total", Help: "Total requests handled, by source runner and status class (2xx/4xx/5xx).", }, []string{"source", "status"}, ) // RequestSeconds is total end-to-end time (Q2 — the "whose time is // it" numerator). Wide buckets span cached/error responses through // long generation runs. RequestSeconds = prometheus.NewHistogramVec( prometheus.HistogramOpts{ Namespace: Namespace, Name: "request_seconds", Help: "End-to-end request latency (handler entry to response closed).", Buckets: []float64{ 0.001, 0.005, 0.01, 0.05, 0.1, 0.25, 0.5, 1, 2, 5, 10, 30, 60, 120, }, }, []string{"source"}, ) // OverheadSeconds is THE wedge metric (Q1): Relay's own time, total // minus the upstream call. The performance contract puts a hard SLO // here (p99 < 10ms live). Tight sub-second buckets: 100µs → 500ms. OverheadSeconds = prometheus.NewHistogramVec( prometheus.HistogramOpts{ Namespace: Namespace, Name: "overhead_seconds", Help: "Relay overhead = total handler time minus the upstream call. Observed only when upstream was reached.", Buckets: []float64{0.0001, 0.0002, 0.0005, 0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.1, 0.25, 0.5}, }, []string{"source"}, ) )
The request-flow metrics (Rate / Errors / Duration) plus the Relay-vs- upstream split. See docs/metrics.md for the questions these answer.
Labelled by `source` (the runner: pipeline/proxy/ws/batch) — the lowest-cardinality dimension already on the lifecycle Context. Wire `shape` is deliberately NOT a label: it isn't carried on the Context and plumbing it would touch the inference handler. Add it the day the question "openai vs anthropic traffic split" is actually asked.
Functions ¶
func ProviderKeyDown ¶
func ProviderKeyDown(reason string)
ProviderKeyDown is the one-liner keypool calls when a key transitions into cooldown.
func RecordLost ¶
func RecordLost(kind string)
RecordLost is the one-liner an emitter calls at its drop site.
func RecordServed ¶
RecordServed is the one-liner the post-flight metrics observer calls once per finalized request. total is the full handler time; upstream is the upstream-call duration (zero when upstream wasn't reached, in which case the overhead split is not observed — it'd be meaningless).
func Register ¶
func Register(collectors ...prometheus.Collector)
Register adds collectors to the shared registry. Panics on duplicate registration (consistent with prometheus.MustRegister).
func SafeLabel ¶
SafeLabel returns "unknown" if s is empty, preventing empty-string Prometheus labels.
func StatusClass ¶
StatusClass returns a bounded status label ("2xx", "4xx", "5xx", etc.) for an HTTP status code. Using a class avoids high-cardinality label explosion.
Types ¶
This section is empty.