ctrlmetrics

package
v0.8.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 19, 2026 License: Apache-2.0 Imports: 25 Imported by: 0

Documentation

Overview

Package ctrlmetrics scrapes and interprets the kgateway controller's Prometheus metrics. All metric names the UI depends on are declared here.

The metrics are a kgateway-pack collector value: one Results under Key in the collected store. Checks, the scope verifier and the reported version read it from there; nothing else carries metrics.

A one-shot caller, such as a CLI built on pkg/setup, collects once:

  1. Build with setup.Build, start the cache and wait as the setup package says: Settled, then each optional watch and list-only kind OK or failed. Take one Snapshot.
  2. install.Discover(snap, reg.Detectors), keeping the installs whose Impl is "kgateway".
  3. Publish(cfg.Collected, CollectOnce(ctx, PodProxyScraper{Kube: kube}, insts, setup.MetricsMaxAge(interval))).
  4. Set cfg.Engine.Snapshots to return that snapshot, then engine.New(cfg.Engine).Compute().

Without step 3 the checks that read metrics report them not collected.

Index

Constants

View Source
const (
	XDSRejectsActive = "kgateway_envoy_xds_rejects_active"
	ResourcesManaged = "kgateway_resources_managed"
	// XDSRejectsTotal counts rejected xDS responses. On v2.3 and v2.4 it grows
	// on every re-push; on v2.5 and later it counts episodes.
	XDSRejectsTotal = "kgateway_envoy_xds_rejects_total"
	// UpdatesDropped counts dropped resources-metric updates. Above zero, the
	// kgateway_resources_* gauges are invalid until the controller restarts.
	UpdatesDropped = "kgateway_resources_updates_dropped_total"
	// XDSSnapshotResources is the controller's xDS snapshot per Gateway and
	// resource type: what it built, not what proxies accepted.
	XDSSnapshotResources = "kgateway_xds_snapshot_resources"
	// BuildInfo carries the controller version in its version label (v2.5 and later; pkg/version/metrics.go at v2.5.0).
	BuildInfo = "kgateway_build_info"
	// PerClientWithheld counts withheld clients per Gateway and state (v2.5 and later).
	PerClientWithheld = "kgateway_xds_snapshot_perclient_withheld_clients"
	// PerClientBounded counts publications a time bound released before every cluster was ready, per Gateway and mode.
	PerClientBounded = "kgateway_xds_snapshot_perclient_bounded_publishes_total"
	// PerClientDefers counts per-client snapshots built while a referenced cluster was not ready, per Gateway and reason.
	PerClientDefers = "kgateway_xds_snapshot_perclient_defers_total"
	// DeferredClients counts connected clients without a snapshot yet, per Gateway.
	DeferredClients = "kgateway_xds_snapshot_deferred_clients"
	// WarmWithheld is the PerClientWithheld state of a warm client withheld past the publish budget.
	WarmWithheld = "warm_withheld"
)

Metric families the UI depends on (verified on kgateway v2.3.x, v2.4.x and main).

View Source
const (
	CollectorName = "controller-metrics"
	ValueKey      = "results"
	Key           = CollectorName + "/" + ValueKey
)

The collected value: the collector publishes one Results under Key.

Variables

This section is empty.

Functions

func NewHTTPClient

func NewHTTPClient() *http.Client

NewHTTPClient is the in-cluster client: a 5 s timeout, no proxy from the environment and no redirects.

func Publish added in v0.6.0

func Publish(store *collect.Store, r Results)

Publish stores r under Key with no store expiry, so the next engine Compute over store reads it; staleness stays per entry. The store owns r afterwards.

func ReportedVersion added in v0.6.0

func ReportedVersion(col collect.Snapshot, id string) (v string, ok bool)

ReportedVersion is the version install id's controller reports in kgateway_build_info, from col's value under Key. ok is false when col does not tell: no OK entry, a failed scrape or replicas that disagree.

Types

type Collector

type Collector struct {
	// contains filtered or unexported fields
}

Collector scrapes installs' ready controller Pods: once with Collect, or every interval with Run, which publishes Results under Key.

func NewCollector

func NewCollector(s Scraper, opts ...CollectorOption) *Collector

NewCollector returns a Collector using s and the real clock unless an option says otherwise.

func (*Collector) Collect

func (c *Collector) Collect(ctx context.Context, in install.Install) Result

Collect scrapes all ready pods of in and aggregates them. It fails if any ready pod cannot be scraped: a partial sum of rejects must not look like zero.

func (*Collector) Run

func (c *Collector) Run(ctx context.Context, every time.Duration, installs func() []install.Install,
	publish func(key string, v any, fp uint64, ttl time.Duration),
) error

Run collects every interval until ctx is done and returns nil then. It returns an error if every <= 0. Installs are scraped concurrently, each pod under a deadline of min(every/2, 5s); a tick waits for all installs.

After each install's result Run publishes every install's latest result as one Results under ValueKey, with no expiry. The fingerprint changes, so the store notifies, when a result is the install's first, differs in content from the previous one, or follows one that had gone stale; a result that only renews CollectedAt keeps it.

type CollectorOption

type CollectorOption func(*Collector)

CollectorOption adjusts a Collector at construction.

func WithClock

func WithClock(clk clock.WithTicker) CollectorOption

WithClock makes the collector stamp results and tick with clk (tests).

func WithMaxAge

func WithMaxAge(d time.Duration) CollectorOption

WithMaxAge sets the staleness limit Run publishes as Results.MaxAge (3 × the metrics interval), so a result after a stale one notifies.

func WithResultHook

func WithResultHook(f func(id string, r Result, notified bool)) CollectorOption

WithResultHook makes Run call f after each result's notify decision (tests).

type GatewayKey

type GatewayKey struct{ Namespace, Name string }

GatewayKey identifies a Gateway in metric labels.

type HTTPScraper

type HTTPScraper struct{ Client *http.Client }

HTTPScraper scrapes the pod IP directly (in-cluster mode). A nil Client means NewHTTPClient(); a supplied one keeps its transport but never follows redirects.

func (HTTPScraper) Scrape

func (s HTTPScraper) Scrape(ctx context.Context, pod *corev1.Pod, port int32) (io.ReadCloser, error)

Scrape implements Scraper.

type ManagedKey

type ManagedKey struct{ Namespace, Resource string }

ManagedKey identifies a resources_managed series after summing over "parent".

type Metrics

type Metrics struct {
	// Rejects is the number of active xDS rejects per Gateway, summed over type_url.
	Rejects map[GatewayKey]float64
	// Managed is kgateway_resources_managed summed over the parent label.
	Managed map[ManagedKey]float64
	// ManagedPresent is true if the family appeared in the scrape. An absent
	// family with resources in scope suggests a rename upstream.
	ManagedPresent bool
	// PerPod holds each controller Pod's samples, sorted by Pod name.
	PerPod []PodMetrics
	// Snapshot is XDSSnapshotResources per Gateway; never a finding.
	Snapshot map[GatewayKey]SnapshotResources
}

Metrics is the interpreted subset of one or more controller scrapes.

func Aggregate

func Aggregate(replicas []Metrics) Metrics

Aggregate combines per-replica scrapes. Rejects are summed, because each proxy's xDS stream is served by one replica. Managed counts are maxed, because every replica computes the full set (D-012), and so is each snapshot count. PerPod is concatenated and sorted by Pod name, then UID.

func Parse

func Parse(r io.Reader) (Metrics, error)

Parse interprets a text-format /metrics payload. A gauge that has never been set has no series, so an absent family means zero, not an error.

func (Metrics) BuildVersion

func (m Metrics) BuildVersion() (v string, agreed bool)

BuildVersion is the build_info version every reporting Pod shares; "" when no Pod reports one. agreed is false when two Pods disagree, as during a rollout.

type ModeKey

type ModeKey struct {
	Gateway GatewayKey
	Mode    string
}

ModeKey identifies a bounded-publish series: one Gateway and one mode.

type PodBounded

type PodBounded map[string]map[ModeKey]float64

PodBounded is the last seen bounded_publishes_total per Pod UID.

func BoundedGrowth

func BoundedGrowth(prev PodBounded, cur []PodMetrics) (growth, next PodBounded)

BoundedGrowth is TotalGrowth for bounded_publishes_total, kept per Pod UID so callers can name the Pods: a first sight is a baseline and a lower value a reset.

type PodMetrics

type PodMetrics struct {
	UID, Name string
	// Active and Total are rejects_active and rejects_total per Gateway and type.
	Active, Total map[TypeKey]float64
	// Dropped is updates_dropped_total; DroppedPresent is false when the
	// family is absent, which never means zero.
	Dropped        float64
	DroppedPresent bool
	// BuildVersion is build_info's version label, such as "v2.5.0"; "" when absent.
	BuildVersion string
	// PerClientPresent is true when the scrape has a per-client family or
	// build_info (both kgateway 2.5). Withheld and bounded series exist only
	// while a client is held, so their absence alone proves nothing.
	PerClientPresent bool
	// Withheld is the warm_withheld client count per Gateway; nil when none.
	Withheld map[GatewayKey]float64
	// Bounded is bounded_publishes_total per Gateway and mode; nil when none.
	Bounded map[ModeKey]float64
}

PodMetrics is what one controller Pod reports. Parse leaves UID and Name empty; the collector fills them from the scraped Pod.

type PodProxyScraper

type PodProxyScraper struct{ Kube kubernetes.Interface }

PodProxyScraper scrapes through the API server's pods/proxy subresource (local mode; uses the user's kubeconfig rights). It sends one GET over the REST client's HTTP client without redirects; Request.Stream is not used, as it reads error bodies unbounded and retries on Retry-After.

func (PodProxyScraper) Scrape

func (s PodProxyScraper) Scrape(ctx context.Context, pod *corev1.Pod, port int32) (io.ReadCloser, error)

Scrape implements Scraper.

type PodTotals

type PodTotals map[string]map[TypeKey]float64

PodTotals is the last seen rejects_total per Pod UID.

func TotalGrowth

func TotalGrowth(prev PodTotals, cur []PodMetrics) (growth map[TypeKey]float64, next PodTotals)

TotalGrowth returns the rejects_total growth since prev, summed over Pods per type, and the totals to pass next time. Growth is taken per Pod UID: a Pod seen for the first time sets a baseline, a counter that went down is a reset, and neither counts as growth, so a restarted replica is never negative. A UID absent from cur is forgotten, so callers skip Err results (an empty cur forgets every UID). A series new under a seen UID counts its full value.

type Result

type Result struct {
	Metrics     Metrics
	CollectedAt time.Time
	// Err is non-empty when the metrics could not be collected completely.
	Err string
}

Result is the outcome of collecting one install's metrics.

func Lookup added in v0.6.0

func Lookup(v collect.Value, id string) (Result, time.Duration, bool)

Lookup returns install id's entry in v, a value collected under Key, and the max age. It is false unless v is OK and holds an entry for id.

func (Result) SameContent

func (r Result) SameContent(o Result) bool

SameContent reports whether r and o carry the same Err and Metrics. CollectedAt is not content: freshness is a deadline, not a change (D-019).

func (Result) Stale

func (r Result) Stale(now time.Time, maxAge time.Duration) bool

Stale reports whether r was collected more than maxAge before now. A zero maxAge never makes a result stale. Consumers use 3 × the metrics interval, so a stalled collector cannot keep an old zero-reject result green.

type Results added in v0.6.0

type Results struct {
	ByInstall map[string]Result
	// MaxAge is 3 × the collection interval; zero never makes an entry stale.
	MaxAge time.Duration
}

Results is one scrape per install ID, with the age beyond which an entry is stale. Entries age on their own: each Result keeps its CollectedAt.

func CollectOnce added in v0.6.0

func CollectOnce(ctx context.Context, s Scraper, insts []install.Install, maxAge time.Duration) Results

CollectOnce scrapes each install's ready controller Pods once, concurrently, and returns the results by install ID with maxAge. Pass only the installs of the kgateway pack (install.Install.Impl == "kgateway").

func (Results) Fingerprint added in v0.6.0

func (r Results) Fingerprint() uint64

Fingerprint identifies r's content: each entry's install ID, Err and metrics, and MaxAge. CollectedAt is left out, as in Result.SameContent.

func (Results) StalePrint added in v0.6.0

func (r Results) StalePrint(now time.Time) uint64

StalePrint implements collect.Aging: it covers the installs whose entry is stale at now, so an entry turning stale changes the engine's source print.

func (Results) StampFor added in v0.6.0

func (r Results) StampFor(install string) (time.Time, bool)

StampFor implements collect.Stamped: install's entry's CollectedAt, which checks 001 and 024 stamp their results with.

type Scraper

type Scraper interface {
	Scrape(ctx context.Context, pod *corev1.Pod, port int32) (io.ReadCloser, error)
}

Scraper fetches the /metrics payload of one controller pod.

func ScraperFor added in v0.6.0

func ScraperFor(kube kubernetes.Interface, inCluster bool) Scraper

ScraperFor is the controller metrics scraper: the Pod IP in-cluster, pods/proxy with the user's rights otherwise.

type SnapshotResources

type SnapshotResources struct {
	Listeners, Routes, Clusters, Endpoints, Secrets int
}

SnapshotResources counts the resources of one Gateway's xDS snapshot.

type TypeKey

type TypeKey struct {
	Gateway GatewayKey
	TypeURL string
}

TypeKey identifies an xDS reject series: one Gateway and one resource type.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL