metricspush

package
v0.17.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 4, 2026 License: Apache-2.0 Imports: 30 Imported by: 0

Documentation

Overview

Package metricspush lets a workload export metrics by pushing them, for workloads that have nothing a scraper could reach, such as a background worker with no HTTP listener.

A push travels in two hops. The workload sends it to a relay on its own node's bridge router, which identifies the sandbox the same way the token server does and attaches a workload token minted for it. The relay hands the push to the coordinator, which verifies that token, derives every miren_* label from the sandbox it names, and forwards the samples to the managed-metrics vmagent. Nothing the workload sends can choose which app its samples are filed under.

Index

Constants

View Source
const (
	// Audience scopes the token a relay attaches to a push. A sandbox token
	// minted for anything else, including the default "miren" audience that
	// every sandbox has on disk, does not verify here.
	Audience = "miren-metrics-push"

	// TokenHeader carries the sandbox's workload token from relay to
	// coordinator. Not Authorization, for the same reason runnertelemetry
	// avoids it: on a cloud-registered cluster the listener's authenticator
	// treats an Authorization header as a cloud JWT and fails hard on anything
	// else, which would reject a relay's otherwise-valid mTLS request.
	TokenHeader = "Miren-Workload-Token"

	// ScopeHeader, FormatHeader and GroupingHeader describe the push on the
	// relay-to-coordinator hop. The workload never sets them; the relay derives
	// them from the path the workload pushed to.
	ScopeHeader    = "Miren-Push-Scope"
	FormatHeader   = "Miren-Push-Format"
	GroupingHeader = "Miren-Push-Grouping"

	// IngestPath is where a coordinator accepts pushes from relays.
	IngestPath = "/_push/metrics"
)
View Source
const (
	// RelayBasePath is where a workload finds the relay on its bridge router.
	// Each scope is a complete base for the client that uses it: a Pushgateway
	// client is pointed at <base>/<scope>, an OTLP exporter at
	// <base>/<scope>/otlp.
	RelayBasePath = "/v1/metrics"
)

Variables

View Source
var IngestPattern = http.MethodPost + " " + IngestPath

IngestPattern is the ServeMux pattern the coordinator mounts. It pins one method and one path, so the route cannot be used to reach anything else vmagent serves.

View Source
var StatusPattern = http.MethodGet + " " + IngestPath

StatusPattern answers whether this cluster accepts pushes at all, so a runner can decide whether to advertise push to the sandboxes it starts.

Functions

This section is empty.

Types

type Backend

type Backend struct {
	// ImportURL is the managed-metrics vmagent, normally on loopback.
	ImportURL string

	// Transport carries the credential vmagent requires at ImportURL. Nil
	// uses http.DefaultTransport.
	Transport http.RoundTripper

	// ClusterID is stamped as miren_cluster, the same value the scrape path
	// uses.
	ClusterID string

	// Resolver looks up the sandbox a token names.
	Resolver SandboxResolver
	// contains filtered or unexported fields
}

Backend is everything a push needs that only exists once managed metrics is running.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client forwards pushes from a runner's relay to the coordinator's ingest.

It is its own client rather than runnertelemetry's, even though both ride the runner's certificate: that one stamps the runner's system token on every request under the same header this one fills with a sandbox's token.

func NewClient

func NewClient(cfg ClientConfig) (*Client, error)

func (*Client) Available

func (c *Client) Available(ctx context.Context) bool

Available asks the coordinator whether the cluster accepts pushes, reusing the answer for a minute.

func (*Client) Close

func (c *Client) Close() error

Close tears down the underlying QUIC connections.

func (*Client) Push

func (c *Client) Push(ctx context.Context, p Push) error

type ClientConfig

type ClientConfig struct {
	// CoordinatorAddress is the coordinator's API host:port.
	CoordinatorAddress string

	// ClientCertPEM and ClientKeyPEM are the runner's certificate from Join.
	// The listener demands a client identity before it routes anything, and
	// the runner's is the one it has. It opens the door; the sandbox token
	// inside the request is what the ingest actually trusts.
	ClientCertPEM []byte
	ClientKeyPEM  []byte

	// CACertPEM verifies the coordinator.
	CACertPEM []byte

	// Timeout bounds a single push.
	Timeout time.Duration
}

ClientConfig describes how a runner's relay reaches the coordinator.

type EntityResolver

type EntityResolver struct {
	// contains filtered or unexported fields
}

EntityResolver resolves sandboxes from the entity store, caching each for a few minutes so a workload pushing every few seconds costs a handful of lookups per sandbox rather than four per push.

func (*EntityResolver) Resolve

func (r *EntityResolver) Resolve(ctx context.Context, sandboxID string) (Sandbox, error)

type Error

type Error struct {
	Status  int
	Message string
}

Error is a push failure with the HTTP status it should surface as, so a relay can hand the workload the coordinator's verdict rather than a generic failure.

func (*Error) Error

func (e *Error) Error() string

type Format

type Format string

Format is the wire format of a push body.

const (
	// FormatPrometheus is the Prometheus exposition format a Pushgateway
	// accepts, as text or as delimited protobuf.
	FormatPrometheus Format = "prometheus"

	// FormatOTLP is an OTLP/HTTP metrics export request in protobuf.
	FormatOTLP Format = "otlp"
)

type Ingest

type Ingest struct {
	// contains filtered or unexported fields
}

Ingest is the coordinator's half of the push path. It is where a push is trusted or refused, so it assumes nothing about how the push arrived: the relay in this process and a relay on a distributed runner reach the same checks.

func NewIngest

func NewIngest(log *slog.Logger, verifier Verifier, enabled bool) *Ingest

func (*Ingest) Arm

func (i *Ingest) Arm(backend Backend)

Arm points the ingest at a running vmagent. It arrives after the ingest is mounted, because the API listener has to be up before the entity store and managed metrics can start behind it. Until then a push is refused as unavailable rather than dropped.

func (*Ingest) Available

func (i *Ingest) Available(context.Context) bool

Available reports whether this cluster accepts pushes: it has a remote-write destination configured, and managed metrics has not failed to start.

func (*Ingest) Disarm

func (i *Ingest) Disarm()

Disarm stops forwarding, for when managed metrics shuts down.

func (*Ingest) Fail

func (i *Ingest) Fail()

Fail stops advertising push, for when managed metrics could not start and will not be arming this ingest. Without it, every sandbox started afterward would be told push works and have each push refused.

func (*Ingest) Handler

func (i *Ingest) Handler() http.Handler

Handler serves IngestPath for relays on the coordinator's API listener.

Reaching it takes an authenticated caller, which in practice is a runner's certificate. That is not what authorizes the push: a certificate identity bypasses authorization entirely, so the sandbox token in TokenHeader is what Push checks, and it is the token alone that decides whose labels the samples get.

func (*Ingest) Push

func (i *Ingest) Push(ctx context.Context, p Push) error

Push checks p and forwards it to vmagent labeled for the sandbox its token names.

func (*Ingest) StatusHandler

func (i *Ingest) StatusHandler() http.Handler

StatusHandler serves StatusPattern: 204 when the cluster accepts pushes, 503 when it has no remote-write destination.

type Push

type Push struct {
	// Token is the sandbox's workload token, minted for Audience.
	Token string

	Scope  Scope
	Format Format

	// Grouping is the Pushgateway grouping key from the push path, job
	// included. Every sample in the push gets these labels. Prometheus only.
	Grouping map[string]string

	// ContentType and ContentEncoding are the workload's, passed through.
	ContentType     string
	ContentEncoding string

	Body []byte
}

Push is one batch of samples on its way from a relay to the coordinator.

type Pusher

type Pusher interface {
	Push(ctx context.Context, p Push) error

	// Available reports whether the cluster accepts pushes at all. Push is
	// only advertised to a sandbox when it does, so an app that finds the
	// push URLs in its environment can rely on them.
	Available(ctx context.Context) bool
}

Pusher delivers a push to the coordinator. The coordinator's own relay hands pushes straight to its *Ingest; a runner's relay sends them over the network with a *Client.

type Relay

type Relay struct {
	// contains filtered or unexported fields
}

Relay is the workload-facing half of the push path, served on a node's bridge router beside the token server.

It asks the workload for nothing that changes: the per-sandbox secret it authenticates with is set once in the sandbox's environment, so a Pushgateway client or OTel exporter can carry it as a static header. The token the coordinator needs rotates hourly, and minting it is the relay's job rather than the workload's.

func NewRelay

func NewRelay(log *slog.Logger, auth SandboxAuthenticator, issuer TokenIssuer, pusher Pusher) *Relay

func (*Relay) Register

func (r *Relay) Register(mux *http.ServeMux)

Register mounts the relay's routes on mux.

The Pushgateway routes follow its API exactly, so an unmodified client works against a scope's base URL. PUT and POST differ on a Pushgateway only in how they replace what it has stored; nothing is stored here, so both forward. DELETE has nothing to remove and is accepted so a client's cleanup call does not fail.

type Sandbox

type Sandbox struct {
	ID      string
	App     string
	Version string
	Service string
	Runner  string
}

Sandbox is what the runtime knows about a pushing sandbox, and so what its samples are labeled with.

type SandboxAuthenticator

type SandboxAuthenticator func(remoteHost, secret string) (sandboxID, app string, ok bool)

SandboxAuthenticator identifies the sandbox behind a request from the caller's source address and the secret it presented. It is the token server's check, shared so both endpoints stand on the same ground.

type SandboxResolver

type SandboxResolver interface {
	Resolve(ctx context.Context, sandboxID string) (Sandbox, error)
}

SandboxResolver looks up the sandbox a verified token names.

type Scope

type Scope string

Scope says which series a push lands on.

const (
	// ScopeSandbox files samples under the sandbox that pushed them, with the
	// same labels a scrape of that sandbox would carry. It is correct for every
	// kind of metric.
	ScopeSandbox Scope = "sandbox"

	// ScopeApp files samples under the app and service alone, with no
	// per-sandbox labels, so every writer lands on the same series. It is for
	// values every writer agrees on, like a row count read from a shared
	// database: any number of replicas can push one and sum() still reads it
	// once. Counters and histograms are refused, because interleaving
	// cumulative values from different processes on one series is garbage.
	ScopeApp Scope = "app"
)

type TokenIssuer

type TokenIssuer interface {
	IssueTokenWithOptions(app, sandboxID string, opts workloadidentity.TokenOptions) (string, error)
}

TokenIssuer mints the token the relay attaches for a sandbox. On a runner it is the remote issuer, which has the coordinator mint the token and refuses a sandbox that is not scheduled to this runner.

type Verifier

type Verifier interface {
	VerifyToken(token, audience string) (*workloadidentity.WorkloadClaims, error)
}

Verifier checks a sandbox's workload token. It is satisfied by *workloadidentity.Issuer, which verifies in-process with the keys it signs with.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL