scrape

package
v0.315.0-rc.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 14, 2026 License: Apache-2.0 Imports: 54 Imported by: 380

Documentation

Index

Constants

View Source
const UnixSocketLabel = "__unix_socket__"

UnixSocketLabel is the name of the label that holds the unix socket path to connect to for scraping. When set, the scrape client will connect via the specified Unix domain socket instead of the target's __address__.

Variables

View Source
var AlignScrapeTimestamps = true

AlignScrapeTimestamps enables the tolerance for scrape appends timestamps described above.

View Source
var ScrapeTimestampTolerance = 2 * time.Millisecond

ScrapeTimestampTolerance is the tolerance for scrape appends timestamps alignment, to enable better compression at the TSDB level. See https://github.com/prometheus/prometheus/issues/7846

Functions

func AddMetricFamilyToProtobuf added in v0.45.0

func AddMetricFamilyToProtobuf(buffer *bytes.Buffer, metricFamily *dto.MetricFamily) error

AddMetricFamilyToProtobuf appends a MetricFamily protobuf representation to a buffer. This function is intended for testing scraping by providing protobuf serialized input.

func ContextWithMetricMetadataStore added in v0.36.0

func ContextWithMetricMetadataStore(ctx context.Context, s MetricMetadataStore) context.Context

func ContextWithTarget added in v0.36.0

func ContextWithTarget(ctx context.Context, t *Target) context.Context

func MetricFamilyToProtobuf added in v0.45.0

func MetricFamilyToProtobuf(metricFamily *dto.MetricFamily) ([]byte, error)

MetricFamilyToProtobuf writes a MetricFamily into a protobuf. This function is intended for testing scraping by providing protobuf serialized input.

func PopulateDiscoveredLabels added in v0.302.0

func PopulateDiscoveredLabels(lb *labels.Builder, cfg *config.ScrapeConfig, tLabels, tgLabels model.LabelSet)

PopulateDiscoveredLabels sets base labels on lb from target and group labels and scrape configuration, before relabeling.

func PopulateLabels

func PopulateLabels(lb *labels.Builder, cfg *config.ScrapeConfig, tLabels, tgLabels model.LabelSet) (res labels.Labels, err error)

PopulateLabels builds labels from target and group labels and scrape configuration, performs defined relabeling, checks validity, and adds Prometheus standard labels such as 'instance'. A return of empty labels and nil error means the target was dropped by relabeling.

Types

type FailureLogger added in v0.302.0

type FailureLogger interface {
	slog.Handler
	io.Closer
}

FailureLogger is an interface that can be used to log all failed scrapes.

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager maintains a set of scrape pools and manages start/stop cycles when receiving new target groups from the discovery manager.

func NewManager

func NewManager(
	o *Options,
	logger *slog.Logger,
	newScrapeFailureLogger func(string) (*logging.JSONFileLogger, error),
	appendable storage.Appendable,
	appendableV2 storage.AppendableV2,
	registerer prometheus.Registerer,
) (*Manager, error)

NewManager is the Manager constructor using storage.Appendable or storage.AppendableV2.

If unsure which one to use/implement, implement AppendableV2 as it significantly simplifies implementation and allows more (passing ST, always-on metadata, exemplars per sample).

NewManager returns error if both appendable and appendableV2 are specified.

Switch to AppendableV2 is in progress (https://github.com/prometheus/prometheus/issues/17632). storage.Appendable will be removed soon (ETA: Q2 2026).

func (*Manager) ApplyConfig

func (m *Manager) ApplyConfig(cfg *config.Config) error

ApplyConfig resets the manager's target providers and job configurations as defined by the new cfg.

func (*Manager) DisableEndOfRunStalenessMarkers added in v0.308.0

func (m *Manager) DisableEndOfRunStalenessMarkers(targetSet string, targets []*Target)

DisableEndOfRunStalenessMarkers disables the end-of-run staleness markers for the provided targets in the given targetSet. When the end-of-run staleness is disabled for a target, when it goes away, there will be no staleness markers written for its series.

func (*Manager) Run

func (m *Manager) Run(tsets <-chan map[string][]*targetgroup.Group) error

Run receives and saves target set updates and triggers the scraping loops reloading. Reloading happens in the background so that it doesn't block receiving targets updates.

func (*Manager) ScrapePoolConfig added in v0.308.0

func (m *Manager) ScrapePoolConfig(scrapePool string) (*config.ScrapeConfig, error)

func (*Manager) ScrapePools added in v0.42.0

func (m *Manager) ScrapePools() []string

ScrapePools returns the list of all scrape pool names.

func (*Manager) Stop

func (m *Manager) Stop()

Stop cancels all running scrape pools and blocks until all have exited.

func (*Manager) TargetsActive

func (m *Manager) TargetsActive() map[string][]*Target

TargetsActive returns the active targets currently being scraped.

func (*Manager) TargetsAll

func (m *Manager) TargetsAll() map[string][]*Target

TargetsAll returns active and dropped targets grouped by job_name.

func (*Manager) TargetsDropped

func (m *Manager) TargetsDropped() map[string][]*Target

TargetsDropped returns the dropped targets during relabelling, subject to KeepDroppedTargets limit.

func (*Manager) TargetsDroppedCounts added in v0.47.0

func (m *Manager) TargetsDroppedCounts() map[string]int

func (*Manager) UnregisterMetrics added in v0.52.0

func (m *Manager) UnregisterMetrics()

UnregisterMetrics unregisters manager metrics.

type MetadataMetricsCollector

type MetadataMetricsCollector struct {
	CacheEntries    *prometheus.Desc
	CacheBytes      *prometheus.Desc
	TargetsGatherer TargetsGatherer
}

MetadataMetricsCollector is a Custom Collector for the metadata cache metrics.

func (*MetadataMetricsCollector) Collect

func (mc *MetadataMetricsCollector) Collect(ch chan<- prometheus.Metric)

Collect creates and sends the metrics for the metadata cache.

func (*MetadataMetricsCollector) Describe

func (mc *MetadataMetricsCollector) Describe(ch chan<- *prometheus.Desc)

Describe sends the metrics descriptions to the channel.

type MetricMetadata

type MetricMetadata struct {
	MetricFamily string
	Type         model.MetricType
	Help         string
	Unit         string
}

MetricMetadata is a piece of metadata for a metric family.

type MetricMetadataStore

type MetricMetadataStore interface {
	ListMetadata() []MetricMetadata
	GetMetadata(mfName string) (MetricMetadata, bool)
	SizeMetadata() int
	LengthMetadata() int
}

MetricMetadataStore represents a storage for metadata.

func MetricMetadataStoreFromContext added in v0.36.0

func MetricMetadataStoreFromContext(ctx context.Context) (MetricMetadataStore, bool)

type Options

type Options struct {
	// Option used by downstream scraper users like OpenTelemetry Collector
	// to help lookup metric metadata. Should be false for Prometheus.
	PassMetadataInContext bool
	// Option to enable appending of scraped Metadata to the TSDB/other appenders. Individual appenders
	// can decide what to do with metadata, but for practical purposes this flag exists so that metadata
	// can be written to the WAL and thus read for remote write.
	AppendMetadata bool
	// Option to increase the interval used by scrape manager to throttle target groups updates.
	DiscoveryReloadInterval model.Duration

	// Option to enable the ingestion of the created timestamp as a synthetic zero sample.
	// See: https://github.com/prometheus/proposals/blob/main/proposals/2023-06-13_created-timestamp.md
	//
	// NOTE: This option has no effect for AppenderV2 and will be removed with the AppenderV1
	// removal.
	EnableStartTimestampZeroIngestion bool

	// ParseST controls if ST should be parsed and appended from the scrape formats.
	// This should be by default true, but it's opt-in for OpenMetrics (OM) 1.0 reasons and might be moved
	// to OM  1.0 only flow.
	//
	// Specifically for OpenMetrics 1.0 flow, it can have some additional effects that might not be desired for non-ST users:
	//
	// * OpenMetrics 1.0 <metric>_created series will be parsed as ST instead of normal sample. Could be breaking
	// if downstream user depends on _created metric. TODO(bwplotka): Add "preserveOMLines" hidden option?
	// * Add relatively small (but still) overhead.
	// * Can yield wrong ST values in rare edge cases (unknown metadata and metric name collisions).
	//
	// This only applies to AppenderV2 flow (Prometheus default).
	// TODO: Move this option to OM1 parser and use only on OM1 flow.
	ParseST bool

	// SynthesizeST enables the synthesis of start timestamp for cumulative metrics.
	SynthesizeST bool

	// EnableTypeAndUnitLabels represents type-and-unit-labels feature flag.
	EnableTypeAndUnitLabels bool

	// EnableOpenMetrics2 represents the openmetrics2 feature flag. It enables
	// the experimental OpenMetrics 2.0 parser.
	EnableOpenMetrics2 bool

	// EnableZstdScrape represents the zstd-scrape feature flag.
	EnableZstdScrape bool

	// Optional HTTP client options to use when scraping.
	HTTPClientOptions []config_util.HTTPClientOption

	// FeatureRegistry is the registry for tracking enabled/disabled features.
	FeatureRegistry features.Collector

	// ScrapeOnShutdown enables a final scrape before the manager closes. This is useful
	// for Prometheus in agent mode or OTel's prometheusreceiver when used in serverless
	// job scenarios, allowing an extra scrape for the short-living edge cases.
	//
	// NOTE: This final scrape ignores the configured scrape interval.
	ScrapeOnShutdown bool

	// DiscoveryReloadOnStartup enables discovering targets immediately on start up as opposed
	// to waiting for the interval defined in DiscoveryReloadInterval before
	// initializing the scrape pools. Disabled by default. Useful for serverless
	// flavors of OpenTelemetry contrib's prometheusreceiver where we're
	// sensitive to start up delays.
	DiscoveryReloadOnStartup bool

	// InitialScrapeOffset applies an additional baseline delay before we begin
	// scraping targets. By default, Prometheus calculates a specific offset for
	// each target to spread the scraping load evenly across the server. Configuring
	// this option adds a fixed duration to that target-specific offset. This allows
	// tuning the initial startup delay without overriding the underlying target
	// jitter, preserving proper load balancing across the scraper pools.
	//
	// Setting this offset (e.g., to 10s) is particularly useful in Prometheus
	// agent mode and OTel's prometheusreceiver when used in serverless job
	// scenarios. It helps avoid readiness races where targets might not be fully
	// initialized immediately upon startup. It also prevents capturing
	// intermediate state (such as applications crashing shortly after booting),
	// and ensures backend rate limits don't drop valuable shutdown scrapes
	// because of an early startup scrape.
	InitialScrapeOffset time.Duration
	// contains filtered or unexported fields
}

Options are the configuration parameters to the scrape manager.

type Target

type Target struct {
	// contains filtered or unexported fields
}

Target refers to a singular HTTP or HTTPS endpoint.

func NewTarget

func NewTarget(labels labels.Labels, scrapeConfig *config.ScrapeConfig, tLabels, tgLabels model.LabelSet) *Target

NewTarget creates a reasonably configured target for querying.

func TargetFromContext added in v0.36.0

func TargetFromContext(ctx context.Context) (*Target, bool)

func TargetsFromGroup

func TargetsFromGroup(tg *targetgroup.Group, cfg *config.ScrapeConfig, targets []*Target, lb *labels.Builder) ([]*Target, []error)

TargetsFromGroup builds targets based on the given TargetGroup and config.

func (*Target) DiscoveredLabels

func (t *Target) DiscoveredLabels(lb *labels.Builder) labels.Labels

DiscoveredLabels returns a copy of the target's labels before any processing.

func (*Target) GetMetadata added in v0.50.0

func (t *Target) GetMetadata(mfName string) (MetricMetadata, bool)

GetMetadata returns type and help metadata for the given metric.

func (*Target) GetValue

func (t *Target) GetValue(name string) string

GetValue gets a label value from the entire label set.

func (*Target) Health

func (t *Target) Health() TargetHealth

Health returns the last known health state of the target.

func (*Target) Labels

func (t *Target) Labels(b *labels.Builder) labels.Labels

Labels returns a copy of the set of all public labels of the target.

func (*Target) LabelsRange added in v0.44.0

func (t *Target) LabelsRange(f func(l labels.Label))

LabelsRange calls f on each public label of the target.

func (*Target) LastError

func (t *Target) LastError() error

LastError returns the error encountered during the last scrape.

func (*Target) LastScrape

func (t *Target) LastScrape() time.Time

LastScrape returns the time of the last scrape.

func (*Target) LastScrapeDuration

func (t *Target) LastScrapeDuration() time.Duration

LastScrapeDuration returns how long the last scrape of the target took.

func (*Target) LengthMetadata added in v0.50.0

func (t *Target) LengthMetadata() int

func (*Target) ListMetadata added in v0.50.0

func (t *Target) ListMetadata() []MetricMetadata

func (*Target) Report

func (t *Target) Report(start time.Time, dur time.Duration, err error)

Report sets target data about the last scrape.

func (*Target) SetMetadataStore

func (t *Target) SetMetadataStore(s MetricMetadataStore)

func (*Target) SetScrapeConfig added in v0.302.0

func (t *Target) SetScrapeConfig(scrapeConfig *config.ScrapeConfig, tLabels, tgLabels model.LabelSet)

SetScrapeConfig sets new ScrapeConfig.

func (*Target) SizeMetadata added in v0.50.0

func (t *Target) SizeMetadata() int

func (*Target) String

func (t *Target) String() string

func (*Target) URL

func (t *Target) URL() *url.URL

URL returns a copy of the target's URL.

type TargetHealth

type TargetHealth string

TargetHealth describes the health state of a target.

const (
	HealthUnknown TargetHealth = "unknown"
	HealthGood    TargetHealth = "up"
	HealthBad     TargetHealth = "down"
)

The possible health states of a target based on the last performed scrape.

type Targets

type Targets []*Target

Targets is a sortable list of targets.

func (Targets) Len

func (ts Targets) Len() int

func (Targets) Less

func (ts Targets) Less(i, j int) bool

func (Targets) Swap

func (ts Targets) Swap(i, j int)

type TargetsGatherer added in v0.49.0

type TargetsGatherer interface {
	TargetsActive() map[string][]*Target
}

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL