Documentation
¶
Overview ¶
Package metrics provides Prometheus metrics collection for the proxy.
Index ¶
- Constants
- Variables
- func DecrementActiveRequests()
- func Handler() http.Handler
- func IncrementActiveRequests()
- func RecordArtifactCacheHit(ecosystem string)
- func RecordArtifactCacheMiss(ecosystem string)
- func RecordArtifactEvicted(reason, ecosystem string)
- func RecordCircuitBreakerTrip(registry string)
- func RecordHealthProbeFailure(step string, detail FailureDetail)
- func RecordIntegrityFailure(ecosystem string, detail FailureDetail)
- func RecordMetadataCacheHit(ecosystem string, bytes int)
- func RecordMetadataCacheMiss(ecosystem string)
- func RecordMetadataEvicted(ecosystem string)
- func RecordRequest(ecosystem string, status int, duration time.Duration)
- func RecordResponse(ecosystem, client string, bytes int64)
- func RecordScanError(ecosystem, scannerName, errorType string, detail FailureDetail)
- func RecordScanResult(ecosystem, scannerName string, allowed bool, duration time.Duration)
- func RecordStorageError(operation string, detail FailureDetail)
- func RecordStorageOperation(operation string, duration time.Duration)
- func RecordUpstreamError(ecosystem, errorType string, detail FailureDetail)
- func RecordUpstreamFetch(ecosystem string, duration time.Duration)
- func RecordUpstreamRateLimitHold(host string)
- func UpdateCacheStats(sizeBytes, artifactCount int64)
- func UpdateCircuitBreakerState(registry string, state int)
- func UpdateEcosystemStats(stats []EcosystemStats)
- func UpdateMetadataCacheStats(stats []MetadataCacheStats)
- type EcosystemStats
- type Failure
- type FailureDetail
- type MetadataCacheStats
- type Sample
- type Snapshot
Constants ¶
const ( CacheKindArtifact = "artifact" CacheKindMetadata = "metadata" )
Values of the kind label on proxy_cache_hits_total and proxy_cache_misses_total. Artifacts and metadata are separate caches with very different costs, so a hit on one says nothing about the other.
Variables ¶
var ( // Request metrics RequestsTotal = prometheus.NewCounterVec( prometheus.CounterOpts{ Name: "proxy_requests_total", Help: "Total number of requests by ecosystem and status", }, []string{"ecosystem", "status"}, ) RequestDuration = prometheus.NewHistogramVec( prometheus.HistogramOpts{ Name: "proxy_request_duration_seconds", Help: "Request duration in seconds", Buckets: prometheus.DefBuckets, }, []string{"ecosystem", "status"}, ) // Cache metrics CacheHits = prometheus.NewCounterVec( prometheus.CounterOpts{ Name: "proxy_cache_hits_total", Help: "Total number of cache hits by ecosystem and kind (artifact or metadata)", }, []string{"ecosystem", "kind"}, ) CacheMisses = prometheus.NewCounterVec( prometheus.CounterOpts{ Name: "proxy_cache_misses_total", Help: "Total number of cache misses by ecosystem and kind (artifact or metadata)", }, []string{"ecosystem", "kind"}, ) MetadataServedBytes = prometheus.NewCounterVec( prometheus.CounterOpts{ Name: "proxy_metadata_cache_served_bytes_total", Help: "Total metadata bytes served from the metadata cache, by ecosystem", }, []string{"ecosystem"}, ) CacheSize = prometheus.NewGauge( prometheus.GaugeOpts{ Name: "proxy_cache_size_bytes", Help: "Total size of cached artifacts in bytes", }, ) CachedArtifacts = prometheus.NewGauge( prometheus.GaugeOpts{ Name: "proxy_cached_artifacts_total", Help: "Total number of cached artifacts", }, ) ArtifactsEvicted = prometheus.NewCounterVec( prometheus.CounterOpts{ Name: "proxy_artifacts_evicted_total", Help: "Total number of cached artifacts evicted, by reason (lru or retention) and ecosystem", }, []string{"reason", "ecosystem"}, ) MetadataEvicted = prometheus.NewCounterVec( prometheus.CounterOpts{ Name: "proxy_metadata_evicted_total", Help: "Total number of cached metadata documents evicted by the size limit, by ecosystem", }, []string{"ecosystem"}, ) // Upstream metrics UpstreamFetchDuration = prometheus.NewHistogramVec( prometheus.HistogramOpts{ Name: "proxy_upstream_fetch_duration_seconds", Help: "Upstream fetch duration in seconds", Buckets: []float64{.1, .25, .5, 1, 2.5, 5, 10, 30}, }, []string{"ecosystem"}, ) UpstreamErrors = prometheus.NewCounterVec( prometheus.CounterOpts{ Name: "proxy_upstream_errors_total", Help: "Total number of upstream fetch errors by type", }, []string{"ecosystem", "error_type"}, ) // Circuit breaker metrics CircuitBreakerState = prometheus.NewGaugeVec( prometheus.GaugeOpts{ Name: "proxy_circuit_breaker_state", Help: "Circuit breaker state (0=closed, 1=half-open, 2=open)", }, []string{"registry"}, ) CircuitBreakerTrips = prometheus.NewCounterVec( prometheus.CounterOpts{ Name: "proxy_circuit_breaker_trips_total", Help: "Total number of circuit breaker trips", }, []string{"registry"}, ) UpstreamRateLimitHolds = prometheus.NewCounterVec( prometheus.CounterOpts{ Name: "proxy_upstream_rate_limit_holds_total", Help: "Holds placed on an upstream host after it rate limited the proxy", }, []string{"host"}, ) // Storage metrics StorageOperationDuration = prometheus.NewHistogramVec( prometheus.HistogramOpts{ Name: "proxy_storage_operation_duration_seconds", Help: "Storage operation duration in seconds", Buckets: []float64{.001, .005, .01, .025, .05, .1, .25, .5, 1}, }, []string{"operation"}, ) StorageErrors = prometheus.NewCounterVec( prometheus.CounterOpts{ Name: "proxy_storage_errors_total", Help: "Total number of storage errors by operation", }, []string{"operation"}, ) // Active requests ActiveRequests = prometheus.NewGauge( prometheus.GaugeOpts{ Name: "proxy_active_requests", Help: "Number of currently active requests", }, ) IntegrityFailures = prometheus.NewCounterVec( prometheus.CounterOpts{ Name: "proxy_integrity_failures_total", Help: "Cached artifacts that failed hash verification on read", }, []string{"ecosystem"}, ) HealthProbeFailures = prometheus.NewCounterVec( prometheus.CounterOpts{ Name: "proxy_health_probe_failures_total", Help: "Total number of storage health probe failures, by step (write|size|read|verify|delete).", }, []string{"step"}, ) // Per-ecosystem gauges, derived from the database rather than incremented // in the request path, and refreshed on the same tick as the cache gauges // above. EcosystemDownloadedBytes = prometheus.NewGaugeVec( prometheus.GaugeOpts{ Name: "proxy_ecosystem_downloaded_bytes", Help: "Accumulated bytes served from cache per ecosystem (cache hits x artifact size)", }, []string{"ecosystem"}, ) EcosystemDownloads = prometheus.NewGaugeVec( prometheus.GaugeOpts{ Name: "proxy_ecosystem_artifact_downloads", Help: "Accumulated artifact downloads served from cache per ecosystem", }, []string{"ecosystem"}, ) EcosystemCacheSize = prometheus.NewGaugeVec( prometheus.GaugeOpts{ Name: "proxy_ecosystem_cache_size_bytes", Help: "Size of cached artifacts per ecosystem in bytes", }, []string{"ecosystem"}, ) EcosystemCachedArtifacts = prometheus.NewGaugeVec( prometheus.GaugeOpts{ Name: "proxy_ecosystem_cached_artifacts", Help: "Number of cached artifacts per ecosystem", }, []string{"ecosystem"}, ) EcosystemPackages = prometheus.NewGaugeVec( prometheus.GaugeOpts{ Name: "proxy_ecosystem_packages", Help: "Number of known packages per ecosystem", }, []string{"ecosystem"}, ) EcosystemVersions = prometheus.NewGaugeVec( prometheus.GaugeOpts{ Name: "proxy_ecosystem_versions", Help: "Number of known package versions per ecosystem", }, []string{"ecosystem"}, ) // Metadata cache gauges, derived from the metadata_cache table on the // same tick as the per-ecosystem gauges above. MetadataCacheSize = prometheus.NewGaugeVec( prometheus.GaugeOpts{ Name: "proxy_metadata_cache_size_bytes", Help: "Size of cached metadata documents per ecosystem in bytes", }, []string{"ecosystem"}, ) MetadataCacheEntries = prometheus.NewGaugeVec( prometheus.GaugeOpts{ Name: "proxy_metadata_cache_entries", Help: "Number of cached metadata documents per ecosystem", }, []string{"ecosystem"}, ) ResponseBytes = prometheus.NewCounterVec( prometheus.CounterOpts{ Name: "proxy_response_bytes_total", Help: "Total response body bytes written to clients, by ecosystem", }, []string{"ecosystem"}, ) ClientRequests = prometheus.NewCounterVec( prometheus.CounterOpts{ Name: "proxy_client_requests_total", Help: "Total requests by client tool, as identified from the User-Agent", }, []string{"client"}, ) ClientResponseBytes = prometheus.NewCounterVec( prometheus.CounterOpts{ Name: "proxy_client_response_bytes_total", Help: "Total response body bytes written to clients, by client tool", }, []string{"client"}, ) // Scanning metrics ScanDuration = prometheus.NewHistogramVec( prometheus.HistogramOpts{ Name: "proxy_scan_duration_seconds", Help: "Pre-cache artifact scan duration in seconds, by ecosystem and scanner", Buckets: prometheus.DefBuckets, }, []string{"ecosystem", "scanner"}, ) ScanBlocked = prometheus.NewCounterVec( prometheus.CounterOpts{ Name: "proxy_scan_blocked_total", Help: "Total number of artifacts blocked by a pre-cache scan, by ecosystem and scanner", }, []string{"ecosystem", "scanner"}, ) ScanErrors = prometheus.NewCounterVec( prometheus.CounterOpts{ Name: "proxy_scan_errors_total", Help: "Total number of pre-cache scan errors, by ecosystem, scanner, and error type", }, []string{"ecosystem", "scanner", "error_type"}, ) )
Functions ¶
func DecrementActiveRequests ¶
func DecrementActiveRequests()
DecrementActiveRequests decrements the active request counter.
func IncrementActiveRequests ¶
func IncrementActiveRequests()
IncrementActiveRequests increments the active request counter.
func RecordArtifactCacheHit ¶ added in v0.10.0
func RecordArtifactCacheHit(ecosystem string)
RecordArtifactCacheHit counts an artifact served from storage.
func RecordArtifactCacheMiss ¶ added in v0.10.0
func RecordArtifactCacheMiss(ecosystem string)
RecordArtifactCacheMiss counts an artifact that had to be fetched upstream.
func RecordArtifactEvicted ¶ added in v0.10.0
func RecordArtifactEvicted(reason, ecosystem string)
RecordArtifactEvicted counts one evicted artifact. ecosystem is the package record's value, normalized like the cache hit counter; an artifact without a package record counts as "other".
func RecordCircuitBreakerTrip ¶
func RecordCircuitBreakerTrip(registry string)
RecordCircuitBreakerTrip increments circuit breaker trip counter.
func RecordHealthProbeFailure ¶ added in v0.4.0
func RecordHealthProbeFailure(step string, detail FailureDetail)
RecordHealthProbeFailure increments the health probe failure counter and keeps the detail for /ui/analytics. step is one of: "write", "size", "read", "verify", "delete".
func RecordIntegrityFailure ¶ added in v0.3.2
func RecordIntegrityFailure(ecosystem string, detail FailureDetail)
RecordIntegrityFailure increments the integrity failure counter and keeps the detail for /ui/analytics.
func RecordMetadataCacheHit ¶ added in v0.10.0
RecordMetadataCacheHit counts a metadata document served from the metadata cache, along with the bytes it served.
func RecordMetadataCacheMiss ¶ added in v0.10.0
func RecordMetadataCacheMiss(ecosystem string)
RecordMetadataCacheMiss counts a metadata lookup the cache could not serve.
func RecordMetadataEvicted ¶ added in v0.10.0
func RecordMetadataEvicted(ecosystem string)
RecordMetadataEvicted counts one metadata document evicted by the size limit. ecosystem is the metadata cache's, normalized like the cache hit counter.
func RecordRequest ¶
RecordRequest tracks request metrics with timing.
func RecordResponse ¶ added in v0.9.1
RecordResponse tracks what a client actually downloaded.
Distinct from proxy_ecosystem_downloaded_bytes: that gauge is derived from the database and counts cache hits multiplied by artifact size, while this counts body bytes as they are written, including metadata responses and cache misses.
client must come from a closed set -- a User-Agent is attacker-controlled, so passing it through raw would mint a time series per request.
func RecordScanError ¶ added in v0.8.0
func RecordScanError(ecosystem, scannerName, errorType string, detail FailureDetail)
RecordScanError increments the scan error counter. errorType is one of: "error" (scanner call failed), "timeout", "cancelled". The detail is kept for /ui/analytics.
func RecordScanResult ¶ added in v0.8.0
RecordScanResult tracks a completed pre-cache scan call.
func RecordStorageError ¶
func RecordStorageError(operation string, detail FailureDetail)
RecordStorageError increments storage error counter and keeps the detail for /ui/analytics.
func RecordStorageOperation ¶
RecordStorageOperation tracks storage operation duration.
func RecordUpstreamError ¶
func RecordUpstreamError(ecosystem, errorType string, detail FailureDetail)
RecordUpstreamError increments upstream error counter and keeps the detail for /ui/analytics.
func RecordUpstreamFetch ¶
RecordUpstreamFetch tracks upstream fetch duration.
func RecordUpstreamRateLimitHold ¶ added in v0.10.0
func RecordUpstreamRateLimitHold(host string)
RecordUpstreamRateLimitHold counts a hold placed on host.
func UpdateCacheStats ¶
func UpdateCacheStats(sizeBytes, artifactCount int64)
UpdateCacheStats updates cache size and artifact count gauges.
func UpdateCircuitBreakerState ¶
UpdateCircuitBreakerState updates circuit breaker state gauge. state: 0=closed, 1=half-open, 2=open
func UpdateEcosystemStats ¶ added in v0.9.1
func UpdateEcosystemStats(stats []EcosystemStats)
UpdateEcosystemStats republishes the per-ecosystem gauges from a fresh snapshot.
Rows are summed by label before anything is published, because normalizing collapses aliases: a database holding both "gem" and "rubygems" rows -- the proxy writes the former, git-pkgs the latter -- arrives as two rows belonging to one label, and publishing them one at a time would leave only the last.
The vectors are not Reset() first. Reset followed by a repopulating loop leaves a window in which a scrape sees the families empty or half filled, which renders as a spurious gap on any panel built from them. Each series is Set instead, and only labels that have actually disappeared are deleted.
func UpdateMetadataCacheStats ¶ added in v0.10.0
func UpdateMetadataCacheStats(stats []MetadataCacheStats)
UpdateMetadataCacheStats republishes the metadata cache gauges from a fresh snapshot. Like UpdateEcosystemStats it sums rows by normalized label, sets each series in place and deletes only labels that have disappeared.
Types ¶
type EcosystemStats ¶ added in v0.9.1
type EcosystemStats struct {
Ecosystem string
Packages int64
Versions int64
Artifacts int64
CacheSize int64
Downloads int64
DownloadedBytes int64
}
EcosystemStats is one ecosystem's row of the snapshot published as gauges.
type Failure ¶ added in v0.10.0
type Failure struct {
At time.Time
PURL string
Filename string
URL string
Path string
Error string
}
Failure is one recorded failure, with credentials already stripped from the URL and the error text.
type FailureDetail ¶ added in v0.10.0
FailureDetail is what a failure counter's call site knows beyond the counter's labels: which artifact failed, where it came from or was going, and why. Every field is optional.
None of it becomes a metric label. Package names and URLs are unbounded, so they are kept in a small in-memory store for /ui/analytics instead.
type MetadataCacheStats ¶ added in v0.10.0
MetadataCacheStats is one ecosystem's share of the metadata cache.
type Sample ¶ added in v0.9.1
Sample is one time series read out of the registry at a point in time.
Counters and gauges carry their value in Value. Histograms carry their observation count and total in Count and Sum, and leave Value at zero — there is no single "value" for a histogram, and the UI shows a mean derived from Sum/Count rather than pretending otherwise.
type Snapshot ¶ added in v0.9.1
type Snapshot struct {
// contains filtered or unexported fields
}
Snapshot is the value of every registered proxy metric at one instant.
These are process-lifetime figures held in the Prometheus registry, not database state: they start at zero when the proxy starts and are lost on restart. Anything that needs to survive a restart, or needs history, has to come from a Prometheus server scraping /metrics.
func Gather ¶ added in v0.9.1
Gather reads the default Prometheus registry. It is the same data /metrics serves, shaped for rendering rather than for scraping.
func GatherFrom ¶ added in v0.9.1
func GatherFrom(g prometheus.Gatherer) (*Snapshot, error)
GatherFrom reads an explicit gatherer, so tests can supply their own registry.
func (*Snapshot) Count ¶ added in v0.9.1
Count adds up the observation counts of a histogram across every series.
func (*Snapshot) Mean ¶ added in v0.9.1
Mean returns the average observation of a histogram across every series, or 0 when nothing has been observed.
func (*Snapshot) Samples ¶ added in v0.9.1
Samples returns every series of a metric, or nil when it has never been observed. Series are ordered by their label values so that rendering is stable between scrapes.