metrics

package
v0.0.322 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 31, 2026 License: Apache-2.0 Imports: 3 Imported by: 0

Documentation

Overview

Package metrics defines the storage server's Prometheus metrics and registers them with the k8s.io/component-base legacyregistry, the same registry the generic apiserver's built-in "/metrics" endpoint already serves (see k8s.io/apiserver/pkg/server, EnableMetrics).

This currently covers Phase 0 of the storage-locking investigation (docs/features/storage-lock-pool-metrics.md): lock-hold and connection-pool-wait durations, so that contention which used to be visible only via Debug-level log lines becomes a real, always-on, percentile-queryable metric.

Index

Constants

View Source
const (
	OutcomeAcquired = "acquired"
	OutcomeTimeout  = "timeout"
)

Outcome label values for LockWaitDuration / PoolWaitDuration: whether the acquisition succeeded or hit its timeout.

View Source
const (
	PriorityHigh = "high"
	PriorityLow  = "low"
)

Priority label values for the single-writer priority queue metrics: which lane a commitJob traveled through.

View Source
const (
	CommitOutcomeCommitted = "committed"
	CommitOutcomeConflict  = "conflict"
	CommitOutcomeError     = "error"
)

Outcome label values for SingleWriterCommitTotal: how a commit attempt concluded.

Variables

View Source
var (
	// LockWaitDuration observes how long a caller waited to acquire the
	// per-key in-process lock (pkg/utils.MapMutex), labeled by resource
	// "kind" (see file.resourceFromKey) and "outcome" (acquired/timeout).
	LockWaitDuration = metrics.NewHistogramVec(
		&metrics.HistogramOpts{
			Subsystem:      "storage",
			Name:           "lock_wait_duration_seconds",
			Help:           "Time spent waiting to acquire the per-key in-process lock, by resource kind and outcome.",
			Buckets:        waitBuckets,
			StabilityLevel: metrics.ALPHA,
		},
		[]string{"kind", "outcome"},
	)

	// PoolWaitDuration observes how long a caller waited to acquire a SQLite
	// connection from the pool (sqlitemigration.Pool.Take), labeled by
	// resource "kind" and "outcome" (acquired/timeout).
	PoolWaitDuration = metrics.NewHistogramVec(
		&metrics.HistogramOpts{
			Subsystem:      "storage",
			Name:           "pool_wait_duration_seconds",
			Help:           "Time spent waiting to acquire a SQLite connection from the pool, by resource kind and outcome.",
			Buckets:        waitBuckets,
			StabilityLevel: metrics.ALPHA,
		},
		[]string{"kind", "outcome"},
	)

	// SingleWriterQueueWaitDuration observes how long a commitJob sat in the
	// single writer's high/low priority channel before the writer goroutine
	// (singleWriter.run) picked it up and started processing it. This is the
	// new contention point the single-writer design introduces by
	// centralizing every write through one goroutine, labeled by resource
	// "kind" and queue "priority" (high/low).
	SingleWriterQueueWaitDuration = metrics.NewHistogramVec(
		&metrics.HistogramOpts{
			Subsystem:      "storage",
			Name:           "single_writer_queue_wait_duration_seconds",
			Help:           "Time a commit job spent waiting in the single writer's priority queue before being picked up, by resource kind and priority.",
			Buckets:        waitBuckets,
			StabilityLevel: metrics.ALPHA,
		},
		[]string{"kind", "priority"},
	)

	// SingleWriterCommitTotal counts single-writer commit attempts, labeled by
	// resource "kind", queue "priority" (high/low), and "outcome"
	// (committed/conflict/error). This shows how often optimistic commits
	// succeed vs. hit a resourceVersion conflict vs. fail for another reason,
	// and whether high- vs low-priority jobs are serviced as expected.
	SingleWriterCommitTotal = metrics.NewCounterVec(
		&metrics.CounterOpts{
			Subsystem:      "storage",
			Name:           "single_writer_commit_total",
			Help:           "Count of single-writer commit attempts, by resource kind, priority, and outcome (committed/conflict/error).",
			StabilityLevel: metrics.ALPHA,
		},
		[]string{"kind", "priority", "outcome"},
	)

	// SingleWriterConflictRetryTotal counts how many times
	// guaranteedUpdateSingleWriter's retry loop redoes the prepare phase
	// because a commit was rejected with errWriteConflict, labeled by
	// resource "kind". This directly measures retry-storm risk under real
	// contention.
	SingleWriterConflictRetryTotal = metrics.NewCounterVec(
		&metrics.CounterOpts{
			Subsystem:      "storage",
			Name:           "single_writer_conflict_retry_total",
			Help:           "Count of GuaranteedUpdate prepare-phase retries caused by a resourceVersion conflict at commit time, by resource kind.",
			StabilityLevel: metrics.ALPHA,
		},
		[]string{"kind"},
	)

	// SingleWriterQueueDepth gauges the current number of commitJobs waiting
	// in each single-writer priority lane, updated on enqueue and dequeue.
	// This shows whether a backlog is building up under real load, labeled
	// by queue "priority" (high/low).
	SingleWriterQueueDepth = metrics.NewGaugeVec(
		&metrics.GaugeOpts{
			Subsystem:      "storage",
			Name:           "single_writer_queue_depth",
			Help:           "Current number of commit jobs waiting in the single writer's priority queue, by priority.",
			StabilityLevel: metrics.ALPHA,
		},
		[]string{"priority"},
	)
)

Functions

func IncSingleWriterCommit

func IncSingleWriterCommit(kind, priority, outcome string)

IncSingleWriterCommit records one single-writer commit attempt for the given resource kind, priority (PriorityHigh / PriorityLow), and outcome (CommitOutcomeCommitted / CommitOutcomeConflict / CommitOutcomeError).

func IncSingleWriterConflictRetry

func IncSingleWriterConflictRetry(kind string)

IncSingleWriterConflictRetry records one GuaranteedUpdate prepare-phase retry caused by a resourceVersion conflict at commit time, for the given resource kind.

func ObserveLockWait

func ObserveLockWait(kind, outcome string, d time.Duration)

ObserveLockWait records a lock-hold-wait observation for the given resource kind and outcome (OutcomeAcquired / OutcomeTimeout).

func ObservePoolWait

func ObservePoolWait(kind, outcome string, d time.Duration)

ObservePoolWait records a connection-pool-wait observation for the given resource kind and outcome (OutcomeAcquired / OutcomeTimeout).

func ObserveSingleWriterQueueWait

func ObserveSingleWriterQueueWait(kind, priority string, d time.Duration)

ObserveSingleWriterQueueWait records how long a commitJob waited in the single writer's priority queue for the given resource kind and priority (PriorityHigh / PriorityLow) before being picked up by the writer goroutine.

func SetSingleWriterQueueDepth

func SetSingleWriterQueueDepth(priority string, depth int)

SetSingleWriterQueueDepth sets the current number of commit jobs waiting in the given priority lane (PriorityHigh / PriorityLow).

Types

This section is empty.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL