metrics

package
v19.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 16, 2026 License: MIT Imports: 23 Imported by: 0

Documentation

Overview

Package metrics declares the performance metrics compared between builds of the autocore workflow engine, and resolves them against Prometheus.

The declaration is data, in manifest.yaml, validated by the buf.validate rules on manifest.proto. Three failure modes seen in prior measurement campaigns are handled structurally rather than per metric: a removed metric silently resolving to null, a saturated histogram reporting a bucket boundary as though it were a latency, and a delta smaller than its instrument's repeatability being reported as a finding.

Index

Constants

View Source 
const AbsentSeriesNote = "no series matched the query; absence is defined as zero for this metric"

AbsentSeriesNote is recorded on an absent_means_zero metric whose query matched nothing, so that a defined zero stays distinguishable from a measured one in the result file.

View Source 
const OverflowUnreliableThreshold = 0.05

OverflowUnreliableThreshold is the share of samples in a histogram's +Inf bucket above which its quantiles are not a measurement. Above this, a quantile reads as the last finite boundary rather than as a latency.

Variables

View Source 
var (
	Tier_name = map[int32]string{
		0: "TIER_UNSPECIFIED",
		1: "TIER_OUTCOME",
		2: "TIER_COST",
		3: "TIER_SATURATION",
		4: "TIER_LATENCY",
		5: "TIER_BACKLOG",
		6: "TIER_GATE",
	}
	Tier_value = map[string]int32{
		"TIER_UNSPECIFIED": 0,
		"TIER_OUTCOME":     1,
		"TIER_COST":        2,
		"TIER_SATURATION":  3,
		"TIER_LATENCY":     4,
		"TIER_BACKLOG":     5,
		"TIER_GATE":        6,
	}
)

Enum value maps for Tier.

View Source 
var (
	Kind_name = map[int32]string{
		0: "KIND_UNSPECIFIED",
		1: "KIND_COUNTER",
		2: "KIND_GAUGE",
		3: "KIND_HISTOGRAM",
	}
	Kind_value = map[string]int32{
		"KIND_UNSPECIFIED": 0,
		"KIND_COUNTER":     1,
		"KIND_GAUGE":       2,
		"KIND_HISTOGRAM":   3,
	}
)

Enum value maps for Kind.

View Source 
var File_scripts_loadtests_autoflow_metrics_manifest_proto protoreflect.FileDescriptor

Functions

func ShardSkew 

func ShardSkew(perPodPerDB map[string]int, dbCount int) (float64, error)

ShardSkew is the largest relative deviation of any pod's per-database shard ownership from an even split across databases. Zero is even.

Keys are "<pod>/<db_id>". Global balance is not enough: rebalance computes one target over all databases, so a pod can own its exact target while holding almost all of it on one database, which moves that pod's pool acquire wait by orders of magnitude.

The map is normally dense. autocore_shards_owned is an observable instrument whose callback runs per pod and database on every scrape and reports its count unconditionally, so a pod owning nothing on a database still has a series, with an explicit zero. The missing-entry branch is therefore defensive: it scores an absent pair at maximum deviation (1.0), which is what an explicit zero already scores, so a map that is sparse for any reason cannot read as balanced.

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

func NewClient 

func NewClient(baseURL string, hc *http.Client, evalAt time.Time) *Client

NewClient builds a Prometheus client. A non-zero evalAt pins every instant query to that instant through the API's time parameter. Without it each query is evaluated at its own wall-clock arrival time, so an increase() over the capture window measures an interval that slides with however long the caller took to issue the query: pod scheduling and image pull alone move it by tens of seconds. SeriesExists is unaffected, because whether a metric is still exported is a question about the retention window rather than about the measurement window.

func (*Client) Instant 

func (c *Client) Instant(ctx context.Context, query string) ([]Sample, error)

func (*Client) SeriesExists 

func (c *Client) SeriesExists(ctx context.Context, name string) (bool, error)

SeriesExists reports whether the metric family currently has any series. It is the guard against a renamed or removed metric resolving to null, which is indistinguishable from "the feature is switched off".

type Comparison 

type Comparison struct {
	Key          string            `json:"key"`
	Labels       map[string]string `json:"labels,omitempty"`
	Quantile     float64           `json:"quantile,omitempty"`
	BaseN        int               `json:"base_n"`
	TipN         int               `json:"tip_n"`
	BaseMean     float64           `json:"base_mean"`
	TipMean      float64           `json:"tip_mean"`
	BaseSE       float64           `json:"base_se"`
	TipSE        float64           `json:"tip_se"`
	DeltaPct     float64           `json:"delta_pct"`
	SeparationSE float64           `json:"separation_se"`
	// The floor this delta was judged against, so a reader of the output can
	// see that a metric had none rather than having to know.
	NoiseFloorPct float64 `json:"noise_floor_pct"`
	// The smallest delta this run could have called, as a percentage of the
	// base mean. Without it a null result says nothing about how large an
	// effect could have hidden behind the sample size.
	MinDetectablePct float64 `json:"min_detectable_pct"`
	Resolvable       bool    `json:"resolvable"`
	Note             string  `json:"note,omitempty"`
}

func Compare 

func Compare(m *Manifest, base, tip []*Result) ([]Comparison, error)

type FloorProvenance 

type FloorProvenance struct {
	MeasuredAt           string `protobuf:"bytes,1,opt,name=measured_at" json:"measured_at,omitempty"`
	Protocol             string `protobuf:"bytes,2,opt,name=protocol" json:"protocol,omitempty"`
	Campaign             string `protobuf:"bytes,3,opt,name=campaign" json:"campaign,omitempty"`
	Points               int32  `protobuf:"varint,4,opt,name=points" json:"points,omitempty"`
	WindowSeconds        int32  `protobuf:"varint,5,opt,name=window_seconds" json:"window_seconds,omitempty"`
	ShardsTotal          int32  `protobuf:"varint,6,opt,name=shards_total" json:"shards_total,omitempty"`
	WorkflowDbCount      int32  `protobuf:"varint,7,opt,name=workflow_db_count" json:"workflow_db_count,omitempty"`
	WorkerReplicas       int32  `protobuf:"varint,8,opt,name=worker_replicas" json:"worker_replicas,omitempty"`
	GeneratorReplicas    int32  `protobuf:"varint,9,opt,name=generator_replicas" json:"generator_replicas,omitempty"`
	WorkflowPoolMaxConns int32  `protobuf:"varint,10,opt,name=workflow_pool_max_conns" json:"workflow_pool_max_conns,omitempty"`
	WorkloadConfigSha256 string `protobuf:"bytes,11,opt,name=workload_config_sha256" json:"workload_config_sha256,omitempty"`
	// contains filtered or unexported fields
}

func (*FloorProvenance) Descriptor deprecated 

func (*FloorProvenance) Descriptor() ([]byte, []int)

Deprecated: Use FloorProvenance.ProtoReflect.Descriptor instead.

func (*FloorProvenance) GetCampaign 

func (x *FloorProvenance) GetCampaign() string

func (*FloorProvenance) GetGeneratorReplicas 

func (x *FloorProvenance) GetGeneratorReplicas() int32

func (*FloorProvenance) GetMeasuredAt 

func (x *FloorProvenance) GetMeasuredAt() string

func (*FloorProvenance) GetPoints 

func (x *FloorProvenance) GetPoints() int32

func (*FloorProvenance) GetProtocol 

func (x *FloorProvenance) GetProtocol() string

func (*FloorProvenance) GetShardsTotal 

func (x *FloorProvenance) GetShardsTotal() int32

func (*FloorProvenance) GetWindowSeconds 

func (x *FloorProvenance) GetWindowSeconds() int32

func (*FloorProvenance) GetWorkerReplicas 

func (x *FloorProvenance) GetWorkerReplicas() int32

func (*FloorProvenance) GetWorkflowDbCount 

func (x *FloorProvenance) GetWorkflowDbCount() int32

func (*FloorProvenance) GetWorkflowPoolMaxConns 

func (x *FloorProvenance) GetWorkflowPoolMaxConns() int32

func (*FloorProvenance) GetWorkloadConfigSha256 

func (x *FloorProvenance) GetWorkloadConfigSha256() string

func (*FloorProvenance) ProtoMessage 

func (*FloorProvenance) ProtoMessage()

func (*FloorProvenance) ProtoReflect 

func (x *FloorProvenance) ProtoReflect() protoreflect.Message

func (*FloorProvenance) Reset 

func (x *FloorProvenance) Reset()

func (*FloorProvenance) String 

func (x *FloorProvenance) String() string

type FloorReport 

type FloorReport struct {
	Protocol      string        `json:"protocol"`
	WindowSeconds int           `json:"window_seconds"`
	Points        int           `json:"points"`
	Shape         Shape         `json:"shape"`
	Metrics       []MetricFloor `json:"metrics"`
}

func Floors 

func Floors(m *Manifest, results []*Result) (*FloorReport, error)

Floors estimates each metric's repeatability from points captured against a single unchanged build. Rather than reading one delta out of a base arm against a tip arm, it splits the points into two arms every way it can and takes a high quantile of the resulting delta distribution: an A/A campaign of ten points yields 252 splits rather than a single sample of the quantity being estimated.

type Kind 

type Kind int32
const (
	Kind_KIND_UNSPECIFIED Kind = 0
	Kind_KIND_COUNTER     Kind = 1
	Kind_KIND_GAUGE       Kind = 2
	Kind_KIND_HISTOGRAM   Kind = 3
)

func (Kind) Descriptor 

func (Kind) Descriptor() protoreflect.EnumDescriptor

func (Kind) Enum 

func (x Kind) Enum() *Kind

func (Kind) EnumDescriptor deprecated 

func (Kind) EnumDescriptor() ([]byte, []int)

Deprecated: Use Kind.Descriptor instead.

func (Kind) Number 

func (x Kind) Number() protoreflect.EnumNumber

func (Kind) String 

func (x Kind) String() string

func (Kind) Type 

func (Kind) Type() protoreflect.EnumType

type Manifest 

type Manifest struct {
	Metrics         []*Metric        `protobuf:"bytes,1,rep,name=metrics" json:"metrics,omitempty"`
	FloorProvenance *FloorProvenance `protobuf:"bytes,2,opt,name=floor_provenance" json:"floor_provenance,omitempty"`
	// contains filtered or unexported fields
}

func Load 

func Load() (*Manifest, error)

func (*Manifest) Descriptor deprecated 

func (*Manifest) Descriptor() ([]byte, []int)

Deprecated: Use Manifest.ProtoReflect.Descriptor instead.

func (*Manifest) GetFloorProvenance 

func (x *Manifest) GetFloorProvenance() *FloorProvenance

func (*Manifest) GetMetrics 

func (x *Manifest) GetMetrics() []*Metric

func (*Manifest) ProtoMessage 

func (*Manifest) ProtoMessage()

func (*Manifest) ProtoReflect 

func (x *Manifest) ProtoReflect() protoreflect.Message

func (*Manifest) Reset 

func (x *Manifest) Reset()

func (*Manifest) String 

func (x *Manifest) String() string

type Metric 

type Metric struct {
	Key              string    `protobuf:"bytes,1,opt,name=key" json:"key,omitempty"`
	Name             string    `protobuf:"bytes,2,opt,name=name" json:"name,omitempty"`
	Query            string    `protobuf:"bytes,3,opt,name=query" json:"query,omitempty"`
	Description      string    `protobuf:"bytes,4,opt,name=description" json:"description,omitempty"`
	Tier             Tier      `protobuf:"varint,5,opt,name=tier,enum=gitlab.agent.loadtest.metrics.Tier" json:"tier,omitempty"`
	Kind             Kind      `protobuf:"varint,6,opt,name=kind,enum=gitlab.agent.loadtest.metrics.Kind" json:"kind,omitempty"`
	Unit             string    `protobuf:"bytes,7,opt,name=unit" json:"unit,omitempty"`
	NoiseFloorPct    float64   `protobuf:"fixed64,8,opt,name=noise_floor_pct" json:"noise_floor_pct,omitempty"`
	Quantiles        []float64 `protobuf:"fixed64,9,rep,packed,name=quantiles" json:"quantiles,omitempty"`
	TopFiniteBucket  string    `protobuf:"bytes,10,opt,name=top_finite_bucket" json:"top_finite_bucket,omitempty"`
	StructurallyZero bool      `protobuf:"varint,11,opt,name=structurally_zero" json:"structurally_zero,omitempty"`
	ZeroReason       string    `protobuf:"bytes,12,opt,name=zero_reason" json:"zero_reason,omitempty"`
	AbsentMeansZero  bool      `protobuf:"varint,13,opt,name=absent_means_zero" json:"absent_means_zero,omitempty"`
	// contains filtered or unexported fields
}

func (*Metric) Descriptor deprecated 

func (*Metric) Descriptor() ([]byte, []int)

Deprecated: Use Metric.ProtoReflect.Descriptor instead.

func (*Metric) GetAbsentMeansZero 

func (x *Metric) GetAbsentMeansZero() bool

func (*Metric) GetDescription 

func (x *Metric) GetDescription() string

func (*Metric) GetKey 

func (x *Metric) GetKey() string

func (*Metric) GetKind 

func (x *Metric) GetKind() Kind

func (*Metric) GetName 

func (x *Metric) GetName() string

func (*Metric) GetNoiseFloorPct 

func (x *Metric) GetNoiseFloorPct() float64

func (*Metric) GetQuantiles 

func (x *Metric) GetQuantiles() []float64

func (*Metric) GetQuery 

func (x *Metric) GetQuery() string

func (*Metric) GetStructurallyZero 

func (x *Metric) GetStructurallyZero() bool

func (*Metric) GetTier 

func (x *Metric) GetTier() Tier

func (*Metric) GetTopFiniteBucket 

func (x *Metric) GetTopFiniteBucket() string

func (*Metric) GetUnit 

func (x *Metric) GetUnit() string

func (*Metric) GetZeroReason 

func (x *Metric) GetZeroReason() string

func (*Metric) ProtoMessage 

func (*Metric) ProtoMessage()

func (*Metric) ProtoReflect 

func (x *Metric) ProtoReflect() protoreflect.Message

func (*Metric) Reset 

func (x *Metric) Reset()

func (*Metric) String 

func (x *Metric) String() string

type MetricFloor 

type MetricFloor struct {
	Key string `json:"key"`
	// The widest of the metric's series, because the manifest carries one floor
	// per metric while a comparison reports one row per series. Taking the
	// narrowest would publish another series' noise as a finding.
	FloorPct float64       `json:"floor_pct"`
	Series   []SeriesFloor `json:"series"`
}

type Provenance 

type Provenance struct {
	GitSHA               string         `json:"git_sha"`
	ImageDigest          string         `json:"image_digest"`
	WorkloadConfigSHA256 string         `json:"workload_config_sha256"`
	ShardsTotal          int            `json:"shards_total"`
	WorkflowDBCount      int            `json:"workflow_db_count"`
	ShardsPerPodPerDB    map[string]int `json:"shards_per_pod_per_db"`
	ShardSkewRatio       float64        `json:"shard_skew_ratio"`
	WorkerReplicas       int            `json:"worker_replicas"`
	GeneratorReplicas    int            `json:"generator_replicas"`
	WorkflowPoolMaxConns int            `json:"workflow_pool_max_conns"`
	ChaosCRCount         int            `json:"chaos_cr_count"`
	CapturedAt           time.Time      `json:"captured_at"`
}

Provenance is stamped into every captured result.

type Result 

type Result struct {
	Protocol       string     `json:"protocol"`
	WindowSeconds  int        `json:"window_seconds"`
	Provenance     Provenance `json:"provenance"`
	Values         []Value    `json:"values"`
	Invalid        bool       `json:"invalid"`
	InvalidReasons []string   `json:"invalid_reasons,omitempty"`
	BacklogStart   float64    `json:"backlog_start"`
	BacklogEnd     float64    `json:"backlog_end"`
}

func Capture 

func Capture(ctx context.Context, c *Client, m *Manifest, window time.Duration) (*Result, error)

type Sample 

type Sample struct {
	Labels map[string]string
	Value  float64
}

type SeriesFloor 

type SeriesFloor struct {
	Labels    map[string]string `json:"labels,omitempty"`
	Quantile  float64           `json:"quantile,omitempty"`
	Points    int               `json:"points"`
	Splits    int               `json:"splits"`
	MedianPct float64           `json:"median_delta_pct"`
	FloorPct  float64           `json:"floor_delta_pct"`
	MaxPct    float64           `json:"max_delta_pct"`
}

type Shape 

type Shape struct {
	ShardsTotal          int    `json:"shards_total"`
	WorkflowDBCount      int    `json:"workflow_db_count"`
	WorkerReplicas       int    `json:"worker_replicas"`
	GeneratorReplicas    int    `json:"generator_replicas"`
	WorkflowPoolMaxConns int    `json:"workflow_pool_max_conns"`
	WorkloadConfigSHA256 string `json:"workload_config_sha256"`
}

Shape is the part of a point's provenance that a floor is only valid under. It is the same field set the campaign driver compares across a campaign's points, because "these points are comparable with each other" and "this floor applies to these points" are the same question.

type Tier 

type Tier int32
const (
	Tier_TIER_UNSPECIFIED Tier = 0
	Tier_TIER_OUTCOME     Tier = 1
	Tier_TIER_COST        Tier = 2
	Tier_TIER_SATURATION  Tier = 3
	Tier_TIER_LATENCY     Tier = 4
	Tier_TIER_BACKLOG     Tier = 5
	Tier_TIER_GATE        Tier = 6
)

func (Tier) Descriptor 

func (Tier) Descriptor() protoreflect.EnumDescriptor

func (Tier) Enum 

func (x Tier) Enum() *Tier

func (Tier) EnumDescriptor deprecated 

func (Tier) EnumDescriptor() ([]byte, []int)

Deprecated: Use Tier.Descriptor instead.

func (Tier) Number 

func (x Tier) Number() protoreflect.EnumNumber

func (Tier) String 

func (x Tier) String() string

func (Tier) Type 

func (Tier) Type() protoreflect.EnumType

type Value 

type Value struct {
	Key              string            `json:"key"`
	Labels           map[string]string `json:"labels,omitempty"`
	Value            float64           `json:"value"`
	Quantile         float64           `json:"quantile,omitempty"`
	OverflowFraction float64           `json:"overflow_fraction,omitempty"`
	Reliable         bool              `json:"reliable"`
	Note             string            `json:"note,omitempty"`
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.