stats

package module
v0.83.0-devel Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 10, 2026 License: Apache-2.0 Imports: 27 Imported by: 12

Documentation

Overview

Package stats contains the logic to process APM stats.

Index

Constants

This section is empty.

Variables

View Source
var KindsComputed = map[string]struct{}{
	"server":   {},
	"consumer": {},
	"client":   {},
	"producer": {},
}

KindsComputed is the list of span kinds that will have stats computed on them when computeStatsByKind is enabled in the concentrator.

Functions

This section is empty.

Types

type Aggregation

type Aggregation struct {
	BucketsAggregationKey
	PayloadAggregationKey
}

Aggregation contains all the dimension on which we aggregate statistics.

func NewAggregationFromGroup

func NewAggregationFromGroup(g *pb.ClientGroupedStats) Aggregation

NewAggregationFromGroup gets the Aggregation key of grouped stats.

func NewAggregationFromSpan

func NewAggregationFromSpan(s *StatSpan, origin string, aggKey PayloadAggregationKey) Aggregation

NewAggregationFromSpan creates a new aggregation from the provided span and env

type BlockCounts added in v0.81.0

type BlockCounts struct {
	// LengthBlocks counts additional_metric_tags values that exceeded the 200-char length cap.
	LengthBlocks int64
	// CapBlocks counts additional_metric_tags entries that exceeded the per-bucket cardinality cap.
	CapBlocks int64
	// WholeKeyCollapses counts spans collapsed due to the whole-key cardinality limit.
	WholeKeyCollapses int64
	// ResourceCollapses counts spans whose resource field was collapsed to the sentinel.
	ResourceCollapses int64
	// HTTPEndpointCollapses counts spans whose http_endpoint field was collapsed to the sentinel.
	HTTPEndpointCollapses int64
	// PeerTagsCollapses counts spans whose peer_tags were collapsed to the sentinel.
	PeerTagsCollapses int64
	// OriginCollapses counts spans whose origin was collapsed.
	OriginCollapses int64
}

BlockCounts reports cardinality collapse events since the last drain.

type BucketCardinalityLimits

type BucketCardinalityLimits struct {
	AdditionalTags int
	Resource       int
	HTTPEndpoint   int
	PeerTags       int
	Origin         int
	WholeKey       int
}

BucketCardinalityLimits holds per-field and whole-key cardinality limits for a RawBucket. A value of 0 disables the cap for that field. These are tracer-only controls; the Agent intentionally leaves them all at 0 (no-op).

type BucketsAggregationKey

type BucketsAggregationKey struct {
	Service                  string
	Name                     string
	Resource                 string
	Type                     string
	SpanKind                 string
	StatusCode               uint32
	Synthetics               bool
	PeerTagsHash             uint64
	AdditionalMetricTagsHash uint64
	ServiceSource            string
	IsTraceRoot              pb.Trilean
	GRPCStatusCode           string
	HTTPMethod               string
	HTTPEndpoint             string
}

BucketsAggregationKey specifies the key by which a bucket is aggregated.

type ClientStatsAggregator

type ClientStatsAggregator struct {
	In chan *pb.ClientStatsPayload
	// contains filtered or unexported fields
}

ClientStatsAggregator aggregates client stats payloads on buckets of bucketDuration If a single payload is received on a bucket, this Aggregator is a passthrough. If two or more payloads collide, their counts will be aggregated into one bucket. Multiple payloads will be sent: - Original payloads with their distributions will be sent with counts zeroed. - A single payload with the bucket aggregated counts will be sent. This and the aggregator timestamp alignment ensure that all counts will have at most one point per second per agent for a specific granularity. While distributions are not tied to the agent.

func NewClientStatsAggregator

func NewClientStatsAggregator(conf *config.AgentConfig, writer Writer, statsd statsd.ClientInterface) *ClientStatsAggregator

NewClientStatsAggregator initializes a new aggregator ready to be started

func (*ClientStatsAggregator) Start

func (a *ClientStatsAggregator) Start()

Start starts the aggregator.

func (*ClientStatsAggregator) Stop

func (a *ClientStatsAggregator) Stop()

Stop stops the aggregator. Calling Stop twice will panic.

type Concentrator

type Concentrator struct {
	Writer Writer
	// contains filtered or unexported fields
}

Concentrator produces time bucketed statistics from a stream of raw traces. https://en.wikipedia.org/wiki/Knelson_concentrator Gets an imperial shitton of traces, and outputs pre-computed data structures allowing to find the gold (stats) amongst the traces.

func NewConcentrator

func NewConcentrator(conf *config.AgentConfig, writer Writer, now time.Time, statsd statsd.ClientInterface) *Concentrator

NewConcentrator initializes a new concentrator ready to be started

func (*Concentrator) Add

func (c *Concentrator) Add(t Input)

Add applies the given input to the concentrator.

func (*Concentrator) AddV1

func (c *Concentrator) AddV1(t InputV1)

AddV1 applies the given input to the concentrator.

func (*Concentrator) Flush

func (c *Concentrator) Flush(force bool) *pb.StatsPayload

Flush deletes and returns complete statistic buckets. The force boolean guarantees flushing all buckets if set to true.

func (*Concentrator) Run

func (c *Concentrator) Run()

Run runs the main loop of the concentrator goroutine. Traces are received through `Add`, this loop only deals with flushing.

func (*Concentrator) Start

func (c *Concentrator) Start()

Start starts the concentrator.

func (*Concentrator) Stop

func (c *Concentrator) Stop()

Stop stops the main Run loop.

type Input

type Input struct {
	Traces        []traceutil.ProcessedTrace
	ContainerID   string
	ContainerTags []string
	ProcessTags   string
}

Input specifies a set of traces originating from a certain payload.

func NewStatsInput

func NewStatsInput(numChunks int, containerID string, clientComputedStats bool, processTags string) Input

NewStatsInput allocates a stats input for an incoming trace payload

type InputV1

type InputV1 struct {
	Traces        []traceutil.ProcessedTraceV1
	ContainerID   string
	ContainerTags []string
	ProcessTags   string
}

InputV1 specifies a set of traces originating from a certain payload.

func NewStatsInputV1

func NewStatsInputV1(numChunks int, containerID string, clientComputedStats bool, processTags string) InputV1

NewStatsInputV1 allocates a stats input for an incoming trace payload

type PayloadAggregationKey

type PayloadAggregationKey struct {
	Env             string
	Hostname        string
	Version         string
	ContainerID     string
	GitCommitSha    string
	ImageTag        string
	Lang            string
	ProcessTagsHash uint64
	BaseService     string
}

PayloadAggregationKey specifies the key by which a payload is aggregated.

type RawBucket

type RawBucket struct {
	// contains filtered or unexported fields
}

RawBucket is used to compute span data and aggregate it within a time-framed bucket. This should not be used outside the agent, use ClientStatsBucket for this.

func NewRawBucket

func NewRawBucket(ts, d uint64, limits BucketCardinalityLimits) *RawBucket

NewRawBucket opens a new calculation bucket for time ts and initializes it properly

func (*RawBucket) Export

Export transforms a RawBucket into a ClientStatsBucket, typically used before communicating data to the API, as RawBucket is the internal type while ClientStatsBucket is the public, shared one.

func (*RawBucket) HandleSpan

func (sb *RawBucket) HandleSpan(s *StatSpan, weight float64, origin string, aggKey PayloadAggregationKey) SpanCollapseResult

HandleSpan adds the span to this bucket stats, aggregated with the finest grain matching given aggregators. It returns a SpanCollapseResult indicating which cardinality collapses were applied. HandleSpan does not modify s.

type SpanCollapseResult

type SpanCollapseResult struct {
	WholeKeyCollapsed      bool
	ResourceCollapsed      bool
	HTTPEndpointCollapsed  bool
	PeerTagsCollapsed      bool
	OriginCollapsed        bool
	AdditionalTagsCapBlock bool
}

SpanCollapseResult reports which cardinality collapses were applied to a span in HandleSpan.

type SpanConcentrator

type SpanConcentrator struct {
	// contains filtered or unexported fields
}

SpanConcentrator produces time bucketed statistics from a stream of raw spans.

func NewSpanConcentrator

func NewSpanConcentrator(cfg *SpanConcentratorConfig, now time.Time) *SpanConcentrator

NewSpanConcentrator builds a new SpanConcentrator object

func (*SpanConcentrator) AddSpan

func (sc *SpanConcentrator) AddSpan(s *StatSpan, aggKey PayloadAggregationKey, containerID string, containerTags []string, origin string)

AddSpan to the SpanConcentrator, appending the new data to the appropriate internal bucket. todo:raphael migrate dd-trace-go API to not depend on containerID/containerTags and add processTags at encoding layer

func (*SpanConcentrator) DrainBlockCounts added in v0.81.0

func (sc *SpanConcentrator) DrainBlockCounts() BlockCounts

DrainBlockCounts atomically reads and zeroes all collapse counters.

func (*SpanConcentrator) Flush

func (sc *SpanConcentrator) Flush(now int64, force bool) []*pb.ClientStatsPayload

Flush deletes and returns complete ClientStatsPayloads. The force boolean guarantees flushing all buckets if set to true.

func (*SpanConcentrator) NewStatSpan

func (sc *SpanConcentrator) NewStatSpan(
	service, resource, name string,
	typ string,
	parentID uint64,
	start, duration int64,
	error int32,
	meta map[string]string,
	metrics map[string]float64,
	peerTags []string,
) (statSpan *StatSpan, ok bool)

NewStatSpan builds a StatSpan from the required fields for stats calculation peerTags is the configured list of peer tags to look for returns (nil,false) if the provided fields indicate a span should not have stats calculated Deprecated: use NewStatSpanWithConfig instead

func (*SpanConcentrator) NewStatSpanFromPB

func (sc *SpanConcentrator) NewStatSpanFromPB(s *pb.Span, peerTags []string, additionalMetricTagKeys []string) (statSpan *StatSpan, ok bool)

NewStatSpanFromPB is a helper version of NewStatSpanWithConfig that builds a StatSpan from a pb.Span.

func (*SpanConcentrator) NewStatSpanFromV1

func (sc *SpanConcentrator) NewStatSpanFromV1(s *idx.InternalSpan, peerTags []string, additionalMetricTagKeys []string) (statSpan *StatSpan, ok bool)

NewStatSpanFromV1 is a helper version of NewStatSpan that builds a StatSpan from an idx.InternalSpan.

func (*SpanConcentrator) NewStatSpanWithConfig

func (sc *SpanConcentrator) NewStatSpanWithConfig(config StatSpanConfig) (statSpan *StatSpan, ok bool)

NewStatSpanWithConfig builds a StatSpan from the required fields for stats calculation peerTags is the configured list of peer tags to look for returns (nil,false) if the provided fields indicate a span should not have stats calculated

func (*SpanConcentrator) SetObfuscationEnabled

func (sc *SpanConcentrator) SetObfuscationEnabled(enabled bool, bigResource bool)

SetObfuscationEnabled updates whether string length caps are applied during stat span creation. Should be called when the agent's obfuscation version is known (e.g. after /info is fetched). bigResource should be true when the agent's /info feature_flags contains "big_resource".

type SpanConcentratorConfig

type SpanConcentratorConfig struct {
	// ComputeStatsBySpanKind enables/disables the computing of stats based on a span's `span.kind` field
	ComputeStatsBySpanKind bool
	// BucketInterval the size of our pre-aggregation per bucket
	BucketInterval int64

	// AdditionalMetricTagsCardinalityLimit caps distinct additional_metric_tags entries per bucket. 0 = no cap.
	AdditionalMetricTagsCardinalityLimit int
	// ResourceCardinalityLimit caps distinct resource values per bucket. 0 = no cap.
	ResourceCardinalityLimit int
	// HTTPEndpointCardinalityLimit caps distinct http_endpoint values per bucket. 0 = no cap.
	HTTPEndpointCardinalityLimit int
	// PeerTagsCardinalityLimit caps distinct peer_tags combinations per bucket. 0 = no cap.
	PeerTagsCardinalityLimit int
	// OriginCardinalityLimit caps distinct origin values per bucket. 0 = no cap.
	OriginCardinalityLimit int
	// WholeKeyCardinalityLimit caps the total distinct BucketsAggregationKeys per bucket. 0 = no cap.
	// This is the backstop that guarantees a hard memory bound regardless of which field causes explosion.
	WholeKeyCardinalityLimit int

	// ObfuscationEnabled signals that the tracer is performing obfuscation/normalization.
	// When true, string length caps (service ≤ 100, name ≤ 100, type ≤ 100, resource ≤ ResourceMaxBytes) are applied.
	ObfuscationEnabled bool
	// ResourceMaxBytes is the max byte length for resource strings when ObfuscationEnabled is true.
	// Defaults to 5000; tracers should set to 15000 when the agent /info endpoint advertises the big_resource feature flag.
	ResourceMaxBytes int
}

SpanConcentratorConfig exposes configuration options for a SpanConcentrator

type StatSpan

type StatSpan struct {
	// contains filtered or unexported fields
}

StatSpan holds all the required fields from a span needed to calculate stats

type StatSpanConfig

type StatSpanConfig struct {
	Service                 string
	Resource                string
	Name                    string
	Type                    string
	ParentID                uint64
	Start                   int64
	Duration                int64
	Error                   int32
	Meta                    map[string]string
	Metrics                 map[string]float64
	PeerTags                []string
	AdditionalMetricTagKeys []string
	HTTPMethod              string
	HTTPEndpoint            string
}

StatSpanConfig holds the configuration options for creating a StatSpan using NewStatSpanWithConfig

type Writer

type Writer interface {
	// Write this payload
	Write(*pb.StatsPayload)
}

Writer is an interface for something that can Write Stats Payloads

Directories

Path Synopsis
oteltest module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL