Documentation
¶
Overview ¶
Package stats contains the logic to process APM stats.
Index ¶
- Variables
- type Aggregation
- type BlockCounts
- type BucketCardinalityLimits
- type BucketsAggregationKey
- type ClientStatsAggregator
- type Concentrator
- type Input
- type InputV1
- type PayloadAggregationKey
- type RawBucket
- type SpanCollapseResult
- type SpanConcentrator
- func (sc *SpanConcentrator) AddSpan(s *StatSpan, aggKey PayloadAggregationKey, containerID string, ...)
- func (sc *SpanConcentrator) DrainBlockCounts() BlockCounts
- func (sc *SpanConcentrator) Flush(now int64, force bool) []*pb.ClientStatsPayload
- func (sc *SpanConcentrator) NewStatSpan(service, resource, name string, typ string, parentID uint64, ...) (statSpan *StatSpan, ok bool)
- func (sc *SpanConcentrator) NewStatSpanFromPB(s *pb.Span, peerTags []string, additionalMetricTagKeys []string) (statSpan *StatSpan, ok bool)
- func (sc *SpanConcentrator) NewStatSpanFromV1(s *idx.InternalSpan, peerTags []string, additionalMetricTagKeys []string) (statSpan *StatSpan, ok bool)
- func (sc *SpanConcentrator) NewStatSpanWithConfig(config StatSpanConfig) (statSpan *StatSpan, ok bool)
- func (sc *SpanConcentrator) SetObfuscationEnabled(enabled bool, bigResource bool)
- type SpanConcentratorConfig
- type StatSpan
- type StatSpanConfig
- type Writer
Constants ¶
This section is empty.
Variables ¶
var KindsComputed = map[string]struct{}{
"server": {},
"consumer": {},
"client": {},
"producer": {},
}
KindsComputed is the list of span kinds that will have stats computed on them when computeStatsByKind is enabled in the concentrator.
Functions ¶
This section is empty.
Types ¶
type Aggregation ¶
type Aggregation struct {
BucketsAggregationKey
PayloadAggregationKey
}
Aggregation contains all the dimension on which we aggregate statistics.
func NewAggregationFromGroup ¶
func NewAggregationFromGroup(g *pb.ClientGroupedStats) Aggregation
NewAggregationFromGroup gets the Aggregation key of grouped stats.
func NewAggregationFromSpan ¶
func NewAggregationFromSpan(s *StatSpan, origin string, aggKey PayloadAggregationKey) Aggregation
NewAggregationFromSpan creates a new aggregation from the provided span and env
type BlockCounts ¶ added in v0.81.0
type BlockCounts struct {
// LengthBlocks counts additional_metric_tags values that exceeded the 200-char length cap.
LengthBlocks int64
// CapBlocks counts additional_metric_tags entries that exceeded the per-bucket cardinality cap.
CapBlocks int64
// WholeKeyCollapses counts spans collapsed due to the whole-key cardinality limit.
WholeKeyCollapses int64
// ResourceCollapses counts spans whose resource field was collapsed to the sentinel.
ResourceCollapses int64
// HTTPEndpointCollapses counts spans whose http_endpoint field was collapsed to the sentinel.
HTTPEndpointCollapses int64
// PeerTagsCollapses counts spans whose peer_tags were collapsed to the sentinel.
PeerTagsCollapses int64
// OriginCollapses counts spans whose origin was collapsed.
OriginCollapses int64
}
BlockCounts reports cardinality collapse events since the last drain.
type BucketCardinalityLimits ¶
type BucketCardinalityLimits struct {
AdditionalTags int
Resource int
HTTPEndpoint int
PeerTags int
Origin int
WholeKey int
}
BucketCardinalityLimits holds per-field and whole-key cardinality limits for a RawBucket. A value of 0 disables the cap for that field. These are tracer-only controls; the Agent intentionally leaves them all at 0 (no-op).
type BucketsAggregationKey ¶
type BucketsAggregationKey struct {
Service string
Name string
Resource string
Type string
SpanKind string
StatusCode uint32
Synthetics bool
PeerTagsHash uint64
AdditionalMetricTagsHash uint64
ServiceSource string
IsTraceRoot pb.Trilean
GRPCStatusCode string
HTTPMethod string
HTTPEndpoint string
}
BucketsAggregationKey specifies the key by which a bucket is aggregated.
type ClientStatsAggregator ¶
type ClientStatsAggregator struct {
In chan *pb.ClientStatsPayload
// contains filtered or unexported fields
}
ClientStatsAggregator aggregates client stats payloads on buckets of bucketDuration If a single payload is received on a bucket, this Aggregator is a passthrough. If two or more payloads collide, their counts will be aggregated into one bucket. Multiple payloads will be sent: - Original payloads with their distributions will be sent with counts zeroed. - A single payload with the bucket aggregated counts will be sent. This and the aggregator timestamp alignment ensure that all counts will have at most one point per second per agent for a specific granularity. While distributions are not tied to the agent.
func NewClientStatsAggregator ¶
func NewClientStatsAggregator(conf *config.AgentConfig, writer Writer, statsd statsd.ClientInterface) *ClientStatsAggregator
NewClientStatsAggregator initializes a new aggregator ready to be started
func (*ClientStatsAggregator) Start ¶
func (a *ClientStatsAggregator) Start()
Start starts the aggregator.
func (*ClientStatsAggregator) Stop ¶
func (a *ClientStatsAggregator) Stop()
Stop stops the aggregator. Calling Stop twice will panic.
type Concentrator ¶
type Concentrator struct {
Writer Writer
// contains filtered or unexported fields
}
Concentrator produces time bucketed statistics from a stream of raw traces. https://en.wikipedia.org/wiki/Knelson_concentrator Gets an imperial shitton of traces, and outputs pre-computed data structures allowing to find the gold (stats) amongst the traces.
func NewConcentrator ¶
func NewConcentrator(conf *config.AgentConfig, writer Writer, now time.Time, statsd statsd.ClientInterface) *Concentrator
NewConcentrator initializes a new concentrator ready to be started
func (*Concentrator) Add ¶
func (c *Concentrator) Add(t Input)
Add applies the given input to the concentrator.
func (*Concentrator) AddV1 ¶
func (c *Concentrator) AddV1(t InputV1)
AddV1 applies the given input to the concentrator.
func (*Concentrator) Flush ¶
func (c *Concentrator) Flush(force bool) *pb.StatsPayload
Flush deletes and returns complete statistic buckets. The force boolean guarantees flushing all buckets if set to true.
func (*Concentrator) Run ¶
func (c *Concentrator) Run()
Run runs the main loop of the concentrator goroutine. Traces are received through `Add`, this loop only deals with flushing.
type Input ¶
type Input struct {
Traces []traceutil.ProcessedTrace
ContainerID string
ContainerTags []string
ProcessTags string
}
Input specifies a set of traces originating from a certain payload.
type InputV1 ¶
type InputV1 struct {
Traces []traceutil.ProcessedTraceV1
ContainerID string
ContainerTags []string
ProcessTags string
}
InputV1 specifies a set of traces originating from a certain payload.
type PayloadAggregationKey ¶
type PayloadAggregationKey struct {
Env string
Hostname string
Version string
ContainerID string
GitCommitSha string
ImageTag string
Lang string
ProcessTagsHash uint64
BaseService string
}
PayloadAggregationKey specifies the key by which a payload is aggregated.
type RawBucket ¶
type RawBucket struct {
// contains filtered or unexported fields
}
RawBucket is used to compute span data and aggregate it within a time-framed bucket. This should not be used outside the agent, use ClientStatsBucket for this.
func NewRawBucket ¶
func NewRawBucket(ts, d uint64, limits BucketCardinalityLimits) *RawBucket
NewRawBucket opens a new calculation bucket for time ts and initializes it properly
func (*RawBucket) Export ¶
func (sb *RawBucket) Export() map[PayloadAggregationKey]*pb.ClientStatsBucket
Export transforms a RawBucket into a ClientStatsBucket, typically used before communicating data to the API, as RawBucket is the internal type while ClientStatsBucket is the public, shared one.
func (*RawBucket) HandleSpan ¶
func (sb *RawBucket) HandleSpan(s *StatSpan, weight float64, origin string, aggKey PayloadAggregationKey) SpanCollapseResult
HandleSpan adds the span to this bucket stats, aggregated with the finest grain matching given aggregators. It returns a SpanCollapseResult indicating which cardinality collapses were applied. HandleSpan does not modify s.
type SpanCollapseResult ¶
type SpanCollapseResult struct {
WholeKeyCollapsed bool
ResourceCollapsed bool
HTTPEndpointCollapsed bool
PeerTagsCollapsed bool
OriginCollapsed bool
AdditionalTagsCapBlock bool
}
SpanCollapseResult reports which cardinality collapses were applied to a span in HandleSpan.
type SpanConcentrator ¶
type SpanConcentrator struct {
// contains filtered or unexported fields
}
SpanConcentrator produces time bucketed statistics from a stream of raw spans.
func NewSpanConcentrator ¶
func NewSpanConcentrator(cfg *SpanConcentratorConfig, now time.Time) *SpanConcentrator
NewSpanConcentrator builds a new SpanConcentrator object
func (*SpanConcentrator) AddSpan ¶
func (sc *SpanConcentrator) AddSpan(s *StatSpan, aggKey PayloadAggregationKey, containerID string, containerTags []string, origin string)
AddSpan to the SpanConcentrator, appending the new data to the appropriate internal bucket. todo:raphael migrate dd-trace-go API to not depend on containerID/containerTags and add processTags at encoding layer
func (*SpanConcentrator) DrainBlockCounts ¶ added in v0.81.0
func (sc *SpanConcentrator) DrainBlockCounts() BlockCounts
DrainBlockCounts atomically reads and zeroes all collapse counters.
func (*SpanConcentrator) Flush ¶
func (sc *SpanConcentrator) Flush(now int64, force bool) []*pb.ClientStatsPayload
Flush deletes and returns complete ClientStatsPayloads. The force boolean guarantees flushing all buckets if set to true.
func (*SpanConcentrator) NewStatSpan ¶
func (sc *SpanConcentrator) NewStatSpan( service, resource, name string, typ string, parentID uint64, start, duration int64, error int32, meta map[string]string, metrics map[string]float64, peerTags []string, ) (statSpan *StatSpan, ok bool)
NewStatSpan builds a StatSpan from the required fields for stats calculation peerTags is the configured list of peer tags to look for returns (nil,false) if the provided fields indicate a span should not have stats calculated Deprecated: use NewStatSpanWithConfig instead
func (*SpanConcentrator) NewStatSpanFromPB ¶
func (sc *SpanConcentrator) NewStatSpanFromPB(s *pb.Span, peerTags []string, additionalMetricTagKeys []string) (statSpan *StatSpan, ok bool)
NewStatSpanFromPB is a helper version of NewStatSpanWithConfig that builds a StatSpan from a pb.Span.
func (*SpanConcentrator) NewStatSpanFromV1 ¶
func (sc *SpanConcentrator) NewStatSpanFromV1(s *idx.InternalSpan, peerTags []string, additionalMetricTagKeys []string) (statSpan *StatSpan, ok bool)
NewStatSpanFromV1 is a helper version of NewStatSpan that builds a StatSpan from an idx.InternalSpan.
func (*SpanConcentrator) NewStatSpanWithConfig ¶
func (sc *SpanConcentrator) NewStatSpanWithConfig(config StatSpanConfig) (statSpan *StatSpan, ok bool)
NewStatSpanWithConfig builds a StatSpan from the required fields for stats calculation peerTags is the configured list of peer tags to look for returns (nil,false) if the provided fields indicate a span should not have stats calculated
func (*SpanConcentrator) SetObfuscationEnabled ¶
func (sc *SpanConcentrator) SetObfuscationEnabled(enabled bool, bigResource bool)
SetObfuscationEnabled updates whether string length caps are applied during stat span creation. Should be called when the agent's obfuscation version is known (e.g. after /info is fetched). bigResource should be true when the agent's /info feature_flags contains "big_resource".
type SpanConcentratorConfig ¶
type SpanConcentratorConfig struct {
// ComputeStatsBySpanKind enables/disables the computing of stats based on a span's `span.kind` field
ComputeStatsBySpanKind bool
// BucketInterval the size of our pre-aggregation per bucket
BucketInterval int64
// AdditionalMetricTagsCardinalityLimit caps distinct additional_metric_tags entries per bucket. 0 = no cap.
AdditionalMetricTagsCardinalityLimit int
// ResourceCardinalityLimit caps distinct resource values per bucket. 0 = no cap.
ResourceCardinalityLimit int
// HTTPEndpointCardinalityLimit caps distinct http_endpoint values per bucket. 0 = no cap.
HTTPEndpointCardinalityLimit int
// PeerTagsCardinalityLimit caps distinct peer_tags combinations per bucket. 0 = no cap.
PeerTagsCardinalityLimit int
// OriginCardinalityLimit caps distinct origin values per bucket. 0 = no cap.
OriginCardinalityLimit int
// WholeKeyCardinalityLimit caps the total distinct BucketsAggregationKeys per bucket. 0 = no cap.
// This is the backstop that guarantees a hard memory bound regardless of which field causes explosion.
WholeKeyCardinalityLimit int
// ObfuscationEnabled signals that the tracer is performing obfuscation/normalization.
// When true, string length caps (service ≤ 100, name ≤ 100, type ≤ 100, resource ≤ ResourceMaxBytes) are applied.
ObfuscationEnabled bool
// ResourceMaxBytes is the max byte length for resource strings when ObfuscationEnabled is true.
// Defaults to 5000; tracers should set to 15000 when the agent /info endpoint advertises the big_resource feature flag.
ResourceMaxBytes int
}
SpanConcentratorConfig exposes configuration options for a SpanConcentrator
type StatSpan ¶
type StatSpan struct {
// contains filtered or unexported fields
}
StatSpan holds all the required fields from a span needed to calculate stats
type StatSpanConfig ¶
type StatSpanConfig struct {
Service string
Resource string
Name string
Type string
ParentID uint64
Start int64
Duration int64
Error int32
Meta map[string]string
Metrics map[string]float64
PeerTags []string
AdditionalMetricTagKeys []string
HTTPMethod string
HTTPEndpoint string
}
StatSpanConfig holds the configuration options for creating a StatSpan using NewStatSpanWithConfig
type Writer ¶
type Writer interface {
// Write this payload
Write(*pb.StatsPayload)
}
Writer is an interface for something that can Write Stats Payloads