Documentation
¶
Overview ¶
Package metricsbp provides metrics related features for baseplate.go, based on go-kit metrics package.
There are three main parts of this package:
1. Wrappers of go-kit metrics to provide easy to use create on-the-fly metrics, similar to what we have in Baseplate.py.
2. Helper function for use cases of pre-create the metrics before using them.
3. Sampled counter/histogram implementations.
This package comes with benchmark test to show the performance difference between pre-created metrics, on-the-fly metrics, and on-the-fly with additional tags metrics:
$ go test -bench . -benchmem goos: darwin goarch: amd64 pkg: github.com/reddit/baseplate.go/metricsbp BenchmarkStatsd/pre-create/histogram-8 8583646 124 ns/op 44 B/op 0 allocs/op BenchmarkStatsd/pre-create/timing-8 10221859 120 ns/op 47 B/op 0 allocs/op BenchmarkStatsd/pre-create/counter-8 10205341 120 ns/op 47 B/op 0 allocs/op BenchmarkStatsd/pre-create/gauge-8 96462238 12.4 ns/op 0 B/op 0 allocs/op BenchmarkStatsd/on-the-fly/histogram-8 4665778 256 ns/op 99 B/op 2 allocs/op BenchmarkStatsd/on-the-fly/timing-8 4784816 273 ns/op 126 B/op 2 allocs/op BenchmarkStatsd/on-the-fly/counter-8 4818908 259 ns/op 125 B/op 2 allocs/op BenchmarkStatsd/on-the-fly/gauge-8 28754060 40.6 ns/op 0 B/op 0 allocs/op BenchmarkStatsd/on-the-fly-with-tags/histogram-8 2624264 453 ns/op 192 B/op 4 allocs/op BenchmarkStatsd/on-the-fly-with-tags/timing-8 2639377 449 ns/op 192 B/op 4 allocs/op BenchmarkStatsd/on-the-fly-with-tags/counter-8 2600418 457 ns/op 193 B/op 4 allocs/op BenchmarkStatsd/on-the-fly-with-tags/gauge-8 3429901 339 ns/op 112 B/op 3 allocs/op PASS ok github.com/reddit/baseplate.go/metricsbp 18.675s
Index ¶
- Constants
- Variables
- func CheckNilFields(root interface{}) []string
- func Float64Ptr(v float64) *float64
- func InitFromConfig(ctx context.Context, cfg Config) io.Closer
- func LogWrapper(args LogWrapperArgs) log.Wrapper
- type Config
- type CreateServerSpanHook
- type LogWrapperArgs
- type SampledCounter
- type SampledHistogram
- type Statsd
- func (st *Statsd) Close() error
- func (st *Statsd) Counter(name string) metrics.Counter
- func (st *Statsd) CounterWithRate(name string, rate float64) metrics.Counter
- func (st *Statsd) Ctx() context.Context
- func (st *Statsd) Gauge(name string) metrics.Gauge
- func (st *Statsd) Histogram(name string) metrics.Histogram
- func (st *Statsd) HistogramWithRate(name string, rate float64) metrics.Histogram
- func (st *Statsd) RunSysStats()
- func (st *Statsd) RuntimeGauge(name string) metrics.Gauge
- func (st *Statsd) Timing(name string) metrics.Histogram
- func (st *Statsd) TimingWithRate(name string, rate float64) metrics.Histogram
- func (st *Statsd) WriteTo(w io.Writer) (n int64, err error)
- type StatsdConfig
- type Tags
- type Timer
Examples ¶
Constants ¶
const DefaultSampleRate = 1
DefaultSampleRate is the default value to be used when *SampleRate in StatsdConfig is nil (zero value).
Variables ¶
var M = NewStatsd(context.Background(), StatsdConfig{})
M is short for "Metrics".
This is the global Statsd to use. It's pre-initialized with one that does not send metrics anywhere, so it won't cause panic even if you don't initialize it before using it (for example, it's safe to be used in test code).
But in production code you should still properly initialize it to actually send your metrics to your statsd collector, usually early in your main function:
func main() {
flag.Parse()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
metricsbp.M = metricsbp.NewStatsd{
ctx,
metricsbp.StatsdConfig{
...
},
}
metricsbp.M.RunSysStats()
...
}
func someOtherFunction() {
...
metricsbp.M.Counter("my-counter").Add(1)
...
}
var ReporterTickerInterval = time.Minute
ReporterTickerInterval is the interval the reporter sends data to statsd server. Default is one minute.
var SysStatsTickerInterval = time.Second * 10
SysStatsTickerInterval is the interval we pull and report sys stats. Default is 10 seconds.
Functions ¶
func CheckNilFields ¶
func CheckNilFields(root interface{}) []string
CheckNilFields returns all the nil value fields inside root.
root should be a value to a struct, or a pointer to a struct, otherwise this function will panic. The return value would be the field names of all the uninitialized fields recursively.
For example, for the following code:
type Bar struct {
A io.Reader
B io.Reader
c io.Reader
D struct{
A io.Reader
B io.Reader
}
}
func main() {
fields := CheckNilInterfaceFields(
&Bar{
A: strings.NewReader("foo"),
D: {
B: bytes.NewReader([]bytes("bar")),
},
},
)
}
fields should contain 3 strings: "Bar.B", "Bar.c", and "Bar.D.A".
Special case: When root itself is nil, or pointer to nil pointer, a single, empty string ("") will be returned.
The main use case of this function is for pre-created metrics. A common pattern is to define a struct contains all the metrics, initialize them in main function, then pass the struct down to the handler functions to use. It has better performance over creating metrics on the fly when using, but comes with a down side that if you added a new metric to the struct but forgot to initialize it in the main function, the code would panic when it's first used.
Use this function to check the metrics struct after initialization could help you panic earlier and in a more predictable way.
Example ¶
This example demonstrate how to use CheckNilFields in your microservice code to pre-create frequently used metrics.
package main
import (
"context"
"fmt"
"github.com/reddit/baseplate.go/metricsbp"
"github.com/go-kit/kit/metrics"
)
type SubMetrics struct {
MyHistogram metrics.Histogram
MyGauge metrics.Gauge
}
type PreCreatedMetrics struct {
MyCounter metrics.Counter
MySubMetrics SubMetrics
}
// This example demonstrate how to use CheckNilFields in your microservice code
// to pre-create frequently used metrics.
func main() {
// In reality these should come from flag or other configurations.
const (
prefix = "myservice"
statsdAddr = "localhost:1234"
sampleRate = 1
)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
metricsbp.M = metricsbp.NewStatsd(
ctx,
metricsbp.StatsdConfig{
Prefix: prefix,
Address: statsdAddr,
CounterSampleRate: metricsbp.Float64Ptr(sampleRate),
HistogramSampleRate: metricsbp.Float64Ptr(sampleRate),
},
)
// Initialize metrics
m := PreCreatedMetrics{
MyCounter: metricsbp.M.Counter("my.counter"),
MySubMetrics: SubMetrics{
MyHistogram: metricsbp.M.Histogram("my.histogram"),
// Forgot to initialize MyGauge here
},
}
missingFields := metricsbp.CheckNilFields(m)
if len(missingFields) > 0 {
panic(fmt.Sprintf("Uninitialized metrics: %v", missingFields))
}
// Other initializations.
// Replace with your actual service starter
startService := func(m PreCreatedMetrics /* and other args */) {}
startService(
m,
// other args
)
}
func Float64Ptr ¶
Float64Ptr converts float64 value into pointer.
func InitFromConfig ¶
InitFromConfig initializes the global metricsbp.M with the given context and Config and returns an io.Closer to use to close out the metrics client when your server exits.
It also registers CreateServerSpanHook and ConcurrencyCreateServerSpanHook with the global tracing hook registry.
func LogWrapper ¶ added in v0.6.0
func LogWrapper(args LogWrapperArgs) log.Wrapper
LogWrapper creates a log.Wrapper implementation with metrics emitting.
Types ¶
type Config ¶
type Config struct {
// Namespace is the standard prefix applied to all of your metrics, it should
// include the name of your service.
Namespace string `yaml:"namespace"`
// Endpoint is the endpoint for your metrics backend.
Endpoint string `yaml:"endpoint"`
// Tags are the base tags that will be applied to all metrics.
Tags Tags `yaml:"tags"`
// CounterSampleRate is the fraction of counters that you want to send to your
// metrics backend.
//
// Optional, defaults to 1.0
CounterSampleRate *float64 `yaml:"counterSampleRate"`
// CounterSampleRate is the fraction of histograms that you want to send to
// your metrics backend.
//
// Optional, defaults to 1.0
HistogramSampleRate *float64 `yaml:"histogramSampleRate"`
// RunSysStats indicates that you want to publish system stats.
//
// Optional, defaults to false
RunSysStats bool `yaml:"runSysStats"`
}
Config is the confuration struct for the metricsbp package.
Can be deserialized from YAML.
type CreateServerSpanHook ¶
type CreateServerSpanHook struct {
// Optional, will fallback to M when it's nil.
Metrics *Statsd
}
CreateServerSpanHook registers each Server Span with a MetricsSpanHook.
Example ¶
This example demonstrates how to use CreateServerSpanHook.
package main
import (
"github.com/reddit/baseplate.go/metricsbp"
"github.com/reddit/baseplate.go/tracing"
)
func main() {
const prefix = "service.server"
// initialize the CreateServerSpanHook
hook := metricsbp.CreateServerSpanHook{}
// register the hook with Baseplate
tracing.RegisterCreateServerSpanHooks(hook)
}
func (CreateServerSpanHook) OnCreateServerSpan ¶
func (h CreateServerSpanHook) OnCreateServerSpan(span *tracing.Span) error
OnCreateServerSpan registers MetricSpanHooks on a server Span.
type LogWrapperArgs ¶ added in v0.6.0
type LogWrapperArgs struct {
// The metrics path of the counter.
//
// Optional. If it's non-empty,
// every time the generaged log.Wrapper is called, counter will be added by 1.
Counter string
// Additional tags to be applied to the metrics, optional.
Tags Tags
// Statsd to use.
//
// Optional. If this is nil, metricsbp.M will be used instead.
Statsd *Statsd
// The base log.Wrapper implementation.
//
// Optional. If this is nil,
// then LogWrapper implementation will only emit metrics without logging.
Wrapper log.Wrapper
}
LogWrapperArgs defines the args used by LogWrapper.
type SampledCounter ¶
SampledCounter is a metrics.Counter implementation that actually sample the Add calls.
func (SampledCounter) Add ¶
func (c SampledCounter) Add(delta float64)
Add implements metrics.Counter.
type SampledHistogram ¶
SampledHistogram is a metrics.Histogram implementation that actually sample the Observe calls.
func (SampledHistogram) Observe ¶
func (h SampledHistogram) Observe(value float64)
Observe implements metrics.Histogram.
type Statsd ¶
type Statsd struct {
// contains filtered or unexported fields
}
Statsd defines a statsd reporter (with influx extension) and the root of the metrics.
It can be used to create metrics, and also maintains the background reporting goroutine,
It supports metrics tags in Influxstatsd format.
Please use NewStatsd to initialize it.
When a *Statsd is nil, any function calls to it will fallback to use M instead, so they are gonna be safe to use (unless M was explicitly overridden as nil). For example:
st := (*metricsbp.Statsd)(nil)
st.Counter("my-counter").Add(1) // does not panic unless metricsbp.M is nil
func NewStatsd ¶
func NewStatsd(ctx context.Context, cfg StatsdConfig) *Statsd
NewStatsd creates a Statsd object.
It also starts a background reporting goroutine when Address is not empty. The goroutine will be stopped when the passed in context is canceled.
NewStatsd never returns nil.
func (*Statsd) Close ¶
Close flushes all metrics not written to collector (if Address was set), and cancel the context, thus stop all background goroutines started by this Statsd.
After Close() is called, no more metrics will be send to the remote collector, similar to the situation that this Statsd was initialized without Address set, but the difference is that calling Close() again will do the manual flush again.
After Close() is called, Ctx() will always return an already canceled context.
This function is useful for jobs that exit, to make sure that all metrics are flushed before exiting. For server code, there's usually no need to call Close(), just cancel the context object passed in is sufficient. But server code can also choose to pass in a background context, and use Close() call to do the cleanup instead of canceling the context.
func (*Statsd) Counter ¶
Counter returns a counter metrics to the name, with sample rate inherited from StatsdConfig.
func (*Statsd) CounterWithRate ¶ added in v0.2.0
CounterWithRate returns a counter metrics to the name, with sample rate passed in instead of inherited from StatsdConfig.
func (*Statsd) Ctx ¶
Ctx provides a read-only access to the context object this Statsd holds.
It's useful when you need to implement your own goroutine to report some metrics (usually gauges) periodically, and be able to stop that goroutine gracefully. For example:
func reportGauges() {
gauge := metricsbp.M.Gauge("my-gauge")
go func() {
ticker := time.NewTicker(time.Minute)
defer ticker.Stop()
for {
select {
case <- metricsbp.M.Ctx().Done():
return
case <- ticker.C:
gauge.Set(getValue())
}
}
}
}
func (*Statsd) Gauge ¶
Gauge returns a gauge metrics to the name.
Please note that gauges are considered "low level". In most cases when you use a Gauge, you want to use RuntimeGauge instead.
func (*Statsd) Histogram ¶
Histogram returns a histogram metrics to the name with no specific unit, with sample rate inherited from StatsdConfig.
func (*Statsd) HistogramWithRate ¶ added in v0.2.0
HistogramWithRate returns a histogram metrics to the name with no specific unit, with sample rate passed in instead of inherited from StatsdConfig.
func (*Statsd) RunSysStats ¶
func (st *Statsd) RunSysStats()
RunSysStats starts a goroutine to periodically pull and report sys stats.
All the sys stats will be reported as RuntimeGauges.
Canceling the context passed into NewStatsd will stop this goroutine.
func (*Statsd) RuntimeGauge ¶ added in v0.4.1
RuntimeGauge returns a Gauge that's suitable to report runtime data.
It will be applied with "runtime." prefix and instance+pid tags automatically.
All gauges reported from RunSysStats are runtime gauges.
func (*Statsd) Timing ¶
Timing returns a histogram metrics to the name with milliseconds as the unit, with sample rate inherited from StatsdConfig.
func (*Statsd) TimingWithRate ¶ added in v0.2.0
TimingWithRate returns a histogram metrics to the name with milliseconds as the unit, with sample rate passed in instead of inherited from StatsdConfig.
func (*Statsd) WriteTo ¶ added in v0.2.1
WriteTo calls the underlying statsd implementation's WriteTo function.
Doing this will flush all the buffered metrics to the writer, so in most cases you shouldn't be using it in production code. But it's useful in unit tests to verify that you have the correct metrics you want to report.
type StatsdConfig ¶
type StatsdConfig struct {
// Prefix is the common metrics path prefix shared by all metrics managed by
// (created from) this Metrics object.
//
// If it's not ending with a period ("."), a period will be added.
Prefix string
// The reporting sample rate used when creating counters and
// timings/histograms, respectively.
//
// DefaultSampleRate will be used when they are nil (zero value).
//
// Use Float64Ptr to convert literals or other values that you can't get the
// pointer directly.
//
// To override global sample rate set here for particular counters/histograms,
// use CounterWithRate/HistogramWithRate/TimingWithRate.
CounterSampleRate *float64
HistogramSampleRate *float64
// Address is the UDP address (in "host:port" format) of the statsd service.
//
// It could be empty string, in which case we won't start the background
// reporting goroutine.
//
// When Address is the empty string,
// the Statsd object and the metrics created under it will not be reported
// anywhere,
// so it can be used in lieu of discarded metrics in test code.
// But the metrics are still stored in memory,
// so it shouldn't be used in lieu of discarded metrics in prod code.
Address string
// The log level used by the reporting goroutine.
LogLevel log.Level
// Tags are the tags to be attached to every metrics created from this Statsd
// object. For tags only needed by some metrics, use Counter/Gauge/Timing.With()
// instead.
Tags Tags
}
StatsdConfig is the configs used in NewStatsd.
type Tags ¶ added in v0.4.0
Tags allows you to specify tags as a convenient map and provides helpers to convert them into other formats.
func (Tags) AsStatsdTags ¶ added in v0.4.0
AsStatsdTags returns the tags in the format expected by the statsd metrics client, that is a slice of strings.
This method is nil-safe and will just return nil if the receiver is nil.
type Timer ¶
Timer is a timer wraps a histogram.
It's very similar to go-kit's Timer, with a few differences:
1. The reporting unit is millisecond and non-changeable.
2. It's nil-safe (zero values of *Timer or Timer will be safe to call, but they are no-ops)
Example ¶
This example demonstrates how to use Timer.
package main
import (
"context"
"github.com/reddit/baseplate.go/metricsbp"
)
// This example demonstrates how to use Timer.
func main() {
type timerContextKeyType struct{}
// variables should be properly initialized in production code
var (
ctx context.Context
timerContextKey timerContextKeyType
)
const metricsPath = "dummy.call.timer"
// initialize and inject a timer into context
ctx = context.WithValue(
ctx,
timerContextKey,
metricsbp.NewTimer(metricsbp.M.Timing(metricsPath)),
)
// do the work
dummyCall(ctx)
// get the timer out of context and report
if t, ok := ctx.Value(timerContextKey).(*metricsbp.Timer); ok {
t.ObserveDuration()
}
}
func dummyCall(_ context.Context) {}
func (*Timer) ObserveDuration ¶
func (t *Timer) ObserveDuration()
ObserveDuration reports the time elapsed via the wrapped histogram.
If either t or *t is zero value, it will be no-op.
The reporting unit is millisecond.
Source Files