telemetry

package
v0.2.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 3, 2026 License: MIT Imports: 43 Imported by: 0

Documentation

Overview

Package telemetry is chok's observability battery: Prometheus metrics, OpenTelemetry traces and, optionally, the App's logs copied to OpenTelemetry Logs, under one Config.

tel := telemetry.MustNew(cfg.Telemetry)
d := db.MustOpen(cfg.DB, ...)
tel.Instrument(d)
app.Use(tel, d, ...)

Registered with app.Use, a Telemetry instruments every request (it is a chok.Instrumenter: its middleware runs between ClientIP and AccessLog) and serves its registry on a listener of its own, not the App's port: GET http://127.0.0.1:9464/metrics by default, reachable from the same host only. Set telemetry.metrics.addr to :9464 for a scraper on another host (a Prometheus in another container or pod), keeping that port away from the public internet. Use it before the components whose shutdown it should still see: Close runs in reverse order, and telemetry's Close flushes the spans and log records the others' Close produced.

Names

The names follow the OpenTelemetry semantic conventions (semconv v1.43.0), written the Prometheus way: dots become underscores and the unit is a suffix.

  • http_server_request_duration_seconds{http_request_method, http_route, http_response_status_code}: a histogram, http.server.request.duration.
  • http_server_active_requests: requests in flight.
  • db_client_operation_duration_seconds{db_system_name, db_operation_name, db_collection_name, error_type}: a histogram, db.client.operation.duration, after Instrument.
  • db_client_slow_operations_total{db_system_name, db_operation_name, db_collection_name}: statements at or over db.slow_threshold.
  • go_sql_* by db_name: the connection pools, after Instrument.
  • go_* and process_*: the Go runtime and the process.

A server span is named "{method} {route}", as in "GET /api/v1/posts/{id}", and carries http.request.method, http.route, http.response.status_code, url.path, url.scheme, server.address, network.protocol.version, user_agent.original and client.address; a 5xx sets error.type and the error status. A request no route matched is "GET unmatched" (route label unmatched). A statement's span is named "{operation} {table}", as in "SELECT posts".

Traces

Tracing is off by default. On, every request continues the trace of an incoming W3C traceparent header, or starts one for sample_ratio of them, and the spans go to stdout or, over OTLP/HTTP, to the collector at tracing.endpoint. Providers remain local by default. Give Globals() to the single process-wide Telemetry when third-party clients use otel's global defaults, or pass TracerProvider() to those clients explicitly. A binary that links telemetry records the trace id in identity's audit events.

Example

Telemetry next to the App: Use puts its middleware in the default stack, and the running App serves the metrics at http://127.0.0.1:9464/metrics, a port of their own that only this host reaches; Instrument adds the database. A Config from chok.Load has the metrics on; this one is built in code. The App here is not run: the registry is read in process.

package main

import (
	"context"
	"fmt"
	"log/slog"
	"net/http"
	"net/http/httptest"
	"strings"

	"github.com/prometheus/client_golang/prometheus/promhttp"

	"github.com/zynthara/chok"
	"github.com/zynthara/chok/db"
	"github.com/zynthara/chok/telemetry"
)

type greeting struct {
	Name string `path:"name"`
}

// Telemetry next to the App: Use puts its middleware in the default
// stack, and the running App serves the metrics at
// http://127.0.0.1:9464/metrics, a port of their own that only this host
// reaches; Instrument adds the database. A Config from chok.Load has the
// metrics on; this one is built in code. The App here is not run: the
// registry is read in process.
func main() {
	var cfg telemetry.Config
	cfg.Metrics.Enabled = true
	tel := telemetry.MustNew(cfg)
	defer func() { _ = tel.Close(context.Background()) }()

	quiet := slog.New(slog.DiscardHandler)
	d := db.MustOpen(db.Config{Driver: "sqlite", SQLite: db.SQLiteConfig{Path: ":memory:"}}, db.Logger(quiet))
	defer func() { _ = d.Close(context.Background()) }()
	tel.Instrument(d)

	app := chok.New(chok.Base{}, chok.Logger(quiet))
	app.Use(tel, d)
	chok.GET(app.Router(), "/hello/{name}", func(ctx context.Context, req *greeting) (string, error) {
		var one int
		err := d.Unsafe(ctx).Raw("SELECT 1").Scan(&one).Error
		return "hello " + req.Name, err
	})
	app.Handler().ServeHTTP(httptest.NewRecorder(), httptest.NewRequest(http.MethodGet, "/hello/ann", nil))

	rec := httptest.NewRecorder()
	promhttp.HandlerFor(tel.Registry(), promhttp.HandlerOpts{}).ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/metrics", nil))
	for line := range strings.Lines(rec.Body.String()) {
		if strings.HasPrefix(line, "http_server_request_duration_seconds_count") ||
			strings.HasPrefix(line, "db_client_operation_duration_seconds_count") {
			fmt.Print(line)
		}
	}
}
Output:
db_client_operation_duration_seconds_count{db_collection_name="",db_operation_name="SELECT",db_system_name="sqlite",error_type=""} 1
http_server_request_duration_seconds_count{http_request_method="GET",http_response_status_code="200",http_route="/hello/{name}"} 1

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Config

type Config struct {
	Metrics struct {
		// Enabled records the HTTP and database metrics, with the Go
		// runtime and process collectors, and serves them on Addr and
		// Path in the Prometheus text format. Off, the registry stays
		// empty.
		Enabled bool `yaml:"enabled" default:"true"`
		// Addr is the metrics' own listener, apart from http.addr: the
		// loopback default keeps them off the network, and an address
		// such as :9464 lets a scraper on another host reach them. It
		// listens while the App's HTTP server does.
		Addr string `yaml:"addr" default:"127.0.0.1:9464"`
		// Path is where the metrics listener serves the metrics.
		Path string `yaml:"path" default:"/metrics"`
	} `yaml:"metrics"`
	Tracing struct {
		// Enabled records a span for every request and, through
		// Instrument, every SQL statement, and exports them.
		Enabled bool `yaml:"enabled"`
		// Exporter is where spans go: stdout (one JSON object per span,
		// for development) or otlp (OTLP over HTTP to Endpoint).
		Exporter string `yaml:"exporter" default:"stdout" validate:"oneof=stdout otlp"`
		// Endpoint is the OpenTelemetry collector's base URL, as in
		// OTEL_EXPORTER_OTLP_ENDPOINT: spans are posted to
		// <endpoint>/v1/traces and, with Logs.OTel, log records to
		// <endpoint>/v1/logs. https uses TLS with the system roots. The
		// other OTEL_EXPORTER_OTLP_* variables (headers, certificates,
		// timeout) still apply.
		Endpoint string `yaml:"endpoint" default:"http://localhost:4318"`
		// ServiceName is the service.name of the spans and log records;
		// empty takes OTEL_SERVICE_NAME, else the executable's name.
		ServiceName string `yaml:"service_name"`
		// SampleRatio is the fraction of new traces recorded, in (0, 1];
		// a request whose traceparent says sampled is always recorded.
		SampleRatio float64 `yaml:"sample_ratio" default:"1"`
	} `yaml:"tracing"`
	Logs struct {
		// OTel copies every record the App's loggers write (those New
		// builds from the log section, the access log included) to
		// OpenTelemetry Logs, exported over OTLP to Tracing.Endpoint
		// with the trace and span id of the request; the log output
		// itself is unchanged.
		OTel bool `yaml:"otel"`
	} `yaml:"logs"`
}

Config is the telemetry section of an application's configuration:

telemetry:
  metrics:                   # on by default
    addr: 127.0.0.1:9464     # the default: this host only
    path: /metrics
  tracing:
    enabled: true
    exporter: otlp
    endpoint: http://otel-collector:4318
    service_name: blog
    sample_ratio: 0.25
  logs:
    otel: true

New gives the zero fields of a Config built in code the defaults below: Metrics.Addr, Metrics.Path, Tracing.Exporter, Tracing.Endpoint and Tracing.SampleRatio. The switches have no zero-value default: a Config built in code sets Metrics.Enabled to record and serve the metrics.

func (*Config) Validate

func (c *Config) Validate() error

Validate checks the parts of the section that are switched on; the exporter's name, a validate tag, is checked either way. It runs when chok.Load reads the configuration and again in New.

type Option

type Option func(*options)

Option configures New.

func Globals

func Globals() Option

Globals installs this Telemetry's provider and propagator as process defaults for third-party clients. Use at most one per process; the default keeps providers local to the Telemetry instance.

func Logger

func Logger(l *slog.Logger) Option

Logger is where telemetry reports its own trouble, such as an export that failed; the default is slog.Default().

type Telemetry

type Telemetry struct {
	// contains filtered or unexported fields
}

Telemetry is the observability battery: a Prometheus registry, an OpenTelemetry tracer provider and, with logs.otel, a bridge from the App's logs to OpenTelemetry Logs. Register it with app.Use: its middleware records every request (chok.Instrumenter), its Start serves the metrics on a listener of their own, its Close flushes what is buffered.

func MustNew

func MustNew(cfg Config, opts ...Option) *Telemetry

MustNew is New that panics on error.

func New

func New(cfg Config, opts ...Option) (*Telemetry, error)

New builds the battery from cfg. Providers remain instance-local unless Globals is given. With Globals, OpenTelemetry's own errors are routed through Logger. App.Use binds the Logs bridge only to that App. Spans and log records are exported in the background, in batches; Close flushes what is buffered and stops the exporters.

func (*Telemetry) BindLogger

func (t *Telemetry) BindLogger(l *slog.Logger)

BindLogger attaches the log bridge to an App-owned logger. App.Use calls it automatically. Injected, unwrapped loggers remain application-owned.

func (*Telemetry) Close

func (t *Telemetry) Close(ctx context.Context) error

Close stops copying the logs, then flushes the spans and log records still buffered and shuts the exporters down, within ctx. The global tracer provider goes back to a no-op one if it is still this Telemetry's. Close is idempotent.

func (*Telemetry) Health

func (t *Telemetry) Health(context.Context) error

Health reports nil: telemetry's trouble, such as an unreachable collector, never takes the process out of rotation; failed exports are logged.

func (*Telemetry) Instrument

func (t *Telemetry) Instrument(d *db.DB)

Instrument observes GORM statement execution and RunInTx transaction phases. Query callbacks include result scanning; Raw().Row()/Rows() callbacks end when the cursor is obtained and exclude later Scan/Next/Close work. Instrument that application-owned streaming work with a span when its duration matters. Statements produce a client span named "{operation} {table}", such as "SELECT posts", carrying the SQL as gorm built it, with placeholders for the bound values (the text of the slow-query log), and its outcome: a failed statement's or transaction phase's error status words the error as the statement log does, by its driver, code and names, never with a value the driver's message quotes; the statement's duration in db_client_operation_duration_seconds, by operation, table and error type (canceled, timeout or error; a missing record is not an error); and db_client_slow_operations_total, counting the statements that took db.slow_threshold or longer, the threshold of the slow-query log (0 counts none). d's connection pools are exported as the go_sql_* metrics, labelled db_name with the dialect (a SQLite file's read pool with _read; a second handle of the same dialect with _2, and so on).

Instrument may be called at any time, even while d is in use; calling it again for d does nothing. With tracing and metrics off it records nothing.

func (*Telemetry) Middleware

func (t *Telemetry) Middleware() chok.Middleware

Middleware returns the request instrumentation the App puts between ClientIP and AccessLog (t implements chok.Instrumenter): it continues the trace of an incoming W3C traceparent, records a server span named "{method} {route}", such as "GET /api/v1/posts/{id}", and observes the request's duration in http_server_request_duration_seconds by method, route and status code. A request no route matched is named "GET unmatched"; a method outside the standard ones is recorded as _OTHER (the span's name says HTTP), so neither can grow the label sets. A panic counts as a 500 and goes on to the App's Recover.

func (*Telemetry) Name

func (t *Telemetry) Name() string

Name is "telemetry", its name in the health report.

func (*Telemetry) Registry

func (t *Telemetry) Registry() *prometheus.Registry

Registry returns the Prometheus registry the metrics listener serves: the HTTP, database, Go runtime and process metrics, and whatever the application registers next to them. With metrics off it stays empty and is not served. To serve it on the App's own port as well, behind the application's own guard:

app.Router().Handle(http.MethodGet, "/metrics", promhttp.HandlerFor(tel.Registry(), promhttp.HandlerOpts{}))

func (*Telemetry) Start

func (t *Telemetry) Start(ctx context.Context) error

Start serves the registry on GET metrics.path at metrics.addr, a listener of its own, apart from the App's port: the loopback default keeps the metrics off the network. It listens only while the App serves HTTP — a worker process (http.enabled false) and an App under choktest open no port — and with metrics on; otherwise it just waits for ctx. An address already in use fails the App's start.

func (*Telemetry) TracerProvider

func (t *Telemetry) TracerProvider() trace.TracerProvider

TracerProvider returns the tracer provider, for spans of the application's own; with tracing off it is a no-op provider, so instrumenting code needs no switch.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL