Documentation
¶
Overview ¶
Package observe provides observability primitives for forge-generated services: Connect interceptors for logging, tracing, metrics, recovery, and request-id correlation, plus opt-in helpers (LogCall, TraceCall, RecordCall) for explicit per-method instrumentation inside internal packages.
Why interceptors, not method-by-method codegen ¶
Earlier forge versions emitted per-package middleware_gen.go, tracing_gen.go and metrics_gen.go wrappers around every contract.go interface. That covered the case where one internal Service called another and you wanted observability at the inner call boundary — but the cost was four generated files per internal package, plus a regeneration step every time a contract changed.
In practice almost every observability need is a request-scoped one: "log this RPC", "trace this RPC", "count this RPC". Connect interceptors capture all of those at the handler boundary, once, without per-package codegen. Internal-package observability — when one Service calls another and you want a child span — is expressed opt-in via the Trace/Log/Record helpers: explicit, greppable, and only paid when the user actually wants it.
The interceptor chain ¶
Most projects want a canonical chain. DefaultMiddlewares returns it:
interceptors := observe.DefaultMiddlewares(observe.DefaultMiddlewareDeps{
Logger: logger,
Tracer: tracer,
Meter: meter,
})
Order matters; see the DefaultMiddlewares docstring for the rationale.
Projects that want a custom chain compose interceptors directly:
interceptors := []connect.Interceptor{
observe.RecoveryInterceptor(logger),
observe.RequestIDInterceptor(),
observe.LoggingInterceptor(logger),
auth.Interceptor(...),
}
Per-method opt-in inside internal packages ¶
When one Service method calls another, wrap the inner call with a helper to produce a child span / log / metric:
func (s *svc) DoThing(ctx context.Context, req Req) (Resp, error) {
return observe.TraceCall(ctx, tracer, "userstore.Get", func(ctx context.Context) (User, error) {
return s.userStore.Get(ctx, req.UserID)
})
}
This replaces the auto-generated per-method wrapper with an explicit call site. The mock_gen.go file is still emitted by forge generate (greppable test seam) — only the middleware/tracing/metrics codegen is gone.
Index ¶
- Constants
- func Chain(deps Deps) []connect.Interceptor
- func ContextWithRequestID(ctx context.Context, id string) context.Context
- func DefaultMiddlewares(deps DefaultMiddlewareDeps) []connect.Interceptor
- func FromContext(ctx context.Context) *slog.Logger
- func LogCall(ctx context.Context, logger *slog.Logger, method string, start time.Time, ...)
- func LoggingInterceptor(logger *slog.Logger) connect.Interceptor
- func MetricsInterceptor(meter metric.Meter) connect.Interceptor
- func RecoveryInterceptor(logger *slog.Logger) connect.Interceptor
- func RequestIDFromContext(ctx context.Context) string
- func RequestIDInterceptor() connect.Interceptor
- func Setup(ctx context.Context, cfg Config) (func(context.Context) error, http.Handler, error)
- func TraceCall[T any](ctx context.Context, tracer trace.Tracer, operationName string, ...) (T, error)
- func TraceVoidCall(ctx context.Context, tracer trace.Tracer, operationName string, ...) error
- func TracingInterceptor(tracer trace.Tracer) connect.Interceptor
- func WithLogger(ctx context.Context, logger *slog.Logger) context.Context
- type CallMetrics
- type Config
- type DefaultMiddlewareDeps
- type Deps
Constants ¶
const RequestIDHeader = "X-Request-Id"
RequestIDHeader is the canonical correlation header read on inbound requests and echoed onto responses. Mirrors the value used by the scaffolded pkg/middleware.RequestIDMiddleware (HTTP layer) so the two stay consistent end-to-end.
Variables ¶
This section is empty.
Functions ¶
func Chain ¶
func Chain(deps Deps) []connect.Interceptor
Chain returns the canonical Connect interceptor chain for a project composed with EXPLICIT collaborators — the per-server composition-root counterpart to DefaultMiddlewares.
The order is the canonical forge order, with the application layer (auth → audit → rate-limit) sitting INNER to the observability layer and OUTER to the handler, exactly the position DefaultMiddlewares documents for Extras:
- RecoveryInterceptor — outermost; observes panics from everything.
- RequestIDInterceptor — mints/propagates the correlation id early.
- LoggingInterceptor — one record per RPC.
- TracingInterceptor — one OTel span per RPC.
- MetricsInterceptor — calls/errors/duration.
- Auth — authenticate (when non-nil). Inner to observability so auth failures are still logged / traced / counted.
- Audit — durable audit record (when non-nil); runs after auth so it sees the authenticated principal.
- RateLimit — throttle (when non-nil); keyed off the authenticated subject auth attached, so it follows auth.
- Extras — remaining project interceptors, in order.
nil collaborator fields are simply skipped — the chain has no fixed length (unlike the observability-only DefaultMiddlewares, which keeps a stable length via no-op interceptors). A worker process passing all-nil application collaborators gets the pure observability chain.
func ContextWithRequestID ¶
ContextWithRequestID attaches id to ctx so downstream handlers and log call sites can correlate work across goroutines.
func DefaultMiddlewares ¶
func DefaultMiddlewares(deps DefaultMiddlewareDeps) []connect.Interceptor
DefaultMiddlewares returns the canonical Connect interceptor chain for forge-generated services.
Order, and why ¶
The order is:
RecoveryInterceptor — outermost; observes panics from every subsequent layer + the handler. If you put it later, an interceptor crash propagates to the client as a torn connection instead of a clean Internal error.
RequestIDInterceptor — runs early so log records, traces, and metrics from later layers can attribute themselves to the same request ID. Trusts an inbound RequestIDHeader when present, mints a fresh ID otherwise.
LoggingInterceptor — emits one record per RPC. Sits before tracing/metrics so its timing reflects ALL inner cost (including the OTel work itself). Logging is cheap; the placement is about "this is what the user paid".
TracingInterceptor — wraps the handler in an OTel span. Inner to logging so the span name is stable per procedure even when auth/tenant rewrites context.
MetricsInterceptor — innermost observability layer. Records calls/errors/duration. Sitting after tracing means the duration histogram measures handler-only time (excluding upstream observability cost), which is the more meaningful number.
Extras — project-specific interceptors. The canonical position for auth, tenant, rate-limit, idempotency is INNER to observability (so failures from those layers still get logged, traced and counted) and OUTER to the handler. DefaultMiddlewares appends Extras in the order supplied; callers control inter-Extra ordering.
Auth-first vs auth-last ¶
A reasonable alternative is "auth at position 1, before everything else" — the case being that observability of unauthenticated traffic is noise. The forge default is auth-after-observability for two reasons:
- Operators want to see authentication failures (count, rate, source) in the same dashboards they see successful traffic. Logging and metrics need to run.
- The Connect handler's procedure routing (which the observability layer attributes against) happens BEFORE any interceptor, so observability sees procedure regardless of order.
Projects that disagree can build a custom chain — DefaultMiddlewares is opinionated, not mandatory.
nil deps ¶
nil tracer / nil meter degrade to pass-through interceptors. nil logger falls back to slog.Default. This makes DefaultMiddlewares safe to wire from a test harness that doesn't configure OTel.
func FromContext ¶
FromContext returns the *slog.Logger previously stored by WithLogger, or slog.Default() when none is present (or a nil one was stored). It never returns nil, so call sites can log unconditionally:
observe.FromContext(ctx).Info("doing the thing", "id", id)
This is the read half of the logger-from-context convention that lets non-server shapes (CLI commands, standalone binaries) share the server logger without a global or an extra parameter.
func LogCall ¶
LogCall emits a single slog.Info record summarising a wrapped operation that returns an error. Use it for opt-in per-method observability when one Service calls another and you want a log line at the inner boundary:
start := time.Now() user, err := s.userStore.Get(ctx, id) observe.LogCall(ctx, logger, "userstore.Get", start, err)
The record shape (msg = method, attrs = duration + error) matches what the now-removed middleware_gen.go used to emit, so dashboards keyed on those attribute names keep working.
nil-safe on logger.
func LoggingInterceptor ¶
func LoggingInterceptor(logger *slog.Logger) connect.Interceptor
LoggingInterceptor returns a Connect interceptor that emits one slog.Info record per RPC: procedure, duration, request_id, and (on failure) error. Matches the shape long-emitted by the scaffolded pkg/middleware.LoggingInterceptor — projects that adopt the chain via DefaultMiddlewares get the same log records without keeping a copy of the interceptor in their tree.
func MetricsInterceptor ¶
func MetricsInterceptor(meter metric.Meter) connect.Interceptor
MetricsInterceptor returns a Connect interceptor that records three OpenTelemetry metrics per RPC:
- rpc.server.calls (counter, attribute: procedure)
- rpc.server.errors (counter, attribute: procedure)
- rpc.server.duration (histogram seconds, attribute: procedure)
Streaming handlers record one duration sample per stream end. A nil meter disables metrics (interceptor is a pass-through), matching TracingInterceptor's behaviour for tracer == nil.
func RecoveryInterceptor ¶
func RecoveryInterceptor(logger *slog.Logger) connect.Interceptor
RecoveryInterceptor returns a Connect interceptor that recovers from panics inside downstream handlers, logs the recovered value plus the stack, and returns connect.CodeInternal so the client never sees a torn connection.
Place this FIRST in the chain so it observes panics from every subsequent interceptor and the handler itself.
func RequestIDFromContext ¶
RequestIDFromContext returns the request ID stored on ctx (empty when absent). Nil-context safe.
func RequestIDInterceptor ¶
func RequestIDInterceptor() connect.Interceptor
RequestIDInterceptor returns a Connect interceptor that ensures every inbound request has a correlation ID:
- If the inbound request carries a non-empty RequestIDHeader, that value is trusted and propagated. This lets edge proxies and upstream services stitch a single trace across hops.
- Otherwise a fresh 16-byte hex token is minted via crypto/rand.
The chosen ID is stored on ctx (RequestIDFromContext) and echoed onto the response header so clients can log it for later correlation.
Place this AFTER RecoveryInterceptor (so panics still get the ID in their log line) and BEFORE LoggingInterceptor (so log records inherit the ID).
func Setup ¶
Setup initializes OpenTelemetry trace and metric providers from an explicit Config and installs them as the global providers. A Prometheus exporter is always registered so the returned metricsHandler (mount at /metrics) is available regardless of OTLP. When Config.OTLPEndpoint is non-empty, an OTLP/gRPC trace exporter and an OTLP/gRPC metric exporter are also configured for push-based collection, the global text-map propagator is set to TraceContext+Baggage, and a resource describing the service is attached.
Setup performs NO environment reads: the OTLP endpoint, service name/version, and instance id all come from Config. This is the library form of the code that forge previously generated into each app's cmd/otel.go.
It returns a shutdown function (flushes/stops the providers), an http.Handler for /metrics, and any error.
func TraceCall ¶
func TraceCall[T any](ctx context.Context, tracer trace.Tracer, operationName string, fn func(context.Context) (T, error)) (T, error)
TraceCall executes fn inside a new OpenTelemetry span named operationName and returns the inner result. Span errors are recorded when fn returns a non-nil error.
Use TraceCall to express the previous tracing_gen.go behaviour at explicit call sites:
user, err := observe.TraceCall(ctx, tracer, "userstore.Get", func(ctx context.Context) (User, error) {
return s.userStore.Get(ctx, id)
})
nil-safe on tracer (executes fn directly, no span).
func TraceVoidCall ¶
func TraceVoidCall(ctx context.Context, tracer trace.Tracer, operationName string, fn func(context.Context) error) error
TraceVoidCall is the no-result variant of TraceCall, for operations that return only an error. Same semantics, no value.
func TracingInterceptor ¶
func TracingInterceptor(tracer trace.Tracer) connect.Interceptor
TracingInterceptor returns a Connect interceptor that creates one OpenTelemetry span per RPC. The span name is the full procedure ("/service.v1.Foo/Bar"); errors are recorded via span.RecordError + span.SetStatus(codes.Error, …).
A nil tracer disables tracing (interceptor is a pass-through). This keeps DefaultMiddlewares safe to wire in test harnesses that don't configure OTel.
func WithLogger ¶
WithLogger returns a copy of ctx carrying logger. The runtime (serverkit.Run, a cobra RunE, a worker fan-out) calls this once at the top of a request/command so downstream code can recover the request-scoped logger without threading it through every signature.
A nil logger is stored as-is; FromContext substitutes a usable default when it reads one back, so callers never have to nil-check.
Types ¶
type CallMetrics ¶
type CallMetrics struct {
// contains filtered or unexported fields
}
CallMetrics is a triple of OpenTelemetry instruments for opt-in per-method instrumentation inside internal packages. Use NewCallMetrics to construct; reuse a single CallMetrics per package (creating instruments per-call is expensive).
func NewCallMetrics ¶
func NewCallMetrics(meter metric.Meter, namespace string) *CallMetrics
NewCallMetrics builds a CallMetrics on meter using the canonical "<namespace>.{calls,errors,duration}" names — matching the metric names the now-removed metrics_gen.go template emitted, so dashboards keyed on those names keep working.
Errors from meter.Int64Counter / Float64Histogram are silently dropped (the same posture as contractkit.NewMetrics). meter == nil returns a CallMetrics whose RecordCall / RecordError methods are no-ops, so callers can wire a CallMetrics in test harnesses without configuring OTel.
func (*CallMetrics) RecordCall ¶
RecordCall increments the call counter and (if err != nil) the error counter, and records duration on the histogram. method is attached as the "method" attribute on each instrument.
Idiomatic call site:
start := time.Now() out, err := s.inner.Do(ctx, req) s.metrics.RecordCall(ctx, "Do", start, err) return out, err
nil-safe on the receiver.
type Config ¶
type Config struct {
// ServiceName is the logical service name reported on traces and metrics
// (semconv service.name). Empty falls back to "unknown".
ServiceName string
// ServiceVersion is reported as semconv service.version when non-empty and
// not "dev". The "dev" sentinel is treated as "no version" to match the
// prior env-driven behaviour.
ServiceVersion string
// OTLPEndpoint is the OTLP/gRPC collector endpoint (e.g.
// "http://localhost:4317" or "collector:4317"). Empty means no OTLP
// exporter is configured: only the always-on Prometheus reader is wired,
// so /metrics still works.
OTLPEndpoint string
// InstanceID is reported as semconv service.instance.id when non-empty.
// Callers typically pass os.Hostname() (the env-free input stays in the
// app, not in this library).
InstanceID string
}
Config is the explicit, typed configuration for Setup. Every field that influences exporter or resource behaviour is named here; Setup never reads os.Getenv or uses resource.WithFromEnv, so the caller (a forge app) owns the full configuration surface via its typed config.
type DefaultMiddlewareDeps ¶
type DefaultMiddlewareDeps struct {
// Logger is the slog.Logger used by RecoveryInterceptor and
// LoggingInterceptor. nil falls back to slog.Default.
Logger *slog.Logger
// Tracer feeds TracingInterceptor. nil disables tracing.
Tracer trace.Tracer
// Meter feeds MetricsInterceptor. nil disables metrics.
Meter metric.Meter
// Extras are appended to the canonical chain in the order supplied
// — useful for project-specific interceptors (auth, tenant,
// rate-limit, idempotency, audit) that the canonical chain doesn't
// know about. Order is preserved relative to each other and they
// run AFTER the observability layer.
Extras []connect.Interceptor
}
DefaultMiddlewareDeps is the dependency bag for DefaultMiddlewares. Every field is optional — a nil Logger, Tracer, or Meter cleanly degrades the corresponding interceptor (slog.Default for the logger, pass-through interceptors for tracer/meter), so projects that haven't configured OTel can still use the helper.
type Deps ¶
type Deps struct {
// Logger feeds RecoveryInterceptor and LoggingInterceptor. nil falls
// back to slog.Default — same degradation as DefaultMiddlewareDeps.
Logger *slog.Logger
// Tracer feeds TracingInterceptor. nil disables tracing.
Tracer trace.Tracer
// Meter feeds MetricsInterceptor. nil disables metrics.
Meter metric.Meter
// Auth is the project's authentication interceptor — the value
// returned by authn.NewInterceptor(policy), with the token validator
// and identity enricher already threaded through the Policy. This is
// the field that retires middleware.SetTokenValidator /
// SetIdentityEnricher: the composition root builds the interceptor
// with its validator in hand and passes it here. nil skips the auth
// layer entirely (e.g. a worker-only process, or a test harness).
Auth connect.Interceptor
// Audit is the project's audit interceptor — the value returned by
// middleware.AuditInterceptor(sink, ...), with the durable audit sink
// already bound. This retires middleware.SetAuditStore: the
// composition root opens the store, builds the interceptor around it,
// and passes it here. nil skips audit.
Audit connect.Interceptor
// RateLimit is the project's rate-limit interceptor — the value
// returned by middleware.RateLimitInterceptor(opts, claimsLookup), or
// nil when rate limiting is disabled (RateLimitInterceptor itself
// returns nil for Rps <= 0, so the composition root can pass its
// result through unconditionally and a disabled limiter cleanly drops
// out of the chain).
RateLimit connect.Interceptor
// Extras are additional project-specific interceptors (tenant
// scoping, idempotency, anything the named fields don't cover),
// appended AFTER the named application layer in the order supplied.
// Inter-Extra order is the caller's to control.
Extras []connect.Interceptor
}
Deps is the EXPLICIT-COLLABORATOR dependency bag for Chain, the composition-root variant of DefaultMiddlewares.
Why a second bag ¶
DefaultMiddlewareDeps models the observability layer plus an opaque Extras slice the caller assembles by hand. That worked when the cmd shim built the auth / audit / rate-limit interceptors itself and append- ordered them into Extras. Under explicit per-server composition the composition root has those collaborators as LOCALS (it just constructed the token validator, the audit sink interceptor, the rate limiter) and wants to hand them in by NAME — not pack them into a positional slice and rely on getting the canonical inner-order right. Deps names each project concern so a project NEVER reaches for a package-global (SetTokenValidator / SetAuditStore / SetIdentityEnricher): it passes the already-built interceptor straight into the field.
Every collaborator field is a connect.Interceptor (or nil to skip it), which is exactly what the project's builders already produce — authn.NewInterceptor for Auth, middleware.AuditInterceptor for Audit, middleware.RateLimitInterceptor for RateLimit. Passing the constructed VALUE (not a setter) is the whole point: the wiring is visible at the call site and testable without globals.