Documentation
¶
Overview ¶
File: client.go — the OUTBOUND Connect client stack.
This is the client-side counterpart to Chain/DefaultMiddlewares (the server-side interceptor chain). When a service that used to be bound in-process is split out into its own deployment, the consumer's wiring in internal/app/providers.go changes from "construct the package and hand the interface down" to "construct a generated Connect client over HTTP". THIS is the canonical way to build that client's HTTP stack:
stack, err := observe.NewClientStack(observe.ClientStackDeps{})
if err != nil { ... }
infra.Users = usersv1connect.NewUsersServiceClient(
stack.HTTPClient, cfg.UsersBaseURL, stack.ClientOptions...,
)
The stack bundles the three things every outbound Connect client needs and that are easy to forget individually:
- the otelconnect CLIENT interceptor — one OTel client span per RPC, rpc.client.* metrics, and W3C trace-context propagation onto the wire (the global propagator installed by observe.Setup), so the callee's server span joins the caller's trace;
- request-ID forwarding — the X-Request-Id already on ctx (put there by the server-side RequestIDInterceptor / HTTP middleware) is copied onto the outbound request, so logs correlate across the hop;
- a default timeout on the http.Client, so a hung downstream can never wedge the caller indefinitely.
File: component_chain.go — the IN-PROCESS component middleware chain.
This is the in-process twin of the Connect edge interceptor chain (Chain / DefaultMiddlewares in middleware.go). Those interceptors wrap every RPC at the transport boundary; a ComponentChain wraps every method of an internal component (a contract.go Service) at the in-process call boundary — one span, one metric sample and one log record per method call (successes sampled only when sampling is turned on; see log_policy.go) — without the method's body knowing anything about it.
A forge-generated decorator (middleware_gen.go) routes each interface method through the chain with a single line:
func (o *forgeMiddlewareService) Do(ctx context.Context, in DoInput) (DoResult, error) {
return o.chain.Around(ctx, "checkout.Do", func(ctx context.Context) (DoResult, error) {
return o.inner.Do(ctx, in)
})
}
Every contract package whose constructor returns its Service interface gets the decorator unless it opts out with `// forge:no-observe`. Its chain is DefaultComponentChain, or — when the package declares the optional owned seam observe_chain.go (newObserveChain) — whatever that seam assembles from the middlewares below plus any the user adds. The generated decorator never names a middleware itself: the chain is a runtime / composition concern, exactly like the Connect chain.
Package observe provides observability primitives for forge-generated services: Connect interceptors for logging, tracing, metrics, recovery, and request-id correlation at the RPC edge, and the in-process ComponentChain that forge's generated decorators route every internal Service method through.
Where spans come from ¶
A request should read as a tree: the RPC, the business methods it calls, and the SQL and outbound calls those make. Each layer is instrumented where forge owns the boundary, so none of it depends on someone remembering to add a span:
- the RPC edge: the Connect interceptor chain below (Chain / DefaultMiddlewares), one server span per call;
- business methods: every contract.go package whose constructor returns its Service interface gets a generated decorator (middleware_gen.go) that opens one span per method, records call metrics and logs failures, through DefaultComponentChain or the package's own observe_chain.go seam. It is the default; `// forge:no-observe` on the package (or one method) opts out. compose.go constructs the package wrapped;
- outbound calls: NewClientStack for Connect clients.
The per-call helpers (TraceCall, LogCall, RecordCall) remain for code that is not a contract package. They are the exception: an earlier version of this package recommended them over generated decorators, and in practice they were not written — most traces were a single span.
The interceptor chain ¶
Most projects want a canonical chain. DefaultMiddlewares returns it:
interceptors := observe.DefaultMiddlewares(observe.DefaultMiddlewareDeps{
Logger: logger,
Tracer: tracer,
Meter: meter,
})
Order matters; see the DefaultMiddlewares docstring for the rationale. Both DefaultMiddlewares and Chain end with svcerr.Interceptor, so a handler's svcerr error reaches the client with its Connect code.
Projects that want a custom chain compose interceptors directly, and add svcerr.Interceptor() last themselves:
interceptors := []connect.Interceptor{
observe.RecoveryInterceptor(logger),
observe.RequestIDInterceptor(),
observe.LoggingInterceptor(logger),
auth.Interceptor(...),
}
The environment contract ¶
The interface between a forge app and whatever collector exists is the standard OpenTelemetry environment. The app never learns the backend: the platform renders these variables and Setup obeys them.
OTEL_EXPORTER_OTLP_ENDPOINT base URL of the collector, e.g. http://127.0.0.1:4318
OTEL_EXPORTER_OTLP_PROTOCOL http/protobuf (default, port 4318) or grpc (port 4317)
OTEL_EXPORTER_OTLP_HEADERS k=v,k2=v2 — e.g. authorization=<key>
OTEL_EXPORTER_OTLP_{TRACES,METRICS,LOGS}_{ENDPOINT,PROTOCOL,HEADERS}
per-signal overrides of the three above
OTEL_SERVICE_NAME wins over Config.ServiceName (the compile-time default)
OTEL_RESOURCE_ATTRIBUTES merged into the resource; carries
deployment.environment.name=<forge env name>
OTEL_SDK_DISABLED=true every export off
OTEL_{TRACES,METRICS}_EXPORTER=none that signal off
OTEL_LOGS_EXPORTER=otlp the explicit switch that turns OTLP logs ON
Timeout, compression, TLS and the rest of the OTLP exporter variables are read by the exporters themselves. Config.OTLPEndpoint, when set, overrides the endpoint variables (it is a base URL; /v1/<signal> is appended for http/protobuf); headers and protocol always come from the environment. otelenv.go is the only file in this package that calls os.Getenv.
With no endpoint from any source nothing is exported — only the Prometheus /metrics reader is wired — but the W3C TraceContext+Baggage propagator is installed regardless, so an incoming traceparent is still honoured and forwarded by every observe client.
With export on, Setup also records Go runtime metrics (go.opentelemetry.io/contrib/instrumentation/runtime).
Logs ¶
serverkit's logger writes JSON lines to stdout. NewLogHandler adds trace_id and span_id to every record logged with a context that holds a valid span (use the *Context methods: logger.InfoContext(ctx, ...)). Log shipping is ONE path per environment: the collector reads the stdout/file stream. The OTLP logs exporter (OTEL_LOGS_EXPORTER=otlp or Config.OTLPLogs) is off by default and is MUTUALLY EXCLUSIVE with that collection: it does not replace stdout, so enabling both ships every line twice.
A line read from stdout has no SDK resource, so each JSON line carries the process resource under LogKeyResource ("resource"): the LogResourceKeys (service.name, service.version, service.namespace, deployment.environment.name, service.instance.id) of the resource Setup builds for traces and metrics, encoded once per logger. Collectors lift them into the record's resource (forge's node agent and local forwarder do; see pkg/deploy). A panic or any line without the field carries no version.
Error records ¶
A log record at ERROR or above is a server error event, and NewLogHandler gives it what an issue is grouped by, on both shipping paths (the stdout line and the OTLP bridge), and nothing at lower levels:
- code.function.name, code.file.path, code.line.number: the call site, read from the record's PC, so a logger that builds its own record and calls Handler().Handle directly (Temporal's slog adapter) names its caller rather than a handler frame.
- exception.type and exception.message: the innermost error of the record's first error-valued attribute (svcerr's withheld cause first, then Unwrap to the end, the first branch of a multi-error). Type is %T; the message is bounded to MaxExceptionMessage (1 KiB). Attributes the call site already set are kept.
- exception.stacktrace, from LogPanic and from svcerr. LogPanic is what every recover path in forge calls; its record's PC is the panicking frame, so panics group by where they happened, not by where they were recovered. svcerr captures a bounded stack (32 frames, 8 KiB, module-relative paths) where a server fault — Internal, Unknown, DataLoss or Unavailable — is first built or wrapped, and the record carries it, with its innermost application frame as the call site above (so an "rpc failed" issue groups by the handler's failing code, not by the interceptor that logged it). Client-class errors capture none, and a stack never crosses the wire to the client. The same stack rides the span's exception event and the log.error event.
- error.group, from ErrorGroup(ctx, name) only: an explicit issue name for the failure whose derived grouping is wrong.
A record at any level becomes an issue event of a named source when it carries issue.source (IssueSource(ctx, name)). Such issues are listed apart from errors (`forge obs issues --source <name>`), and never count as errors unless the record is also at ERROR.
File: transport.go — request-ID forwarding for PLAIN outbound HTTP.
ClientRequestIDInterceptor (client.go) forwards the correlation ID for outbound CONNECT calls. A service that talks to a third-party REST API does so through a plain *http.Client, which never sees a Connect interceptor — so without this the hop leaves the process with no X-Request-Id and the two sides' logs cannot be stitched together.
This is a RoundTripper rather than a whole client so it composes with the OTel instrumentation instead of competing with it. otelhttp goes OUTSIDE, so its client span covers the full round trip:
transport := otelhttp.NewTransport(observe.NewRequestIDTransport(http.DefaultTransport))
client := &http.Client{Transport: transport, Timeout: 30 * time.Second}
Note this is the opposite of ClientStackDeps.Transport, which must NOT be wrapped in otelhttp: there the otelconnect interceptor already opens the client span, and wrapping would nest a duplicate span under every RPC.
Index ¶
- Constants
- Variables
- func Chain(deps Deps) []connect.Interceptor
- func ClientRequestIDInterceptor() connect.Interceptor
- func ContextWithRequestID(ctx context.Context, id string) context.Context
- func DefaultMiddlewares(deps DefaultMiddlewareDeps) []connect.Interceptor
- func ErrorGroup(_ context.Context, name string) slog.Attr
- func FromContext(ctx context.Context) *slog.Logger
- func IssueSource(_ context.Context, name string) slog.Attr
- func LevelForError(err error) slog.Level
- func LogCall(ctx context.Context, logger *slog.Logger, method string, start time.Time, ...)
- func LogPanic(ctx context.Context, logger *slog.Logger, msg string, recovered any, ...)
- func LoggingInterceptor(logger *slog.Logger, opts ...LogOption) connect.Interceptor
- func MetricsInterceptor(meter metric.Meter) connect.Interceptor
- func NewErrorClassHandler(inner slog.Handler, opts ...LogOption) slog.Handler
- func NewHTTPClient(opts HTTPClientOptions) *http.Client
- func NewHTTPTransport(base http.RoundTripper) http.RoundTripper
- func NewLogHandler(next slog.Handler, cfg Config) slog.Handler
- func NewRequestIDTransport(base http.RoundTripper) http.RoundTripper
- func NewTraceHandler(next slog.Handler) slog.Handler
- func ProfileLabelsMiddleware(next http.Handler) http.Handler
- func RecordSpanError(span trace.Span, err error)
- func RecoveryInterceptor(logger *slog.Logger) connect.Interceptor
- func RequestIDFromContext(ctx context.Context) string
- func RequestIDInterceptor() connect.Interceptor
- func RunRoot(ctx context.Context, name string, fn func(context.Context) error, ...) (err error)
- func SetAppVersion(v string)
- func Setup(ctx context.Context, cfg Config) (func(context.Context) error, http.Handler, error)
- func TraceCall[T any](ctx context.Context, tracer trace.Tracer, operationName string, ...) (T, error)
- func TraceResponseInterceptor() connect.Interceptor
- func TraceResponseValue(sc trace.SpanContext) string
- func TraceVoidCall(ctx context.Context, tracer trace.Tracer, operationName string, ...) error
- func TracingInterceptor(tracer trace.Tracer) connect.Interceptor
- func Track(ctx context.Context, name string, attrs ...slog.Attr)
- func TrustRemoteBy(classify TraceTrustFunc, opts ...otelconnect.Option) (connect.Interceptor, error)
- func TrustRemoteFrom(trusted TrustPredicate, opts ...otelconnect.Option) (connect.Interceptor, error)
- func ValidEventName(name string) bool
- func WithLogger(ctx context.Context, logger *slog.Logger) context.Context
- type CallMetrics
- type ClientStack
- type ClientStackDeps
- type ComponentChain
- type ComponentMiddleware
- func LogMiddleware(logger *slog.Logger, level slog.Level, opts ...LogOption) ComponentMiddleware
- func MetricsMiddleware(meter metric.Meter, namespace string) ComponentMiddleware
- func RecoverMiddleware(logger *slog.Logger) ComponentMiddleware
- func TraceMiddleware(tracer trace.Tracer) ComponentMiddleware
- type ComponentOp
- type Config
- type Deadline
- func (d Deadline) Validate() error
- func (Deadline) WrapStreamingClient(next connect.StreamingClientFunc) connect.StreamingClientFunc
- func (d Deadline) WrapStreamingHandler(next connect.StreamingHandlerFunc) connect.StreamingHandlerFunc
- func (d Deadline) WrapUnary(next connect.UnaryFunc) connect.UnaryFunc
- type DefaultMiddlewareDeps
- type Deps
- type HTTPClientOptions
- type LogOption
- func WithErrorClassifier(classify func(err error) svcerr.Class) LogOption
- func WithExpectedErrorFunc(isExpected func(err error) bool) LogOption
- func WithExpectedErrors(targets ...error) LogOption
- func WithSlowThreshold(threshold time.Duration) LogOption
- func WithSuccessLevel(name string, level slog.Level) LogOption
- func WithSuccessSampling(window time.Duration) LogOption
- func WithUserErrorLevel(level slog.Level) LogOption
- type ProfileType
- type ProfilesConfig
- type SlowCall
- type TraceTrust
- type TraceTrustFunc
- type TrustPredicate
Constants ¶
const ( // DefaultUnaryDeadline bounds a unary RPC that Deadline.Procedures does // not name. Thirty seconds is longer than any interactive call should // take and shorter than the proxies and load balancers in front of most // servers give up after, so the server — not a middlebox — answers. DefaultUnaryDeadline = 30 * time.Second // DefaultSlowAfter is when the watchdog first reports a unary RPC that is // still running. DefaultSlowAfter = 10 * time.Second // DefaultSlowRepeat is how often the watchdog reports it again while it // keeps running, so a multi-minute hang stays in the log tail. DefaultSlowRepeat = 30 * time.Second )
The Deadline slot's defaults.
const ( LogKeyExceptionType = "exception.type" LogKeyExceptionMessage = "exception.message" LogKeyExceptionStacktrace = "exception.stacktrace" LogKeyErrorGroup = "error.group" LogKeyIssueSource = "issue.source" )
Log attribute keys an ERROR record carries for issue grouping.
const ( LogKeyTraceID = "trace_id" LogKeySpanID = "span_id" )
Log attribute keys carrying the active span, matching what the audit middleware already writes.
const ( LogEventWarn = "log.warn" LogEventError = "log.error" )
Span event names NewTraceHandler adds for a WARN or ERROR record. The gateway's tail-sampling policy matches them, so they are part of the contract between the SDK and the collector config forge renders.
const ( EnvOTLPEndpoint = "OTEL_EXPORTER_OTLP_ENDPOINT" EnvOTLPProtocol = "OTEL_EXPORTER_OTLP_PROTOCOL" EnvOTLPHeaders = "OTEL_EXPORTER_OTLP_HEADERS" // EnvLogsExporter is the explicit switch for the OTLP logs exporter. The // OpenTelemetry specification defaults it to "otlp"; forge defaults it OFF // because logs already have a shipping path (stdout/file collection), and // only the value "otlp" turns the exporter on. EnvLogsExporter = "OTEL_LOGS_EXPORTER" // EnvSDKDisabled turns every export off. Context propagation stays on. EnvSDKDisabled = "OTEL_SDK_DISABLED" // EnvProfilesExporter is the switch for continuous profiling. Profiles are // exported by default whenever an OTLP endpoint is configured; the value // "none" turns them off (the same spelling OTEL_TRACES_EXPORTER=none uses). EnvProfilesExporter = "OTEL_PROFILES_EXPORTER" // EnvOTLPProfilesEndpoint is the per-signal profiles endpoint. Unlike the // base OTEL_EXPORTER_OTLP_ENDPOINT it is a FULL URL used as given, so a // platform can send profiles to a different collector than traces. It // exists because profiles need a collector built with the profiles feature // gate and a profiles-capable exporter, which the traces collector may not be. EnvOTLPProfilesEndpoint = "OTEL_EXPORTER_OTLP_PROFILES_ENDPOINT" // EnvTracesSampler and EnvTracesSamplerArg are the SDK's sampler switch and // its argument. forge renders parentbased_traceidratio + the ratio when an // env's sampling.traces_ratio is below 1. EnvTracesSampler = "OTEL_TRACES_SAMPLER" EnvTracesSamplerArg = "OTEL_TRACES_SAMPLER_ARG" )
Environment variable names of the contract. Exported so docs, doctor checks and renderers can refer to one spelling.
const ( // ProtocolHTTPProtobuf is OTLP over HTTP with protobuf bodies (port 4318). // It is the default. ProtocolHTTPProtobuf = "http/protobuf" // ProtocolGRPC is OTLP over gRPC (port 4317). ProtocolGRPC = "grpc" )
const ( // EnvProfiles selects the profile types, as a comma-separated list of // ProfileType values ("cpu,heap,alloc"), or "none" for no profile at all. // It replaces the default set; it does not add to it. EnvProfiles = "FORGE_PROFILES" // EnvProfilesHeapInterval is how often the heap (and, when on, alloc) // snapshot is taken, as a Go duration ("15m", "1m"). EnvProfilesHeapInterval = "FORGE_PROFILES_HEAP_INTERVAL" // EnvProfilesCPUInterval is the time between capture cycles (the CPU // profile, and the opt-in goroutine / mutex / block profiles), as a Go // duration ("5m", "1m"), at least MinCPUInterval. EnvProfilesCPUInterval = "FORGE_PROFILES_CPU_INTERVAL" // EnvProfilesCPUDuration is how long each CPU profile samples, as a Go // duration ("5s", "10s"), at least MinCPUDuration and at most half the // cycle. EnvProfilesCPUDuration = "FORGE_PROFILES_CPU_DURATION" // DefaultCPUInterval is the time between capture cycles by default. DefaultCPUInterval = 5 * time.Minute // DefaultCPUDuration is how long each CPU profile samples by default. DefaultCPUDuration = 5 * time.Second // DefaultHeapInterval is how often the heap snapshot is taken by default. DefaultHeapInterval = 2 * time.Hour // MinCPUInterval is the shortest capture cycle. MinCPUInterval = 30 * time.Second // MinCPUDuration is the shortest CPU profile window. MinCPUDuration = time.Second )
const ( // EventNamePrefix starts every product event's event.name. EventNamePrefix = "product." // AttrEventName is the OTel event semconv attribute carrying the name. AttrEventName = "event.name" // AttrUserID is the authenticated user id; absent for anonymous callers. AttrUserID = "user.id" // AttrAppClient says which kind of emitter wrote the event. AttrAppClient = "app.client" // AttrAppVersion is the emitting app's version. AttrAppVersion = "app.version" // AttrPropPrefix prefixes every event property. AttrPropPrefix = "product." // AppClientBackend is the app.client value for events emitted by Track. AppClientBackend = "backend" // MaxProductProps is the most properties one event keeps; the rest are dropped. MaxProductProps = 32 // MaxProductPropValueLen is the longest property value in characters; // longer values are truncated. MaxProductPropValueLen = 256 )
The product-event contract. A product event is one INFO log record whose event.name attribute is "product.<name>"; the telemetry pipeline selects on that attribute, so the scope and logger name do not matter. The attribute keys below are the cross-repo seam: the web runtime, the Go emitter and the ClickHouse view must agree on them.
const DefaultSlowThreshold = time.Second
DefaultSlowThreshold is the duration at which a successful call is written regardless of sampling. Interactive calls finish well under it; one that does not is exactly the outlier a sample would hide.
const DefaultUserErrorLevel = slog.LevelInfo
DefaultUserErrorLevel is the level a user error or a cancellation logs at unless WithUserErrorLevel says otherwise: visible under a production INFO handler, below anything that pages.
const EnvOTLPHeadersFile = "OTEL_EXPORTER_OTLP_HEADERS_FILE"
EnvOTLPHeadersFile names a file holding OTLP request headers in the OTEL_EXPORTER_OTLP_HEADERS format, re-read when it changes. The per-signal OTEL_EXPORTER_OTLP_<SIGNAL>_HEADERS_FILE overrides it for that signal.
const ErrorClassKey = "error_class"
ErrorClassKey is the attribute every error record carries: the svcerr.Class of its error ("server", "user", "canceled").
const LogKeyResource = "resource"
LogKeyResource is the field every JSON log line carries the workload's OpenTelemetry resource identity under:
{"time":…,"level":"INFO","msg":"…","resource":{"service.name":"api","service.version":"v1.7.58",…},…}
A log line read from stdout by a collector (filelog) has no SDK resource of its own: without this field its record would carry only what the collector derives from the pod, and none of what the traces and metrics of the same process carry. The node agents forge renders (and any collector following the same contract) lift the keys under this field into the record's RESOURCE, never its attributes, and leave a key a trusted source (the Kubernetes API, the log file's path) already set alone.
Lines with no such field — a panic, a runtime fatal error, a library that writes to stdout directly, a non-forge image — carry no version: they pass through the collector unchanged.
const ( // MaxExceptionMessage bounds exception.message, in bytes, ellipsis // included. MaxExceptionMessage = 1 << 10 )
const RequestIDHeader = "X-Request-Id"
RequestIDHeader is the canonical correlation header read on inbound requests and echoed onto responses. Mirrors the value used by the scaffolded pkg/middleware.RequestIDMiddleware (HTTP layer) so the two stay consistent end-to-end.
const RootTracerName = "github.com/reliant-labs/forge/pkg/observe/root"
RootTracerName is the instrumentation scope of RunRoot's spans.
const TraceResponseHeader = "Traceresponse"
TraceResponseHeader is the W3C Trace Context Level 2 response header: `00-<trace-id>-<span-id>-<flags>`, naming the server span.
Variables ¶
var LogResourceKeys = []string{
"service.name",
"service.version",
"service.namespace",
"deployment.environment.name",
"service.instance.id",
}
LogResourceKeys are the resource attributes stamped on every log line, in the order written. They are the workload's identity across signals: what the release, the environment and the replica a line came from are. Any other resource attribute stays on the SDK signals only.
Functions ¶
func Chain ¶
func Chain(deps Deps) []connect.Interceptor
Chain returns the canonical Connect interceptor chain for a project composed with EXPLICIT collaborators — the per-server composition-root counterpart to DefaultMiddlewares.
The order is the canonical forge order, with the application layer (deadline → auth → audit → rate-limit) sitting INNER to the observability layer and OUTER to the handler, exactly the position DefaultMiddlewares documents for Extras:
- Trace — the server span (Deps.Trace, when non-nil), then the pprof labels naming it and the traceresponse header returning its context to the caller (TraceResponseInterceptor). Outside everything below, so the span covers recovery, logging and auth, and their log lines carry its ids.
- RecoveryInterceptor — observes panics from everything below.
- RequestIDInterceptor — mints/propagates the correlation id early.
- LoggingInterceptor — every failure; every success, or a sample of them when sampling is turned on.
- TracingInterceptor — one OTel span per RPC.
- MetricsInterceptor — calls/errors/duration.
- Deadline — the RPC's time budget: Deps.Deadline, Deadline{} when nil, none for serverkit.NoDeadline. Ahead of auth, so authenticating runs inside the RPC's deadline.
- Auth — authenticate (when non-nil). Inner to observability so auth failures are still logged / traced / counted.
- Audit — durable audit record (when non-nil); runs after auth so it sees the authenticated principal.
- RateLimit — throttle (when non-nil); keyed off the authenticated subject auth attached, so it follows auth.
- Extras — remaining project interceptors, in order.
- svcerr.Interceptor — always last, next to the handler, so a handler's svcerr error becomes its Connect error (PermissionDenied is permission_denied, not unknown) before audit, logging and metrics read it. Not a slot: an svcerr.Interceptor passed in Extras is moved here, not run twice.
nil collaborator fields are simply skipped — the chain has no fixed length (unlike the observability-only DefaultMiddlewares, which keeps a stable length via no-op interceptors) — except Deadline, whose nil is forge's default budget. A worker process passing all-nil application collaborators gets the observability chain plus that budget.
func ClientRequestIDInterceptor ¶
func ClientRequestIDInterceptor() connect.Interceptor
ClientRequestIDInterceptor returns a Connect interceptor for OUTBOUND clients that forwards the correlation ID already on ctx (see RequestIDFromContext) as the RequestIDHeader on the outgoing request, so the callee's logs carry the same request_id as the caller's. An ID already present on the outgoing header is left untouched; a ctx without an ID forwards nothing (the callee mints its own).
It is included in NewClientStack's canonical chain; it is exported for projects assembling a bespoke client chain.
func ContextWithRequestID ¶
ContextWithRequestID attaches id to ctx so downstream handlers and log call sites can correlate work across goroutines.
func DefaultMiddlewares ¶
func DefaultMiddlewares(deps DefaultMiddlewareDeps) []connect.Interceptor
DefaultMiddlewares returns the canonical Connect interceptor chain for forge-generated services.
Order, and why ¶
The order is:
RecoveryInterceptor — outermost; observes panics from every subsequent layer + the handler. If you put it later, an interceptor crash propagates to the client as a torn connection instead of a clean Internal error.
RequestIDInterceptor — runs early so log records, traces, and metrics from later layers can attribute themselves to the same request ID. Trusts an inbound RequestIDHeader when present, mints a fresh ID otherwise.
LoggingInterceptor — logs every failed RPC and every successful one, or a per-procedure sample of the successes when sampling is turned on (see LoggingInterceptor). Sits before tracing/metrics so its timing reflects ALL inner cost (including the OTel work itself). Logging is cheap; the placement is about "this is what the user paid".
TracingInterceptor — wraps the handler in an OTel span. Inner to logging so the span name is stable per procedure even when auth rewrites context.
MetricsInterceptor — innermost observability layer. Records calls/errors/duration. Sitting after tracing means the duration histogram measures handler-only time (excluding upstream observability cost), which is the more meaningful number.
Extras — project-specific interceptors. The canonical position for auth, rate-limit, idempotency is INNER to observability (so failures from those layers still get logged, traced and counted) and OUTER to the handler. DefaultMiddlewares appends Extras in the order supplied; callers control inter-Extra ordering.
svcerr.Interceptor — always last, next to the handler: a handler's svcerr error becomes its Connect error before any layer above reads it. Not a slot; an svcerr.Interceptor passed in Extras is moved here, not run twice.
Auth-first vs auth-last ¶
A reasonable alternative is "auth at position 1, before everything else" — the case being that observability of unauthenticated traffic is noise. The forge default is auth-after-observability for two reasons:
- Operators want to see authentication failures (count, rate, source) in the same dashboards they see successful traffic. Logging and metrics need to run.
- The Connect handler's procedure routing (which the observability layer attributes against) happens BEFORE any interceptor, so observability sees procedure regardless of order.
Projects that disagree can build a custom chain — DefaultMiddlewares is opinionated, not mandatory.
nil deps ¶
nil tracer / nil meter degrade to pass-through interceptors. nil logger falls back to slog.Default. This makes DefaultMiddlewares safe to wire from a test harness that doesn't configure OTel.
func ErrorGroup ¶ added in v1.0.0
ErrorGroup returns the attribute that pins a record to a named issue, whatever its message, call site or error type: the escape hatch when the derived grouping is wrong for one failure.
logger.ErrorContext(ctx, "reconcile drift", "err", err, observe.ErrorGroup(ctx, "billing-reconcile-drift"))
func FromContext ¶
FromContext returns the *slog.Logger previously stored by WithLogger, or slog.Default() when none is present (or a nil one was stored). It never returns nil, so call sites can log unconditionally:
observe.FromContext(ctx).Info("doing the thing", "id", id)
This is the read half of the logger-from-context convention that lets non-server shapes (CLI commands, standalone binaries) share the server logger without a global or an extra parameter.
func IssueSource ¶ added in v1.0.0
IssueSource returns the attribute that makes a record an issue event of the named source, at whatever level it is logged: the issues pipeline groups it (by its fingerprint, which ErrorGroup can pin) into issues of that source, listed apart from errors (`forge obs issues --source <name>`). Errors carry no source. Use it for a report that must be triaged like an error without counting as one — an error rate, an error alert or a release-health gate never sees it unless it is also logged at ERROR.
logger.WarnContext(ctx, "[reliant] message truncated", observe.IssueSource(ctx, "llm_bug_report"), observe.ErrorGroup(ctx, "llm_bug_report/reliant/message truncated"))
func LevelForError ¶
LevelForError is the single place that decides how loud a failed call is, under the default policy (see error_class.go).
Every failure used to log at WARN, including the ones that mean the SERVER is broken. A total database outage produced a stream of WARN records and a 500 for every request, so the standard alert rule — level=ERROR — stayed silent through the whole incident. Meanwhile a client sending a malformed field also logged WARN, which is why raising everything to ERROR is not the fix either: it just moves the noise.
The split is fault attribution — svcerr.Classify — not HTTP status:
- ERROR: the server failed and someone should be paged. Internal (a bug or a dependency down), Unavailable (a dependency refused), DataLoss, Unknown, and any error nothing classified.
- WARN: a server-side failure that says "retry", not "page" — DeadlineExceeded, Aborted, Unimplemented.
- DefaultUserErrorLevel (INFO): the request failed for a reason the server correctly detected, or the caller went away. NotFound, InvalidArgument, PermissionDenied, FailedPrecondition, ResourceExhausted, cancellations, and anything marked svcerr.WithClass(err, svcerr.ClassUser). These are the API working.
Exported so the audit interceptor and any project-owned logging site classify identically — two log streams disagreeing about severity for the same RPC is its own incident. A layer configured WithUserErrorLevel uses its own level for the user half.
func LogCall ¶
LogCall emits a single slog.Info record summarising a wrapped operation that returns an error. Use it for opt-in per-method observability when one Service calls another and you want a log line at the inner boundary:
start := time.Now() user, err := s.userStore.Get(ctx, id) observe.LogCall(ctx, logger, "userstore.Get", start, err)
The record shape (msg = method, attrs = duration + error) matches what the now-removed middleware_gen.go used to emit, so dashboards keyed on those attribute names keep working.
nil-safe on logger.
func LogPanic ¶ added in v1.0.0
func LogPanic(ctx context.Context, logger *slog.Logger, msg string, recovered any, attrs ...slog.Attr)
LogPanic logs a recovered panic at ERROR on logger. Call it from the deferred function that called recover():
defer func() {
if rec := recover(); rec != nil {
observe.LogPanic(ctx, logger, "panic recovered", rec, slog.String("path", p))
}
}()
The record's PC is the PANICKING frame, not the recover site, so a handler reading it (otelslog's code.function.name, slog's AddSource) groups panics by where they happened. The stack goes in exception.stacktrace; "panic" is the recovered value.
func LoggingInterceptor ¶
func LoggingInterceptor(logger *slog.Logger, opts ...LogOption) connect.Interceptor
LoggingInterceptor returns a Connect interceptor that logs RPCs: procedure, duration, request_id, and (on failure) error.
- Every FAILED call is written — "rpc failed" / "stream failed" — with the error, its cause, its error_class and its code, at the level its class earns (see error_class.go): ERROR for a server fault, the user-error level (INFO; WithUserErrorLevel) for a user error or a cancellation.
- Every SUCCESSFUL call is written — "rpc completed" / "stream completed", at INFO — unless WithSuccessSampling turns success sampling on for this layer. Sampled, a procedure's first success is written, then at most one per window, carrying `suppressed`, the successes of that procedure since the previous record that were not written. A unary success at or above DefaultSlowThreshold is always written, with slow=true. See log_policy.go.
- A failure whose error WithExpectedErrors declared an expected outcome is still "rpc failed" with its error — the client did get one — but is written like a success: at the procedure's success level, sampled with it, carrying expected=true.
opts tune the success half — WithSuccessSampling, WithSlowThreshold, WithSuccessLevel for one procedure, WithExpectedErrors — and the failure levels: WithUserErrorLevel, WithErrorClassifier. Through Chain / DefaultMiddlewares they are passed as Deps.LogOptions.
func MetricsInterceptor ¶
func MetricsInterceptor(meter metric.Meter) connect.Interceptor
MetricsInterceptor returns a Connect interceptor that records three OpenTelemetry metrics per RPC:
- rpc.server.calls (counter, attribute: procedure)
- rpc.server.errors (counter, attribute: procedure)
- rpc.server.duration (histogram seconds, attribute: procedure)
Streaming handlers record one duration sample per stream end. A nil meter disables metrics (interceptor is a pass-through), matching TracingInterceptor's behaviour for tracer == nil.
func NewErrorClassHandler ¶
NewErrorClassHandler wraps inner so that EVERY record in the process obeys the error-class policy, not only the ones forge's interceptors write. A record above the user-error level that carries an error is classified:
- it gains error_class=<class> (unless it already carries one — a layer that classified it, such as LoggingInterceptor, is trusted);
- if the error is not a server fault, its level is lowered to the user-error level. A level is only ever lowered: a server error logged at WARN stays WARN, because the call site chose that.
The error is the first attribute whose value is an `error`, whatever its key ("error", "err", logr's "err"). Attributes bound earlier with Logger.With are not searched; an error belongs on the record that reports it. A record with no error attribute, or at or below the user-error level, passes through untouched and is not scanned.
Wrap it OUTSIDE anything that routes on level — a Sentry forwarder, an error counter — so they see the classified level:
handler := slog.NewJSONHandler(os.Stdout, opts) handler = sentryHandler(handler) // forwards >= ERROR handler = observe.NewErrorClassHandler(handler) slog.SetDefault(slog.New(handler))
opts are WithUserErrorLevel and WithErrorClassifier; other LogOptions concern call outcomes and are ignored here.
func NewHTTPClient ¶ added in v1.0.0
func NewHTTPClient(opts HTTPClientOptions) *http.Client
NewHTTPClient returns a fresh *http.Client whose every request is an OTel CLIENT span ("<METHOD> <host>"), carries the caller's trace context (W3C traceparent, through the global propagator observe.Setup installs) and its X-Request-Id. Fresh per call: a consumer may change CheckRedirect or Jar on it without affecting anyone else, while the default transport, and so the connection pool, is shared.
func NewHTTPTransport ¶ added in v1.0.0
func NewHTTPTransport(base http.RoundTripper) http.RoundTripper
NewHTTPTransport wraps base (nil: http.DefaultTransport) so every request through it is traced and forwards the request ID. otelhttp sits OUTSIDE the request-ID transport, so its span covers the whole round trip including the header the inner layer adds. Prefer NewHTTPClient; this is for a client whose construction you do not own.
func NewLogHandler ¶
NewLogHandler is the handler serverkit builds for the process logger: next (the stdout JSON or text handler) wrapped by NewTraceHandler. Every record at ERROR or above also gains the attributes an issue is grouped by — code.function.name, code.file.path, code.line.number, and for a record carrying an error exception.type and exception.message (see error_inputs.go) — on both shipping paths. When the OTLP logs switch is on (Config.OTLPLogs or OTEL_LOGS_EXPORTER=otlp) it is also teed into the OpenTelemetry logs bridge, which Setup connects to an OTLP exporter.
OTLP logs export is OFF by default and is the SECOND shipping path, not an addition to the first: the stdout stream is still written, so a workload that enables the OTLP exporter must not also be collected from its stdout/log files (the local collector's file reader, a cluster node agent), or every line arrives twice. Exactly one shipping path per environment.
Every line next writes (a JSON handler; a *slog.TextHandler is left alone) also carries the process resource under LogKeyResource: service.name, service.version, service.namespace, deployment.environment.name and service.instance.id, from the resource Setup builds for the traces and metrics — Setup's own once it has run, the same inputs (cfg and the OTEL_* environment) before. Pass the Config you pass Setup. The collector reading stdout lifts them into the record's resource; the OTLP bridge needs no stamp, because its provider carries the resource itself.
func NewRequestIDTransport ¶
func NewRequestIDTransport(base http.RoundTripper) http.RoundTripper
NewRequestIDTransport returns an http.RoundTripper that copies the correlation ID already on the request context (RequestIDFromContext — put there by the inbound RequestIDInterceptor or the HTTP RequestIDMiddleware) onto the outgoing RequestIDHeader, so the callee's logs carry the same request_id as the caller's.
An ID already present on the outgoing header is left untouched, and a context without an ID forwards nothing (the callee mints its own). A nil base uses http.DefaultTransport.
Wrap it in otelhttp for traces — request-ID forwarding and OTel spans are independent concerns and you want both; see the file header for the canonical composition.
func NewTraceHandler ¶
NewTraceHandler wraps next so every record logged with a context holding a valid span carries trace_id and span_id. Records without a span, and records that already carry a trace_id attribute, pass through unchanged. The ids are only available to the *Context logging methods (InfoContext, LogAttrs, ...): slog.Info and friends have no context to read.
Under logger.WithGroup the ids are nested in the group, as with any attribute added after the group opens.
A record at WARN or above, logged with a recording span in its context, is also added to that span as an event — "log.warn" or "log.error", carrying the message, the level and the error (see LogEventWarn). The gateway's tail sampler keeps a trace that has one, so a request whose logs complained is never sampled away while its log lines survive. Wrap the handler INSIDE NewErrorClassHandler (as serverkit does) so the event follows the level the line is written at: a user error lowered to INFO adds none.
func ProfileLabelsMiddleware ¶
ProfileLabelsMiddleware labels the goroutine serving a plain net/http request with its span, for handlers that are not Connect RPCs (TracingInterceptor already covers those). Mount it INSIDE the otelhttp handler (or anything else that starts the server span).
func RecordSpanError ¶
RecordSpanError marks span as failed when err is non-nil: the error is recorded as a span event and the span status is set to Error. This is the one-liner the owned per-package observe.go wrappers call after each inner method invocation:
ctx, span := o.tracer.Start(ctx, "userstore.Get") defer span.End() out, err := o.inner.Get(ctx, id) observe.RecordSpanError(span, err)
A server fault that svcerr captured a stack for (see svcerr.StackOf) carries it on the span's exception event as exception.stacktrace; a client-class error never does.
nil-safe on both span and err (a nil err is the no-op success path).
func RecoveryInterceptor ¶
func RecoveryInterceptor(logger *slog.Logger) connect.Interceptor
RecoveryInterceptor returns a Connect interceptor that recovers from panics inside downstream handlers, logs the recovered value plus the stack, and returns connect.CodeInternal so the client never sees a torn connection.
Place this FIRST in the chain so it observes panics from every subsequent interceptor and the handler itself.
func RequestIDFromContext ¶
RequestIDFromContext returns the request ID stored on ctx (empty when absent). Nil-context safe.
func RequestIDInterceptor ¶
func RequestIDInterceptor() connect.Interceptor
RequestIDInterceptor returns a Connect interceptor that ensures every inbound request has a correlation ID. It resolves the ID in this order, and the order is the whole point:
- An ID ALREADY ON THE CONTEXT wins. The HTTP edge (pkg/middleware.RequestIDMiddleware) sits outside this interceptor, and it has already picked the ID, written it to the RESPONSE header, and put it on ctx. It does not write it back onto the INBOUND request header, so an interceptor that only consulted req.Header() found nothing and minted a SECOND id — the response came back with two X-Request-Id values, every standard client read the first, and only the second ever reached the logs. Quoting an ID at support and grepping for it returned nothing.
- Otherwise the inbound RequestIDHeader, so an edge proxy or calling service can stitch one ID across hops.
- Otherwise a fresh 16-byte crypto/rand hex token.
ECHO OWNERSHIP follows from the same rule: whichever layer CHOSE the ID echoes it. When the ID came off the context, an outer layer chose it and this interceptor writes no response header at all — that is what keeps exactly one X-Request-Id on the wire. When this interceptor chose the ID (no HTTP middleware in the stack, e.g. a bare Connect mux) it echoes, so the client is never left without one.
Place this AFTER RecoveryInterceptor (so panics still get the ID in their log line) and BEFORE LoggingInterceptor (so log records inherit the ID).
func RunRoot ¶ added in v1.0.0
func RunRoot(ctx context.Context, name string, fn func(context.Context) error, opts ...trace.SpanStartOption) (err error)
RunRoot runs fn under a NEW ROOT span named name and ends it when fn returns. It is a new trace even when ctx already carries a span: a loop started inside a request must not file every later iteration under that request. The span records fn's error, and a panic in fn — recorded, then re-raised, so the caller's recovery policy still applies.
Name it for the work, not the instance: "worker.metering.tick", "reconcile Workspace". Put ids in attributes (opts), never in the name.
func SetAppVersion ¶ added in v1.0.0
func SetAppVersion(v string)
SetAppVersion sets the app.version stamped on product events. Setup calls it with Config.ServiceVersion; call it directly in a process that does not use Setup.
func Setup ¶
Setup initializes OpenTelemetry from Config plus the standard OTEL_* environment and installs the providers globally.
Always, even with export off: the W3C TraceContext+Baggage propagator is installed (so an incoming traceparent is honoured and forwarded by every observe client), and a Prometheus reader backs the returned /metrics handler.
When an OTLP endpoint is configured and OTEL_SDK_DISABLED is not "true": traces and metrics are exported over OTLP, http/protobuf by default or grpc when OTEL_EXPORTER_OTLP_PROTOCOL says so, with OTEL_EXPORTER_OTLP_HEADERS (and the per-signal variants) applied by the exporter; Go runtime metrics are recorded; and, only when explicitly switched on, logs are exported too.
It returns a shutdown function (flushes/stops the providers), an http.Handler for /metrics, and any error.
func TraceCall ¶
func TraceCall[T any](ctx context.Context, tracer trace.Tracer, operationName string, fn func(context.Context) (T, error)) (T, error)
TraceCall executes fn inside a new OpenTelemetry span named operationName and returns the inner result. Span errors are recorded when fn returns a non-nil error.
Use TraceCall to express the previous tracing_gen.go behaviour at explicit call sites:
user, err := observe.TraceCall(ctx, tracer, "userstore.Get", func(ctx context.Context) (User, error) {
return s.userStore.Get(ctx, id)
})
nil-safe on tracer (executes fn directly, no span).
func TraceResponseInterceptor ¶ added in v1.0.0
func TraceResponseInterceptor() connect.Interceptor
TraceResponseInterceptor writes the traceresponse header for the server span on ctx. Place it directly inside the interceptor that opens that span — observe.Chain does — so the header names the span every log line and child span of the request hangs off.
Unary: on the response, and on a *connect.Error's metadata, which Connect sends as response headers, so a FAILED call names its trace too (the case that matters). An error that is not a *connect.Error is left untouched rather than rewrapped; forge handlers return svcerr-wrapped connect errors. Streaming: on the response headers before the handler runs.
func TraceResponseValue ¶ added in v1.0.0
func TraceResponseValue(sc trace.SpanContext) string
TraceResponseValue renders sc as a traceresponse value, or "" when sc is not a valid span context.
func TraceVoidCall ¶
func TraceVoidCall(ctx context.Context, tracer trace.Tracer, operationName string, fn func(context.Context) error) error
TraceVoidCall is the no-result variant of TraceCall, for operations that return only an error. Same semantics, no value.
func TracingInterceptor ¶
func TracingInterceptor(tracer trace.Tracer) connect.Interceptor
TracingInterceptor returns a Connect interceptor that creates one OpenTelemetry span per RPC. The span name is the full procedure ("/service.v1.Foo/Bar"); errors are recorded via span.RecordError + span.SetStatus(codes.Error, …).
A nil tracer disables tracing (interceptor is a pass-through). This keeps DefaultMiddlewares safe to wire in test harnesses that don't configure OTel.
func Track ¶ added in v1.0.0
Track records one product event: a named, countable thing a user did or that happened to them ("project_created", "subscription_started"). It writes an INFO record through the context's logger (FromContext) carrying event.name="product.<name>", app.client="backend", app.version, user.id when ctx holds an authenticated user (the claims forge's auth interceptor installs), and each attr as product.<key>.
Properties are bounded the same way the telemetry pipeline bounds them: at most MaxProductProps are kept and each value is truncated to MaxProductPropValueLen characters. Group attrs are flattened to "group.key". Property values are stringified; keep them low-cardinality and never put secrets or personal data in them.
Track never fails the caller. A name that is not snake_case is dropped with a warning, because the pipeline would drop it anyway.
observe.Track(ctx, "project_created", slog.String("template", "go"), slog.Int("repos", 2))
func TrustRemoteBy ¶ added in v1.0.0
func TrustRemoteBy(classify TraceTrustFunc, opts ...otelconnect.Option) (connect.Interceptor, error)
TrustRemoteBy returns the server tracing interceptor (otelconnect) that adopts as much of each caller's trace context as classify grants it: all of it (TrustAll), its trace and parent span ids only (TrustIDs), or none — a new trace linked to the caller's (TrustNone). Pass it as observe.Deps.Trace so it runs outermost. opts are applied to every underlying interceptor (do not pass otelconnect.WithTrustRemote yourself). A nil classify trusts no one, which is otelconnect's default.
A TrustIDs request is sampled by the server's own head-sampling policy (OTEL_TRACES_SAMPLER / OTEL_TRACES_SAMPLER_ARG, as the SDK reads them), whatever the caller's sampled flag says.
func TrustRemoteFrom ¶ added in v1.0.0
func TrustRemoteFrom(trusted TrustPredicate, opts ...otelconnect.Option) (connect.Interceptor, error)
TrustRemoteFrom returns the server tracing interceptor (otelconnect) that continues the caller's whole trace context (TrustAll) when trusted reports the caller authenticated, and otherwise starts a new trace linked to the caller's. Pass it as observe.Deps.Trace so it runs outermost. opts are applied to both underlying interceptors (do not pass otelconnect.WithTrustRemote yourself). A nil trusted trusts no one, which is otelconnect's default. TrustRemoteBy is the three-level form.
func ValidEventName ¶ added in v1.0.0
ValidEventName reports whether name is a legal product event name: snake_case, starting with a letter, at most 64 characters.
func WithLogger ¶
WithLogger returns a copy of ctx carrying logger. The runtime (serverkit.Run, a cobra RunE, a worker fan-out) calls this once at the top of a request/command so downstream code can recover the request-scoped logger without threading it through every signature.
A nil logger is stored as-is; FromContext substitutes a usable default when it reads one back, so callers never have to nil-check.
Types ¶
type CallMetrics ¶
type CallMetrics struct {
// contains filtered or unexported fields
}
CallMetrics is a triple of OpenTelemetry instruments for opt-in per-method instrumentation inside internal packages. Use NewCallMetrics to construct; reuse a single CallMetrics per package (creating instruments per-call is expensive).
func NewCallMetrics ¶
func NewCallMetrics(meter metric.Meter, namespace string) *CallMetrics
NewCallMetrics builds a CallMetrics on meter using the canonical "<namespace>.{calls,errors,duration}" names — matching the metric names the now-removed metrics_gen.go template emitted, so dashboards keyed on those names keep working.
Errors from meter.Int64Counter / Float64Histogram are silently dropped (the same posture as contractkit.NewMetrics). meter == nil returns a CallMetrics whose RecordCall / RecordError methods are no-ops, so callers can wire a CallMetrics in test harnesses without configuring OTel.
func (*CallMetrics) RecordCall ¶
RecordCall increments the call counter and (if err != nil) the error counter, and records duration on the histogram. method is attached as the "method" attribute on each instrument.
Idiomatic call site:
start := time.Now() out, err := s.inner.Do(ctx, req) s.metrics.RecordCall(ctx, "Do", start, err) return out, err
nil-safe on the receiver.
type ClientStack ¶
type ClientStack struct {
// HTTPClient carries the timeout and the base transport. It is freshly
// constructed per stack — callers may tweak it (CheckRedirect, Jar)
// without affecting other stacks.
HTTPClient *http.Client
// ClientOptions carries the interceptor chain as connect.ClientOption
// values, ready to splat into the generated New<Svc>Client call.
ClientOptions []connect.ClientOption
}
ClientStack is what a generated Connect client constructor consumes:
foov1connect.NewFooServiceClient(stack.HTTPClient, baseURL, stack.ClientOptions...)
func NewClientStack ¶
func NewClientStack(deps ClientStackDeps) (ClientStack, error)
NewClientStack builds the canonical outbound Connect client stack. See the file header for what it bundles and the providers.go usage shape. The error surface is otelconnect interceptor construction (never fails under a correctly-installed OTel SDK, but is not swallowed — an uninstrumented outbound edge should be loud).
type ClientStackDeps ¶
type ClientStackDeps struct {
// Timeout bounds each outbound request end-to-end (http.Client.Timeout).
// Zero applies the 30s default. Negative disables the timeout entirely —
// required for long-lived server-streaming RPCs, which the default
// timeout would sever mid-stream.
Timeout time.Duration
// Transport is the base RoundTripper under the client. nil uses
// http.DefaultTransport — the process-wide shared connection pool, which
// is what you want unless the target needs special TLS/proxy settings.
// Do NOT wrap this in otelhttp: the otelconnect interceptor already
// creates the client span and injects propagation headers; doubling up
// yields nested duplicate client spans per RPC.
Transport http.RoundTripper
// Extras are appended after the canonical client interceptors
// (otelconnect, request-ID forwarding), in the order supplied — the slot
// for project-specific concerns like auth-token injection.
Extras []connect.Interceptor
}
ClientStackDeps configures NewClientStack. The zero value is a working production default: shared http.DefaultTransport pool, 30s timeout, otelconnect + request-ID forwarding interceptors.
type ComponentChain ¶
type ComponentChain struct {
// contains filtered or unexported fields
}
ComponentChain is an ordered, reusable stack of ComponentMiddleware shared by every method of one component decorator. Build it once in the owned observe_chain.go seam (NewComponentChain) and route every method through it via Around / Run. The zero value (and a nil *ComponentChain) is a valid pass-through: methods still execute, just without instrumentation — so a decorator wired in a test harness that skipped chain construction is safe.
mw[0] is the OUTERMOST layer (runs first, sees everything inner); the inner method call runs last. This matches the Connect chain's outer-to-inner ordering.
func DefaultComponentChain ¶ added in v1.0.0
func DefaultComponentChain(scope, name string, successLevel slog.Level) *ComponentChain
DefaultComponentChain is the chain a generated decorator uses when its package declares no observe_chain.go seam of its own: panic recovery, one span per call (tracer named scope), the <name>.calls/.errors/.duration metrics, and a log line per failed call — successes at successLevel, which forge renders from forge.yaml's observability.log_level (default Debug, so quiet under a production Info handler). It is exactly what the scaffolded seam assembles; write the seam only to change it.
The logger is slog.Default() as of the call, and the tracer and meter come from the global providers, which delegate to whatever the process installs later — so a decorator built before observe.Setup still reports.
func NewComponentChain ¶
func NewComponentChain(mw ...ComponentMiddleware) *ComponentChain
NewComponentChain builds a ComponentChain from mw in outer-to-inner order. nil entries are dropped so the owned seam can pass a conditionally-disabled middleware straight through (e.g. `enabled ? mw : nil`) without a length change, mirroring how Chain treats nil collaborators.
func (*ComponentChain) Around ¶
func (c *ComponentChain) Around[T any](ctx context.Context, method string, next func(context.Context) (T, error)) (T, error)
Around routes a value-returning method through the chain. T is the method's single non-error result; the value is captured out of the closure so the chain only ever sees the error. It keeps the generated per-method body a one-liner for the common (T, error) shape.
T is a parameter of the METHOD, not of ComponentChain: one chain instance is shared by every method of a component, and those methods return different types. Before Go 1.27 that forced this to be a package-level function taking the chain as an argument, since only functions could carry type parameters.
func (*ComponentChain) Run ¶
func (c *ComponentChain) Run(ctx context.Context, method string, next ComponentOp) error
Run routes an error-only (or result-captured) operation through the chain and returns its error. It is the general primitive: the generated decorator uses it directly for error-only methods and — capturing results in the closure — for methods with multiple non-error results. A nil chain runs next directly.
func (*ComponentChain) WithMethodIDs ¶ added in v1.0.0
func (c *ComponentChain) WithMethodIDs(ids map[string]string) *ComponentChain
WithMethodIDs returns an independent chain with generated, bounded method metadata. Legacy method names, tracer scopes and metric dimensions are kept. A nil chain remains a pass-through. No request data enters these identities.
type ComponentMiddleware ¶
type ComponentMiddleware interface {
WrapComponent(ctx context.Context, method string, next ComponentOp) error
}
ComponentMiddleware wraps one in-process method call. Implementations start a span, time the call, recover a panic, emit a log line, etc., then invoke next(ctx) and observe the returned error. method is the fully-qualified operation name ("<pkg>.<Method>").
A middleware MUST call next(ctx) exactly once (passing a possibly- enriched ctx) and return its error — the chain has no notion of short-circuiting a component call.
func LogMiddleware ¶
LogMiddleware logs component calls with the method name as the message and the duration as an attribute.
- Every FAILED call is written with the error and its error_class, at the level its class earns (see error_class.go): ERROR for a server fault — which is every error nothing classified — and the user-error level (INFO; WithUserErrorLevel) for a svcerr user error or a cancellation.
- A call whose error the seam declared an EXPECTED OUTCOME (WithExpectedErrors / WithExpectedErrorFunc — a storage adapter's "no such object") is not a failure: it is logged like a success, below, with the error and expected=true attached.
- Every SUCCESSFUL call is written at level — which the owned seam wires from forge.yaml's observability.log_level (default Debug: quiet under a production Info handler) — unless success sampling is on. Sampling is opt-in, exactly as at the RPC edge: WithSuccessSampling turns it on for this layer. Sampled, a method's first success is written, then at most one per window, carrying `suppressed`, the successes of that method since the previous record that were not written. A success at or above DefaultSlowThreshold is always written, with slow=true.
Sampling is what keeps this layer readable at DEBUG when a method runs on a tight loop: one method on a 10s work loop was two thirds of a dev stack's log. See log_policy.go.
opts are the RPC edge's: WithSuccessSampling, WithSlowThreshold, WithSuccessLevel keyed by the "<pkg>.<Method>" operation name, WithExpectedErrors, WithUserErrorLevel and WithErrorClassifier. nil logger falls back to slog.Default.
func MetricsMiddleware ¶
func MetricsMiddleware(meter metric.Meter, namespace string) ComponentMiddleware
MetricsMiddleware records call / error / duration metrics under the "<namespace>.{calls,errors,duration}" instruments (see NewCallMetrics), tagging each with the method attribute set to the operation name. nil meter yields a no-op middleware, so a chain built without a configured OTel SDK is safe.
func RecoverMiddleware ¶
func RecoverMiddleware(logger *slog.Logger) ComponentMiddleware
RecoverMiddleware converts a panic in the inner call (or any inner middleware) into an error, logs it with a stack trace, and returns it — so one component's panic can't tear down the caller. It is the OUTERMOST layer in the canonical seam so it observes panics from every inner middleware too.
func TraceMiddleware ¶
func TraceMiddleware(tracer trace.Tracer) ComponentMiddleware
TraceMiddleware opens one OTel span per call named method, threads the span's context into the inner call, and records the returned error onto the span. nil tracer disables tracing (pass-through) — same degradation as TraceCall.
type ComponentOp ¶
ComponentOp is the terminal in-process operation a ComponentChain wraps: it runs the inner method call and returns only its error. A method's non-error results are captured out of the caller's closure (see Around / Run), so a middleware never has to be generic over the return type — it observes identity, duration, and error status only, which is all a span, metric, or log line needs.
type Config ¶
type Config struct {
// ServiceName is the compile-time default service.name. OTEL_SERVICE_NAME,
// when set, wins over it. Empty falls back to "unknown".
ServiceName string
// ServiceVersion is reported as semconv service.version when non-empty and
// not "dev". The "dev" sentinel is treated as "no version".
ServiceVersion string
// OTLPEndpoint overrides OTEL_EXPORTER_OTLP_ENDPOINT (e.g.
// "http://localhost:4318" or "collector:4318"). It is a BASE endpoint: for
// http/protobuf the per-signal path (/v1/traces, ...) is appended. When both
// this field and every endpoint variable are empty, no OTLP exporter is
// configured and only the always-on Prometheus reader is wired, so /metrics
// still works. The protocol, headers, timeout and TLS settings always come
// from the environment.
OTLPEndpoint string
// OTLPLogs turns the OTLP logs exporter on, like OTEL_LOGS_EXPORTER=otlp. It
// is off by default and only takes effect with an endpoint. It is a second
// shipping path for logs: do not enable it where stdout/file collection
// already ships the same lines (see NewLogHandler).
OTLPLogs bool
// ProfilesDisabled turns continuous profiling off, like
// OTEL_PROFILES_EXPORTER=none. Profiling is ON by default whenever an OTLP
// endpoint is configured: a 5s CPU profile every 5 minutes plus a
// live-heap snapshot every two hours. See profiler.go.
ProfilesDisabled bool
// Profiles chooses which profiles are captured and how often the heap is
// snapshotted. The zero value is the default (CPU and the live heap);
// allocation, goroutine, mutex and block profiles are opt-in. The
// FORGE_PROFILES and FORGE_PROFILES_HEAP_INTERVAL variables win over it.
// See profiles_config.go.
Profiles ProfilesConfig
// Development makes the declared-metrics registry panic on an attribute a
// metric did not declare, where production corrects it and counts it. See
// pkg/observe/metrics.
Development bool
// InstanceID is reported as semconv service.instance.id when non-empty.
// Callers typically pass os.Hostname().
InstanceID string
}
Config is the typed configuration for Setup. The standard OTEL_* environment variables are the DEFAULT source for everything exporter-related (see otelenv.go and the package doc); a non-zero field here overrides its variable.
type Deadline ¶ added in v1.0.0
type Deadline struct {
// Unary bounds a unary RPC that Procedures does not name. Zero is
// DefaultUnaryDeadline; negative is no server deadline.
Unary time.Duration
// Procedures overrides the budget per procedure ("/pkg.v1.Service/Method",
// the connect procedure constants). A positive entry is that RPC's
// budget; zero or negative is no server deadline — an rpc that waits on a
// person, such as an OAuth sign-in. An entry applies to a streaming
// procedure too; streams have no default, since a long-lived stream is
// healthy by design. A scaffolded project passes
// middleware.ProcedureDeadlines, which forge projects from each rpc's
// `(forge.v1.method).timeout`.
Procedures map[string]time.Duration
// SlowAfter is how long a unary RPC runs before the watchdog reports it
// as still in flight. Zero is DefaultSlowAfter; negative turns the
// watchdog off. Streams are never watched.
SlowAfter time.Duration
// SlowRepeat is how often a still-running RPC is reported again. Zero
// is DefaultSlowRepeat.
SlowRepeat time.Duration
// OnSlow receives every report: one per SlowAfter/SlowRepeat tick while
// the RPC runs (SlowCall.Done false, from the watchdog's goroutine), and
// one when an RPC that ran past SlowAfter completes (Done true). It must
// be safe for concurrent use. nil logs each at WARN through Logger;
// setting it replaces that log, so a project that wants a metric too
// logs and counts in the one func.
OnSlow func(context.Context, SlowCall)
// Logger is what a nil OnSlow logs to. nil is the chain's logger
// (ChainDeps.Logger), or slog.Default() outside a chain.
Logger *slog.Logger
}
Deadline is the Deadline slot of a Chain: the time budget of every RPC and the watchdog that reports a slow one. It is the slot's configuration and its interceptor at once, so a chain is configured in its literal:
serverkit.ChainDeps{
Auth: serverkit.AuthFor(serverkit.AudienceUser, authInterceptor),
Deadline: serverkit.Deadline{Procedures: middleware.ProcedureDeadlines},
}
The zero value is the default a nil slot gets — ChainDeps.Deadline and Deps.Deadline alike: every unary RPC bounded by DefaultUnaryDeadline, streams unbounded, and a unary RPC still running after DefaultSlowAfter logged at WARN.
The budget is a ceiling. A caller's own deadline (Connect-Timeout-Ms, grpc-timeout) still applies when it is shorter; a longer one does not lift the server's.
func (Deadline) Validate ¶ added in v1.0.0
Validate rejects a Deadline that would silently do something other than it says: a procedure key that can never match a connect procedure, or a negative repeat interval. serverkit.NewChain runs it at boot; Chain, which returns no error, does not.
func (Deadline) WrapStreamingClient ¶ added in v1.0.0
func (Deadline) WrapStreamingClient(next connect.StreamingClientFunc) connect.StreamingClientFunc
WrapStreamingClient is a passthrough: the slot bounds RPCs this server answers, not the ones it makes.
func (Deadline) WrapStreamingHandler ¶ added in v1.0.0
func (d Deadline) WrapStreamingHandler(next connect.StreamingHandlerFunc) connect.StreamingHandlerFunc
WrapStreamingHandler bounds a stream only when Procedures names it; streams are not watched.
type DefaultMiddlewareDeps ¶
type DefaultMiddlewareDeps struct {
// Logger is the slog.Logger used by RecoveryInterceptor and
// LoggingInterceptor. nil falls back to slog.Default.
Logger *slog.Logger
// Tracer feeds TracingInterceptor. nil disables tracing.
Tracer trace.Tracer
// Meter feeds MetricsInterceptor. nil disables metrics.
Meter metric.Meter
// LogOptions tune how LoggingInterceptor logs successful RPCs
// (sampling window, slow threshold, per-procedure level). nil keeps
// the defaults: every success logged. WithSuccessSampling here is the
// only thing that samples them — pass it the app's typed config value.
// Failures are always logged in full.
LogOptions []LogOption
// Extras are appended to the canonical chain in the order supplied
// — useful for project-specific interceptors (auth, rate-limit,
// idempotency, audit) that the canonical chain doesn't
// know about. Order is preserved relative to each other and they
// run AFTER the observability layer.
Extras []connect.Interceptor
}
DefaultMiddlewareDeps is the dependency bag for DefaultMiddlewares. Every field is optional — a nil Logger, Tracer, or Meter cleanly degrades the corresponding interceptor (slog.Default for the logger, pass-through interceptors for tracer/meter), so projects that haven't configured OTel can still use the helper.
type Deps ¶
type Deps struct {
// Logger feeds RecoveryInterceptor and LoggingInterceptor. nil falls
// back to slog.Default — same degradation as DefaultMiddlewareDeps.
Logger *slog.Logger
// Tracer feeds TracingInterceptor. nil disables tracing.
Tracer trace.Tracer
// Meter feeds MetricsInterceptor. nil is the global MeterProvider's
// meter (otel.Meter), the one observe.Setup installs, so every RPC is
// counted (rpc.server.calls / .errors / .duration) without wiring: the
// generated dashboards and the RPC alerts read those metrics, and a chain
// that silently counted nothing left all of them empty. A process that
// never installed a provider records into the no-op global.
Meter metric.Meter
// LogOptions tune how LoggingInterceptor logs successful RPCs
// (sampling window, slow threshold, per-procedure level). nil keeps
// the defaults: every success logged. WithSuccessSampling here is the
// only thing that samples them — pass it the app's typed config value.
// Failures are always logged in full.
LogOptions []LogOption
// Trace is the interceptor that opens the SERVER SPAN for each RPC —
// otelconnect.NewInterceptor(...). Chain puts it OUTERMOST, ahead of
// recovery, logging, auth and audit, so that:
//
// - a call auth rejects still has a span (the failed and unauthenticated
// calls are the ones you most want to see);
// - every log line any layer writes for the request, including the
// "rpc failed" line and the audit event, carries the span's trace_id;
// - the pprof labels that tie profile samples to this span (see
// profiles_labels.go) wrap everything inside it.
//
// Passing it as an Extra instead lands it AFTER auth, which loses all
// three. Build it where you can decide whether to trust an inbound
// traceparent (otelconnect.WithTrustRemote). nil opens no span here.
// Set this OR Tracer, not both: two spans per RPC.
Trace connect.Interceptor
// Deadline is the RPC's time budget — serverkit's Deadline slot: a
// per-RPC deadline and a slow-call watchdog. It runs after the
// observability layer, so a DeadlineExceeded is logged and counted with
// its full duration, and BEFORE Auth, so authentication's own lookups
// (a token store query, an introspection call) run inside the budget.
// Passing it as an Extra lands it after Auth, which bounds nothing Auth
// does.
//
// The slot has serverkit.NewChain's rule, because NewChain builds on
// Chain: nil is Deadline{} — every unary RPC bounded by
// DefaultUnaryDeadline (30s), streams unbounded, a unary RPC still
// running after DefaultSlowAfter logged at WARN through Logger — and
// serverkit.NoDeadline drops it. A Deadline (or *Deadline) that names no
// Logger gets Logger. Any other interceptor here replaces forge's and
// runs in the same position. Chain does not validate a Deadline; NewChain
// does (Deadline.Validate).
Deadline connect.Interceptor
// Auth is the project's authentication interceptor — the value
// returned by authn.NewInterceptor(policy), with the token validator
// and identity enricher already threaded through the Policy. This is
// the field that retires middleware.SetTokenValidator /
// SetIdentityEnricher: the composition root builds the interceptor
// with its validator in hand and passes it here. nil skips the auth
// layer entirely (e.g. a worker-only process, or a test harness).
Auth connect.Interceptor
// Audit is the project's audit interceptor — the value returned by
// middleware.AuditInterceptor(sink, ...), with the durable audit sink
// already bound. This retires middleware.SetAuditStore: the
// composition root opens the store, builds the interceptor around it,
// and passes it here. nil skips audit.
Audit connect.Interceptor
// RateLimit is the project's rate-limit interceptor — the value
// returned by middleware.RateLimitInterceptor(opts, claimsLookup), or
// nil when rate limiting is disabled (RateLimitInterceptor itself
// returns nil for Rps <= 0, so the composition root can pass its
// result through unconditionally and a disabled limiter cleanly drops
// out of the chain).
RateLimit connect.Interceptor
// Extras are additional project-specific interceptors (idempotency,
// anything the named fields don't cover),
// appended AFTER the named application layer in the order supplied.
// Inter-Extra order is the caller's to control.
Extras []connect.Interceptor
}
Deps is the EXPLICIT-COLLABORATOR dependency bag for Chain, the composition-root variant of DefaultMiddlewares.
Why a second bag ¶
DefaultMiddlewareDeps models the observability layer plus an opaque Extras slice the caller assembles by hand. That worked when the cmd shim built the auth / audit / rate-limit interceptors itself and append- ordered them into Extras. Under explicit per-server composition the composition root has those collaborators as LOCALS (it just constructed the token validator, the audit sink interceptor, the rate limiter) and wants to hand them in by NAME — not pack them into a positional slice and rely on getting the canonical inner-order right. Deps names each project concern so a project NEVER reaches for a package-global (SetTokenValidator / SetAuditStore / SetIdentityEnricher): it passes the already-built interceptor straight into the field.
Every collaborator field is a connect.Interceptor (or nil to skip it), which is exactly what the project's builders already produce — authn.NewInterceptor for Auth, middleware.AuditInterceptor for Audit, middleware.RateLimitInterceptor for RateLimit. Passing the constructed VALUE (not a setter) is the whole point: the wiring is visible at the call site and testable without globals.
type HTTPClientOptions ¶ added in v1.0.0
type HTTPClientOptions struct {
// Timeout bounds each request end to end (http.Client.Timeout). Zero
// applies the 30s default; negative disables it (a long-poll or a
// streaming download).
Timeout time.Duration
// Base is the RoundTripper under the instrumentation. nil shares one
// process-wide traced transport over http.DefaultTransport — one
// connection pool. Set it only for a target that needs its own TLS or
// proxy settings; it is wrapped, never replaced, so it is traced too.
Base http.RoundTripper
}
HTTPClientOptions configures NewHTTPClient. The zero value is the production default.
type LogOption ¶
type LogOption func(*logPolicy)
LogOption tunes how a logging layer — LoggingInterceptor at the RPC edge, LogMiddleware at the in-process component boundary, NewErrorClassHandler under every other log call — logs SUCCESSFUL calls, which errors are expected outcomes rather than failures (WithExpectedErrors), and how loud an error that is not a server fault may be (WithUserErrorLevel, WithErrorClassifier). Every failure is still written, with full fields.
func WithErrorClassifier ¶
WithErrorClassifier adds fault attribution svcerr.Classify cannot derive — an error type from a library the app does not own, such as a workflow engine's application error whose category travelled across a serialization boundary the svcerr marker cannot cross.
classify returns svcerr.ClassNone for "no opinion": the next classifier is asked, and svcerr.Classify last. Classifiers accumulate across options, in order; each must be safe for concurrent use. A nil classify is ignored.
func WithExpectedErrorFunc ¶
WithExpectedErrorFunc is WithExpectedErrors for a classification no sentinel expresses — a typed vendor error whose status field decides it, say. isExpected is called with every non-nil error the layer sees and must be safe for concurrent use; a nil isExpected is ignored.
func WithExpectedErrors ¶
WithExpectedErrors declares errors that are an EXPECTED OUTCOME of the calls a layer logs — an answer the caller acts on, not a failure of the call. An error matching any target (errors.Is) is logged the way a success is: at the call name's success level, sampled with WithSuccessSampling in a slot of its own, written regardless when slow, and carrying its error plus expected=true. Every other error is a failure and is logged as before.
It is how a component says what only it knows. A storage adapter's "no such object" is the answer a static-site 404 is built from, and a registry adapter's "name unknown" is how a caller learns a repository is empty; logged at ERROR, those buried a prod proxy's real failures under 288 ERROR records in five hours. Declare them in the package's owned observe_chain.go, on its LogMiddleware:
observe.LogMiddleware(logger, slog.LevelDebug,
observe.WithExpectedErrors(ErrNotFound)),
Nothing is expected unless declared — forge cannot tell an answer from a fault, so an undeclared not-found stays a failure — except a drain (svcerr.ErrServerDraining), which every layer treats as expected without being told. Declarations accumulate across options. The declaration scopes to the layer it is passed to — the error itself is not marked, so a caller that turns the answer into a failure ("the bundle this deploy needs is gone") still logs a failure.
func WithSlowThreshold ¶
WithSlowThreshold sets the duration at or above which a successful call is always written, with slow=true (default DefaultSlowThreshold). Zero or less disables the rule. It applies to unary calls only: a stream's duration is its session length, not its latency.
func WithSuccessLevel ¶
WithSuccessLevel sets the level of ONE call name's success records, overriding the layer's default (INFO at the RPC edge; the seam's level for LogMiddleware). For LoggingInterceptor the name is the full procedure, "/pkg.v1.Service/Method" — the generated <pkg>connect.<Service><Method>Procedure constant; for LogMiddleware it is the "<pkg>.<Method>" operation name. Sampling, when on, still applies.
slog.LevelDebug is how to silence a known-noisy procedure (a poll, a health check) under an INFO handler; a higher level makes an important one stand out. Failures and slow calls of that name are logged as before.
func WithSuccessSampling ¶
WithSuccessSampling sets the sampling window for one layer: at most one success record per call name per window, carrying `suppressed`. A window of zero or less — or no WithSuccessSampling at all — writes every success.
It is the only way a layer samples. A forge app passes its typed config's log_success_sample_window here (the scaffolded serve.go does, through Deps.LogOptions), so a deployment sets the window in its config.
func WithUserErrorLevel ¶
WithUserErrorLevel sets the level for errors that are not a server fault — svcerr.ClassUser and svcerr.ClassCanceled — on one layer. The default is DefaultUserErrorLevel (INFO); slog.LevelWarn is the usual alternative for a deployment that wants them in a WARN view. It is a CEILING: a call site that logged a user error below it (DEBUG) keeps its level.
type ProfileType ¶ added in v1.0.0
type ProfileType string
ProfileType is one kind of profile the continuous profiler can capture.
const ( // ProfileCPU is a CPU profile of a DefaultCPUDuration window, captured // every cycle (DefaultCPUInterval). On by default. ProfileCPU ProfileType = "cpu" // ProfileHeap is the live heap (inuse_space, inuse_objects), captured every // HeapInterval. On by default. ProfileHeap ProfileType = "heap" // ProfileAlloc is the cumulative allocation counters since process start // (alloc_space, alloc_objects), captured with the heap every HeapInterval. // Off by default: each snapshot carries every allocation site the process // ever had, so it grows with uptime. Values stay cumulative; they are never // converted to deltas. ProfileAlloc ProfileType = "alloc" // ProfileGoroutine is every goroutine's stack, captured every cycle. Off by // default. ProfileGoroutine ProfileType = "goroutine" // ProfileMutex is mutex contention, captured every cycle. Off by default. // Enabling it also sets runtime.SetMutexProfileFraction while the profiler // runs, because the runtime records nothing without a sampling rate. ProfileMutex ProfileType = "mutex" // ProfileBlock is time blocked on channels and sync primitives, captured // every cycle. Off by default. Enabling it also sets // runtime.SetBlockProfileRate while the profiler runs. ProfileBlock ProfileType = "block" )
func DefaultProfileTypes ¶ added in v1.0.0
func DefaultProfileTypes() []ProfileType
DefaultProfileTypes is what the profiler captures when neither Config.Profiles.Types nor FORGE_PROFILES says otherwise: CPU and the live heap.
type ProfilesConfig ¶ added in v1.0.0
type ProfilesConfig struct {
// Types is the set to capture. Nil means DefaultProfileTypes; an empty,
// non-nil slice captures nothing, so no profiler starts.
Types []ProfileType
// HeapInterval is how often the heap (and alloc) snapshot is taken. Zero
// means DefaultHeapInterval. A value shorter than the capture cycle
// (CPUInterval) is raised to it.
HeapInterval time.Duration
// CPUInterval is the time between capture cycles. Zero means
// DefaultCPUInterval; the shortest is MinCPUInterval.
CPUInterval time.Duration
// CPUDuration is how long each CPU profile samples. Zero means
// DefaultCPUDuration; it is between MinCPUDuration and half the cycle.
CPUDuration time.Duration
}
ProfilesConfig chooses which profiles the continuous profiler captures. The zero value is the default: a 5s CPU profile every 5 minutes and the live heap every two hours. The FORGE_PROFILES* variables, when set and valid, win over it: they are the deploy-time override.
type SlowCall ¶ added in v1.0.0
type SlowCall struct {
// Procedure is the RPC ("/pkg.v1.Service/Method").
Procedure string
// Elapsed is how long it had run when reported.
Elapsed time.Duration
// Done is false while it is still running, and true for the one report
// made when it completes.
Done bool
// Err is what it returned, when Done.
Err error
}
SlowCall is one watchdog report of a unary RPC that ran past SlowAfter.
type TraceTrust ¶ added in v1.0.0
type TraceTrust uint8
TraceTrust is how much of one inbound request's trace context a server adopts. See the file header for what each level hands the caller.
const ( // TrustNone starts a new trace linked to the caller's. The zero value. TrustNone TraceTrust = iota // TrustIDs continues the caller's trace (trace id and parent span id) but // makes the server's own sampling decision and drops the caller's // tracestate and baggage. For an authenticated end user. TrustIDs // TrustAll adopts the caller's whole trace context. For an authenticated // internal service. TrustAll )
type TraceTrustFunc ¶ added in v1.0.0
TraceTrustFunc classifies the caller of one inbound request. It sees the request's peer and headers only; see the file header for what it must verify before returning anything but TrustNone.
type TrustPredicate ¶ added in v1.0.0
TrustPredicate reports whether the caller of one inbound request is an authenticated internal service whose trace context may be adopted whole. It sees the request's peer and headers only; see the file header for what it must verify.
Source Files
¶
- client.go
- component_chain.go
- component_identity.go
- deadline.go
- declared_metrics.go
- doc.go
- error_class.go
- error_inputs.go
- headers_file.go
- helpers.go
- httpclient.go
- interceptors.go
- log_policy.go
- log_resource.go
- log_trace.go
- logger_context.go
- middleware.go
- otelenv.go
- otlp_exporters.go
- profiler.go
- profiles_config.go
- profiles_convert.go
- profiles_labels.go
- root.go
- setup.go
- traceresponse.go
- track.go
- transport.go
- trust_remote.go
Directories
¶
| Path | Synopsis |
|---|---|
|
Package chtrace makes a ClickHouse connection's statements OpenTelemetry spans, the way dbtrace does for Postgres.
|
Package chtrace makes a ClickHouse connection's statements OpenTelemetry spans, the way dbtrace does for Postgres. |
|
Package dbtrace opens a *sql.DB whose queries are OpenTelemetry spans.
|
Package dbtrace opens a *sql.DB whose queries are OpenTelemetry spans. |
|
Package metrics is forge's declared metrics registry: a typed layer over the OpenTelemetry metrics API in which every instrument is DECLARED before it is created.
|
Package metrics is forge's declared metrics registry: a typed layer over the OpenTelemetry metrics API in which every instrument is DECLARED before it is created. |
|
Package natsotel carries trace context across NATS, so a message's consumer continues the trace of the code that published it.
|
Package natsotel carries trace context across NATS, so a message's consumer continues the trace of the code that published it. |
|
Package observetest holds test helpers that pin a process's observability wiring from a unit test.
|
Package observetest holds test helpers that pin a process's observability wiring from a unit test. |
|
Package temporalotel carries trace context through Temporal, so the request that starts or signals a workflow, the workflow, and every activity it runs are ONE trace.
|
Package temporalotel carries trace context through Temporal, so the request that starts or signals a workflow, the workflow, and every activity it runs are ONE trace. |