observability

package
v0.12.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 19, 2026 License: MIT Imports: 21 Imported by: 0

Documentation

Overview

Package observability provides OpenTelemetry tracing for the ox CLI.

Creates one trace per CLI command invocation. All HTTP requests within a command share the same trace ID, appearing as children of the root span in Honeycomb. This gives CLI↔server timing visibility.

Index

Constants

View Source 
const (
	// AttrOXVersion is the ox CLI/daemon version string. Always set on the
	// root command span so traces can be sliced by release without joining
	// against build metadata.
	AttrOXVersion = "ox.version"

	// AttrHostOS is the operating system the binary is running on
	// (linux/darwin/windows). Always set on the root command span.
	AttrHostOS = "host.os"

	// AttrHostArch is the CPU architecture the binary was built for
	// (amd64/arm64/...). Cheap to capture and useful for narrowing
	// platform-specific regressions.
	AttrHostArch = "host.arch"

	// AttrCLIExitCode is the integer exit code the CLI process returned.
	// Set unconditionally from main() after Execute returns, on both
	// success (0) and error (non-zero) paths.
	AttrCLIExitCode = "cli.exit_code"

	// AttrCLIArgs is the redacted command-line arguments. Flag NAMES are
	// preserved; flag VALUES and positional arguments are replaced with
	// "<REDACTED>" by RedactArgs. The intent is to make "which flags do
	// users pass to ox code search" queryable without ever exporting user
	// content.
	AttrCLIArgs = "ox.command.args"

	// AttrResultCount is the size of the result set returned by a command
	// that produces results (search, query, list). Set best-effort by
	// individual commands via SetResultCount; not every command has one.
	AttrResultCount = "result.count"

	// AttrAgentEnv identifies the AI coworker that invoked ox, sourced
	// from the AGENT_ENV environment variable that adapter hooks set
	// (claude-code, aider, gemini, codex, ...). Only present when the
	// command is being driven by an AI coworker — human-driven
	// invocations leave this attribute off the span entirely so a
	// `ox.agent.env exists` query in the trace backend cleanly isolates
	// agent traffic from human traffic.
	AttrAgentEnv = "ox.agent.env"

	// AttrCommandSubcommand is the resolved subcommand path for commands
	// that go through a dispatcher. Cobra's PreRunE creates the root
	// span before the dispatcher resolves what was actually invoked, so
	// every dispatcher hit would otherwise share one bland span name
	// (e.g. "ox agent"). RenameRootSpan attaches this attribute alongside
	// the renamed span so backends can group by it even if a future
	// refactor changes the span-name scheme — and so high-cardinality
	// suffixes like hook phase names don't bloat span-name buckets.
	AttrCommandSubcommand = "ox.command.subcommand"
)

Span attribute keys for ox CLI traces.

These are exported as constants so call sites in cmd/ox don't typo a key and end up with two attributes for the same concept. New keys belong here.

Naming follows two conventions:

  • OTel semantic conventions where they exist (host.os, cli.exit_code).
  • The "ox.*" namespace for everything else, mirroring how the daemon scopes its task spans.
View Source 
const RedactedPlaceholder = "<REDACTED>"

RedactedPlaceholder is the string that replaces every redacted token in the output of RedactArgs. Exposed as a constant so tests and downstream code can match against it without hardcoding the literal.

Variables

This section is empty.

Functions

func AddSpanProcessor added in v0.9.0 

func AddSpanProcessor(p sdktrace.SpanProcessor)

AddSpanProcessor registers an additional SpanProcessor to be installed alongside the OTLP batch exporter on the next Init() call. Must be called BEFORE Init — processors added after Init are ignored, because the TracerProvider is already built.

Used by callers (CLI bootstrap, daemon bootstrap) to wire in internal/perf's TreeCollectorProcessor so per-phase timing renders locally in addition to flowing to the OTLP backend.

Safe to call multiple times; each processor is registered once.

func CommandName 

func CommandName(parts ...string) string

CommandName extracts a readable command path for span naming. e.g., "ox login", "ox agent prime", "ox session stop"

func Enabled 

func Enabled() bool

Enabled returns true if OTel tracing is initialized.

func Init 

func Init(ctx context.Context, serviceName, apiEndpoint string, tokenFunc TokenFunc, attrs ...attribute.KeyValue) error

Init sets up the OTel TracerProvider with OTLP/HTTP export to the SageOx OTLP proxy at {apiEndpoint}/api/v1/otlp/v1/traces.

The proxy is JWT-gated (see apps/api-go/internal/handlers/otlp_proxy.go), so tokenFunc is required for exports to succeed. tokenFunc is invoked per-request rather than baked into a static header so long-running processes (the daemon) pick up rotated tokens without restart.

Extra attrs are merged into the OTel resource alongside service.name. The daemon uses this to set client.id, client.class, os.type, etc.

Safe to call with empty apiEndpoint or nil tokenFunc — tracing is disabled (noop). When tokenFunc returns "" at export time (logged-out / expired), the batch is dropped client-side (synthetic 2xx, nothing sent) rather than 401'd by the server. Either is fine for tests and for users who are logged out.

func RedactArgs 

func RedactArgs(args []string) []string

RedactArgs returns a copy of args suitable for tracing or telemetry, preserving flag *names* but replacing every flag *value* and every positional argument with RedactedPlaceholder.

The goal is to make `ox.command.args` queryable ("which flags are users passing to ox code search?") without leaking any user content ("what is the user searching for?"). The conservative default is to redact aggressively: free-form input belongs in command-specific attributes (e.g. ox.code.search.query) where redaction can be tuned to the field, not at the args level.

Behavior:

  • "--name=value" -> "--name=<REDACTED>"
  • "--name", "value" -> "--name", "<REDACTED>"
  • "--name", "-5" -> "--name", "<REDACTED>" (closes the "value starts with -" leak)
  • "-n", "value" -> "-n", "<REDACTED>"
  • "positional" -> "<REDACTED>"
  • "--bool", "--other" -> "--bool", "--other" (chained long flags are preserved as flag NAMES)
  • "--", "rest..." -> "--", "<REDACTED>", "<REDACTED>", ...

We do NOT have a flag schema here, so we cannot distinguish a boolean flag from a value-taking flag. The discriminator we use is whether the next token starts with "--" — a true long flag begins with "--" by convention, so this lets us preserve chained boolean long flags (`--verbose --json`) without keeping anything that could plausibly be a value. Tokens starting with a single "-" (e.g. "-5", "-k", "-foo") are treated as values for the preceding flag and redacted.

Known residual: a value that *literally* begins with "--" (e.g. `--message --secret-token` where `--secret-token` is meant as a string value, not a flag) will be preserved verbatim. This is statistically vanishingly rare for real secrets and was an explicit trade-off to keep `--verbose --json`-style chained long flags queryable. A follow-up PR can take a flag schema from cobra at the call site and disambiguate exactly.

Pure function. Returns a fresh slice — the input is never mutated.

func RenameRootSpan 

func RenameRootSpan(name, subcommand string)

RenameRootSpan rewrites the name of the root command span and attaches an ox.command.subcommand attribute. Used by command dispatchers — most notably the `ox agent <agent_id> <cmd>` dispatcher in cmd/ox/agent.go — that need to refine the span name once they have resolved the actual invocation.

Why this exists: cobra's PersistentPreRunE creates the root span before the dispatcher's RunE runs, so the span starts life with a bland name like "ox agent" because cobra only sees the agentCmd. Without renaming, every dispatcher hit ends up in one giant bucket in the trace backend and you can't answer "which dispatcher path is slow / erroring?".

The subcommand string is also attached as an attribute (AttrCommandSubcommand) so dashboards can group by it independently of the span name. Empty subcommand strings are not attached.

Should be called BEFORE any early-return paths in the dispatcher (unknown agent_id, validation errors, help) so even fast failures end up under the right span name.

Safe to call when tracing is disabled.

func SetCommandAttrs 

func SetCommandAttrs(version string, args []string)

SetCommandAttrs sets the universal attributes that should be present on every CLI root span: ox.version, host.os, host.arch, ox.command.args, and (when set) ox.agent.env.

Safe to call when tracing is disabled (no-op if there is no root span). Args are redacted via RedactArgs before being attached.

Should be called once per command, immediately after StartCommand. The CLI bootstrap in internal/cli.NewContext does this; downstream code does not need to.

func SetCommandStatus 

func SetCommandStatus(err error)

SetCommandStatus records whether the command succeeded or failed on the root span. Call before Shutdown.

func SetExitCode 

func SetExitCode(code int)

SetExitCode records the process exit code on the root command span. Called from main() after rootCmd.Execute returns, so the attribute is present on both success and error paths — even when PersistentPostRunE never ran (cobra skips PostRunE on RunE errors).

On non-zero exit codes the span status is also marked Error so backends can filter for failed commands without inspecting the exit code field.

Safe to call when tracing is disabled.

func SetResultCount 

func SetResultCount(n int)

SetResultCount attaches a result.count attribute to the root command span. Best-effort: callers in commands that produce a result set (search, query, list) call this once they know the count. Commands without a natural result set should not call it.

Safe to call when tracing is disabled.

func Shutdown 

func Shutdown(ctx context.Context)

Shutdown ends the root span and flushes pending exports. Blocks up to 3s for flush. Safe to call if Init was never called.

func StartCommand 

func StartCommand(ctx context.Context, commandName string) (context.Context, trace.Span)

StartCommand creates a root span for a CLI command invocation. All HTTP requests within this command will be children of this span.

func TraceParent 

func TraceParent() string

TraceParent returns the W3C traceparent header value from the current root span. Returns empty string if tracing is not active.

Types

type DaemonTracer 

type DaemonTracer struct {
	// contains filtered or unexported fields
}

DaemonTracer provides per-task trace creation for the daemon. Unlike the CLI (one root span per command), the daemon creates many independent traces — one per scheduled task execution.

All methods are nil-safe: callers never need to check whether tracing is enabled.

func NewDaemonTracer 

func NewDaemonTracer() *DaemonTracer

NewDaemonTracer returns a tracer for daemon tasks. Must be called after Init() has set up the TracerProvider. Returns nil if tracing is not enabled.

func (*DaemonTracer) StartSubtask 

func (dt *DaemonTracer) StartSubtask(ctx context.Context, name string, attrs ...attribute.KeyValue) (context.Context, trace.Span)

StartSubtask creates a child span under the current span in ctx.

func (*DaemonTracer) StartTask 

func (dt *DaemonTracer) StartTask(ctx context.Context, taskName string, attrs ...attribute.KeyValue) (context.Context, trace.Span)

StartTask creates a new root span for a daemon task. Each call produces an independent trace (no parent), so the daemon generates many short traces over its lifetime rather than one long one.

Example: 

ctx, span := dt.StartTask(ctx, "daemon:gc_check")
defer span.End()

type HistogramSummary added in v0.9.0 

type HistogramSummary struct {
	Count   uint64    `json:"count"`
	SumMs   float64   `json:"sum_ms"`
	Buckets []uint64  `json:"buckets"`    // parallel to Boundaries + overflow
	Bounds  []float64 `json:"boundaries"` // canonical bucket boundaries (ms)
}

HistogramSummary is the snapshot view of one histogram series.

type InMemoryMetrics added in v0.9.0 

type InMemoryMetrics struct {
	// contains filtered or unexported fields
}

InMemoryMetrics is an adapterruntime.Metrics implementation that counts events in process-local maps. Designed for early-rollout visibility today, before OTLP metric exporters get wired into internal/observability/otel.go alongside the existing tracer.

Two reasons this ships before a real OTLP path:

  1. The existing observability package only initializes a TracerProvider (otel.go:Init), not a MeterProvider. Building a MeterProvider with the OTLP/HTTP metric exporter, JWT auth, and correct shutdown semantics is a non-trivial change that deserves its own PR for review.
  2. The Metrics interface lets us swap the implementation later without touching adapter or detector code — the detector wiring in pkg/adapterruntime never needs to learn about OTLP.

Snapshot() returns a serialized view of every counter and histogram observed so far, suitable for dumping in `ox doctor`, a JSON debug endpoint, or a periodic flush to logs while the OTLP integration is in flight.

Thread-safe: counters/histograms guarded by a single mutex. The detector currently fires one event per matched entry plus one per silence-window tick, so contention is negligible.

func NewInMemoryMetrics added in v0.9.0 

func NewInMemoryMetrics() *InMemoryMetrics

NewInMemoryMetrics constructs an empty in-memory metrics sink.

func (*InMemoryMetrics) PatternConfirmed added in v0.9.0 

func (m *InMemoryMetrics) PatternConfirmed(adapter, patternID string)

PatternConfirmed increments ox.adapter.terminal.pattern_confirmed{adapter, pattern_id}.

func (*InMemoryMetrics) PatternHit added in v0.9.0 

func (m *InMemoryMetrics) PatternHit(adapter, patternID string, source adapterruntime.PatternSource, reason string)

PatternHit increments ox.adapter.terminal.pattern_hit{adapter, pattern_id, source, reason}.

func (*InMemoryMetrics) PatternRevoked added in v0.9.0 

func (m *InMemoryMetrics) PatternRevoked(adapter, patternID string)

PatternRevoked increments ox.adapter.terminal.pattern_revoked{adapter, pattern_id}.

func (*InMemoryMetrics) ResetsAtParseFailure added in v0.9.0 

func (m *InMemoryMetrics) ResetsAtParseFailure(adapter, patternID string)

ResetsAtParseFailure increments ox.adapter.terminal.resets_at_parse_failure{adapter, pattern_id}.

func (*InMemoryMetrics) SilenceWindowObservedMs added in v0.9.0 

func (m *InMemoryMetrics) SilenceWindowObservedMs(adapter, patternID string, observed time.Duration)

SilenceWindowObservedMs records ox.adapter.terminal.silence_window_observed_ms {adapter, pattern_id} as a histogram with silenceWindowBuckets boundaries.

func (*InMemoryMetrics) Snapshot added in v0.9.0 

func (m *InMemoryMetrics) Snapshot() MetricsSnapshot

Snapshot returns a deep copy of the metrics state. Safe to call from any goroutine. The returned maps are independent and can be mutated without affecting the in-memory store.

type MetricsSnapshot added in v0.9.0 

type MetricsSnapshot struct {
	PatternHit       map[string]uint64           `json:"pattern_hit,omitempty"`
	PatternConfirmed map[string]uint64           `json:"pattern_confirmed,omitempty"`
	PatternRevoked   map[string]uint64           `json:"pattern_revoked,omitempty"`
	ResetsParseFail  map[string]uint64           `json:"resets_at_parse_failure,omitempty"`
	SilenceWindowMs  map[string]HistogramSummary `json:"silence_window_observed_ms,omitempty"`
}

MetricsSnapshot is a serialized view of every counter and histogram observed by an InMemoryMetrics. Keys are the joined label tuple (adapter|pattern_id|source|reason) — split on "|" to recover labels.

func (MetricsSnapshot) SortedKeys added in v0.9.0 

func (s MetricsSnapshot) SortedKeys() []string

SortedKeys returns the sorted union of every metric key, useful for deterministic display (table rendering in `ox doctor` or test assertions).

type TokenFunc added in v0.9.0 

type TokenFunc func() string

TokenFunc returns the current bearer token to attach to OTLP requests. Called per-export so long-running processes (daemon) pick up rotated tokens. Returning "" means no bearer is available (logged out / token expired): the batch is dropped client-side (see bearerRoundTripper.RoundTrip) rather than sent, because the JWT-gated proxy would only 401 it anyway.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.