telemetry

package
v0.38.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 27, 2026 License: MIT Imports: 15 Imported by: 0

Documentation

Overview

Package telemetry provides opt-in OpenTelemetry trace export for waza runs. It is off by default. When disabled, the package returns no-op tracers and adds no runtime cost to users who do not enable it.

Spans follow the OpenTelemetry GenAI semantic conventions (https://opentelemetry.io/docs/specs/semconv/gen-ai/). The span hierarchy emitted by waza is: 

eval (root)
└── task (per test case)
    └── turn (per Execute call: initial + follow-ups + responder)
        ├── model_call (one per recorded model invocation)
        └── tool_call  (one per recorded tool invocation)

Payload redaction is on by default: prompt text, tool arguments, and model output content are dropped. The --otel-include-payloads flag (Config.IncludePayloads) opts back in.

Index

Constants

View Source 
const (
	SemConvServiceName    = attribute.Key("service.name")
	SemConvServiceVersion = attribute.Key("service.version")

	// GenAI standard keys.
	AttrGenAISystem               = attribute.Key("gen_ai.system")
	AttrGenAIOperationName        = attribute.Key("gen_ai.operation.name")
	AttrGenAIRequestModel         = attribute.Key("gen_ai.request.model")
	AttrGenAIResponseModel        = attribute.Key("gen_ai.response.model")
	AttrGenAIUsageInputTokens     = attribute.Key("gen_ai.usage.input_tokens")
	AttrGenAIUsageOutputTokens    = attribute.Key("gen_ai.usage.output_tokens")
	AttrGenAIUsageCacheReadTokens = attribute.Key("gen_ai.usage.cache_read_tokens")
	AttrGenAIToolName             = attribute.Key("gen_ai.tool.name")
	AttrGenAIToolCallID           = attribute.Key("gen_ai.tool.call.id")
	AttrGenAIPromptText           = attribute.Key("gen_ai.prompt")
	AttrGenAICompletionText       = attribute.Key("gen_ai.completion")
	AttrGenAIToolArguments        = attribute.Key("gen_ai.tool.arguments")
	AttrGenAIToolResult           = attribute.Key("gen_ai.tool.result")

	// Waza-specific keys for shape that GenAI semconv does not yet cover.
	AttrWazaEvalName     = attribute.Key("waza.eval.name")
	AttrWazaEvalSkill    = attribute.Key("waza.eval.skill")
	AttrWazaEvalEngine   = attribute.Key("waza.eval.engine")
	AttrWazaEvalRunID    = attribute.Key("waza.eval.run_id")
	AttrWazaTaskID       = attribute.Key("waza.task.id")
	AttrWazaTaskName     = attribute.Key("waza.task.name")
	AttrWazaTurnNumber   = attribute.Key("waza.turn.number")
	AttrWazaTurnTrial    = attribute.Key("waza.turn.trial")
	AttrWazaTurnKind     = attribute.Key("waza.turn.kind")
	AttrWazaSessionID    = attribute.Key("waza.session.id")
	AttrWazaWorkspaceDir = attribute.Key("waza.workspace.dir")
	AttrWazaToolSuccess  = attribute.Key("waza.tool.success")
	AttrWazaPremiumReqs  = attribute.Key("waza.usage.premium_requests")

	// Payload-redaction surrogate keys. When --otel-include-payloads is
	// off, payloadAttr emits a sha256 + length pair instead of the raw
	// payload. Keys are namespaced per payload slot so multiple redacted
	// payloads on the same span (prompt + completion, or tool args +
	// result) do not collide.
	AttrWazaPromptHash       = attribute.Key("waza.prompt.sha256")
	AttrWazaPromptLength     = attribute.Key("waza.prompt.length")
	AttrWazaCompletionHash   = attribute.Key("waza.completion.sha256")
	AttrWazaCompletionLength = attribute.Key("waza.completion.length")
	AttrWazaToolArgsHash     = attribute.Key("waza.tool.arguments.sha256")
	AttrWazaToolArgsLength   = attribute.Key("waza.tool.arguments.length")
	AttrWazaToolResultHash   = attribute.Key("waza.tool.result.sha256")
	AttrWazaToolResultLength = attribute.Key("waza.tool.result.length")
)

GenAI and waza-specific attribute keys. These follow the OpenTelemetry GenAI semantic conventions where possible (https://opentelemetry.io/docs/specs/semconv/gen-ai/) so spans land in off-the-shelf backends (Aspire, Jaeger, Tempo, App Insights, Honeycomb, Datadog) with familiar names. Waza-specific keys live under the `waza.*` namespace.

We define these as constants instead of importing the upstream `semconv/genai` module so that:

  1. The set of keys we emit is reviewable in one place; and
  2. We avoid pinning to a specific semconv version.
View Source 
const (
	GenAIOperationChat = "chat"
	GenAIOperationTool = "execute_tool"
)

Operation names per GenAI semconv.

View Source 
const GenAISystemCopilot = "github_copilot"

GenAI system value identifying waza-driven runs through the GitHub Copilot engine. We expose a constant so other engines (mock, custom) can override it by passing a different system into StartTurnSpan.

View Source 
const TracerName = "github.com/microsoft/waza"

TracerName is the OpenTelemetry instrumentation scope used by waza spans.

Variables

This section is empty.

Functions

func ParseHeaders 

func ParseHeaders(raw string) (map[string]string, error)

ParseHeaders parses a comma-separated list of key=value pairs into a map. Whitespace around keys and values is trimmed. Empty input returns nil. An empty key or a missing '=' produces an error.

func RecordCompletion 

func RecordCompletion(span trace.Span, p *Provider, output string)

RecordCompletion attaches the assistant's final output to an in-flight turn span. Honors the IncludePayloads policy.

func RecordModelCall 

func RecordModelCall(ctx context.Context, p *Provider, info ModelCallInfo)

RecordModelCall emits a child span describing one model invocation. The span is started and ended in this call because waza records model usage after the fact rather than wrapping the request itself.

func RecordToolCall 

func RecordToolCall(ctx context.Context, p *Provider, info ToolCallInfo)

RecordToolCall emits a child span describing one tool invocation. As with RecordModelCall the span is opened and closed in one go because waza only learns about the call after the engine reports it.

func StartEvalSpan 

func StartEvalSpan(ctx context.Context, p *Provider, info EvalInfo) (context.Context, trace.Span)

StartEvalSpan opens the root span for an eval run. The returned span should be ended with End() (and SetStatus on failure) when the eval finishes.

func StartTaskSpan 

func StartTaskSpan(ctx context.Context, p *Provider, info TaskInfo) (context.Context, trace.Span)

StartTaskSpan opens a child span for one test case.

func StartTurnSpan 

func StartTurnSpan(ctx context.Context, p *Provider, info TurnInfo) (context.Context, trace.Span)

StartTurnSpan opens a child span for one Execute call. The caller is expected to redact the prompt unless p.Config().IncludePayloads is true; this helper does the redaction for them.

Types

type Config 

type Config struct {
	// Exporter selects which span exporter to use. When empty, telemetry is off.
	Exporter ExporterKind

	// Endpoint is the OTLP/HTTP endpoint (e.g. "localhost:4318" or
	// "https://collector.example.com/v1/traces"). Only meaningful when
	// Exporter == ExporterOTLP.
	Endpoint string

	// Headers carries arbitrary OTLP headers (e.g. for authentication).
	// Parsed from --otel-headers=k1=v1,k2=v2.
	Headers map[string]string

	// FilePath is the destination when Exporter == ExporterFile.
	FilePath string

	// IncludePayloads disables payload redaction so prompt text, tool
	// arguments, and model output content are written into span attributes.
	// Default: false (payloads are redacted).
	IncludePayloads bool

	// ServiceName overrides the OTel resource service.name attribute.
	// Defaults to "waza".
	ServiceName string

	// ServiceVersion sets the OTel resource service.version attribute.
	ServiceVersion string
}

Config controls OpenTelemetry trace export. The zero value disables telemetry; callers must explicitly set Exporter for spans to be emitted.

func (Config) Enabled 

func (c Config) Enabled() bool

Enabled reports whether telemetry should be initialized.

func (Config) Validate 

func (c Config) Validate() error

Validate returns an error when the configuration is internally inconsistent. A disabled config is always valid.

type EvalInfo 

type EvalInfo struct {
	Name   string
	Skill  string
	Engine string
	Model  string
	RunID  string
}

EvalInfo carries top-level identifiers that label the root eval span.

type ExporterKind 

type ExporterKind string

ExporterKind selects the span exporter implementation. 

const (
	// ExporterNone disables telemetry. This is the default.
	ExporterNone ExporterKind = ""
	// ExporterOTLP exports via OTLP/HTTP to an OpenTelemetry collector.
	ExporterOTLP ExporterKind = "otlp"
	// ExporterStdout writes spans to stdout (for debugging).
	ExporterStdout ExporterKind = "stdout"
	// ExporterFile writes spans as JSON to a file (for debugging / CI artifacts).
	ExporterFile ExporterKind = "file"
)

type ModelCallInfo 

type ModelCallInfo struct {
	System        string
	RequestModel  string
	ResponseModel string
	InputTokens   int
	OutputTokens  int
	CacheReads    int
	PremiumReqs   float64
}

ModelCallInfo carries identifiers for one model invocation. Engines that expose per-call telemetry should emit one of these per chat completion; engines that only surface aggregate usage can emit a single ModelCall at turn end.

type Provider 

type Provider struct {
	// contains filtered or unexported fields
}

Provider wraps a configured tracer plus its Shutdown hook. Callers should always call Shutdown (best-effort, on a fresh context) to flush spans before the process exits.

func New 

func New(ctx context.Context, cfg Config) (*Provider, error)

New initializes telemetry from cfg and returns a Provider. When cfg is disabled (Exporter == ExporterNone) a no-op provider is returned with a nil shutdown function — calls to Tracer() still succeed and emit nothing.

New does not register the provider globally; callers that want spans created elsewhere in the process (e.g. by the Copilot SDK) can opt in by calling SetGlobal on the returned provider.

func (*Provider) Config 

func (p *Provider) Config() Config

Config returns the configuration the provider was built with. Useful for downstream code that needs to know whether payloads should be included (Config.IncludePayloads).

func (*Provider) Enabled 

func (p *Provider) Enabled() bool

Enabled reports whether the provider is exporting spans to a real backend (as opposed to a no-op tracer).

func (*Provider) SetGlobal 

func (p *Provider) SetGlobal()

SetGlobal registers p as the global OTel TracerProvider. Optional; only useful when downstream libraries (Copilot SDK, etc.) read from the global provider rather than receiving a tracer explicitly. No-op when telemetry is disabled.

func (*Provider) Shutdown 

func (p *Provider) Shutdown(ctx context.Context) error

Shutdown flushes pending spans and releases exporter resources. Safe to call when telemetry is disabled (it is a no-op in that case).

func (*Provider) Tracer 

func (p *Provider) Tracer() trace.Tracer

Tracer returns the tracer that should be used to start spans. When telemetry is disabled, the returned tracer is a no-op so callers can always wrap their work in spans without conditionals.

type TaskInfo 

type TaskInfo struct {
	TestID      string
	DisplayName string
}

TaskInfo carries identifiers for a single test case execution.

type ToolCallInfo 

type ToolCallInfo struct {
	Name      string
	CallID    string
	Arguments string
	Result    string
	Success   bool
}

ToolCallInfo carries identifiers for one tool invocation.

type TurnInfo 

type TurnInfo struct {
	Number       int    // 1-indexed turn within the conversation (initial=1)
	Trial        int    // optional trial number when the same task is re-run
	Kind         string // "initial" | "follow_up" | "responder_reply"
	Model        string
	SessionID    string
	WorkspaceDir string
	Prompt       string
}

TurnInfo carries identifiers for one Execute call (initial prompt, follow-up, or responder reply).

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.