agenttrace

package
v0.9.22 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 22, 2026 License: Apache-2.0 Imports: 26 Imported by: 0

Documentation

Overview

Package agenttrace provides tracing infrastructure for AI agent interactions.

Overview

This package contains the foundational types for tracking agent executions:

  • ExecutionContext: Reconciler-level metadata (PR, path, commit, turn number) for trace enrichment
  • Trace[T]: Complete agent interaction from prompt to result
  • ToolCall[T]: Individual tool invocation within a trace
  • Tracer[T]: Interface for creating and managing traces

Separation of Concerns

The agenttrace package provides low-level tracing primitives. Higher-level evaluation helpers (NoErrors, ExactToolCalls, etc.), observers, and metrics reporters are in the evals package which builds on top of this package.

Usage

Set execution context for trace enrichment: 

ctx = agenttrace.WithExecutionContext(ctx, agenttrace.ExecutionContext{
	ReconcilerKey:  "pr:chainguard-dev/enterprise-packages/41025",
	ReconcilerType: "pr",
	CommitSHA:      "abc123",
	TurnNumber:     1,
	// Optional bounded custom labels stamped on every GenAI metric emitted
	// while this context is in scope (see ExecutionContext.EnrichAttributes).
	Labels: map[string]string{"genai_component": "analyzer", "purl_type": "npm"},
})

Create and use traces: 

tracer := agenttrace.ByCode[string](func(trace *agenttrace.Trace[string]) {
	log.Printf("Trace completed: %s", trace.ID)
})
ctx = agenttrace.WithTracer[string](ctx, tracer)

trace, done := agenttrace.StartTrace[string](ctx, "Analyze the security report")
toolCall := trace.StartToolCall("tc1", "file-reader", map[string]any{
	"path": "/var/logs/security.log",
})
toolCall.Complete("File content here", nil)
done("Analysis complete", nil)

Index

Examples

Constants

View Source 
const (
	SystemAnthropic    = "anthropic"
	SystemGoogleVertex = "google.vertex"
	SystemOpenAI       = "openai"
)

Canonical gen_ai.system values per the OpenTelemetry GenAI semantic conventions. Executors pass these to BeginTurn so downstream eval tools can filter traces by provider without consumers needing to remember the exact spelling.

View Source 
const (
	// AttrDisposition names a trace's terminal disposition when it is neither
	// a plain success nor an error (currently only DispositionSuspended).
	AttrDisposition = "driftlessaf.disposition"
	// AttrSuspensionReason carries the free-form reason a trace suspended.
	AttrSuspensionReason = "driftlessaf.suspension.reason"
	// DispositionSuspended is the AttrDisposition value for a suspended trace.
	DispositionSuspended = "suspended"
	// AttrResumeLinkedTraceID links a resumed run's root span back to the
	// trace that suspended, since a resume mints a fresh trace rather than
	// continuing the suspended one.
	AttrResumeLinkedTraceID = "driftlessaf.resume.linked_trace_id"
)

Suspension span attributes describing a trace's terminal disposition when it halted mid-run instead of completing or failing.

View Source 
const (
	// EventType is the CloudEvent type for agent trace records.
	EventType = "dev.chainguard.driftlessaf.agent.trace.v1"
)
View Source 
const SpanEventType = "dev.chainguard.driftlessaf.agent.span.v1"

SpanEventType is the CloudEvent type for the per-span trace payload. One event is emitted per LLM turn when payloads are enabled, alongside the per-trace event emitted at trace completion.

Per-span emission is gated on WithPayloadsEnabled(ctx, true). The gate is an explicit caller opt-in because the per-span payload carries the full prompt and completion bytes alongside the per-turn metadata that the per-trace event already records. When the gate is off (the default), RecordRequest and RecordResponse are no-ops, no per-span CloudEvent is emitted, and only the per-trace event flows. When the gate is on, the caller takes responsibility for what ends up in the receiving BigQuery table.

Retention

Persisted payloads are stored verbatim. Retention is configured by the receiving cloudevent-recorder module, which sets BigQuery partition expiration on the table this event lands in. Two knobs:

  • Dataset default: set retention-period (in days) on the cloudevent-recorder module call. Applies to every CloudEvent type the module records.
  • Per-event override: set retention_period_days on the "dev.chainguard.driftlessaf.agent.span.v1" entry in the module's types map. Overrides the dataset default for this event type only.

The partition field is recorded_at (see iac/schemas/agent_trace_span.schema.json).

Variables

View Source 
var DefaultRationaleToolNames = []string{
	"edit_file", "write_file", "delete_file", "move_file", "copy_file",
	"chmod", "symlink",
}

DefaultRationaleToolNames lists the mutating tools whose per-call `reasoning` argument reads as a change rationale ("why I made this edit"). Read-only exploration tools (read_file, search_codebase, ...) and result-submission tools are deliberately excluded: their rationales are procedural noise in a change summary.

Functions

func CaptureTrace added in v0.7.43 

func CaptureTrace[T any](ctx context.Context) (context.Context, func() *Trace[T])

CaptureTrace returns a derived context whose Tracer[T] tees every completed trace into an accumulator, plus a function returning the most recently completed trace. The previously-installed tracer (CloudEvent emission, logging, ...) is forwarded unchanged — this only adds a side effect, it never replaces or skips the original behavior.

When several traces of the same T complete under the returned context, the most recent wins: for the meta-reconcilers' use (one agent execution per capture scope) that is the agent run itself, and a retried run cleanly supersedes its predecessor. The accessor reports nil until a trace completes, so downstream rendering (SummarizeTraceReasoning returns "" for nil) degrades to exactly the pre-capture behavior.

Each call installs a fresh accumulator, so concurrent reconciliations (each with their own derived context) never share state.

func GetDefaultAgentName added in v0.6.0 

func GetDefaultAgentName(ctx context.Context) string

GetDefaultAgentName returns the agent name stored in the context by WithDefaultAgentName, or "" if none was set.

func GetDefaultNameFn added in v0.6.0 

func GetDefaultNameFn(ctx context.Context) func(ExecutionContext) string

GetDefaultNameFn returns the name function stored in the context by WithDefaultNameFn, or nil if none was set.

func JoinStringsWithLimit added in v0.7.37 

func JoinStringsWithLimit(blocks []string, maxChars int, separator string, moreNote func(dropped int) string) string

JoinStringsWithLimit joins non-blank, trimmed entries from blocks with separator, stopping before the combined length would exceed maxChars. If any blocks don't fit, moreNote is called with the count of blocks that were dropped entirely and the result is appended.

When even the first block alone (after trimming) exceeds maxChars, it is shown truncated to maxChars rather than omitted, so the result is never empty just because one block is long; that block does not count toward the dropped total passed to moreNote, since it was shown (albeit cut short), not dropped.

Blank entries (empty after trimming) are skipped entirely and never count toward the dropped total. Returns "" if blocks is empty or every entry is blank.

func NewBrokerClient added in v0.4.1 

func NewBrokerClient(ctx context.Context, brokerURL string, opts ...option.ClientOption) cloudevents.Client

NewBrokerClient creates a CloudEvents HTTP client authenticated with an ID token for the given broker URL. Call this once at startup and pass the client to WithCloudEventEmission or middleware that wraps it.

If brokerURL is empty or client construction fails, NewBrokerClient returns nil with a warning log. Callers should treat a nil client as "emission disabled" and skip wrapping the tracer.

The ID token is signed directly from the ambient ADC (the common case: a reconciler running as its own service account). When the identity that authenticates the process differs from the identity authorized to call the broker — and especially when the ambient credential is a federated (external_account) one that cannot mint an ID token directly — use NewBrokerClientImpersonating instead.

opts are forwarded to idtoken.NewTokenSource.

func NewBrokerClientImpersonating added in v0.7.11 

func NewBrokerClientImpersonating(ctx context.Context, brokerURL, targetPrincipal string) cloudevents.Client

NewBrokerClientImpersonating is like NewBrokerClient but mints the broker ID token by impersonating targetPrincipal (a service account email) rather than signing directly with the ambient ADC. The process's ADC must hold roles/iam.serviceAccountTokenCreator on targetPrincipal.

This is the path for a producer whose ambient credential is a federated (external_account) WIF credential: idtoken cannot mint an ID token directly from such a credential, but the ADC can obtain an access token and use it to impersonate targetPrincipal, which generates the ID token. The returned source refreshes automatically. brokerURL empty or source construction failure returns nil (emission disabled), same as NewBrokerClient.

func SetLocalSpanSink added in v0.7.61 

func SetLocalSpanSink(fn SpanEmitter) (restore func())

SetLocalSpanSink installs fn as the process-global span sink. LLMTurn.End calls it for every turn that recorded a request payload (i.e. under WithPayloadsEnabled(ctx, true) — without that opt-in buildRecordedSpan yields nothing and the sink is never called), regardless of the context tracer or its response type. It fires in ADDITION to any per-tracer spanEmitter, so a consumer that installs both receives each span twice; typically only one is in use (the CloudEvent emitter in production, or this local sink in a CLI).

fn must be non-blocking and safe for concurrent use: End invokes it synchronously on the turn-cleanup path, and turns from concurrently-executing agents can call it in parallel. Passing nil clears the sink. The returned restore func reinstalls the previously-registered sink — defer it to scope a sink to one run rather than leaking a file handle into the process global.

func SummarizeReasoning added in v0.7.37 

func SummarizeReasoning(blocks []ReasoningContent, maxChars int) string

SummarizeReasoning renders a concise, human-readable summary of a trace's extended-thinking blocks, suitable for display in a PR body or comment. It performs straightforward truncation rather than a second LLM call: blocks are rendered as a bulleted list and joined until adding the next one would exceed maxChars. If any blocks don't fit, a "+N more reasoning blocks" note replaces them.

Returns "" if blocks is empty or contains only blank entries. Note that maxChars bounds the joined *bulleted* text (each non-blank block prefixed with "- "), so the rendered result can run a couple of characters over maxChars relative to the raw block content.

func SummarizeTraceReasoning added in v0.7.43 

func SummarizeTraceReasoning[T any](t *Trace[T], maxChars int) string

SummarizeTraceReasoning renders a concise, human-readable rationale for a completed trace, suitable for a PR body. It prefers the per-action reasoning recorded on mutating tool calls (see DefaultRationaleToolNames and Trace.AttachToolCallReasoning) — one bullet per distinct rationale, in call order — because those explain individual changes. When the trace carries no tool-call rationales it falls back to the extended-thinking blocks via SummarizeReasoning. Returns "" for a nil trace or when neither source has content.

func WithDefaultAgentName added in v0.6.0 

func WithDefaultAgentName(ctx context.Context, name string) context.Context

WithDefaultAgentName returns a context carrying the given agent name so any subsequent StartTrace call without an explicit WithAgentName option emits gen_ai.agent.name=<name> on the root invoke_agent span. Callers building a reconciler that drives multiple executors can set this once at the top of the chain to attach a stable agent name (e.g. "loganalyzer", "judge", "fixer") to every trace.

func WithDefaultNameFn added in v0.6.0 

func WithDefaultNameFn(ctx context.Context, fn func(ExecutionContext) string) context.Context

WithDefaultNameFn returns a context carrying a name function invoked by newTrace to build the driftlessaf.invocation.label attribute from the ExecutionContext. Callers at the reconciler layer (where the GitHub Resource is available) use this to stamp resource-aware labels such as "autofix: pr:chainguard-dev/mono#38632" on every subsequent trace.

func WithExecutionContext 

func WithExecutionContext(ctx context.Context, execCtx ExecutionContext) context.Context

WithExecutionContext layers non-zero fields from execCtx onto any ExecutionContext already on ctx. Zero-valued fields leave the existing value intact, so a deep call site that only owns TurnNumber can update it without clobbering ReconcilerKey/ReconcilerType/CommitSHA set by the enclosing reconciler. Pass a fully-populated struct to set all fields at once.

Example

ExampleWithExecutionContext demonstrates attaching execution context to a context for trace enrichment. 

package main

import (
	"context"
	"fmt"

	"chainguard.dev/driftlessaf/agents/agenttrace"
)

func main() {
	ctx := context.Background()
	ctx = agenttrace.WithExecutionContext(ctx, agenttrace.ExecutionContext{
		ReconcilerKey:  "pr:chainguard-dev/enterprise-packages/42",
		ReconcilerType: "pr",
		CommitSHA:      "abc123",
		TurnNumber:     1,
	})

	ec := agenttrace.GetExecutionContext(ctx)
	fmt.Printf("key=%s turn=%d\n", ec.ReconcilerKey, ec.TurnNumber)
}

Output:
key=pr:chainguard-dev/enterprise-packages/42 turn=1
Example (Partial)

ExampleWithExecutionContext_partial demonstrates the merge-on-non-zero semantics: a deep call site that only knows about TurnNumber updates it without clobbering ReconcilerKey, ReconcilerType, or CommitSHA set by the enclosing reconciler. 

package main

import (
	"context"
	"fmt"

	"chainguard.dev/driftlessaf/agents/agenttrace"
)

func main() {
	ctx := agenttrace.WithExecutionContext(context.Background(), agenttrace.ExecutionContext{
		ReconcilerKey:  "pr:chainguard-dev/mono/40044",
		ReconcilerType: "pr",
		CommitSHA:      "abc123",
	})

	ctx = agenttrace.WithExecutionContext(ctx, agenttrace.ExecutionContext{TurnNumber: 3})

	ec := agenttrace.GetExecutionContext(ctx)
	fmt.Printf("key=%s sha=%s turn=%d\n", ec.ReconcilerKey, ec.CommitSHA, ec.TurnNumber)
}

Output:
key=pr:chainguard-dev/mono/40044 sha=abc123 turn=3

func WithPayloadsEnabled added in v0.6.0 

func WithPayloadsEnabled(ctx context.Context, enabled bool) context.Context

WithPayloadsEnabled returns a context that opts in (or out) of emitting raw prompt / completion payloads as OTel attributes on the root invoke_agent span (gen_ai.prompt, gen_ai.input.messages, gen_ai.completion, gen_ai.output.messages). The default — when nothing is set on ctx — is false so library consumers don't accidentally leak PII-bearing prompts to a third-party eval backend.

Consumer main packages read their own env var (e.g. DRIFTLESSAF_LLM_PAYLOADS) at startup and set this flag on the base context before handing off to the reconciler or executor. Keeping the decision on ctx (rather than a process-wide env read at package init) matches the repository's go-standards rule that library packages accept configuration as parameters instead of reading the environment directly.

func WithTracer 

func WithTracer[T any](ctx context.Context, tracer Tracer[T]) context.Context

WithTracer returns a new context with the given tracer

Types

type CEOption added in v0.7.49 

type CEOption[T any] func(*ceEmittingTracer[T])

CEOption configures a ceEmittingTracer built by WithCloudEventEmission.

func WithPayloadEncryptor added in v0.7.49 

func WithPayloadEncryptor[T any](enc *payloadcrypt.Encryptor) CEOption[T]

WithPayloadEncryptor seals the sensitive payload fields (input_prompt, result, tool_calls[].params/result, reasoning[].thinking, and per-span prompt_messages/completion) of every emitted event under enc before it is sent. Only structural fields (ids, model, agent_name, source, token counts, timings, exec_context, errors, prompt_hash, and the provider identifier) stay plaintext, so cost views keep working; the payload fields surface as opaque sealed envelopes to every downstream consumer — the agent-trace MCP, UI, and trace replayers — until a reader/break-glass Open path is deployed. Enabling this on a producer before that reader path ships means those consumers serve ciphertext, so order the rollout accordingly.

Emission is fail-closed: if sealing errors (e.g. KMS is unreachable), the event is dropped rather than sent in the clear.

type ExecutionContext 

type ExecutionContext struct {
	ReconcilerKey  string `json:"reconciler_key,omitempty"`  // Primary identifier: "pr:chainguard-dev/enterprise-packages/41025" or "path:chainguard-dev/mono/main/images/nginx"
	ReconcilerType string `json:"reconciler_type,omitempty"` // Type of reconciler: "pr" or "path"
	CommitSHA      string `json:"commit_sha,omitempty"`      // Git commit SHA (optional, for git-based reconcilers)
	TurnNumber     int    `json:"turn_number,omitempty"`     // Turn number for multi-turn agents (optional, 1, 2, 3, ...)

	// RequestID identifies the individual work item / request being handled
	// (e.g. an analyzer vuln-event request ID, or a comma-joined set for a
	// forge series). Like ReconcilerKey/CommitSHA it is HIGH-CARDINALITY and is
	// therefore deliberately kept OFF metrics (not emitted by EnrichAttributes)
	// to avoid minting a new time series per request. It rides on the trace
	// record's exec_context and is emitted as a request_id span attribute, so
	// per-request GenAI usage stays sliceable via traces without inflating
	// metric cardinality. Do NOT move this into Labels.
	RequestID string `json:"request_id,omitempty"`

	// Labels holds additional bounded custom labels that a consumer wants
	// stamped on every GenAI metric emitted while this context is in scope
	// (see EnrichAttributes). Unlike ReconcilerKey/CommitSHA — which are
	// deliberately kept off metrics to avoid unbounded cardinality — these are
	// intended for low-cardinality dimensions the consumer controls, e.g.
	// {"genai_component": "analyzer", "purl_type": "npm"}. Use this for
	// per-request labels that cannot be set at agent-construction time (where
	// metaagent.Config.ResourceLabels is the better fit). Keep the value set
	// bounded — every distinct value creates a new metric time series.
	Labels map[string]string `json:"labels,omitempty"`
}

ExecutionContext provides reconciler-level context for agent executions. This context is used to enrich metrics with labels for tracking token usage and tool calls per reconciler (PR, path, etc.).

func GetExecutionContext 

func GetExecutionContext(ctx context.Context) ExecutionContext

GetExecutionContext retrieves execution context from the Go context

func (ExecutionContext) EnrichAttributes 

func (e ExecutionContext) EnrichAttributes(baseAttrs []attribute.KeyValue) []attribute.KeyValue

EnrichAttributes adds execution context attributes to the provided base attributes. This is used to enrich metrics with reconciler context using only BOUNDED labels.

Note: reconciler_key and commit_sha are NOT included in metrics to prevent unbounded cardinality (every PR and commit creates a new time series). These fields remain in the ExecutionContext for traces where cardinality is not a concern. Use trace exemplars to link from aggregated metrics to detailed per-PR traces.

func (ExecutionContext) Repository 

func (e ExecutionContext) Repository() string

Repository extracts the repository from the reconciler key. For "pr:chainguard-dev/enterprise-packages/41025" returns "chainguard-dev/enterprise-packages" For "path:chainguard-dev/mono/main/images/nginx" returns "chainguard-dev/mono" Returns empty string if the format is invalid.

type LLMTurn added in v0.4.1 

type LLMTurn[T any] struct {
	// contains filtered or unexported fields
}

LLMTurn represents a single LLM call within a trace.

LLMTurn is not safe for concurrent use: callers own a turn for the duration of a single model roundtrip and must not hand it across goroutines. The lifecycle methods (RecordTokens, End) read and mutate the accumulated record without locking because the contract is single-goroutine-per-turn, mirroring ToolCall.

func (*LLMTurn[T]) End added in v0.4.1 

func (lt *LLMTurn[T]) End()

End ends the turn span and restores the trace context to before the turn. It is idempotent: subsequent calls are no-ops. On the first call, it appends the accumulated RecordedTurn to the parent trace's Turns slice and, when a request payload was recorded and the trace has a spanEmitter configured, emits a per-span CloudEvent for the turn.

func (*LLMTurn[T]) Fail added in v0.6.0 

func (lt *LLMTurn[T]) Fail(err error)

Fail marks the turn as having ultimately failed and sets the OTEL span status to Error — mirroring ToolCall.Complete and Trace.complete on the failure path. RecordedTurn.Failed flips to true unconditionally; if err is non-nil it is also appended to Errors and recorded as a span exception event.

Fail(nil) is intentionally NOT symmetric with RecordError(nil). Calling Fail means "this turn ended in failure" — that signal must propagate even when the caller has no concrete error value (e.g. context cancellation surfaced upstream as a sentinel). The alternative (silent no-op) trades a loud false positive for a silent loss of failure signal in BQ; we prefer the loud one because it's discoverable. Callers that don't want to fail the turn must guard the call themselves: `if err != nil { lt.Fail(err) }`, which is exactly what the executor wiring does.

Call before End. Safe to call multiple times: the Failed flag is sticky (subsequent calls don't toggle it off), and each call with non-nil err appends to Errors.

func (*LLMTurn[T]) RecordCacheTokens added in v0.7.1 

func (lt *LLMTurn[T]) RecordCacheTokens(cacheReadTokens, cacheCreationTokens int64)

RecordCacheTokens sets prompt cache token counts as span attributes on the turn span and on the per-turn record. The OTel GenAI semconv attributes (gen_ai.usage.cache_read_input_tokens, gen_ai.usage.cache_creation_input_tokens) belong on the per-call chat span, not the orchestration span. Per-turn cache counts let downstream cost analysis apply the cache discount / surcharge accurately on a per-call basis.

func (*LLMTurn[T]) RecordError added in v0.6.0 

func (lt *LLMTurn[T]) RecordError(err error)

RecordError appends err to the turn's chronological error list and emits an exception event on the OTEL span. It does NOT mark the turn as failed — use Fail for that. RecordError is the right call for transient errors the turn recovered from (e.g. a 503 that succeeded on retry); a non-empty Errors list with Failed=false is exactly that recovery shape.

A nil err is a no-op. Call before End.

func (*LLMTurn[T]) RecordRequest added in v0.7.2 

func (lt *LLMTurn[T]) RecordRequest(messages any) error

RecordRequest captures the per-turn prompt messages. When WithPayloadsEnabled is false on the trace context this is a no-op returning nil. The payload is truncated to maxPayloadBytes before being stored on the turn for later emission in End.

Payloads are persisted verbatim — see the "Security posture" block on SpanEventType for the implications and the canonical handling of a credential-leak event.

func (*LLMTurn[T]) RecordResponse added in v0.7.2 

func (lt *LLMTurn[T]) RecordResponse(content any) error

RecordResponse captures the per-turn completion content. Same gating and truncation as RecordRequest; same verbatim-persistence posture — see the "Security posture" block on SpanEventType.

func (*LLMTurn[T]) RecordTokens added in v0.4.1 

func (lt *LLMTurn[T]) RecordTokens(inputTokens, outputTokens int64)

RecordTokens sets input/output token counts as span attributes on the turn span.

type ReasoningContent 

type ReasoningContent struct {
	Thinking string `json:"thinking"`
}

ReasoningContent represents internal reasoning from an LLM

type RecordedSpan added in v0.7.2 

type RecordedSpan struct {
	TraceID        string          `json:"trace_id"`
	SpanID         string          `json:"span_id"`
	AgentName      string          `json:"agent_name,omitempty"`
	ModelID        string          `json:"model_id,omitempty"`
	RecordedAt     time.Time       `json:"recorded_at"`
	PromptMessages json.RawMessage `json:"prompt_messages,omitempty"`
	Completion     json.RawMessage `json:"completion,omitempty"`
	PromptHash     string          `json:"prompt_hash,omitempty"`
	TokenCounts    json.RawMessage `json:"token_counts,omitempty"`
	JudgeScore     *float64        `json:"judge_score,omitempty"`
	Metadata       json.RawMessage `json:"metadata,omitempty"`
}

RecordedSpan is one LLM turn's payload, shaped to the agent_trace_span BigQuery schema (see iac/schemas/agent_trace_span.schema.json). Field names mirror the BQ column names exactly via json tags. JSON columns use json.RawMessage so the marshal is pass-through and the BQ ingester treats them as native JSON instead of nested STRING.

SpanID is deterministic (trace_id + turn_index) so a CloudEvent retry that re-delivers the same event produces a row with the same span_id. BigQuery does not enforce uniqueness on this field; downstream consumers that need at-most-once semantics must dedup by span_id themselves.

type RecordedTurn added in v0.6.0 

type RecordedTurn struct {
	Index               int       `json:"index"`
	Model               string    `json:"model,omitempty"`
	System              string    `json:"system,omitempty"`
	InputTokens         int64     `json:"input_tokens,omitempty"`
	OutputTokens        int64     `json:"output_tokens,omitempty"`
	CacheReadTokens     int64     `json:"cache_read_tokens,omitempty"`
	CacheCreationTokens int64     `json:"cache_creation_tokens,omitempty"`
	StartTime           time.Time `json:"start_time"`
	EndTime             time.Time `json:"end_time"`
	// Errors is the chronological list of errors the turn encountered, including
	// transients that the turn recovered from. A non-empty list does NOT mean
	// the turn failed — see Failed for the terminal outcome. Populated via
	// LLMTurn.RecordError and LLMTurn.Fail.
	Errors []string `json:"errors,omitempty"`
	// Failed is the terminal outcome flag: true iff the turn ultimately failed.
	// A turn can have non-empty Errors and Failed=false (it recovered from
	// transient errors). Set by LLMTurn.Fail; defaults to false. Serialized
	// without omitempty so successful turns get an explicit `false` in BQ
	// instead of NULL — analytics queries can use `failed = FALSE` directly
	// without the three-valued-logic gotcha of NULL.
	Failed bool `json:"failed"`
}

RecordedTurn is the per-turn data captured on a Trace so each LLM turn surfaces as a queryable row in downstream sinks (e.g. BigQuery via the CloudEvent payload). Token counts reflect a single turn rather than cumulative totals on the parent Trace.

type SpanEmitter added in v0.7.2 

type SpanEmitter func(ctx context.Context, span RecordedSpan) error

SpanEmitter is the callback used by LLMTurn.End to ship a per-turn payload out as a CloudEvent. Implementations should be non-blocking — the existing per-trace emitter pattern uses a bounded errgroup, and span emission must not stall the reconciler.

type StartTraceOption added in v0.6.0 

type StartTraceOption func(*traceOptions)

StartTraceOption configures trace creation. Options let callers attach an agent name (static, for gen_ai.agent.name) and a dynamic name function (for driftlessaf.invocation.label — e.g. "autofix: pr:chainguard-dev/mono#38632").

func WithAgentName added in v0.6.0 

func WithAgentName(name string) StartTraceOption

WithAgentName sets the static gen_ai.agent.name attribute on the root invoke_agent span (e.g. "loganalyzer", "judge", "fixer"). Also used as the fallback for driftlessaf.invocation.label when no nameFn is set.

func WithNameFn added in v0.6.0 

func WithNameFn(fn func(ExecutionContext) string) StartTraceOption

WithNameFn sets a callback that produces the driftlessaf.invocation.label attribute — a per-invocation, free-form label (e.g. "autofix: pr:chainguard-dev/mono#38632") computed from the ExecutionContext. Vendor-neutral; backends that want to use this as the primary trace name (Braintrust, Langfuse, etc.) bridge it in user-side span processors. When nil or unset, driftlessaf.invocation.label falls back to agentName.

type ToolCall 

type ToolCall[T any] struct {
	ID     string         `json:"id"`
	Name   string         `json:"name"`
	Params map[string]any `json:"params,omitempty"`
	Result any            `json:"result,omitempty"`
	Error  error          `json:"-"` // handled by MarshalJSON
	// Recoverable marks an error-carrying call as a designed correction:
	// the handler rejected the call but returned a corrective hint the model
	// can act on, so the record is retryable rather than terminal — the
	// tool-call-level twin of RecordedTurn.Failed's recovered-error
	// semantics. The error stays on the trace for observability; consumers
	// that gate on tool-call errors (e.g. the no-errors eval scorer) skip
	// recoverable records. Producers may set it only on paths where a run
	// that never recovers cannot complete cleanly (e.g. a rejected terminal
	// submit: the run either lands an accepted submission later or fails
	// with a trace-level error).
	Recoverable bool      `json:"recoverable,omitempty"`
	StartTime   time.Time `json:"start_time"`
	EndTime     time.Time `json:"end_time"`
	// contains filtered or unexported fields
}

ToolCall represents a single tool invocation within a trace

func (*ToolCall[T]) Complete 

func (tc *ToolCall[T]) Complete(result any, err error)

Complete marks the tool call as complete and adds it to the parent trace

func (*ToolCall[T]) CompleteRejected added in v0.9.18 

func (tc *ToolCall[T]) CompleteRejected(err error)

CompleteRejected marks the tool call as complete with a rejection the model can correct: the error is recorded for observability, but the call is marked Recoverable because the handler returned a corrective hint and the conversation loop continues. See ToolCall.Recoverable for the contract producers must honor before choosing this over Complete.

func (*ToolCall[T]) Duration 

func (tc *ToolCall[T]) Duration() time.Duration

Duration returns the duration of the tool call

func (*ToolCall[T]) MarshalJSON added in v0.7.77 

func (tc *ToolCall[T]) MarshalJSON() ([]byte, error)

MarshalJSON implements json.Marshaler for ToolCall. It converts the Error field to a string and excludes unexported fields.

type Trace 

type Trace[T any] struct {
	ID          string             `json:"id"`
	OTelTraceID string             `json:"otel_trace_id,omitempty"`
	InputPrompt string             `json:"input_prompt"`
	ExecContext ExecutionContext   `json:"exec_context,omitempty"` // PR/commit metadata
	ToolCalls   []*ToolCall[T]     `json:"tool_calls"`
	Turns       []RecordedTurn     `json:"turns,omitempty"`
	Reasoning   []ReasoningContent `json:"reasoning,omitempty"`
	Result      T                  `json:"result"`
	Error       error              `json:"-"` // handled by MarshalJSON
	StartTime   time.Time          `json:"start_time"`
	EndTime     time.Time          `json:"end_time"`
	Metadata    map[string]any     `json:"metadata,omitempty"`

	// AgentName identifies which logical agent produced this trace (e.g.
	// "materializer", "skillup-reviewer"). Stamped by a middleware that
	// knows the agent identity at tracer construction time.
	AgentName string `json:"agent_name,omitempty"`

	// Source mirrors the CloudEvent Ce-Source header (typically the
	// reconciler's OCTO_IDENTITY). Duplicated into the payload so BigQuery
	// can query by service — the recorder records only event.Data().
	Source string `json:"source,omitempty"`

	// Model identifies the LLM the executor drove this trace against
	// (e.g. "claude-sonnet-4-6", "gemini-2.5-pro"). Populated lazily by
	// BeginTurn from the first turn's model — assumes a single-model
	// trace, which matches every executor in the codebase. Per-turn
	// model lives on Turns[i].Model so multi-model traces can still be
	// reasoned about turn-by-turn.
	Model string `json:"model,omitempty"`

	// Suspended marks a trace that halted mid-run to await an out-of-band
	// signal (e.g. a human answer, a checkpoint wake) rather than completing
	// or failing. A suspended trace is NOT an error: the root span carries
	// status OK and a driftlessaf.suspension.reason attribute, and Error stays
	// nil. Set only by Suspend, so eval graders and dashboards can distinguish
	// an intentional halt from a failure.
	Suspended bool `json:"suspended,omitempty"`

	// SuspensionReason is the free-form reason recorded by Suspend (e.g.
	// "awaiting answer"). Empty unless Suspended is true; also emitted
	// as the driftlessaf.suspension.reason span attribute.
	SuspensionReason string `json:"suspension_reason,omitempty"`
	// contains filtered or unexported fields
}

Trace represents a complete agent interaction from prompt to result.

Trace implements json.Marshaler so it can be serialized directly. The custom MarshalJSON converts the Error field to a string and excludes unexported runtime handles (mutex, context, span). Serialization is intended to happen after Complete — at that point the trace is immutable and no lock is needed.

func StartTrace 

func StartTrace[T any](ctx context.Context, prompt string, opts ...StartTraceOption) (*Trace[T], func(T, error))

StartTrace starts a new trace using the tracer from the context and returns the trace along with a done callback. The caller must invoke done(result, err) when the operation completes; this fills in the trace and records it via the tracer. Capturing the tracer at start time means decorator composition works without a second context lookup.

opts customize the root invoke_agent span attributes — e.g. WithAgentName stamps gen_ai.agent.name, and WithNameFn produces a dynamic driftlessaf.invocation.label based on ExecutionContext.

Example

ExampleStartTrace demonstrates creating and completing a trace. 

package main

import (
	"context"
	"fmt"

	"chainguard.dev/driftlessaf/agents/agenttrace"
)

func main() {
	ctx := context.Background()

	tracer := agenttrace.ByCode[string](func(trace *agenttrace.Trace[string]) {
		fmt.Printf("Trace completed: %s\n", trace.Result)
	})
	ctx = agenttrace.WithTracer[string](ctx, tracer)

	_, done := agenttrace.StartTrace[string](ctx, "Analyze the report")
	done("analysis done", nil)
}

Output:
Trace completed: analysis done

func (*Trace[T]) AppendReasoning added in v0.7.47 

func (t *Trace[T]) AppendReasoning(content ReasoningContent)

AppendReasoning records a block of raw model reasoning (Claude thinking / redacted_thinking blocks, Gemini thought parts) on the trace, but only when the WithPayloadsEnabled opt-in is set on the trace context. Model reasoning is unredacted completion content: under the Derivative Sensitivity Inheritance invariant it inherits the parent submission's confidentiality tier, so it is gated behind the same payload opt-in as raw prompt and completion emission rather than being captured unconditionally. A no-op when the content is empty or the opt-in is off.

func (*Trace[T]) AttachToolCallReasoning added in v0.7.43 

func (t *Trace[T]) AttachToolCallReasoning(id, reasoning string)

AttachToolCallReasoning merges the model-supplied reasoning for the tool call with the given id into that call's recorded params. Executors call this after dispatching a tool: the model passes a universal `reasoning` argument on every call (auto-injected into each tool schema by the executor bridges), but handlers record curated param maps that drop it. Attaching it here preserves the per-action rationale on the trace record — and, since params serialize verbatim, in BigQuery's tool_calls[].params JSON. A no-op when no completed call matches id (e.g. the handler bailed on a parameter error before recording) or when the handler already recorded a reasoning value.

func (*Trace[T]) BadToolCall 

func (t *Trace[T]) BadToolCall(id, name string, params map[string]any, err error)

BadToolCall records a tool call that failed due to bad arguments or unknown tool

func (*Trace[T]) BeginTurn added in v0.4.1 

func (t *Trace[T]) BeginTurn(turn int, system, modelName string) *LLMTurn[T]

BeginTurn starts a new LLM turn span as a child of the trace span. The trace context is updated so subsequent tool call spans are nested under this turn span. Call End() on the returned LLMTurn when the turn completes.

system is the OTel GenAI provider identifier: "openai", "anthropic", "google.vertex", etc. It powers provider filtering in eval tools.

Per-call token usage (input/output and prompt cache) is recorded by LLMTurn.RecordTokens / LLMTurn.RecordCacheTokens — replacing the old trace-level RecordTokenUsage / RecordCacheTokenUsage methods removed in DEV-1140. Per OTel GenAI semconv, gen_ai.usage.* attributes belong on the per-call "chat <model>" span, not the orchestration invoke_agent span. Trace-level token totals are derived from turns[] in downstream consumers (see agent_trace_costs.sql).

Callers MUST call End() on the current turn before calling BeginTurn again. Overlapping turns corrupt the span hierarchy: the later End() restores a stale context, causing subsequent spans to be parented incorrectly.

func (*Trace[T]) Context added in v0.7.2 

func (t *Trace[T]) Context() context.Context

Context returns the context the trace was created with. Eval callbacks run after Complete() and historically derived from context.Background() to detach from any cancellation/deadline on the original request, but that also dropped the reconciler's WithDefaultNameFn / WithDefaultAgentName / WithPayloadsEnabled, causing every eval-emitted trace (e.g. judge) to surface as an orphan root span with no link to the parent trace tree. Callbacks should wrap the returned ctx with context.WithoutCancel before kicking off long-lived work so they inherit the reconciler-set values without inheriting the completed request's cancellation.

func (*Trace[T]) Duration 

func (t *Trace[T]) Duration() time.Duration

Duration returns the total duration of the trace

func (*Trace[T]) MarshalJSON added in v0.4.1 

func (t *Trace[T]) MarshalJSON() ([]byte, error)

MarshalJSON implements json.Marshaler for Trace. It converts the Error field to a string and excludes unexported fields.

func (*Trace[T]) RejectedToolCall added in v0.9.18 

func (t *Trace[T]) RejectedToolCall(id, name string, params map[string]any, err error)

RejectedToolCall records a tool call the handler rejected with a corrective hint the model can act on — the recoverable sibling of BadToolCall. See ToolCall.Recoverable for the contract producers must honor before choosing this over BadToolCall.

func (*Trace[T]) StartToolCall 

func (t *Trace[T]) StartToolCall(id, name string, params map[string]any) *ToolCall[T]

StartToolCall starts a new tool call and returns it

func (*Trace[T]) String 

func (t *Trace[T]) String() string

String returns a structured representation of the trace

func (*Trace[T]) Suspend added in v0.9.8 

func (t *Trace[T]) Suspend(reason string)

Suspend finalizes the trace as suspended: it halted mid-run to await an out-of-band signal (a human answer, a checkpoint wake) rather than completing or failing. Unlike complete's error path, the root span is closed with status OK and stamped with driftlessaf.disposition=suspended and driftlessaf.suspension.reason=<reason>, so eval graders (e.g. NoErrors) and dashboards treat a suspension as a non-error terminal state. Trace.Error stays nil.

Suspend does NOT record the trace; that remains the caller's responsibility (mirroring complete). It is idempotent with complete via the finalized guard: whichever finalizer runs first wins and the second is a no-op, so a suspended trace is never re-closed as an error and the root span is ended exactly once. Executors call this explicitly on the suspend path.

type TraceCallback 

type TraceCallback[T any] func(*Trace[T])

TraceCallback is a function that receives completed traces

type Tracer 

type Tracer[T any] interface {
	// NewTrace creates a new trace with the given prompt. opts customize
	// root-span attributes (agent name, per-invocation label callback, etc.).
	NewTrace(ctx context.Context, prompt string, opts ...StartTraceOption) *Trace[T]
	// RecordTrace records a completed trace
	RecordTrace(trace *Trace[T])
}

Tracer is the interface for creating and managing traces

func ByCode 

func ByCode[T any](callbacks ...TraceCallback[T]) Tracer[T]

ByCode creates a new Tracer for code-based evals that invokes the given callbacks when traces are recorded

func NewDefaultTracer 

func NewDefaultTracer[T any](_ context.Context) Tracer[T]

NewDefaultTracer creates a new default tracer that logs to clog. The trace is logged as a structured JSON document via MarshalJSON so that JSON log sinks (Cloud Logging, etc.) receive a parseable record.

We use trace.ctx (not the startup ctx) so that each log line carries the per-request context — including trace metadata, reconciler key, etc.

func TracerFromContext 

func TracerFromContext[T any](ctx context.Context) Tracer[T]

TracerFromContext returns the tracer from the context, or creates a default tracer

func WithCloudEventEmission added in v0.4.1 

func WithCloudEventEmission[T any](inner Tracer[T], client cloudevents.Client, source string, opts ...CEOption[T]) Tracer[T]

WithCloudEventEmission wraps inner so that each call to RecordTrace also emits the trace as a CloudEvent. The caller provides a pre-built cloudevents.Client (see NewBrokerClient) and a source identifier (e.g. the OCTO_IDENTITY of the reconciler). The CloudEvent type is always EventType.

Call Drain on the returned tracer (via type assertion) before process exit to flush in-flight events.

Source Files

View all Source files

Directories

Path Synopsis
Package payloadcrypt seals the sensitive free-text fields of an agent trace (prompt, completion, tool params/results, reasoning) before they leave the emitting process, so the CloudEvent — and the BigQuery row the recorder derives from it — carries ciphertext rather than plaintext.
 Package payloadcrypt seals the sensitive free-text fields of an agent trace (prompt, completion, tool params/results, reasoning) before they leave the emitting process, so the CloudEvent — and the BigQuery row the recorder derives from it — carries ciphertext rather than plaintext.
kmsseal
Package kmsseal provides the Cloud KMS-backed wrap for payloadcrypt.
 Package kmsseal provides the Cloud KMS-backed wrap for payloadcrypt.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.