metrics

package
v0.9.21 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 21, 2026 License: Apache-2.0 Imports: 8 Imported by: 0

Documentation

Overview

Package metrics provides OpenTelemetry metrics instrumentation for generative AI operations.

This package simplifies the process of tracking token usage and tool calls across AI model interactions. It provides a unified metrics interface that works with any AI model provider (Claude, Gemini, OpenAI, etc.) using OpenTelemetry for observability.

Overview

The metrics package offers the following key features:

  • Token usage tracking (prompt and completion tokens)
  • Tool call counting with model and tool name dimensions
  • Automatic attribute enrichment from execution context (repository, turn, etc.)
  • Graceful degradation when metric creation fails
  • Thread-safe operations

Basic Usage

Create a GenAI metrics instance and record token usage: 

// Create metrics instance with a unified meter name
m := metrics.NewGenAI("chainguard.ai.agents")

// Record token usage for a model interaction
m.RecordTokens(ctx, "claude-3-sonnet", 150, 250)

// Record a tool call
m.RecordToolCall(ctx, "claude-3-sonnet", "read_file")

Attribute Enrichment

Contextual attributes (reconciler_type, repository, turn, etc.) are automatically extracted from the execution context propagated via context.Context using agenttrace.GetExecutionContext. No manual enricher setup is needed.

Graceful Degradation

The package handles metric creation failures gracefully. If a counter cannot be created, a warning is logged and a no-op counter is used instead: 

// Even if OpenTelemetry is not configured, this won't panic
m := metrics.NewGenAI("chainguard.ai.agents")
m.RecordTokens(ctx, "claude-3-sonnet", 100, 200) // Safe to call

Thread Safety

All functions and methods in this package are thread-safe. The GenAI struct can be safely shared across goroutines for concurrent metric recording.

Integration with AI Executors

This package is designed to work with various AI executor implementations:

Claude executor example: 

m := metrics.NewGenAI("chainguard.ai.agents")

// After each Claude API call
m.RecordTokens(ctx, "claude-3-sonnet",
	response.Usage.InputTokens,
	response.Usage.OutputTokens)

for _, toolUse := range response.Content {
	if toolUse.Type == "tool_use" {
		m.RecordToolCall(ctx, "claude-3-sonnet", toolUse.Name)
	}
}

Google Gemini executor example: 

m := metrics.NewGenAI("chainguard.ai.agents")

// After each Gemini API call
m.RecordTokens(ctx, "gemini-pro",
	response.UsageMetadata.PromptTokenCount,
	response.UsageMetadata.CandidatesTokenCount)

Metric Names

The package emits the following OpenTelemetry metrics:

Custom metrics:

  • genai.token.prompt: Counter of prompt tokens used (unit: {tokens})
  • genai.token.completion: Counter of completion tokens used (unit: {tokens})
  • genai.tool.calls: Counter of tool invocations (unit: {calls})

OpenTelemetry GenAI semantic convention metric:

  • gen_ai.client.token.usage: Histogram of token counts with gen_ai.token.type dimension (unit: {token})

All metrics include a "model" attribute. Tool call metrics also include a "tool" attribute. The semconv histogram also includes gen_ai.operation.name and gen_ai.request.model attributes. Callers should pass gen_ai.provider.name via the attrs parameter.

Custom Attributes

Additional attributes can be passed to recording methods: 

m.RecordTokens(ctx, "claude-3-sonnet", 100, 200,
	attribute.String("agent", "code-reviewer"),
	attribute.Int("turn", 3))

m.RecordToolCall(ctx, "claude-3-sonnet", "edit_file",
	attribute.String("file_type", "go"))

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type GenAI 

type GenAI struct {
	// contains filtered or unexported fields
}

GenAI provides OpenTelemetry metrics for generative AI operations. It includes counters for token usage (prompt and completion), tool calls, and prompt cache metrics, with support for graceful degradation if metric creation fails.

Metrics are emitted in both custom format (genai.token.prompt, genai.token.completion) and OpenTelemetry GenAI semantic conventions (gen_ai.client.token.usage histogram with gen_ai.token.type dimension). See: https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-metrics/

Example (MultipleModels)

ExampleGenAI_multipleModels demonstrates tracking metrics across different models. 

package main

import (
	"context"
	"fmt"

	"chainguard.dev/driftlessaf/agents/metrics"
)

func main() {
	ctx := context.Background()

	// Use a single metrics instance for all models
	// The model name serves as a dimension to differentiate between them
	m := metrics.NewGenAI("chainguard.ai.agents")

	// Track Claude usage
	m.RecordTokens(ctx, "claude-3-sonnet", 150, 250)
	m.RecordToolCall(ctx, "claude-3-sonnet", "read_file")

	// Track Gemini usage
	m.RecordTokens(ctx, "gemini-pro", 100, 300)
	m.RecordToolCall(ctx, "gemini-pro", "search_codebase")

	// Track GPT usage
	m.RecordTokens(ctx, "gpt-4", 200, 400)

	fmt.Println("Multi-model metrics recorded")

}

Output:
Multi-model metrics recorded

func NewGenAI 

func NewGenAI(meterName string) *GenAI

NewGenAI creates a new GenAI metrics instance with the specified meter name. Uses graceful degradation: if any metric counter fails to initialize, logs a warning and uses a no-op counter instead of failing entirely.

The meterName should be unified across all agent executors (e.g., "chainguard.ai.agents") with the model name serving as a dimension on the recorded metrics to differentiate between different models (Claude, Gemini, etc.).

Example

ExampleNewGenAI demonstrates creating a new GenAI metrics instance. 

package main

import (
	"fmt"

	"chainguard.dev/driftlessaf/agents/metrics"
)

func main() {
	// Create a metrics instance with a unified meter name
	// The meter name should be consistent across all agent executors
	m := metrics.NewGenAI("chainguard.ai.agents")

	// The metrics instance is ready to use
	fmt.Printf("Metrics instance created: %T\n", m)

}

Output:
Metrics instance created: *metrics.GenAI

func (*GenAI) RecordAPIRequest added in v0.7.0 

func (m *GenAI) RecordAPIRequest(ctx context.Context, model, responseCode string, attrs ...attribute.KeyValue)

RecordAPIRequest counts a single LLM API attempt. Each retry inside an in-process retry loop should call this once with the responseCode it observed — successes pass "200", rate-limit failures pass "429", overloaded failures pass "529", and errors that don't carry an HTTP status pass "unknown". This shape mirrors serviceruntime.googleapis.com/api/request_count's response_code label so PromQL on this metric reads the same as MQL on the GCP-side equivalent.

Example

ExampleGenAI_RecordAPIRequest demonstrates counting LLM API attempts by response code. 

package main

import (
	"context"
	"fmt"

	"chainguard.dev/driftlessaf/agents/metrics"
	"go.opentelemetry.io/otel/attribute"
)

func main() {
	ctx := context.Background()
	m := metrics.NewGenAI("chainguard.ai.agents")

	// Record a successful API call
	m.RecordAPIRequest(ctx, "gemini-2.5-flash", "200")

	// Record a rate-limited API call with provider attribution
	m.RecordAPIRequest(ctx, "gemini-2.5-flash", "429",
		attribute.String("gen_ai.provider.name", "gcp.vertex_ai"))

	// Record an error without an HTTP status
	m.RecordAPIRequest(ctx, "claude-sonnet-4", "unknown")

	fmt.Println("API request metrics recorded")

}

Output:
API request metrics recorded

func (*GenAI) RecordCacheTokens added in v0.2.0 

func (m *GenAI) RecordCacheTokens(ctx context.Context, model string, cacheRead, cacheCreation int64, attrs ...attribute.KeyValue)

RecordCacheTokens records Anthropic prompt cache token usage on the existing prompt tokens counter using gen_ai.token.type dimension values "cache_read" and "cache_creation". This aligns with the OpenTelemetry GenAI semantic conventions pattern from gen_ai.client.token.usage (see #35633).

cacheRead: tokens served from cache (cheap — 0.1x base input price). cacheCreation: tokens written to cache (1.25x base input price, amortized over reads).

A healthy caching setup shows high cache_read and low/zero cache_creation after the first turn.

func (*GenAI) RecordTokens 

func (m *GenAI) RecordTokens(ctx context.Context, model string, promptTokens, completionTokens int64, attrs ...attribute.KeyValue)

RecordTokens records prompt and completion token usage. Enriches attributes from the execution context propagated via context.Context.

Example

ExampleGenAI_RecordTokens demonstrates recording token usage. 

package main

import (
	"context"
	"fmt"

	"chainguard.dev/driftlessaf/agents/metrics"
	"go.opentelemetry.io/otel/attribute"
)

func main() {
	ctx := context.Background()
	m := metrics.NewGenAI("chainguard.ai.agents")

	// Record token usage from an AI model response
	// Parameters: context, model name, prompt tokens, completion tokens
	m.RecordTokens(ctx, "claude-3-sonnet", 150, 250)

	// Record with additional custom attributes
	m.RecordTokens(ctx, "gemini-pro", 100, 200,
		attribute.String("agent", "code-reviewer"),
		attribute.Int("turn", 1))

	fmt.Println("Token metrics recorded")

}

Output:
Token metrics recorded

func (*GenAI) RecordToolCall 

func (m *GenAI) RecordToolCall(ctx context.Context, model, toolName string, attrs ...attribute.KeyValue)

RecordToolCall records a tool invocation. Enriches attributes from the execution context propagated via context.Context.

Example

ExampleGenAI_RecordToolCall demonstrates recording tool invocations. 

package main

import (
	"context"
	"fmt"

	"chainguard.dev/driftlessaf/agents/metrics"
	"go.opentelemetry.io/otel/attribute"
)

func main() {
	ctx := context.Background()
	m := metrics.NewGenAI("chainguard.ai.agents")

	// Record a tool call with model and tool name
	m.RecordToolCall(ctx, "claude-3-sonnet", "read_file")

	// Record with additional custom attributes
	m.RecordToolCall(ctx, "claude-3-sonnet", "edit_file",
		attribute.String("file_type", "go"),
		attribute.String("operation", "modify"))

	fmt.Println("Tool call metrics recorded")

}

Output:
Tool call metrics recorded

func (*GenAI) RecordTurns added in v0.3.0 

func (m *GenAI) RecordTurns(ctx context.Context, model string, turns int, limitExceeded bool, attrs ...attribute.KeyValue)

RecordTurns records the number of LLM round-trips consumed by an agent execution. It always records the turns histogram and, when limitExceeded is true, also increments the turn_limit_exceeded counter. Call this on every exit path of the executor conversation loop so p50/p95/p99 distributions are observable per service.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.