Documentation
¶
Overview ¶
Package middleware provides the tracing and metrics of agent runs, which plug into an agent with agent.WithMiddleware, and model decorators that other middleware builds on. Tracing and metrics follow the OpenTelemetry GenAI semantic conventions, using the interfaces of the telemetry package, without depending on OpenTelemetry; the github.com/nostosx/agentx-contrib/otel module adapts them, and github.com/nostosx/agentx-contrib/middleware provides logging and retries.
For model calls outside the agent loop, such as the model of a actx.Summarize strategy, the package also provides plain decorators: TraceModel and MeterModel wrap a model.Model, and TraceTools and MeterTools are tool.Middleware.
Index ¶
- func MeterTools(m telemetry.Meter, opts ...MetricsOption) tool.Middleware
- func Metrics(m telemetry.Meter, opts ...MetricsOption) loop.Middleware
- func TraceTools(t telemetry.Tracer, opts ...TraceOption) tool.Middleware
- func Tracing(t telemetry.Tracer, opts ...TraceOption) loop.Middleware
- func Wrap(m model.Model, mws ...Model) model.Model
- type Funcs
- type MetricsOption
- type Model
- type TraceOption
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func MeterTools ¶
func MeterTools(m telemetry.Meter, opts ...MetricsOption) tool.Middleware
MeterTools records gen_ai.execute_tool.duration for each tool call.
func Metrics ¶
func Metrics(m telemetry.Meter, opts ...MetricsOption) loop.Middleware
Metrics returns runtime middleware that records the metrics described at telemetry.Meter for each run and its model and tool calls.
Register it after Tracing, so measurements carry the spans, and before retries, so a model call's duration includes them.
func TraceTools ¶
func TraceTools(t telemetry.Tracer, opts ...TraceOption) tool.Middleware
TraceTools records a span for each tool call.
func Tracing ¶
func Tracing(t telemetry.Tracer, opts ...TraceOption) loop.Middleware
Tracing returns runtime middleware that records an invoke_agent span for each run, with the spans of TraceModel and TraceTools as its children. The spans carry the agent name, the session ID as gen_ai.conversation.id, the run ID and the turn.
The run's context carries the tracer, so hooks and compaction add their spans with telemetry.StartSpan, and a sub-agent traced the same way nests under the execute_tool span that runs it. telemetry.Tracer lists the spans and their attributes.
Types ¶
type Funcs ¶
type Funcs struct {
Next model.Model
GenerateFunc func(ctx context.Context, req *model.Request) (*model.Response, error)
StreamFunc func(ctx context.Context, req *model.Request) iter.Seq2[*model.StreamEvent, error]
}
Funcs builds a model from functions. A nil function delegates to Next, and Describe reports Next's description.
func (*Funcs) Describe ¶
func (f *Funcs) Describe() model.Description
Describe implements model.Describer.
type MetricsOption ¶
type MetricsOption func(*metricsConfig)
MetricsOption configures metrics.
func LegacyMetrics ¶
func LegacyMetrics() MetricsOption
LegacyMetrics also records the model metrics that earlier releases of the conventions defined and the current ones replaced, for dashboards that still query them: the gen_ai.client.token.usage histogram, with gen_ai.token.type "input" or "output", and gen_ai.client.operation.duration.
type Model ¶
Model wraps a model.Model.
func MeterModel ¶
func MeterModel(m telemetry.Meter, opts ...MetricsOption) Model
MeterModel records the model metrics described at telemetry.Meter for each call.
func TraceModel ¶
func TraceModel(t telemetry.Tracer, opts ...TraceOption) Model
TraceModel records a span for each model call. The provider, operation and default model come from the model's model.Describer.
type TraceOption ¶
type TraceOption func(*traceConfig)
TraceOption configures tracing.
func RecordContent ¶
func RecordContent() TraceOption
RecordContent records the content of model and tool calls: the system prompt, messages and tool names of model calls and runs (gen_ai.system_instructions, gen_ai.input.messages, gen_ai.output.messages and gen_ai.tool.definitions, as JSON strings), and the arguments and results of tool calls (gen_ai.tool.call.arguments and gen_ai.tool.call.result). They may contain sensitive data and can be large, so the conventions make them opt-in.
Source Files