Documentation
¶
Overview ¶
Package observed wraps an agent.ChatModel with the gridctl observability surface — OTel spans, pricing.CalculateBreakdown for USD cost, and metrics.Accumulator.RecordCost with a synthetic provider name (no MCP envelope spoofing). The wrapped model satisfies the same agent.ChatModel interface, so it is a drop-in replacement at the call site (sandbox bindings, Go skill direct calls, the playground service).
Why a wrapper rather than per-provider instrumentation: each provider (anthropic, openai, google, gateway) implements a single concern — translating a gridctl ChatRequest to/from a vendor wire format. The observability layer is orthogonal to that concern. Centralising it here means one place to maintain span attribute names, one place to keep cost recording synchronised with the metrics package shape, and one place to evolve the contract when (for example) per-client cost attribution lands.
Tracing: Every Generate / Stream call opens a span named `agent.llm.generate` or `agent.llm.stream` under the tracer `gridctl.agent.llm`. The span carries gen_ai.* attributes (model, provider, prompt_tokens, completion_tokens) so it slots into the existing observability stack alongside `mcp.routing` parents.
Pricing & metrics: After a non-error response, the wrapper computes per-component cost via pricing.CalculateBreakdown and records it via metrics.Accumulator.RecordCost using the configured ProviderName (e.g. "llm:anthropic"). Streaming responses bookkeep usage from the final ChatChunk; if the provider does not surface usage on the stream terminator, no cost is recorded — a known limitation rather than a silent miscount.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Accumulator ¶
type Accumulator interface {
RecordCost(serverName string, replicaID int, cost metrics.CostBreakdown)
}
Accumulator is the subset of *metrics.Accumulator the wrapper needs. Defined here so callers can inject a fake in tests without depending on the real Accumulator's atomic internals.
type Option ¶
type Option func(*Provider)
Option configures a Provider during construction.
func WithAccumulator ¶
func WithAccumulator(acc Accumulator) Option
WithAccumulator wires a metrics.Accumulator (or a test fake) for cost recording. Without one, the wrapper still emits spans but skips the RecordCost call. nil is a no-op so callers can pass a possibly- nil accumulator without branching.
func WithTracer ¶
WithTracer overrides the OTel tracer used to open spans. Defaults to otel.Tracer("gridctl.agent.llm"). Useful in tests that want a deterministic tracer attached to a specific provider; nil is a no-op.
type Provider ¶
type Provider struct {
// contains filtered or unexported fields
}
Provider wraps an underlying agent.ChatModel and adds OTel tracing, USD cost calculation, and metrics recording. The zero value is not usable; construct via New.
func New ¶
New wraps an inner agent.ChatModel with observability. providerName is the synthetic server name the wrapper passes to RecordCost (convention: "llm:anthropic", "llm:openai", "llm:google", or "llm:unknown" when the model prefix does not match a known family — callers typically derive it from the provider package they're wiring).
inner must be non-nil; an empty providerName is rejected so misconfigured callers fail at construction rather than recording cost under an empty server bucket.
func (*Provider) Generate ¶
func (p *Provider) Generate(ctx context.Context, req agent.ChatRequest) (agent.ChatResponse, error)
Generate dispatches the request through the inner provider, opens an `agent.llm.generate` span, and records cost via the configured accumulator. Errors are recorded on the span and propagated unchanged.
func (*Provider) Stream ¶
func (p *Provider) Stream(ctx context.Context, req agent.ChatRequest) (*agent.StreamReader[agent.ChatChunk], error)
Stream dispatches the streaming request through the inner provider and returns a wrapped StreamReader that emits the same chunks while accumulating usage. The wrapper records cost when the StreamReader is closed (or fully drained) using the final usage observed on the stream — providers that emit usage only on the terminal chunk are the common case and are handled correctly.
Source Files