Documentation
¶
Overview ¶
ABOUTME: Middleware that emits summaries of LLM call activity for live UI display. ABOUTME: Captures model name, tool calls, response text snippets, and errors.
ABOUTME: Model catalog providing a registry of known LLM models and their capabilities. ABOUTME: Supports lookup by ID/alias, listing by provider, and filtering by capability.
ABOUTME: Core LLM client that routes requests to provider adapters. ABOUTME: Supports multiple providers, default routing, middleware chains, and env-based config.
ABOUTME: Error type hierarchy for the unified LLM client library. ABOUTME: Defines provider errors, retryability, and HTTP status code mapping.
ABOUTME: Pretty-print formatting for Response and Usage types via fmt.Stringer. ABOUTME: Produces concise, human-readable output for CLI and TUI display.
ABOUTME: Middleware types for the unified LLM client's request/response pipeline. ABOUTME: Defines CompleteHandler, Middleware interface, and MiddlewareFunc adapter.
ABOUTME: Cost estimation for LLM usage based on the model catalog. ABOUTME: Prices input, output, and cache tokens using per-model rates from the catalog.
ABOUTME: Defines the ProviderAdapter interface for LLM provider implementations. ABOUTME: Each provider (OpenAI, Anthropic, Gemini, etc.) implements this interface.
ABOUTME: Retry middleware with exponential backoff for the LLM client. ABOUTME: Retries only errors that implement Retryable() bool returning true; respects context and RetryAfter.
ABOUTME: Streaming event types and accumulator for incremental LLM responses. ABOUTME: Defines StreamEvent, StreamEventType enum, and StreamAccumulator.
ABOUTME: Middleware that accumulates per-provider token usage across LLM calls. ABOUTME: Thread-safe; used by the TUI dashboard header for real-time token counts.
ABOUTME: Per-provider cost rollup for the TokenTracker middleware. ABOUTME: Maps accumulated Usage to dollar cost via a caller-supplied model resolver.
ABOUTME: Structured trace events for live LLM introspection across console and TUI surfaces.
ABOUTME: Console logger for structured LLM trace events. ABOUTME: Batches sequential text/reasoning deltas into a single summary line per stream.
ABOUTME: Transform middleware for modifying LLM requests and responses in the pipeline. ABOUTME: Supports request transforms (before call) and optional response transforms (after call).
ABOUTME: Core data types for the unified LLM client library. ABOUTME: Defines Message, ContentPart, Request, Response, Usage, and related types.
Index ¶
- func ErrorFromStatusCode(statusCode int, message, provider string) error
- func EstimateCost(model string, usage Usage) float64
- func FormatCoalescedLine(kind TraceKind, accumulated string) string
- func FormatModelHeader(provider, model string) string
- func FormatTraceLine(evt TraceEvent, verbose bool) string
- type AbortError
- type AccessDeniedError
- type ActivityCallback
- type ActivityEvent
- type ActivityTracker
- type AudioData
- type AuthenticationError
- type Client
- func (c *Client) AddMiddleware(mw Middleware)
- func (c *Client) AddTraceObserver(obs TraceObserver)
- func (c *Client) Close() error
- func (c *Client) Complete(ctx context.Context, req *Request) (*Response, error)
- func (c *Client) DefaultProvider() string
- func (c *Client) Stream(ctx context.Context, req *Request) <-chan StreamEvent
- type ClientOption
- type CompleteHandler
- type ConfigurationError
- type ContentFilterError
- type ContentKind
- type ContentPart
- type ContextLengthError
- type DocumentData
- type FinishReason
- type ImageData
- type InvalidRequestError
- type InvalidToolCallError
- type Message
- type Middleware
- type MiddlewareFunc
- type ModelInfo
- type ModelResolver
- type NetworkError
- type NoObjectGeneratedError
- type NotFoundError
- type ProviderAdapter
- type ProviderCost
- type ProviderError
- type ProviderErrorInterface
- type QuotaExceededError
- type RateLimitError
- type RateLimitInfo
- type Request
- type RequestTimeoutError
- type RequestTransformFunc
- type Response
- type ResponseFormat
- type ResponseTransformFunc
- type RetryMiddleware
- type RetryOption
- type Role
- type SDKError
- type ServerError
- type StreamAccumulator
- type StreamError
- type StreamEvent
- type StreamEventType
- type ThinkingData
- type TokenTracker
- func (t *TokenTracker) AddUsage(provider string, usage Usage, model ...string)
- func (t *TokenTracker) AllProviderUsage() map[string]Usage
- func (t *TokenTracker) CostByProvider(resolve ModelResolver) map[string]ProviderCost
- func (t *TokenTracker) ModelForProvider(provider string) string
- func (t *TokenTracker) ObservedModelResolver(fallback string) ModelResolver
- func (t *TokenTracker) ProviderUsage(provider string) Usage
- func (t *TokenTracker) Providers() []string
- func (t *TokenTracker) TotalCostUSD(resolve ModelResolver) float64
- func (t *TokenTracker) TotalUsage() Usage
- func (t *TokenTracker) WrapComplete(next CompleteHandler) CompleteHandler
- type ToolCallData
- type ToolChoice
- type ToolDefinition
- type ToolResultData
- type TraceBuilder
- type TraceEvent
- type TraceKind
- type TraceLogger
- type TraceLoggerOptions
- type TraceObserver
- type TraceObserverFunc
- type TraceOptions
- type TransformMiddleware
- type TransformOption
- type Usage
- type Warning
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ErrorFromStatusCode ¶
ErrorFromStatusCode maps an HTTP status code to the appropriate error type.
func EstimateCost ¶
EstimateCost returns the estimated dollar cost for the given model and token usage. Looks up pricing from the model catalog (supports ID and aliases). Returns 0 for unknown models, logging a single warning per unknown name so operators notice that their --max-cost ceiling will not apply to usage priced at an unknown rate. Cache read tokens are priced at 10% of input rate; cache write tokens at 25% of input rate (Anthropic pricing convention, applied uniformly since other providers use similar discounts).
func FormatCoalescedLine ¶
FormatCoalescedLine formats an accumulated text or reasoning block for the TUI log. Shows clean content without the `preview="..."` wrapping, for a chat-like display. The provider/model is shown in a separate header line via FormatModelHeader.
func FormatModelHeader ¶
FormatModelHeader returns a display string like "anthropic/claude-opus-4-6" for use as a section header in the activity log.
func FormatTraceLine ¶
func FormatTraceLine(evt TraceEvent, verbose bool) string
FormatTraceLine formats one trace event for console or TUI rendering.
Types ¶
type AbortError ¶
type AbortError struct {
SDKError
}
AbortError indicates the operation was explicitly aborted.
type AccessDeniedError ¶
type AccessDeniedError struct {
ProviderError
}
AccessDeniedError indicates insufficient permissions.
func (*AccessDeniedError) Retryable ¶
func (e *AccessDeniedError) Retryable() bool
type ActivityCallback ¶
type ActivityCallback func(ActivityEvent)
ActivityCallback is called for each LLM completion with an ActivityEvent summary.
type ActivityEvent ¶
type ActivityEvent struct {
Timestamp time.Time
Model string
Provider string
// Request info
ToolCount int
MessageCount int
// Response info (empty on error)
ResponseSnippet string
ToolCalls []string // names of tools called by the model
InputTokens int
OutputTokens int
Latency time.Duration
// Error info
Err error
}
ActivityEvent describes a single LLM call for display in a live activity feed.
func (ActivityEvent) Summary ¶
func (e ActivityEvent) Summary() string
Summary returns a one-line description of the activity event.
type ActivityTracker ¶
type ActivityTracker struct {
// contains filtered or unexported fields
}
ActivityTracker is a middleware that observes LLM calls and emits activity summaries via a callback. It does not modify requests or responses.
func NewActivityTracker ¶
func NewActivityTracker(callback ActivityCallback) *ActivityTracker
NewActivityTracker creates an activity tracking middleware that calls the provided callback after each LLM completion.
func (*ActivityTracker) WrapComplete ¶
func (a *ActivityTracker) WrapComplete(next CompleteHandler) CompleteHandler
WrapComplete implements the Middleware interface.
type AudioData ¶
type AudioData struct {
URL string `json:"url,omitempty"`
Data []byte `json:"data,omitempty"`
MediaType string `json:"media_type,omitempty"`
}
AudioData holds audio content.
type AuthenticationError ¶
type AuthenticationError struct {
ProviderError
}
AuthenticationError indicates invalid or missing credentials.
func (*AuthenticationError) Retryable ¶
func (e *AuthenticationError) Retryable() bool
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client manages LLM provider adapters and routes requests through a middleware chain.
func NewClient ¶
func NewClient(opts ...ClientOption) (*Client, error)
NewClient creates a Client from the given options.
func NewClientFromEnv ¶
func (*Client) AddMiddleware ¶
func (c *Client) AddMiddleware(mw Middleware)
AddMiddleware appends a middleware to the client's existing middleware chain. This allows post-construction addition of middleware (e.g., adding a TokenTracker after the client is built from environment variables).
func (*Client) AddTraceObserver ¶
func (c *Client) AddTraceObserver(obs TraceObserver)
AddTraceObserver appends a live trace observer after client construction.
func (*Client) Complete ¶
Complete sends a completion request through the middleware chain and returns the response.
func (*Client) DefaultProvider ¶
DefaultProvider returns the name of the default provider, or empty string.
func (*Client) Stream ¶
func (c *Client) Stream(ctx context.Context, req *Request) <-chan StreamEvent
Stream sends a streaming request to the resolved provider adapter. Middleware is NOT applied to streaming requests (middleware only wraps Complete). On provider resolution failure, it returns a channel containing a single error event.
type ClientOption ¶
type ClientOption func(*clientConfig)
ClientOption configures a Client during construction.
func WithDefaultProvider ¶
func WithDefaultProvider(name string) ClientOption
WithDefaultProvider sets the provider name used when a request does not specify one.
func WithMiddleware ¶
func WithMiddleware(mw Middleware) ClientOption
WithMiddleware appends a middleware to the client's middleware chain.
func WithProvider ¶
func WithProvider(adapter ProviderAdapter) ClientOption
WithProvider registers a provider adapter with the client.
func WithTraceObserver ¶
func WithTraceObserver(obs TraceObserver) ClientOption
WithTraceObserver registers a live trace observer for completions.
type CompleteHandler ¶
CompleteHandler is a function that processes a completion request and returns a response. It is used as the unit of composition in the middleware chain.
type ConfigurationError ¶
type ConfigurationError struct {
SDKError
}
ConfigurationError indicates a problem with the client configuration.
type ContentFilterError ¶
type ContentFilterError struct {
ProviderError
}
ContentFilterError indicates the request or response was blocked by a content filter.
func (*ContentFilterError) Retryable ¶
func (e *ContentFilterError) Retryable() bool
type ContentKind ¶
type ContentKind string
ContentKind discriminates ContentPart variants.
const ( KindText ContentKind = "text" KindImage ContentKind = "image" KindAudio ContentKind = "audio" KindDocument ContentKind = "document" KindToolCall ContentKind = "tool_call" KindToolResult ContentKind = "tool_result" KindThinking ContentKind = "thinking" KindRedactedThinking ContentKind = "redacted_thinking" )
type ContentPart ¶
type ContentPart struct {
Kind ContentKind `json:"kind"`
Text string `json:"text,omitempty"`
Image *ImageData `json:"image,omitempty"`
Audio *AudioData `json:"audio,omitempty"`
Document *DocumentData `json:"document,omitempty"`
ToolCall *ToolCallData `json:"tool_call,omitempty"`
ToolResult *ToolResultData `json:"tool_result,omitempty"`
Thinking *ThinkingData `json:"thinking,omitempty"`
}
ContentPart is a tagged union representing one piece of message content.
type ContextLengthError ¶
type ContextLengthError struct {
ProviderError
}
ContextLengthError indicates the request exceeded the model's context window.
func (*ContextLengthError) Retryable ¶
func (e *ContextLengthError) Retryable() bool
type DocumentData ¶
type DocumentData struct {
URL string `json:"url,omitempty"`
Data []byte `json:"data,omitempty"`
MediaType string `json:"media_type,omitempty"`
FileName string `json:"file_name,omitempty"`
}
DocumentData holds document content.
type FinishReason ¶
FinishReason indicates why generation stopped.
type ImageData ¶
type ImageData struct {
URL string `json:"url,omitempty"`
Data []byte `json:"data,omitempty"`
MediaType string `json:"media_type,omitempty"`
Detail string `json:"detail,omitempty"`
}
ImageData holds image content as URL or raw bytes.
type InvalidRequestError ¶
type InvalidRequestError struct {
ProviderError
}
InvalidRequestError indicates a malformed or invalid request.
func (*InvalidRequestError) Retryable ¶
func (e *InvalidRequestError) Retryable() bool
type InvalidToolCallError ¶
type InvalidToolCallError struct {
SDKError
}
InvalidToolCallError indicates the model produced an invalid tool call.
type Message ¶
type Message struct {
Role Role `json:"role"`
Content []ContentPart `json:"content"`
Name string `json:"name,omitempty"`
ToolCallID string `json:"tool_call_id,omitempty"`
}
Message is the fundamental unit of conversation.
func AssistantMessage ¶
AssistantMessage creates a message with the assistant role.
func SystemMessage ¶
SystemMessage creates a message with the system role.
func ToolResultMessage ¶
ToolResultMessage creates a tool result message.
func UserMessage ¶
UserMessage creates a message with the user role.
func (Message) ToolCalls ¶
func (m Message) ToolCalls() []ToolCallData
ToolCalls extracts all tool call content parts from the message.
type Middleware ¶
type Middleware interface {
WrapComplete(next CompleteHandler) CompleteHandler
}
Middleware wraps a CompleteHandler to add cross-cutting behavior (logging, retry, etc.).
func NewRetryMiddleware ¶
func NewRetryMiddleware(opts ...RetryOption) Middleware
NewRetryMiddleware creates a retry Middleware with the given options.
type MiddlewareFunc ¶
type MiddlewareFunc func(next CompleteHandler) CompleteHandler
MiddlewareFunc is an adapter that allows ordinary functions to be used as Middleware.
func (MiddlewareFunc) WrapComplete ¶
func (f MiddlewareFunc) WrapComplete(next CompleteHandler) CompleteHandler
WrapComplete implements the Middleware interface for MiddlewareFunc.
type ModelInfo ¶
type ModelInfo struct {
ID string `json:"id"`
Provider string `json:"provider"`
DisplayName string `json:"display_name"`
ContextWindow int `json:"context_window"`
MaxOutput int `json:"max_output"`
SupportsTools bool `json:"supports_tools"`
SupportsVision bool `json:"supports_vision"`
SupportsReasoning bool `json:"supports_reasoning"`
InputCostPerM float64 `json:"input_cost_per_m"`
OutputCostPerM float64 `json:"output_cost_per_m"`
Aliases []string `json:"aliases,omitempty"`
}
ModelInfo describes a known LLM model and its capabilities.
func GetLatestModel ¶
GetLatestModel returns the first model matching the given provider and optional capability filter. Supported capability values are "reasoning", "vision", and "tools". Pass an empty string for no capability filter. Returns nil if no model matches.
func GetModelInfo ¶
GetModelInfo looks up a model by ID or alias. Returns nil if not found.
func ListModels ¶
ListModels returns all known models, optionally filtered by provider. Pass an empty string to return all models.
type ModelResolver ¶
ModelResolver returns the model name that should be used for cost estimation for a given provider. Return "" when unknown — the entry is still included in the result with USD=0.
type NetworkError ¶
type NetworkError struct {
SDKError
}
NetworkError indicates a network-level failure (DNS, TCP, TLS).
func (*NetworkError) Retryable ¶
func (e *NetworkError) Retryable() bool
type NoObjectGeneratedError ¶
type NoObjectGeneratedError struct {
SDKError
}
NoObjectGeneratedError indicates structured output generation failed.
type NotFoundError ¶
type NotFoundError struct {
ProviderError
}
NotFoundError indicates the requested resource does not exist.
func (*NotFoundError) Retryable ¶
func (e *NotFoundError) Retryable() bool
type ProviderAdapter ¶
type ProviderAdapter interface {
// Name returns the provider's identifier (e.g. "openai", "anthropic").
Name() string
// Complete sends a request and returns the full response.
Complete(ctx context.Context, req *Request) (*Response, error)
// Stream sends a request and returns a channel of streaming events.
Stream(ctx context.Context, req *Request) <-chan StreamEvent
// Close releases any resources held by the adapter.
Close() error
}
ProviderAdapter is the interface that LLM provider implementations must satisfy. It supports both synchronous completion and streaming responses.
type ProviderCost ¶
ProviderCost is the per-provider cost rollup returned by TokenTracker.CostByProvider.
type ProviderError ¶
type ProviderError struct {
SDKError
Provider string
StatusCode int
ErrorCode string
RetryAfter *float64
RawBody json.RawMessage
}
ProviderError is the base type for errors returned by LLM providers.
func (*ProviderError) GetProvider ¶
func (e *ProviderError) GetProvider() string
func (*ProviderError) GetStatusCode ¶
func (e *ProviderError) GetStatusCode() int
type ProviderErrorInterface ¶
type ProviderErrorInterface interface {
error
Retryable() bool
GetProvider() string
GetStatusCode() int
}
ProviderErrorInterface describes errors originating from an LLM provider.
type QuotaExceededError ¶
type QuotaExceededError struct {
ProviderError
}
QuotaExceededError indicates the account's quota has been exhausted.
func (*QuotaExceededError) Retryable ¶
func (e *QuotaExceededError) Retryable() bool
type RateLimitError ¶
type RateLimitError struct {
ProviderError
}
RateLimitError indicates the request was rate limited by the provider.
func (*RateLimitError) Retryable ¶
func (e *RateLimitError) Retryable() bool
type RateLimitInfo ¶
type RateLimitInfo struct {
RequestsRemaining *int `json:"requests_remaining,omitempty"`
RequestsLimit *int `json:"requests_limit,omitempty"`
TokensRemaining *int `json:"tokens_remaining,omitempty"`
TokensLimit *int `json:"tokens_limit,omitempty"`
ResetAt *time.Time `json:"reset_at,omitempty"`
}
RateLimitInfo holds rate limit metadata from provider headers.
type Request ¶
type Request struct {
Model string `json:"model"`
Messages []Message `json:"messages"`
Provider string `json:"provider,omitempty"`
Tools []ToolDefinition `json:"tools,omitempty"`
ToolChoice *ToolChoice `json:"tool_choice,omitempty"`
ResponseFormat *ResponseFormat `json:"response_format,omitempty"`
Temperature *float64 `json:"temperature,omitempty"`
TopP *float64 `json:"top_p,omitempty"`
MaxTokens *int `json:"max_tokens,omitempty"`
StopSequences []string `json:"stop_sequences,omitempty"`
ReasoningEffort string `json:"reasoning_effort,omitempty"`
Metadata map[string]string `json:"metadata,omitempty"`
ProviderOptions map[string]any `json:"provider_options,omitempty"`
TraceObservers []TraceObserver `json:"-"`
}
Request is the input for Complete() and Stream().
type RequestTimeoutError ¶
type RequestTimeoutError struct {
ProviderError
}
RequestTimeoutError indicates the provider timed out processing the request.
func (*RequestTimeoutError) Retryable ¶
func (e *RequestTimeoutError) Retryable() bool
type RequestTransformFunc ¶
type RequestTransformFunc func(req *Request)
RequestTransformFunc modifies a Request in place before it is sent downstream.
type Response ¶
type Response struct {
ID string `json:"id"`
Model string `json:"model"`
Provider string `json:"provider"`
Message Message `json:"message"`
FinishReason FinishReason `json:"finish_reason"`
Usage Usage `json:"usage"`
Latency time.Duration `json:"latency"`
Raw json.RawMessage `json:"raw,omitempty"`
Warnings []Warning `json:"warnings,omitempty"`
RateLimit *RateLimitInfo `json:"rate_limit,omitempty"`
}
Response is the output of Complete().
func (Response) ToolCalls ¶
func (r Response) ToolCalls() []ToolCallData
ToolCalls returns tool calls from the response message.
type ResponseFormat ¶
type ResponseFormat struct {
Type string `json:"type"`
JSONSchema json.RawMessage `json:"json_schema,omitempty"`
Strict bool `json:"strict,omitempty"`
}
ResponseFormat specifies output format constraints.
type ResponseTransformFunc ¶
type ResponseTransformFunc func(resp *Response)
ResponseTransformFunc modifies a Response in place after it is received from downstream.
type RetryMiddleware ¶
type RetryMiddleware struct {
// contains filtered or unexported fields
}
RetryMiddleware retries failed requests with exponential backoff when the error is retryable. Non-retryable and unknown errors are returned immediately.
func (*RetryMiddleware) WrapComplete ¶
func (rm *RetryMiddleware) WrapComplete(next CompleteHandler) CompleteHandler
WrapComplete implements the Middleware interface.
type RetryOption ¶
type RetryOption func(*RetryMiddleware)
RetryOption configures a RetryMiddleware.
func WithBaseDelay ¶
func WithBaseDelay(d time.Duration) RetryOption
WithBaseDelay sets the base delay for exponential backoff (default 1s). Actual delay is baseDelay * 2^attempt.
func WithMaxRetries ¶
func WithMaxRetries(n int) RetryOption
WithMaxRetries sets the maximum number of retry attempts (default 3).
type ServerError ¶
type ServerError struct {
ProviderError
}
ServerError indicates a server-side failure from the provider.
func (*ServerError) Retryable ¶
func (e *ServerError) Retryable() bool
type StreamAccumulator ¶
type StreamAccumulator struct {
// contains filtered or unexported fields
}
StreamAccumulator collects streaming events into a complete Response.
func NewStreamAccumulator ¶
func NewStreamAccumulator() *StreamAccumulator
NewStreamAccumulator creates a new StreamAccumulator ready to process events.
func (*StreamAccumulator) Process ¶
func (a *StreamAccumulator) Process(event StreamEvent)
Process handles a single StreamEvent, updating the accumulator state.
func (*StreamAccumulator) Response ¶
func (a *StreamAccumulator) Response() Response
Response builds a complete Response from the accumulated events.
type StreamError ¶
type StreamError struct {
SDKError
}
StreamError indicates a failure during response streaming.
func (*StreamError) Retryable ¶
func (e *StreamError) Retryable() bool
type StreamEvent ¶
type StreamEvent struct {
Type StreamEventType `json:"type"`
Delta string `json:"delta,omitempty"`
TextID string `json:"text_id,omitempty"`
ReasoningDelta string `json:"reasoning_delta,omitempty"`
ReasoningSignature string `json:"reasoning_signature,omitempty"`
ToolCall *ToolCallData `json:"tool_call,omitempty"`
FinishReason *FinishReason `json:"finish_reason,omitempty"`
Usage *Usage `json:"usage,omitempty"`
FullResponse *Response `json:"full_response,omitempty"`
Err error `json:"-"`
Raw json.RawMessage `json:"raw,omitempty"`
}
StreamEvent represents a single event in a streaming response.
type StreamEventType ¶
type StreamEventType string
StreamEventType discriminates the kind of streaming event.
const ( EventStreamStart StreamEventType = "stream_start" EventTextStart StreamEventType = "text_start" EventTextDelta StreamEventType = "text_delta" EventTextEnd StreamEventType = "text_end" EventReasoningStart StreamEventType = "reasoning_start" EventReasoningDelta StreamEventType = "reasoning_delta" EventReasoningSignature StreamEventType = "reasoning_signature" EventReasoningEnd StreamEventType = "reasoning_end" EventRedactedThinking StreamEventType = "redacted_thinking" EventToolCallStart StreamEventType = "tool_call_start" EventToolCallDelta StreamEventType = "tool_call_delta" EventToolCallEnd StreamEventType = "tool_call_end" EventFinish StreamEventType = "finish" EventError StreamEventType = "error" EventProviderEvent StreamEventType = "provider_event" )
type ThinkingData ¶
type ThinkingData struct {
Text string `json:"text"`
Signature string `json:"signature,omitempty"`
Redacted bool `json:"redacted,omitempty"`
}
ThinkingData holds model reasoning content.
type TokenTracker ¶
type TokenTracker struct {
// contains filtered or unexported fields
}
TokenTracker is a middleware that accumulates token usage per provider. It implements Middleware and can be passed to NewClient via WithMiddleware.
func NewTokenTracker ¶
func NewTokenTracker() *TokenTracker
NewTokenTracker creates a new, zeroed token tracking middleware.
func (*TokenTracker) AddUsage ¶
func (t *TokenTracker) AddUsage(provider string, usage Usage, model ...string)
AddUsage manually adds token usage for a provider. Used by backends that bypass the LLM client middleware (e.g., claude-code subprocess backend). The model parameter is optional; pass "" to leave the provider's model unchanged. Model strings are normalized through the catalog so versioned provider-returned IDs resolve to canonical pricing entries — matching WrapComplete's behavior.
func (*TokenTracker) AllProviderUsage ¶
func (t *TokenTracker) AllProviderUsage() map[string]Usage
AllProviderUsage returns a copy of the accumulated usage for all providers.
func (*TokenTracker) CostByProvider ¶
func (t *TokenTracker) CostByProvider(resolve ModelResolver) map[string]ProviderCost
CostByProvider returns a per-provider cost rollup, resolving each provider's model via the caller-supplied resolver and pricing it via EstimateCost. A nil resolver is treated as one that always returns "".
func (*TokenTracker) ModelForProvider ¶
func (t *TokenTracker) ModelForProvider(provider string) string
ModelForProvider returns the last-seen model for a provider. Returns "" if unknown.
func (*TokenTracker) ObservedModelResolver ¶
func (t *TokenTracker) ObservedModelResolver(fallback string) ModelResolver
ObservedModelResolver returns a ModelResolver that uses the tracker's observed per-provider models. Falls back to the provided fallback model for providers where no model was observed.
func (*TokenTracker) ProviderUsage ¶
func (t *TokenTracker) ProviderUsage(provider string) Usage
ProviderUsage returns the accumulated usage for a specific provider. Returns a zero Usage if the provider has not been seen.
func (*TokenTracker) Providers ¶
func (t *TokenTracker) Providers() []string
Providers returns a sorted slice of provider names that have recorded usage.
func (*TokenTracker) TotalCostUSD ¶
func (t *TokenTracker) TotalCostUSD(resolve ModelResolver) float64
TotalCostUSD sums CostByProvider to a single dollar figure using the same resolver.
func (*TokenTracker) TotalUsage ¶
func (t *TokenTracker) TotalUsage() Usage
TotalUsage returns accumulated usage summed across all providers.
func (*TokenTracker) WrapComplete ¶
func (t *TokenTracker) WrapComplete(next CompleteHandler) CompleteHandler
WrapComplete implements the Middleware interface. It calls the next handler and, on success, adds the response's token usage to the per-provider accumulator.
type ToolCallData ¶
type ToolCallData struct {
ID string `json:"id"`
Name string `json:"name"`
Arguments json.RawMessage `json:"arguments"`
ThoughtSigData string `json:"thought_signature,omitempty"`
}
ToolCallData represents a model-initiated tool invocation.
type ToolChoice ¶
ToolChoice controls how the model uses tools.
func ToolChoiceAuto ¶
func ToolChoiceAuto() ToolChoice
ToolChoiceAuto returns a ToolChoice that lets the model decide whether to call tools.
func ToolChoiceNamed ¶
func ToolChoiceNamed(name string) ToolChoice
ToolChoiceNamed creates a ToolChoice that forces the model to call a specific tool.
func ToolChoiceNone ¶
func ToolChoiceNone() ToolChoice
ToolChoiceNone returns a ToolChoice that prevents the model from calling tools.
func ToolChoiceRequired ¶
func ToolChoiceRequired() ToolChoice
ToolChoiceRequired returns a ToolChoice that forces the model to call a tool.
type ToolDefinition ¶
type ToolDefinition struct {
Name string `json:"name"`
Description string `json:"description"`
Parameters json.RawMessage `json:"parameters"`
}
ToolDefinition defines a tool the model can call.
type ToolResultData ¶
type ToolResultData struct {
ToolCallID string `json:"tool_call_id"`
Name string `json:"name,omitempty"`
Content string `json:"content"`
IsError bool `json:"is_error"`
ImageData []byte `json:"image_data,omitempty"`
ImageMediaType string `json:"image_media_type,omitempty"`
}
ToolResultData represents the result of executing a tool call.
type TraceBuilder ¶
type TraceBuilder struct {
// contains filtered or unexported fields
}
TraceBuilder converts streaming events into normalized trace events.
func NewTraceBuilder ¶
func NewTraceBuilder(opts TraceOptions) *TraceBuilder
NewTraceBuilder creates a trace builder for one request.
func (*TraceBuilder) Events ¶
func (b *TraceBuilder) Events() []TraceEvent
Events returns the trace events emitted so far.
func (*TraceBuilder) Process ¶
func (b *TraceBuilder) Process(evt StreamEvent)
Process ingests one stream event and emits any corresponding trace events.
type TraceEvent ¶
type TraceEvent struct {
Kind TraceKind
Provider string
Model string
ToolName string
Preview string
ProviderEvent string
RawPreview string
FinishReason string
Usage Usage
// SessionOwned is true when the originating request carried
// request-level TraceObservers — in tracker only the agent session
// registers those, and it re-emits every trace event as an agent
// llm_* event. Activity-log writers listening at the client level
// skip SessionOwned events to avoid logging the same stream twice
// (issue #354); non-session calls (e.g. the autopilot interviewer)
// stay SessionOwned=false so the trace path remains their only,
// and therefore kept, log surface.
SessionOwned bool
}
TraceEvent is a normalized event for rendering live LLM activity.
type TraceLogger ¶
type TraceLogger struct {
// contains filtered or unexported fields
}
TraceLogger writes trace events to an io.Writer. Sequential text and reasoning deltas are accumulated and flushed as a single summary line when the stream finishes or a different event kind arrives.
func NewTraceLogger ¶
func NewTraceLogger(w io.Writer, opts TraceLoggerOptions) *TraceLogger
NewTraceLogger creates a console trace logger.
func (*TraceLogger) Flush ¶
func (l *TraceLogger) Flush()
Flush forces any pending batched output to be written. Callers should invoke this when a stream terminates without a normal finish event (e.g. on error) to avoid losing buffered trace output.
func (*TraceLogger) HandleTraceEvent ¶
func (l *TraceLogger) HandleTraceEvent(evt TraceEvent)
HandleTraceEvent implements TraceObserver.
type TraceLoggerOptions ¶
type TraceLoggerOptions struct {
Verbose bool
}
TraceLoggerOptions configure console trace logging.
type TraceObserver ¶
type TraceObserver interface {
HandleTraceEvent(evt TraceEvent)
}
TraceObserver receives normalized LLM trace events.
type TraceObserverFunc ¶
type TraceObserverFunc func(evt TraceEvent)
TraceObserverFunc adapts a function into a TraceObserver.
func (TraceObserverFunc) HandleTraceEvent ¶
func (f TraceObserverFunc) HandleTraceEvent(evt TraceEvent)
HandleTraceEvent implements TraceObserver.
type TraceOptions ¶
TraceOptions configure trace building behavior.
type TransformMiddleware ¶
type TransformMiddleware struct {
// contains filtered or unexported fields
}
TransformMiddleware applies request and/or response transforms around a CompleteHandler.
func NewTransformMiddleware ¶
func NewTransformMiddleware(reqFn RequestTransformFunc, opts ...TransformOption) *TransformMiddleware
NewTransformMiddleware creates a TransformMiddleware with the given request transform and optional TransformOptions. If reqFn is nil, the request is passed through unchanged.
func (*TransformMiddleware) WrapComplete ¶
func (tm *TransformMiddleware) WrapComplete(next CompleteHandler) CompleteHandler
WrapComplete implements the Middleware interface. It applies the request transform before calling next, and the response transform after a successful call.
type TransformOption ¶
type TransformOption func(*TransformMiddleware)
TransformOption configures a TransformMiddleware.
func WithResponseTransform ¶
func WithResponseTransform(fn ResponseTransformFunc) TransformOption
WithResponseTransform adds a response transform that runs after a successful downstream call.
type Usage ¶
type Usage struct {
InputTokens int `json:"input_tokens"`
OutputTokens int `json:"output_tokens"`
TotalTokens int `json:"total_tokens"`
ReasoningTokens *int `json:"reasoning_tokens,omitempty"`
CacheReadTokens *int `json:"cache_read_tokens,omitempty"`
CacheWriteTokens *int `json:"cache_write_tokens,omitempty"`
EstimatedCost float64 `json:"estimated_cost,omitempty"`
Raw any `json:"raw,omitempty"`
}
Usage tracks token consumption.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
ABOUTME: Anthropic Messages API adapter implementing the ProviderAdapter interface.
|
ABOUTME: Anthropic Messages API adapter implementing the ProviderAdapter interface. |
|
ABOUTME: Google Gemini API adapter implementing the ProviderAdapter interface.
|
ABOUTME: Google Gemini API adapter implementing the ProviderAdapter interface. |
|
ABOUTME: OpenAI Responses API adapter implementing the ProviderAdapter interface.
|
ABOUTME: OpenAI Responses API adapter implementing the ProviderAdapter interface. |
|
ABOUTME: OpenAI Chat Completions compatible adapter implementing the ProviderAdapter interface.
|
ABOUTME: OpenAI Chat Completions compatible adapter implementing the ProviderAdapter interface. |