llm

package
v0.39.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 12, 2026 License: MIT Imports: 11 Imported by: 2

Documentation

Overview

ABOUTME: Middleware that emits summaries of LLM call activity for live UI display. ABOUTME: Captures model name, tool calls, response text snippets, and errors.

ABOUTME: Model catalog providing a registry of known LLM models and their capabilities. ABOUTME: Supports lookup by ID/alias, listing by provider, and filtering by capability.

ABOUTME: Core LLM client that routes requests to provider adapters. ABOUTME: Supports multiple providers, default routing, middleware chains, and env-based config.

ABOUTME: Error type hierarchy for the unified LLM client library. ABOUTME: Defines provider errors, retryability, and HTTP status code mapping.

ABOUTME: Pretty-print formatting for Response and Usage types via fmt.Stringer. ABOUTME: Produces concise, human-readable output for CLI and TUI display.

ABOUTME: Middleware types for the unified LLM client's request/response pipeline. ABOUTME: Defines CompleteHandler, Middleware interface, and MiddlewareFunc adapter.

ABOUTME: Cost estimation for LLM usage based on the model catalog. ABOUTME: Prices input, output, and cache tokens using per-model rates from the catalog.

ABOUTME: Defines the ProviderAdapter interface for LLM provider implementations. ABOUTME: Each provider (OpenAI, Anthropic, Gemini, etc.) implements this interface.

ABOUTME: Retry middleware with exponential backoff for the LLM client. ABOUTME: Retries only errors that implement Retryable() bool returning true; respects context and RetryAfter.

ABOUTME: Streaming event types and accumulator for incremental LLM responses. ABOUTME: Defines StreamEvent, StreamEventType enum, and StreamAccumulator.

ABOUTME: Middleware that accumulates per-provider token usage across LLM calls. ABOUTME: Thread-safe; used by the TUI dashboard header for real-time token counts.

ABOUTME: Per-provider cost rollup for the TokenTracker middleware. ABOUTME: Maps accumulated Usage to dollar cost via a caller-supplied model resolver.

ABOUTME: Structured trace events for live LLM introspection across console and TUI surfaces.

ABOUTME: Console logger for structured LLM trace events. ABOUTME: Batches sequential text/reasoning deltas into a single summary line per stream.

ABOUTME: Transform middleware for modifying LLM requests and responses in the pipeline. ABOUTME: Supports request transforms (before call) and optional response transforms (after call).

ABOUTME: Core data types for the unified LLM client library. ABOUTME: Defines Message, ContentPart, Request, Response, Usage, and related types.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ErrorFromStatusCode

func ErrorFromStatusCode(statusCode int, message, provider string) error

ErrorFromStatusCode maps an HTTP status code to the appropriate error type.

func EstimateCost

func EstimateCost(model string, usage Usage) float64

EstimateCost returns the estimated dollar cost for the given model and token usage. Looks up pricing from the model catalog (supports ID and aliases). Returns 0 for unknown models, logging a single warning per unknown name so operators notice that their --max-cost ceiling will not apply to usage priced at an unknown rate. Cache read tokens are priced at 10% of input rate; cache write tokens at 25% of input rate (Anthropic pricing convention, applied uniformly since other providers use similar discounts).

func FormatCoalescedLine

func FormatCoalescedLine(kind TraceKind, accumulated string) string

FormatCoalescedLine formats an accumulated text or reasoning block for the TUI log. Shows clean content without the `preview="..."` wrapping, for a chat-like display. The provider/model is shown in a separate header line via FormatModelHeader.

func FormatModelHeader

func FormatModelHeader(provider, model string) string

FormatModelHeader returns a display string like "anthropic/claude-opus-4-6" for use as a section header in the activity log.

func FormatTraceLine

func FormatTraceLine(evt TraceEvent, verbose bool) string

FormatTraceLine formats one trace event for console or TUI rendering.

Types

type AbortError

type AbortError struct {
	SDKError
}

AbortError indicates the operation was explicitly aborted.

type AccessDeniedError

type AccessDeniedError struct {
	ProviderError
}

AccessDeniedError indicates insufficient permissions.

func (*AccessDeniedError) Retryable

func (e *AccessDeniedError) Retryable() bool

type ActivityCallback

type ActivityCallback func(ActivityEvent)

ActivityCallback is called for each LLM completion with an ActivityEvent summary.

type ActivityEvent

type ActivityEvent struct {
	Timestamp time.Time
	Model     string
	Provider  string
	// Request info
	ToolCount    int
	MessageCount int
	// Response info (empty on error)
	ResponseSnippet string
	ToolCalls       []string // names of tools called by the model
	InputTokens     int
	OutputTokens    int
	Latency         time.Duration
	// Error info
	Err error
}

ActivityEvent describes a single LLM call for display in a live activity feed.

func (ActivityEvent) Summary

func (e ActivityEvent) Summary() string

Summary returns a one-line description of the activity event.

type ActivityTracker

type ActivityTracker struct {
	// contains filtered or unexported fields
}

ActivityTracker is a middleware that observes LLM calls and emits activity summaries via a callback. It does not modify requests or responses.

func NewActivityTracker

func NewActivityTracker(callback ActivityCallback) *ActivityTracker

NewActivityTracker creates an activity tracking middleware that calls the provided callback after each LLM completion.

func (*ActivityTracker) WrapComplete

func (a *ActivityTracker) WrapComplete(next CompleteHandler) CompleteHandler

WrapComplete implements the Middleware interface.

type AudioData

type AudioData struct {
	URL       string `json:"url,omitempty"`
	Data      []byte `json:"data,omitempty"`
	MediaType string `json:"media_type,omitempty"`
}

AudioData holds audio content.

type AuthenticationError

type AuthenticationError struct {
	ProviderError
}

AuthenticationError indicates invalid or missing credentials.

func (*AuthenticationError) Retryable

func (e *AuthenticationError) Retryable() bool

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client manages LLM provider adapters and routes requests through a middleware chain.

func NewClient

func NewClient(opts ...ClientOption) (*Client, error)

NewClient creates a Client from the given options.

func NewClientFromEnv

func NewClientFromEnv(constructors map[string]func(apiKey string) (ProviderAdapter, error)) (*Client, error)

func (*Client) AddMiddleware

func (c *Client) AddMiddleware(mw Middleware)

AddMiddleware appends a middleware to the client's existing middleware chain. This allows post-construction addition of middleware (e.g., adding a TokenTracker after the client is built from environment variables).

func (*Client) AddTraceObserver

func (c *Client) AddTraceObserver(obs TraceObserver)

AddTraceObserver appends a live trace observer after client construction.

func (*Client) Close

func (c *Client) Close() error

Close releases resources for all registered provider adapters.

func (*Client) Complete

func (c *Client) Complete(ctx context.Context, req *Request) (*Response, error)

Complete sends a completion request through the middleware chain and returns the response.

func (*Client) DefaultProvider

func (c *Client) DefaultProvider() string

DefaultProvider returns the name of the default provider, or empty string.

func (*Client) Stream

func (c *Client) Stream(ctx context.Context, req *Request) <-chan StreamEvent

Stream sends a streaming request to the resolved provider adapter. Middleware is NOT applied to streaming requests (middleware only wraps Complete). On provider resolution failure, it returns a channel containing a single error event.

type ClientOption

type ClientOption func(*clientConfig)

ClientOption configures a Client during construction.

func WithDefaultProvider

func WithDefaultProvider(name string) ClientOption

WithDefaultProvider sets the provider name used when a request does not specify one.

func WithMiddleware

func WithMiddleware(mw Middleware) ClientOption

WithMiddleware appends a middleware to the client's middleware chain.

func WithProvider

func WithProvider(adapter ProviderAdapter) ClientOption

WithProvider registers a provider adapter with the client.

func WithTraceObserver

func WithTraceObserver(obs TraceObserver) ClientOption

WithTraceObserver registers a live trace observer for completions.

type CompleteHandler

type CompleteHandler func(ctx context.Context, req *Request) (*Response, error)

CompleteHandler is a function that processes a completion request and returns a response. It is used as the unit of composition in the middleware chain.

type ConfigurationError

type ConfigurationError struct {
	SDKError
}

ConfigurationError indicates a problem with the client configuration.

type ContentFilterError

type ContentFilterError struct {
	ProviderError
}

ContentFilterError indicates the request or response was blocked by a content filter.

func (*ContentFilterError) Retryable

func (e *ContentFilterError) Retryable() bool

type ContentKind

type ContentKind string

ContentKind discriminates ContentPart variants.

const (
	KindText             ContentKind = "text"
	KindImage            ContentKind = "image"
	KindAudio            ContentKind = "audio"
	KindDocument         ContentKind = "document"
	KindToolCall         ContentKind = "tool_call"
	KindToolResult       ContentKind = "tool_result"
	KindThinking         ContentKind = "thinking"
	KindRedactedThinking ContentKind = "redacted_thinking"
)

type ContentPart

type ContentPart struct {
	Kind       ContentKind     `json:"kind"`
	Text       string          `json:"text,omitempty"`
	Image      *ImageData      `json:"image,omitempty"`
	Audio      *AudioData      `json:"audio,omitempty"`
	Document   *DocumentData   `json:"document,omitempty"`
	ToolCall   *ToolCallData   `json:"tool_call,omitempty"`
	ToolResult *ToolResultData `json:"tool_result,omitempty"`
	Thinking   *ThinkingData   `json:"thinking,omitempty"`
}

ContentPart is a tagged union representing one piece of message content.

type ContextLengthError

type ContextLengthError struct {
	ProviderError
}

ContextLengthError indicates the request exceeded the model's context window.

func (*ContextLengthError) Retryable

func (e *ContextLengthError) Retryable() bool

type DocumentData

type DocumentData struct {
	URL       string `json:"url,omitempty"`
	Data      []byte `json:"data,omitempty"`
	MediaType string `json:"media_type,omitempty"`
	FileName  string `json:"file_name,omitempty"`
}

DocumentData holds document content.

type FinishReason

type FinishReason struct {
	Reason string `json:"reason"`
	Raw    string `json:"raw,omitempty"`
}

FinishReason indicates why generation stopped.

type ImageData

type ImageData struct {
	URL       string `json:"url,omitempty"`
	Data      []byte `json:"data,omitempty"`
	MediaType string `json:"media_type,omitempty"`
	Detail    string `json:"detail,omitempty"`
}

ImageData holds image content as URL or raw bytes.

type InvalidRequestError

type InvalidRequestError struct {
	ProviderError
}

InvalidRequestError indicates a malformed or invalid request.

func (*InvalidRequestError) Retryable

func (e *InvalidRequestError) Retryable() bool

type InvalidToolCallError

type InvalidToolCallError struct {
	SDKError
}

InvalidToolCallError indicates the model produced an invalid tool call.

type Message

type Message struct {
	Role       Role          `json:"role"`
	Content    []ContentPart `json:"content"`
	Name       string        `json:"name,omitempty"`
	ToolCallID string        `json:"tool_call_id,omitempty"`
}

Message is the fundamental unit of conversation.

func AssistantMessage

func AssistantMessage(text string) Message

AssistantMessage creates a message with the assistant role.

func SystemMessage

func SystemMessage(text string) Message

SystemMessage creates a message with the system role.

func ToolResultMessage

func ToolResultMessage(toolCallID, content string, isError bool) Message

ToolResultMessage creates a tool result message.

func UserMessage

func UserMessage(text string) Message

UserMessage creates a message with the user role.

func (Message) Text

func (m Message) Text() string

Text returns concatenated text from all text content parts.

func (Message) ToolCalls

func (m Message) ToolCalls() []ToolCallData

ToolCalls extracts all tool call content parts from the message.

type Middleware

type Middleware interface {
	WrapComplete(next CompleteHandler) CompleteHandler
}

Middleware wraps a CompleteHandler to add cross-cutting behavior (logging, retry, etc.).

func NewRetryMiddleware

func NewRetryMiddleware(opts ...RetryOption) Middleware

NewRetryMiddleware creates a retry Middleware with the given options.

type MiddlewareFunc

type MiddlewareFunc func(next CompleteHandler) CompleteHandler

MiddlewareFunc is an adapter that allows ordinary functions to be used as Middleware.

func (MiddlewareFunc) WrapComplete

func (f MiddlewareFunc) WrapComplete(next CompleteHandler) CompleteHandler

WrapComplete implements the Middleware interface for MiddlewareFunc.

type ModelInfo

type ModelInfo struct {
	ID                string   `json:"id"`
	Provider          string   `json:"provider"`
	DisplayName       string   `json:"display_name"`
	ContextWindow     int      `json:"context_window"`
	MaxOutput         int      `json:"max_output"`
	SupportsTools     bool     `json:"supports_tools"`
	SupportsVision    bool     `json:"supports_vision"`
	SupportsReasoning bool     `json:"supports_reasoning"`
	InputCostPerM     float64  `json:"input_cost_per_m"`
	OutputCostPerM    float64  `json:"output_cost_per_m"`
	Aliases           []string `json:"aliases,omitempty"`
}

ModelInfo describes a known LLM model and its capabilities.

func GetLatestModel

func GetLatestModel(provider string, capability string) *ModelInfo

GetLatestModel returns the first model matching the given provider and optional capability filter. Supported capability values are "reasoning", "vision", and "tools". Pass an empty string for no capability filter. Returns nil if no model matches.

func GetModelInfo

func GetModelInfo(modelID string) *ModelInfo

GetModelInfo looks up a model by ID or alias. Returns nil if not found.

func ListModels

func ListModels(provider string) []ModelInfo

ListModels returns all known models, optionally filtered by provider. Pass an empty string to return all models.

type ModelResolver

type ModelResolver func(provider string) string

ModelResolver returns the model name that should be used for cost estimation for a given provider. Return "" when unknown — the entry is still included in the result with USD=0.

type NetworkError

type NetworkError struct {
	SDKError
}

NetworkError indicates a network-level failure (DNS, TCP, TLS).

func (*NetworkError) Retryable

func (e *NetworkError) Retryable() bool

type NoObjectGeneratedError

type NoObjectGeneratedError struct {
	SDKError
}

NoObjectGeneratedError indicates structured output generation failed.

type NotFoundError

type NotFoundError struct {
	ProviderError
}

NotFoundError indicates the requested resource does not exist.

func (*NotFoundError) Retryable

func (e *NotFoundError) Retryable() bool

type ProviderAdapter

type ProviderAdapter interface {
	// Name returns the provider's identifier (e.g. "openai", "anthropic").
	Name() string

	// Complete sends a request and returns the full response.
	Complete(ctx context.Context, req *Request) (*Response, error)

	// Stream sends a request and returns a channel of streaming events.
	Stream(ctx context.Context, req *Request) <-chan StreamEvent

	// Close releases any resources held by the adapter.
	Close() error
}

ProviderAdapter is the interface that LLM provider implementations must satisfy. It supports both synchronous completion and streaming responses.

type ProviderCost

type ProviderCost struct {
	Usage Usage
	Model string
	USD   float64
}

ProviderCost is the per-provider cost rollup returned by TokenTracker.CostByProvider.

type ProviderError

type ProviderError struct {
	SDKError
	Provider   string
	StatusCode int
	ErrorCode  string
	RetryAfter *float64
	RawBody    json.RawMessage
}

ProviderError is the base type for errors returned by LLM providers.

func (*ProviderError) GetProvider

func (e *ProviderError) GetProvider() string

func (*ProviderError) GetStatusCode

func (e *ProviderError) GetStatusCode() int

type ProviderErrorInterface

type ProviderErrorInterface interface {
	error
	Retryable() bool
	GetProvider() string
	GetStatusCode() int
}

ProviderErrorInterface describes errors originating from an LLM provider.

type QuotaExceededError

type QuotaExceededError struct {
	ProviderError
}

QuotaExceededError indicates the account's quota has been exhausted.

func (*QuotaExceededError) Retryable

func (e *QuotaExceededError) Retryable() bool

type RateLimitError

type RateLimitError struct {
	ProviderError
}

RateLimitError indicates the request was rate limited by the provider.

func (*RateLimitError) Retryable

func (e *RateLimitError) Retryable() bool

type RateLimitInfo

type RateLimitInfo struct {
	RequestsRemaining *int       `json:"requests_remaining,omitempty"`
	RequestsLimit     *int       `json:"requests_limit,omitempty"`
	TokensRemaining   *int       `json:"tokens_remaining,omitempty"`
	TokensLimit       *int       `json:"tokens_limit,omitempty"`
	ResetAt           *time.Time `json:"reset_at,omitempty"`
}

RateLimitInfo holds rate limit metadata from provider headers.

type Request

type Request struct {
	Model           string            `json:"model"`
	Messages        []Message         `json:"messages"`
	Provider        string            `json:"provider,omitempty"`
	Tools           []ToolDefinition  `json:"tools,omitempty"`
	ToolChoice      *ToolChoice       `json:"tool_choice,omitempty"`
	ResponseFormat  *ResponseFormat   `json:"response_format,omitempty"`
	Temperature     *float64          `json:"temperature,omitempty"`
	TopP            *float64          `json:"top_p,omitempty"`
	MaxTokens       *int              `json:"max_tokens,omitempty"`
	StopSequences   []string          `json:"stop_sequences,omitempty"`
	ReasoningEffort string            `json:"reasoning_effort,omitempty"`
	Metadata        map[string]string `json:"metadata,omitempty"`
	ProviderOptions map[string]any    `json:"provider_options,omitempty"`
	TraceObservers  []TraceObserver   `json:"-"`
}

Request is the input for Complete() and Stream().

type RequestTimeoutError

type RequestTimeoutError struct {
	ProviderError
}

RequestTimeoutError indicates the provider timed out processing the request.

func (*RequestTimeoutError) Retryable

func (e *RequestTimeoutError) Retryable() bool

type RequestTransformFunc

type RequestTransformFunc func(req *Request)

RequestTransformFunc modifies a Request in place before it is sent downstream.

type Response

type Response struct {
	ID           string          `json:"id"`
	Model        string          `json:"model"`
	Provider     string          `json:"provider"`
	Message      Message         `json:"message"`
	FinishReason FinishReason    `json:"finish_reason"`
	Usage        Usage           `json:"usage"`
	Latency      time.Duration   `json:"latency"`
	Raw          json.RawMessage `json:"raw,omitempty"`
	Warnings     []Warning       `json:"warnings,omitempty"`
	RateLimit    *RateLimitInfo  `json:"rate_limit,omitempty"`
}

Response is the output of Complete().

func (Response) Reasoning

func (r Response) Reasoning() string

Reasoning returns concatenated reasoning/thinking text.

func (Response) String

func (r Response) String() string

String formats a Response as a concise, human-readable summary.

func (Response) Text

func (r Response) Text() string

Text returns concatenated text from the response message.

func (Response) ToolCalls

func (r Response) ToolCalls() []ToolCallData

ToolCalls returns tool calls from the response message.

type ResponseFormat

type ResponseFormat struct {
	Type       string          `json:"type"`
	JSONSchema json.RawMessage `json:"json_schema,omitempty"`
	Strict     bool            `json:"strict,omitempty"`
}

ResponseFormat specifies output format constraints.

type ResponseTransformFunc

type ResponseTransformFunc func(resp *Response)

ResponseTransformFunc modifies a Response in place after it is received from downstream.

type RetryMiddleware

type RetryMiddleware struct {
	// contains filtered or unexported fields
}

RetryMiddleware retries failed requests with exponential backoff when the error is retryable. Non-retryable and unknown errors are returned immediately.

func (*RetryMiddleware) WrapComplete

func (rm *RetryMiddleware) WrapComplete(next CompleteHandler) CompleteHandler

WrapComplete implements the Middleware interface.

type RetryOption

type RetryOption func(*RetryMiddleware)

RetryOption configures a RetryMiddleware.

func WithBaseDelay

func WithBaseDelay(d time.Duration) RetryOption

WithBaseDelay sets the base delay for exponential backoff (default 1s). Actual delay is baseDelay * 2^attempt.

func WithMaxRetries

func WithMaxRetries(n int) RetryOption

WithMaxRetries sets the maximum number of retry attempts (default 3).

type Role

type Role string

Role represents who produced a message.

const (
	RoleSystem    Role = "system"
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	RoleTool      Role = "tool"
	RoleDeveloper Role = "developer"
)

type SDKError

type SDKError struct {
	Msg   string
	Cause error
}

SDKError is the base error type for all errors in the LLM package.

func (*SDKError) Error

func (e *SDKError) Error() string

func (*SDKError) Unwrap

func (e *SDKError) Unwrap() error

type ServerError

type ServerError struct {
	ProviderError
}

ServerError indicates a server-side failure from the provider.

func (*ServerError) Retryable

func (e *ServerError) Retryable() bool

type StreamAccumulator

type StreamAccumulator struct {
	// contains filtered or unexported fields
}

StreamAccumulator collects streaming events into a complete Response.

func NewStreamAccumulator

func NewStreamAccumulator() *StreamAccumulator

NewStreamAccumulator creates a new StreamAccumulator ready to process events.

func (*StreamAccumulator) Process

func (a *StreamAccumulator) Process(event StreamEvent)

Process handles a single StreamEvent, updating the accumulator state.

func (*StreamAccumulator) Response

func (a *StreamAccumulator) Response() Response

Response builds a complete Response from the accumulated events.

type StreamError

type StreamError struct {
	SDKError
}

StreamError indicates a failure during response streaming.

func (*StreamError) Retryable

func (e *StreamError) Retryable() bool

type StreamEvent

type StreamEvent struct {
	Type               StreamEventType `json:"type"`
	Delta              string          `json:"delta,omitempty"`
	TextID             string          `json:"text_id,omitempty"`
	ReasoningDelta     string          `json:"reasoning_delta,omitempty"`
	ReasoningSignature string          `json:"reasoning_signature,omitempty"`
	ToolCall           *ToolCallData   `json:"tool_call,omitempty"`
	FinishReason       *FinishReason   `json:"finish_reason,omitempty"`
	Usage              *Usage          `json:"usage,omitempty"`
	FullResponse       *Response       `json:"full_response,omitempty"`
	Err                error           `json:"-"`
	Raw                json.RawMessage `json:"raw,omitempty"`
}

StreamEvent represents a single event in a streaming response.

type StreamEventType

type StreamEventType string

StreamEventType discriminates the kind of streaming event.

const (
	EventStreamStart        StreamEventType = "stream_start"
	EventTextStart          StreamEventType = "text_start"
	EventTextDelta          StreamEventType = "text_delta"
	EventTextEnd            StreamEventType = "text_end"
	EventReasoningStart     StreamEventType = "reasoning_start"
	EventReasoningDelta     StreamEventType = "reasoning_delta"
	EventReasoningSignature StreamEventType = "reasoning_signature"
	EventReasoningEnd       StreamEventType = "reasoning_end"
	EventRedactedThinking   StreamEventType = "redacted_thinking"
	EventToolCallStart      StreamEventType = "tool_call_start"
	EventToolCallDelta      StreamEventType = "tool_call_delta"
	EventToolCallEnd        StreamEventType = "tool_call_end"
	EventFinish             StreamEventType = "finish"
	EventError              StreamEventType = "error"
	EventProviderEvent      StreamEventType = "provider_event"
)

type ThinkingData

type ThinkingData struct {
	Text      string `json:"text"`
	Signature string `json:"signature,omitempty"`
	Redacted  bool   `json:"redacted,omitempty"`
}

ThinkingData holds model reasoning content.

type TokenTracker

type TokenTracker struct {
	// contains filtered or unexported fields
}

TokenTracker is a middleware that accumulates token usage per provider. It implements Middleware and can be passed to NewClient via WithMiddleware.

func NewTokenTracker

func NewTokenTracker() *TokenTracker

NewTokenTracker creates a new, zeroed token tracking middleware.

func (*TokenTracker) AddUsage

func (t *TokenTracker) AddUsage(provider string, usage Usage, model ...string)

AddUsage manually adds token usage for a provider. Used by backends that bypass the LLM client middleware (e.g., claude-code subprocess backend). The model parameter is optional; pass "" to leave the provider's model unchanged. Model strings are normalized through the catalog so versioned provider-returned IDs resolve to canonical pricing entries — matching WrapComplete's behavior.

func (*TokenTracker) AllProviderUsage

func (t *TokenTracker) AllProviderUsage() map[string]Usage

AllProviderUsage returns a copy of the accumulated usage for all providers.

func (*TokenTracker) CostByProvider

func (t *TokenTracker) CostByProvider(resolve ModelResolver) map[string]ProviderCost

CostByProvider returns a per-provider cost rollup, resolving each provider's model via the caller-supplied resolver and pricing it via EstimateCost. A nil resolver is treated as one that always returns "".

func (*TokenTracker) ModelForProvider

func (t *TokenTracker) ModelForProvider(provider string) string

ModelForProvider returns the last-seen model for a provider. Returns "" if unknown.

func (*TokenTracker) ObservedModelResolver

func (t *TokenTracker) ObservedModelResolver(fallback string) ModelResolver

ObservedModelResolver returns a ModelResolver that uses the tracker's observed per-provider models. Falls back to the provided fallback model for providers where no model was observed.

func (*TokenTracker) ProviderUsage

func (t *TokenTracker) ProviderUsage(provider string) Usage

ProviderUsage returns the accumulated usage for a specific provider. Returns a zero Usage if the provider has not been seen.

func (*TokenTracker) Providers

func (t *TokenTracker) Providers() []string

Providers returns a sorted slice of provider names that have recorded usage.

func (*TokenTracker) TotalCostUSD

func (t *TokenTracker) TotalCostUSD(resolve ModelResolver) float64

TotalCostUSD sums CostByProvider to a single dollar figure using the same resolver.

func (*TokenTracker) TotalUsage

func (t *TokenTracker) TotalUsage() Usage

TotalUsage returns accumulated usage summed across all providers.

func (*TokenTracker) WrapComplete

func (t *TokenTracker) WrapComplete(next CompleteHandler) CompleteHandler

WrapComplete implements the Middleware interface. It calls the next handler and, on success, adds the response's token usage to the per-provider accumulator.

type ToolCallData

type ToolCallData struct {
	ID             string          `json:"id"`
	Name           string          `json:"name"`
	Arguments      json.RawMessage `json:"arguments"`
	ThoughtSigData string          `json:"thought_signature,omitempty"`
}

ToolCallData represents a model-initiated tool invocation.

type ToolChoice

type ToolChoice struct {
	Mode     string `json:"mode"`
	ToolName string `json:"tool_name,omitempty"`
}

ToolChoice controls how the model uses tools.

func ToolChoiceAuto

func ToolChoiceAuto() ToolChoice

ToolChoiceAuto returns a ToolChoice that lets the model decide whether to call tools.

func ToolChoiceNamed

func ToolChoiceNamed(name string) ToolChoice

ToolChoiceNamed creates a ToolChoice that forces the model to call a specific tool.

func ToolChoiceNone

func ToolChoiceNone() ToolChoice

ToolChoiceNone returns a ToolChoice that prevents the model from calling tools.

func ToolChoiceRequired

func ToolChoiceRequired() ToolChoice

ToolChoiceRequired returns a ToolChoice that forces the model to call a tool.

type ToolDefinition

type ToolDefinition struct {
	Name        string          `json:"name"`
	Description string          `json:"description"`
	Parameters  json.RawMessage `json:"parameters"`
}

ToolDefinition defines a tool the model can call.

type ToolResultData

type ToolResultData struct {
	ToolCallID     string `json:"tool_call_id"`
	Name           string `json:"name,omitempty"`
	Content        string `json:"content"`
	IsError        bool   `json:"is_error"`
	ImageData      []byte `json:"image_data,omitempty"`
	ImageMediaType string `json:"image_media_type,omitempty"`
}

ToolResultData represents the result of executing a tool call.

type TraceBuilder

type TraceBuilder struct {
	// contains filtered or unexported fields
}

TraceBuilder converts streaming events into normalized trace events.

func NewTraceBuilder

func NewTraceBuilder(opts TraceOptions) *TraceBuilder

NewTraceBuilder creates a trace builder for one request.

func (*TraceBuilder) Events

func (b *TraceBuilder) Events() []TraceEvent

Events returns the trace events emitted so far.

func (*TraceBuilder) Process

func (b *TraceBuilder) Process(evt StreamEvent)

Process ingests one stream event and emits any corresponding trace events.

type TraceEvent

type TraceEvent struct {
	Kind          TraceKind
	Provider      string
	Model         string
	ToolName      string
	Preview       string
	ProviderEvent string
	RawPreview    string
	FinishReason  string
	Usage         Usage
	// SessionOwned is true when the originating request carried
	// request-level TraceObservers — in tracker only the agent session
	// registers those, and it re-emits every trace event as an agent
	// llm_* event. Activity-log writers listening at the client level
	// skip SessionOwned events to avoid logging the same stream twice
	// (issue #354); non-session calls (e.g. the autopilot interviewer)
	// stay SessionOwned=false so the trace path remains their only,
	// and therefore kept, log surface.
	SessionOwned bool
}

TraceEvent is a normalized event for rendering live LLM activity.

type TraceKind

type TraceKind string

TraceKind identifies a normalized LLM trace event.

const (
	TraceRequestStart TraceKind = "request_start"
	TraceReasoning    TraceKind = "reasoning"
	TraceText         TraceKind = "text"
	TraceToolPrepare  TraceKind = "tool_prepare"
	TraceFinish       TraceKind = "finish"
	TraceProviderRaw  TraceKind = "provider_raw"
)

type TraceLogger

type TraceLogger struct {
	// contains filtered or unexported fields
}

TraceLogger writes trace events to an io.Writer. Sequential text and reasoning deltas are accumulated and flushed as a single summary line when the stream finishes or a different event kind arrives.

func NewTraceLogger

func NewTraceLogger(w io.Writer, opts TraceLoggerOptions) *TraceLogger

NewTraceLogger creates a console trace logger.

func (*TraceLogger) Flush

func (l *TraceLogger) Flush()

Flush forces any pending batched output to be written. Callers should invoke this when a stream terminates without a normal finish event (e.g. on error) to avoid losing buffered trace output.

func (*TraceLogger) HandleTraceEvent

func (l *TraceLogger) HandleTraceEvent(evt TraceEvent)

HandleTraceEvent implements TraceObserver.

type TraceLoggerOptions

type TraceLoggerOptions struct {
	Verbose bool
}

TraceLoggerOptions configure console trace logging.

type TraceObserver

type TraceObserver interface {
	HandleTraceEvent(evt TraceEvent)
}

TraceObserver receives normalized LLM trace events.

type TraceObserverFunc

type TraceObserverFunc func(evt TraceEvent)

TraceObserverFunc adapts a function into a TraceObserver.

func (TraceObserverFunc) HandleTraceEvent

func (f TraceObserverFunc) HandleTraceEvent(evt TraceEvent)

HandleTraceEvent implements TraceObserver.

type TraceOptions

type TraceOptions struct {
	Provider string
	Model    string
	Verbose  bool
}

TraceOptions configure trace building behavior.

type TransformMiddleware

type TransformMiddleware struct {
	// contains filtered or unexported fields
}

TransformMiddleware applies request and/or response transforms around a CompleteHandler.

func NewTransformMiddleware

func NewTransformMiddleware(reqFn RequestTransformFunc, opts ...TransformOption) *TransformMiddleware

NewTransformMiddleware creates a TransformMiddleware with the given request transform and optional TransformOptions. If reqFn is nil, the request is passed through unchanged.

func (*TransformMiddleware) WrapComplete

func (tm *TransformMiddleware) WrapComplete(next CompleteHandler) CompleteHandler

WrapComplete implements the Middleware interface. It applies the request transform before calling next, and the response transform after a successful call.

type TransformOption

type TransformOption func(*TransformMiddleware)

TransformOption configures a TransformMiddleware.

func WithResponseTransform

func WithResponseTransform(fn ResponseTransformFunc) TransformOption

WithResponseTransform adds a response transform that runs after a successful downstream call.

type Usage

type Usage struct {
	InputTokens      int     `json:"input_tokens"`
	OutputTokens     int     `json:"output_tokens"`
	TotalTokens      int     `json:"total_tokens"`
	ReasoningTokens  *int    `json:"reasoning_tokens,omitempty"`
	CacheReadTokens  *int    `json:"cache_read_tokens,omitempty"`
	CacheWriteTokens *int    `json:"cache_write_tokens,omitempty"`
	EstimatedCost    float64 `json:"estimated_cost,omitempty"`
	Raw              any     `json:"raw,omitempty"`
}

Usage tracks token consumption.

func (Usage) Add

func (u Usage) Add(other Usage) Usage

Add combines two Usage values.

func (Usage) String

func (u Usage) String() string

String formats Usage as a concise token summary.

type Warning

type Warning struct {
	Message string `json:"message"`
	Code    string `json:"code,omitempty"`
}

Warning represents a non-fatal issue.

Directories

Path Synopsis
ABOUTME: Anthropic Messages API adapter implementing the ProviderAdapter interface.
ABOUTME: Anthropic Messages API adapter implementing the ProviderAdapter interface.
ABOUTME: Google Gemini API adapter implementing the ProviderAdapter interface.
ABOUTME: Google Gemini API adapter implementing the ProviderAdapter interface.
ABOUTME: OpenAI Responses API adapter implementing the ProviderAdapter interface.
ABOUTME: OpenAI Responses API adapter implementing the ProviderAdapter interface.
ABOUTME: OpenAI Chat Completions compatible adapter implementing the ProviderAdapter interface.
ABOUTME: OpenAI Chat Completions compatible adapter implementing the ProviderAdapter interface.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL