Documentation
¶
Overview ¶
ABOUTME: Detects provider credit/quota exhaustion and builds an actionable, ABOUTME: account-attributed message — never printing the raw API key (#487 Phase 1).
ABOUTME: Model catalog derived entirely from dippin-lang/pricing (DRY, #570). ABOUTME: Supports lookup by ID/alias and listing by provider.
ABOUTME: Core LLM client that routes requests to provider adapters. ABOUTME: Supports multiple providers, default routing, middleware chains, and env-based config.
ABOUTME: Error type hierarchy for the unified LLM client library. ABOUTME: Defines provider errors, retryability, and HTTP status code mapping.
ABOUTME: Provider/model failover — switch lanes on a billing/quota exhaustion so ABOUTME: one dead upstream account doesn't end a run when another lane is configured (#486).
ABOUTME: Pretty-print formatting for Response and Usage types via fmt.Stringer. ABOUTME: Produces concise, human-readable output for CLI and TUI display.
ABOUTME: Middleware types for the unified LLM client's request/response pipeline. ABOUTME: Defines CompleteHandler and the Middleware interface.
ABOUTME: Cost estimation for LLM usage — base prices come from dippin-lang/pricing ABOUTME: (the single source of truth, #558); tracker overlays cache rates until dippin ships them.
ABOUTME: Defines the ProviderAdapter interface for LLM provider implementations. ABOUTME: Each provider (OpenAI, Anthropic, Gemini, etc.) implements this interface.
ABOUTME: Builds RateLimitInfo from provider rate-limit response headers so both ABOUTME: the Complete and streamed (traced) paths can surface the same metadata (#617).
ABOUTME: Retry middleware with exponential backoff for the LLM client. ABOUTME: Retries only errors that implement Retryable() bool returning true; respects context and RetryAfter.
ABOUTME: Streaming event types and accumulator for incremental LLM responses. ABOUTME: Defines StreamEvent, StreamEventType enum, and StreamAccumulator.
ABOUTME: Stream-idle deadline guard shared by the provider SSE adapters. ABOUTME: Detects a hung (byte-silent) stream and drives a retryable cancel.
ABOUTME: Middleware that accumulates per-provider token usage across LLM calls. ABOUTME: Thread-safe; used by the TUI dashboard header for real-time token counts.
ABOUTME: Per-provider cost rollup for the TokenTracker middleware. ABOUTME: Maps accumulated Usage to dollar cost via a caller-supplied model resolver.
ABOUTME: Structured trace events for live LLM introspection across console and TUI surfaces.
ABOUTME: Console logger for structured LLM trace events. ABOUTME: Batches sequential text/reasoning deltas into a single summary line per stream.
ABOUTME: Core data types for the unified LLM client library. ABOUTME: Defines Message, ContentPart, Request, Response, Usage, and related types.
ABOUTME: Detects a Claude subscription usage-limit (Max/Team/Pro rolling cap) ABOUTME: — a recoverable "resets at X" pause, distinct from credit/quota exhaustion.
Index ¶
- Constants
- Variables
- func BillingHelp(err error) (string, bool)
- func EmitRequestSent(ch chan<- StreamEvent, body []byte, tracing bool)
- func ErrorFromStatusCode(statusCode int, message, provider string) error
- func ErrorFromStatusCodeRetryAfter(statusCode int, message, provider string, retryAfter *float64) error
- func EstimateCost(model string, usage Usage) float64
- func EstimateCostChecked(model string, usage Usage) (float64, bool)
- func FormatTraceLine(evt TraceEvent, verbose bool) string
- func IsBillingError(err error) bool
- func IsDeprecated(model string) bool
- func IsPriced(model string) bool
- func IsUsageLimit(err error) bool
- func ModelContextWindow(provider, model string) int
- func NewCallID() string
- func ParseRetryAfter(h http.Header) *float64
- func ReadSSELine(reader *bufio.Reader, guard *StreamIdleGuard) ([]byte, error)
- func RequestIsTraced(req *Request) bool
- func UsageLimitResetAt(err error) (time.Time, bool)
- type AbortError
- type AccessDeniedError
- type AudioData
- type AuthenticationError
- type Client
- func (c *Client) AddMiddleware(mw Middleware)
- func (c *Client) AddTraceObserver(obs TraceObserver)
- func (c *Client) Close() error
- func (c *Client) Complete(ctx context.Context, req *Request) (*Response, error)
- func (c *Client) CompleteFailover(ctx context.Context, req *Request, fallbacks []Target, ...) (*Response, error)
- func (c *Client) DefaultProvider() string
- func (c *Client) HasMiddleware(mw Middleware) bool
- func (c *Client) Stream(ctx context.Context, req *Request) <-chan StreamEvent
- type ClientOption
- type CompleteHandler
- type ConfigurationError
- type ContentFilterError
- type ContentKind
- type ContentPart
- type ContextLengthError
- type DocumentData
- type FailoverEvent
- type FinishReason
- type ImageData
- type InvalidRequestError
- type InvalidToolCallError
- type Message
- type Middleware
- type ModelInfo
- type ModelResolver
- type NetworkError
- type NoObjectGeneratedError
- type NotFoundError
- type ProviderAdapter
- type ProviderCost
- type ProviderError
- type ProviderErrorInterface
- type QuotaExceededError
- type RateLimitError
- type RateLimitInfo
- type Request
- type RequestTimeoutError
- type ResetKind
- type Response
- type ResponseFormat
- type RetryMiddleware
- type RetryOption
- type Role
- type SDKError
- type ServerError
- type StreamAccumulator
- type StreamError
- type StreamEvent
- type StreamEventType
- type StreamIdleGuard
- type Target
- type ThinkingData
- type TokenTracker
- func (t *TokenTracker) AddUsage(provider string, usage Usage, model ...string)
- func (t *TokenTracker) AllProviderUsage() map[string]Usage
- func (t *TokenTracker) CostByProvider(resolve ModelResolver) map[string]ProviderCost
- func (t *TokenTracker) ModelForProvider(provider string) string
- func (t *TokenTracker) ObservedModelResolver(fallback string) ModelResolver
- func (t *TokenTracker) ProviderUsage(provider string) Usage
- func (t *TokenTracker) Providers() []string
- func (t *TokenTracker) TotalCostUSD(resolve ModelResolver) float64
- func (t *TokenTracker) TotalUsage() Usage
- func (t *TokenTracker) WrapComplete(next CompleteHandler) CompleteHandler
- type ToolCallData
- type ToolChoice
- type ToolDefinition
- type ToolResultData
- type TraceBuilder
- type TraceEvent
- type TraceKind
- type TraceLogger
- type TraceLoggerOptions
- type TraceObserver
- type TraceObserverFunc
- type TraceOptions
- type Usage
- type Warning
Constants ¶
const DefaultStreamIdleTimeout = 10 * time.Minute
DefaultStreamIdleTimeout bounds how long a streaming SSE socket may go with ZERO bytes before it is treated as hung. It keys on raw socket bytes (any SSE frame — including blank-line keepalives and provider reasoning frames — resets it), NOT on tracker StreamEvents, because reasoning phases emit no tracker events (#577). The default is deliberately generous: legitimate turns reach ~304.5s (sonnet) and gpt-5 has 32s+ silent reasoning gaps, so the threshold sits comfortably above the longest observed turn to never abort a healthy stream. Override per-adapter via WithStreamIdleTimeout.
Variables ¶
var ErrStreamIdle = errors.New("stream idle timeout: no SSE bytes within deadline")
ErrStreamIdle marks a stream cancelled by the idle deadline — distinct from a caller/shutdown context cancel. It is surfaced as a retryable StreamError so the turn-level retry middleware re-issues the completion instead of the channel closing with no error and the turn being silently truncated (#575, #576).
Functions ¶
func BillingHelp ¶ added in v0.47.0
BillingHelp returns an actionable, account-attributed message for a billing error (and true), or ("", false) if err is not one. The message identifies the failing provider, which env var supplied the key, and a masked fingerprint of that key — so a user with multiple accounts knows exactly where to add credit — plus the provider's billing URL. It NEVER includes the raw key.
func EmitRequestSent ¶ added in v0.50.0
func EmitRequestSent(ch chan<- StreamEvent, body []byte, tracing bool)
EmitRequestSent records the verbatim wire body on the event stream. Adapters call it immediately after building the body and before the HTTP call, because that is the only place the body is in scope — the provider's own stream-start arrives several layers deeper, inside SSE parsing.
tracing gates the emit: the body is telemetry, so an untraced caller driving an adapter directly sees the provider event sequence unchanged. A leading synthetic event would otherwise shift every position-indexed assertion in adapter tests and in any embedder reading Stream() by index. Adapters pass the same per-request flag that gates provider events, which the client sets on every request it traces.
func ErrorFromStatusCode ¶
ErrorFromStatusCode maps an HTTP status code to the appropriate error type.
func ErrorFromStatusCodeRetryAfter ¶ added in v0.59.1
func ErrorFromStatusCodeRetryAfter(statusCode int, message, provider string, retryAfter *float64) error
ErrorFromStatusCodeRetryAfter is ErrorFromStatusCode with a server-requested retry delay (seconds) parsed from the `Retry-After` header. It populates ProviderError.RetryAfter so the retry middleware honors the server's backoff on a 429/503 instead of only its local exponential schedule (#549). Adapters that hold the http.Response pass ParseRetryAfter(resp.Header).
func EstimateCost ¶
EstimateCost returns the estimated dollar cost for the given model and token usage. Base input/output prices come from dippin-lang/pricing (the single source of truth, #558); returns 0 for models it doesn't price, logging a single warning per unknown name so operators notice their --max-cost ceiling won't apply to usage priced under an unknown rate.
func EstimateCostChecked ¶ added in v0.52.0
EstimateCostChecked is EstimateCost plus the bit callers need to tell a genuinely-free run apart from an uncatalogued one: priced reports whether the cost was computed from a pricing entry (true) or defaulted to $0 because the model is unknown (false). A found-but-unpriced dippin entry (Priced=false, e.g. Qwen / a free tier) still returns (…, true) — it is priced, at $0.
func FormatTraceLine ¶
func FormatTraceLine(evt TraceEvent, verbose bool) string
FormatTraceLine formats one trace event for console or TUI rendering.
func IsBillingError ¶ added in v0.47.0
IsBillingError reports whether err is a provider credit/quota exhaustion — a recoverable operational condition ("add funds and retry"), distinct from a code bug, an auth failure, or a bad request. It matches the typed QuotaExceededError (OpenAI/compat) and any error in the chain whose message carries a billing signal (Anthropic's credit-balance case).
func IsDeprecated ¶ added in v0.63.0
IsDeprecated reports whether model is retired on its first-party provider API (per dippin's ModelPrice.Deprecated) — it 404s on the first-party endpoint but is still billable through a passthrough platform like Bedrock or Vertex, so it remains in the catalog. A tracker consumer that treats the catalog as a first-party allowlist (e.g. `tracker doctor`) can warn on these unless a gateway/base-URL is routing the model to a passthrough platform. False for an unknown model.
func IsPriced ¶ added in v0.52.0
IsPriced reports whether model has a pricing entry (by ID, alias, or the version-separator fold) and so can be priced. False for an unknown or empty model name — an empty name is the subscription-auth backends' "no model set" case, which a --max-cost ceiling cannot bound either.
func IsUsageLimit ¶ added in v0.65.0
IsUsageLimit reports whether err is a subscription usage-limit signal — the RECOVERABLE "you've hit your plan's cap, it resets at X" condition. It is a companion to IsBillingError, NOT an extension of it: usage-limit is deliberately kept out of IsBillingError so it is never mistaken for insufficient_quota/credit-balance, and — more importantly — so the codergen handler can route it to the resumable OutcomePausedBilling pause instead of letting a retryable-429-shaped failure reach the retry middleware, which would just re-hit the same cap. Returns false for insufficient_quota/credit-balance (that is IsBillingError) and for a plain retryable 429 rate limit.
func ModelContextWindow ¶ added in v0.63.14
ModelContextWindow returns dippin's per-model context window in tokens for the given provider/model, or 0 when dippin doesn't know it (dippin never guesses a window, so an absent value is genuinely unknown, not a real 0-token limit). A provider-scoped lookup wins when provider is non-empty; otherwise a bare model lookup is used. An un-pinned family@selector alias (e.g. "opus@latest") is resolved to a concrete id first so the window comes from the real model.
func NewCallID ¶ added in v0.50.0
func NewCallID() string
NewCallID returns an identifier for one LLM request. Callers stamp it into TraceOptions so every trace event from that request shares it; see TraceEvent.CallID for why grouping matters.
Locally generated rather than taken from Response.ID: the provider's ID only exists once the response arrives, so events emitted before then — including the request-start that carries the wire body — would have nothing to group on. Collision resistance only has to hold within one run's log.
func ParseRetryAfter ¶ added in v0.59.1
ParseRetryAfter extracts a non-negative retry delay in seconds from a `Retry-After` header, supporting both forms the HTTP spec allows: an integer (or float) number of seconds, and an HTTP-date. Returns nil when the header is absent or unparseable, so the caller falls back to local backoff.
func ReadSSELine ¶ added in v0.64.0
func ReadSSELine(reader *bufio.Reader, guard *StreamIdleGuard) ([]byte, error)
ReadSSELine reads one line from reader and re-arms the idle deadline on any byte progress (a successful read — including a blank-line keepalive). It is the single read primitive every adapter SSE loop uses so the idle reset lives in one place.
func RequestIsTraced ¶ added in v0.50.0
RequestIsTraced reports whether the client is tracing this request. The client sets the flag on every request it routes through the streaming trace path; adapters use it to gate telemetry-only emissions (provider events and the wire body) so an untraced caller sees neither.
func UsageLimitResetAt ¶ added in v0.65.0
UsageLimitResetAt extracts the reset time embedded in a usage-limit error's message, returning (zero, false) when none is discoverable. A missing reset is not an error — the pause still happens, just without a scheduled resume hint (PauseError.ResumeAfter stays the zero value = unknown). Recognized forms: an RFC3339 timestamp ("resets at 2026-08-24T15:00:00Z") and a Unix-epoch seconds value (the Claude CLI's "usage limit reached|<epoch>").
Types ¶
type AbortError ¶
type AbortError struct {
SDKError
}
AbortError indicates the operation was explicitly aborted.
type AccessDeniedError ¶
type AccessDeniedError struct {
ProviderError
}
AccessDeniedError indicates insufficient permissions.
func (*AccessDeniedError) Retryable ¶
func (e *AccessDeniedError) Retryable() bool
type AudioData ¶
type AudioData struct {
URL string `json:"url,omitempty"`
Data []byte `json:"data,omitempty"`
MediaType string `json:"media_type,omitempty"`
}
AudioData holds audio content.
type AuthenticationError ¶
type AuthenticationError struct {
ProviderError
}
AuthenticationError indicates invalid or missing credentials.
func (*AuthenticationError) Retryable ¶
func (e *AuthenticationError) Retryable() bool
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client manages LLM provider adapters and routes requests through a middleware chain.
func NewClient ¶
func NewClient(opts ...ClientOption) (*Client, error)
NewClient creates a Client from the given options.
func NewClientFromEnv ¶
func (*Client) AddMiddleware ¶
func (c *Client) AddMiddleware(mw Middleware)
AddMiddleware appends a middleware to the client's existing middleware chain. This allows post-construction addition of middleware (e.g., adding a TokenTracker after the client is built from environment variables).
func (*Client) AddTraceObserver ¶
func (c *Client) AddTraceObserver(obs TraceObserver)
AddTraceObserver appends a live trace observer after client construction.
func (*Client) Complete ¶
Complete sends a completion request through the middleware chain and returns the response.
func (*Client) CompleteFailover ¶ added in v0.47.0
func (c *Client) CompleteFailover(ctx context.Context, req *Request, fallbacks []Target, onFailover func(FailoverEvent)) (*Response, error)
CompleteFailover tries req against its own provider/model first, then each fallback Target in order, switching lanes on a failover-class error. onFailover (optional) is called before each switch so the caller can emit an audit event.
Only a *failover-class* error switches lanes (see isFailoverClass): a billing/ quota exhaustion means the current account is out of credit, so another lane may succeed. A transient error was already retried by the retry middleware; a code/auth/context error would fail identically on every lane, so those are returned as-is without burning the other lanes.
func (*Client) DefaultProvider ¶
DefaultProvider returns the name of the default provider, or empty string.
func (*Client) HasMiddleware ¶ added in v0.46.0
func (c *Client) HasMiddleware(mw Middleware) bool
HasMiddleware reports whether mw is already in the chain, compared by identity. Safe to call with a pointer-typed middleware (e.g. *TokenTracker): interface equality short-circuits on differing dynamic types, so it never compares — and never panics on — a non-comparable value already in the chain.
func (*Client) Stream ¶
func (c *Client) Stream(ctx context.Context, req *Request) <-chan StreamEvent
Stream sends a streaming request to the resolved provider adapter. Middleware is NOT applied to streaming requests (middleware only wraps Complete). On provider resolution failure, it returns a channel containing a single error event.
type ClientOption ¶
type ClientOption func(*clientConfig)
ClientOption configures a Client during construction.
func WithDefaultProvider ¶
func WithDefaultProvider(name string) ClientOption
WithDefaultProvider sets the provider name used when a request does not specify one.
func WithMiddleware ¶
func WithMiddleware(mw Middleware) ClientOption
WithMiddleware appends a middleware to the client's middleware chain.
func WithProvider ¶
func WithProvider(adapter ProviderAdapter) ClientOption
WithProvider registers a provider adapter with the client.
type CompleteHandler ¶
CompleteHandler is a function that processes a completion request and returns a response. It is used as the unit of composition in the middleware chain.
type ConfigurationError ¶
type ConfigurationError struct {
SDKError
}
ConfigurationError indicates a problem with the client configuration.
type ContentFilterError ¶
type ContentFilterError struct {
ProviderError
}
ContentFilterError indicates the request or response was blocked by a content filter.
func (*ContentFilterError) Retryable ¶
func (e *ContentFilterError) Retryable() bool
type ContentKind ¶
type ContentKind string
ContentKind discriminates ContentPart variants.
const ( KindText ContentKind = "text" KindImage ContentKind = "image" KindAudio ContentKind = "audio" KindDocument ContentKind = "document" KindToolCall ContentKind = "tool_call" KindToolResult ContentKind = "tool_result" KindThinking ContentKind = "thinking" KindRedactedThinking ContentKind = "redacted_thinking" )
type ContentPart ¶
type ContentPart struct {
Kind ContentKind `json:"kind"`
Text string `json:"text,omitempty"`
Image *ImageData `json:"image,omitempty"`
Audio *AudioData `json:"audio,omitempty"`
Document *DocumentData `json:"document,omitempty"`
ToolCall *ToolCallData `json:"tool_call,omitempty"`
ToolResult *ToolResultData `json:"tool_result,omitempty"`
Thinking *ThinkingData `json:"thinking,omitempty"`
}
ContentPart is a tagged union representing one piece of message content.
type ContextLengthError ¶
type ContextLengthError struct {
ProviderError
}
ContextLengthError indicates the request exceeded the model's context window.
func (*ContextLengthError) Retryable ¶
func (e *ContextLengthError) Retryable() bool
type DocumentData ¶
type DocumentData struct {
URL string `json:"url,omitempty"`
Data []byte `json:"data,omitempty"`
MediaType string `json:"media_type,omitempty"`
FileName string `json:"file_name,omitempty"`
}
DocumentData holds document content.
type FailoverEvent ¶ added in v0.47.0
type FailoverEvent struct {
From Target // the exhausted lane
To Target // the lane being tried next
Err error // the error that triggered the switch
}
FailoverEvent describes a single lane switch, for the caller to surface in the audit trail (which target served which node).
type FinishReason ¶
FinishReason indicates why generation stopped.
type ImageData ¶
type ImageData struct {
URL string `json:"url,omitempty"`
Data []byte `json:"data,omitempty"`
MediaType string `json:"media_type,omitempty"`
Detail string `json:"detail,omitempty"`
}
ImageData holds image content as URL or raw bytes.
type InvalidRequestError ¶
type InvalidRequestError struct {
ProviderError
}
InvalidRequestError indicates a malformed or invalid request.
func (*InvalidRequestError) Retryable ¶
func (e *InvalidRequestError) Retryable() bool
type InvalidToolCallError ¶
type InvalidToolCallError struct {
SDKError
}
InvalidToolCallError indicates the model produced an invalid tool call.
type Message ¶
type Message struct {
Role Role `json:"role"`
Content []ContentPart `json:"content"`
Name string `json:"name,omitempty"`
ToolCallID string `json:"tool_call_id,omitempty"`
}
Message is the fundamental unit of conversation.
func AssistantMessage ¶
AssistantMessage creates a message with the assistant role.
func SystemMessage ¶
SystemMessage creates a message with the system role.
func ToolResultMessage ¶
ToolResultMessage creates a tool result message.
func UserMessage ¶
UserMessage creates a message with the user role.
func (Message) ToolCalls ¶
func (m Message) ToolCalls() []ToolCallData
ToolCalls extracts all tool call content parts from the message.
type Middleware ¶
type Middleware interface {
WrapComplete(next CompleteHandler) CompleteHandler
}
Middleware wraps a CompleteHandler to add cross-cutting behavior (logging, retry, etc.).
func NewRetryMiddleware ¶
func NewRetryMiddleware(opts ...RetryOption) Middleware
NewRetryMiddleware creates a retry Middleware with the given options.
type ModelInfo ¶
type ModelInfo struct {
ID string `json:"id"`
Provider string `json:"provider"`
DisplayName string `json:"display_name"`
ContextWindow int `json:"context_window"`
MaxOutput int `json:"max_output"`
SupportsTools bool `json:"supports_tools"`
SupportsVision bool `json:"supports_vision"`
SupportsReasoning bool `json:"supports_reasoning"`
Aliases []string `json:"aliases,omitempty"`
// CacheReadMultiplier and CacheWriteMultiplier are a per-model override hook
// for the cache-rate overlay (pricing.go). dippin-lang/pricing now carries
// verified cache read AND write rates for essentially every model, so the
// overlay self-disables and these are never set for a dippin-derived entry
// (0 everywhere). They remain on the struct as the override seam the overlay
// reads through for a model in pricing.CacheGaps().
CacheReadMultiplier float64 `json:"cache_read_multiplier,omitempty"`
CacheWriteMultiplier float64 `json:"cache_write_multiplier,omitempty"`
}
ModelInfo describes a known LLM model and its capabilities. Every field is derived from dippin-lang/pricing (the single source of truth, #558/#570): identity (ID/provider/aliases), display name (#570, dippin v0.68), and capabilities (context window / max output / tool·vision·reasoning, #571). The hand-maintained catalog that used to live here is retired — the model set is now dippin's by construction and can never drift from it.
func GetModelInfo ¶
GetModelInfo looks up a model by ID or alias, resolving through dippin's version-separator fold (claude-haiku-4.5 == claude-haiku-4-5). Returns nil if dippin does not price the model. An exact id wins anywhere in the catalog; otherwise the first alias or version-folded id match resolves to its entry.
func ListModels ¶
ListModels returns all models dippin prices, optionally filtered by provider. Pass an empty string to return all models.
type ModelResolver ¶
ModelResolver returns the model name that should be used for cost estimation for a given provider. Return "" when unknown — the entry is still included in the result with USD=0.
type NetworkError ¶
type NetworkError struct {
SDKError
}
NetworkError indicates a network-level failure (DNS, TCP, TLS).
func (*NetworkError) Retryable ¶
func (e *NetworkError) Retryable() bool
type NoObjectGeneratedError ¶
type NoObjectGeneratedError struct {
SDKError
}
NoObjectGeneratedError indicates structured output generation failed.
type NotFoundError ¶
type NotFoundError struct {
ProviderError
}
NotFoundError indicates the requested resource does not exist.
func (*NotFoundError) Retryable ¶
func (e *NotFoundError) Retryable() bool
type ProviderAdapter ¶
type ProviderAdapter interface {
// Name returns the provider's identifier (e.g. "openai", "anthropic").
Name() string
// Complete sends a request and returns the full response.
Complete(ctx context.Context, req *Request) (*Response, error)
// Stream sends a request and returns a channel of streaming events.
Stream(ctx context.Context, req *Request) <-chan StreamEvent
// Close releases any resources held by the adapter.
Close() error
}
ProviderAdapter is the interface that LLM provider implementations must satisfy. It supports both synchronous completion and streaming responses.
type ProviderCost ¶
ProviderCost is the per-provider cost rollup returned by TokenTracker.CostByProvider. Usage and USD are aggregated across every model the provider ran; Model names the model used for display (the provider's last-observed model — see the note on ModelForProvider). USD is the sum of each (provider, model) bucket priced at its own rate, so a multi-model provider is no longer mispriced by a single last-write-wins model (#527).
type ProviderError ¶
type ProviderError struct {
SDKError
Provider string
StatusCode int
ErrorCode string
RetryAfter *float64
RawBody json.RawMessage
}
ProviderError is the base type for errors returned by LLM providers.
func (*ProviderError) GetProvider ¶
func (e *ProviderError) GetProvider() string
func (*ProviderError) GetStatusCode ¶
func (e *ProviderError) GetStatusCode() int
type ProviderErrorInterface ¶
type ProviderErrorInterface interface {
error
Retryable() bool
GetProvider() string
GetStatusCode() int
}
ProviderErrorInterface describes errors originating from an LLM provider.
type QuotaExceededError ¶
type QuotaExceededError struct {
ProviderError
}
QuotaExceededError indicates the account's quota has been exhausted.
func (*QuotaExceededError) Retryable ¶
func (e *QuotaExceededError) Retryable() bool
type RateLimitError ¶
type RateLimitError struct {
ProviderError
}
RateLimitError indicates the request was rate limited by the provider.
func (*RateLimitError) Retryable ¶
func (e *RateLimitError) Retryable() bool
type RateLimitInfo ¶
type RateLimitInfo struct {
RequestsRemaining *int `json:"requests_remaining,omitempty"`
RequestsLimit *int `json:"requests_limit,omitempty"`
TokensRemaining *int `json:"tokens_remaining,omitempty"`
TokensLimit *int `json:"tokens_limit,omitempty"`
ResetAt *time.Time `json:"reset_at,omitempty"`
}
RateLimitInfo holds rate limit metadata from provider headers.
func RateLimitFromHeaders ¶ added in v0.71.0
func RateLimitFromHeaders(h http.Header, reqRemaining, reqLimit, tokRemaining, tokLimit, reset string, resetKind ResetKind) *RateLimitInfo
RateLimitFromHeaders builds a *RateLimitInfo from a provider's rate-limit headers using the given header names, or nil when none are present. It lets the Anthropic and OpenAI adapters share one parser across their Complete and stream paths so a traced (production) call surfaces the same rate-limit metadata a direct Complete call does (#605 / #617). Providers without standard rate-limit headers (Gemini, openai-compat) simply pass empty names and get nil.
type Request ¶
type Request struct {
Model string `json:"model"`
Messages []Message `json:"messages"`
Provider string `json:"provider,omitempty"`
Tools []ToolDefinition `json:"tools,omitempty"`
ToolChoice *ToolChoice `json:"tool_choice,omitempty"`
ResponseFormat *ResponseFormat `json:"response_format,omitempty"`
Temperature *float64 `json:"temperature,omitempty"`
TopP *float64 `json:"top_p,omitempty"`
MaxTokens *int `json:"max_tokens,omitempty"`
StopSequences []string `json:"stop_sequences,omitempty"`
ReasoningEffort string `json:"reasoning_effort,omitempty"`
Metadata map[string]string `json:"metadata,omitempty"`
ProviderOptions map[string]any `json:"provider_options,omitempty"`
TraceObservers []TraceObserver `json:"-"`
}
Request is the input for Complete() and Stream().
type RequestTimeoutError ¶
type RequestTimeoutError struct {
ProviderError
}
RequestTimeoutError indicates the provider timed out processing the request.
func (*RequestTimeoutError) Retryable ¶
func (e *RequestTimeoutError) Retryable() bool
type ResetKind ¶ added in v0.71.0
type ResetKind int
ResetKind selects how a rate-limit reset header value is interpreted.
type Response ¶
type Response struct {
ID string `json:"id"`
Model string `json:"model"`
Provider string `json:"provider"`
Message Message `json:"message"`
FinishReason FinishReason `json:"finish_reason"`
Usage Usage `json:"usage"`
Latency time.Duration `json:"latency"`
Raw json.RawMessage `json:"raw,omitempty"`
Warnings []Warning `json:"warnings,omitempty"`
RateLimit *RateLimitInfo `json:"rate_limit,omitempty"`
}
Response is the output of Complete().
func (Response) ToolCalls ¶
func (r Response) ToolCalls() []ToolCallData
ToolCalls returns tool calls from the response message.
type ResponseFormat ¶
type ResponseFormat struct {
Type string `json:"type"`
JSONSchema json.RawMessage `json:"json_schema,omitempty"`
Strict bool `json:"strict,omitempty"`
}
ResponseFormat specifies output format constraints.
type RetryMiddleware ¶
type RetryMiddleware struct {
// contains filtered or unexported fields
}
RetryMiddleware retries failed requests with exponential backoff when the error is retryable. Non-retryable and unknown errors are returned immediately.
func (*RetryMiddleware) WrapComplete ¶
func (rm *RetryMiddleware) WrapComplete(next CompleteHandler) CompleteHandler
WrapComplete implements the Middleware interface.
type RetryOption ¶
type RetryOption func(*RetryMiddleware)
RetryOption configures a RetryMiddleware.
func WithBaseDelay ¶
func WithBaseDelay(d time.Duration) RetryOption
WithBaseDelay sets the base delay for exponential backoff (default 1s). Actual delay is baseDelay * 2^attempt.
func WithMaxRetries ¶
func WithMaxRetries(n int) RetryOption
WithMaxRetries sets the maximum number of retry attempts (default 3).
type ServerError ¶
type ServerError struct {
ProviderError
}
ServerError indicates a server-side failure from the provider.
func (*ServerError) Retryable ¶
func (e *ServerError) Retryable() bool
type StreamAccumulator ¶
type StreamAccumulator struct {
// contains filtered or unexported fields
}
StreamAccumulator collects streaming events into a complete Response.
func NewStreamAccumulator ¶
func NewStreamAccumulator() *StreamAccumulator
NewStreamAccumulator creates a new StreamAccumulator ready to process events.
func (*StreamAccumulator) Process ¶
func (a *StreamAccumulator) Process(event StreamEvent)
Process handles a single StreamEvent, updating the accumulator state.
func (*StreamAccumulator) Response ¶
func (a *StreamAccumulator) Response() Response
Response builds a complete Response from the accumulated events.
type StreamError ¶
type StreamError struct {
SDKError
}
StreamError indicates a failure during response streaming.
func (*StreamError) Retryable ¶
func (e *StreamError) Retryable() bool
type StreamEvent ¶
type StreamEvent struct {
Type StreamEventType `json:"type"`
Delta string `json:"delta,omitempty"`
TextID string `json:"text_id,omitempty"`
ReasoningDelta string `json:"reasoning_delta,omitempty"`
ReasoningSignature string `json:"reasoning_signature,omitempty"`
ToolCall *ToolCallData `json:"tool_call,omitempty"`
FinishReason *FinishReason `json:"finish_reason,omitempty"`
Usage *Usage `json:"usage,omitempty"`
FullResponse *Response `json:"full_response,omitempty"`
Err error `json:"-"`
Raw json.RawMessage `json:"raw,omitempty"`
// RequestRaw is the verbatim wire body the adapter sent, carried on the
// EventRequestSent event that EmitRequestSent emits below. EventStreamStart
// only ever sees it via TraceBuilder's fallback, never set by an adapter.
// The normalized Request records what tracker asked for; this records what
// actually went out after provider-specific translation and ProviderOptions
// merging — which is what a post-hoc reader needs to reproduce the call.
RequestRaw json.RawMessage `json:"request_raw,omitempty"`
}
StreamEvent represents a single event in a streaming response.
type StreamEventType ¶
type StreamEventType string
StreamEventType discriminates the kind of streaming event.
const ( EventStreamStart StreamEventType = "stream_start" EventTextStart StreamEventType = "text_start" EventTextDelta StreamEventType = "text_delta" EventTextEnd StreamEventType = "text_end" EventReasoningStart StreamEventType = "reasoning_start" EventReasoningDelta StreamEventType = "reasoning_delta" EventReasoningSignature StreamEventType = "reasoning_signature" EventReasoningEnd StreamEventType = "reasoning_end" EventRedactedThinking StreamEventType = "redacted_thinking" EventToolCallStart StreamEventType = "tool_call_start" EventToolCallDelta StreamEventType = "tool_call_delta" EventToolCallEnd StreamEventType = "tool_call_end" EventFinish StreamEventType = "finish" EventError StreamEventType = "error" EventProviderEvent StreamEventType = "provider_event" // EventRequestSent carries the verbatim wire body in RequestRaw. Adapters // emit it once, immediately after building the body and before the HTTP // call, because that is the only place the body is in scope — the // provider's own stream-start arrives several layers deeper, inside SSE // parsing. TraceBuilder folds it into the TraceRequestStart it emits, and // StreamAccumulator ignores it, so it adds no event to either output. EventRequestSent StreamEventType = "request_sent" )
type StreamIdleGuard ¶ added in v0.64.0
type StreamIdleGuard struct {
// contains filtered or unexported fields
}
StreamIdleGuard arms an idle timer over an SSE read loop. It is created with the CancelFunc of an internal stream context (derived from the caller context): when no bytes arrive within the timeout the timer fires that cancel, unblocking the in-flight socket read. Each byte of progress calls Reset to re-arm the timer. Fired reports whether the timer — rather than the caller — triggered the cancel, letting the classifier distinguish an idle hang (surface a retryable error) from a genuine caller/shutdown cancel (stop cleanly).
A non-positive timeout disables the guard (Reset/Stop are no-ops, Fired is always false).
func NewStreamIdleGuard ¶ added in v0.64.0
func NewStreamIdleGuard(timeout time.Duration, cancel context.CancelFunc) *StreamIdleGuard
NewStreamIdleGuard arms an idle timer that invokes cancel after timeout of byte-silence. A non-positive timeout returns a disabled guard.
func (*StreamIdleGuard) Fired ¶ added in v0.64.0
func (g *StreamIdleGuard) Fired() bool
Fired reports whether the idle timer triggered the cancel.
func (*StreamIdleGuard) Reset ¶ added in v0.64.0
func (g *StreamIdleGuard) Reset()
Reset re-arms the idle timer after byte progress on the socket. It is a no-op once the timer has fired or when the guard is disabled.
func (*StreamIdleGuard) Stop ¶ added in v0.64.0
func (g *StreamIdleGuard) Stop()
Stop halts the idle timer. Call it when the read loop ends.
type ThinkingData ¶
type ThinkingData struct {
Text string `json:"text"`
Signature string `json:"signature,omitempty"`
Redacted bool `json:"redacted,omitempty"`
}
ThinkingData holds model reasoning content.
type TokenTracker ¶
type TokenTracker struct {
// contains filtered or unexported fields
}
TokenTracker is a middleware that accumulates token usage per (provider, model) pair. It implements Middleware and can be passed to NewClient via WithMiddleware.
func NewTokenTracker ¶
func NewTokenTracker() *TokenTracker
NewTokenTracker creates a new, zeroed token tracking middleware.
func (*TokenTracker) AddUsage ¶
func (t *TokenTracker) AddUsage(provider string, usage Usage, model ...string)
AddUsage manually adds token usage for a provider. Used by backends that bypass the LLM client middleware (e.g., claude-code subprocess backend). The model parameter is optional; pass "" to leave the provider's model unchanged. Model strings are normalized through the catalog so versioned provider-returned IDs resolve to canonical pricing entries — matching WrapComplete's behavior.
func (*TokenTracker) AllProviderUsage ¶
func (t *TokenTracker) AllProviderUsage() map[string]Usage
AllProviderUsage returns a copy of the accumulated usage aggregated per provider (summed across each provider's models).
func (*TokenTracker) CostByProvider ¶
func (t *TokenTracker) CostByProvider(resolve ModelResolver) map[string]ProviderCost
CostByProvider returns a per-provider cost rollup. Each (provider, model) bucket is priced at its own model's rate and the results are summed per provider, so a provider that ran multiple models is priced correctly and the total no longer depends on call ordering (#527). The caller-supplied resolver only prices buckets whose model was never observed ("" — e.g. subscription backends); a nil resolver treats those as $0.
func (*TokenTracker) ModelForProvider ¶
func (t *TokenTracker) ModelForProvider(provider string) string
ModelForProvider returns the last-seen model for a provider. Returns "" if unknown. NOTE: a provider that ran multiple models has multiple buckets; this reports only the most-recently-observed one and is retained for back-compat (fallback resolution / display). For accurate multi-model pricing use CostByProvider, which prices each model bucket at its own rate.
func (*TokenTracker) ObservedModelResolver ¶
func (t *TokenTracker) ObservedModelResolver(fallback string) ModelResolver
ObservedModelResolver returns a ModelResolver that uses the tracker's last-observed per-provider model, falling back to the provided fallback model for providers where no model was observed. Since CostByProvider now prices each (provider, model) bucket at its own rate, this resolver only supplies a price for buckets whose model is unknown ("" — e.g. subscription-auth backends). For a provider that ran multiple models it reports the last one.
func (*TokenTracker) ProviderUsage ¶
func (t *TokenTracker) ProviderUsage(provider string) Usage
ProviderUsage returns the accumulated usage for a specific provider, summed across every model that provider ran. Returns a zero Usage if the provider has not been seen.
func (*TokenTracker) Providers ¶
func (t *TokenTracker) Providers() []string
Providers returns a sorted, de-duplicated slice of provider names that have recorded usage (a provider that ran multiple models appears once).
func (*TokenTracker) TotalCostUSD ¶
func (t *TokenTracker) TotalCostUSD(resolve ModelResolver) float64
TotalCostUSD sums CostByProvider to a single dollar figure using the same resolver.
func (*TokenTracker) TotalUsage ¶
func (t *TokenTracker) TotalUsage() Usage
TotalUsage returns accumulated usage summed across all providers and models.
func (*TokenTracker) WrapComplete ¶
func (t *TokenTracker) WrapComplete(next CompleteHandler) CompleteHandler
WrapComplete implements the Middleware interface. It calls the next handler and, on success, adds the response's token usage to the per-(provider, model) accumulator.
type ToolCallData ¶
type ToolCallData struct {
ID string `json:"id"`
Name string `json:"name"`
Arguments json.RawMessage `json:"arguments"`
ThoughtSigData string `json:"thought_signature,omitempty"`
}
ToolCallData represents a model-initiated tool invocation.
type ToolChoice ¶
ToolChoice controls how the model uses tools.
func ToolChoiceAuto ¶
func ToolChoiceAuto() ToolChoice
ToolChoiceAuto returns a ToolChoice that lets the model decide whether to call tools.
func ToolChoiceNamed ¶
func ToolChoiceNamed(name string) ToolChoice
ToolChoiceNamed creates a ToolChoice that forces the model to call a specific tool.
func ToolChoiceNone ¶
func ToolChoiceNone() ToolChoice
ToolChoiceNone returns a ToolChoice that prevents the model from calling tools.
func ToolChoiceRequired ¶
func ToolChoiceRequired() ToolChoice
ToolChoiceRequired returns a ToolChoice that forces the model to call a tool.
type ToolDefinition ¶
type ToolDefinition struct {
Name string `json:"name"`
Description string `json:"description"`
Parameters json.RawMessage `json:"parameters"`
}
ToolDefinition defines a tool the model can call.
type ToolResultData ¶
type ToolResultData struct {
ToolCallID string `json:"tool_call_id"`
Name string `json:"name,omitempty"`
Content string `json:"content"`
IsError bool `json:"is_error"`
ImageData []byte `json:"image_data,omitempty"`
ImageMediaType string `json:"image_media_type,omitempty"`
}
ToolResultData represents the result of executing a tool call.
type TraceBuilder ¶
type TraceBuilder struct {
// contains filtered or unexported fields
}
TraceBuilder converts streaming events into normalized trace events.
func NewTraceBuilder ¶
func NewTraceBuilder(opts TraceOptions) *TraceBuilder
NewTraceBuilder creates a trace builder for one request.
func (*TraceBuilder) Events ¶
func (b *TraceBuilder) Events() []TraceEvent
Events returns the trace events emitted so far.
func (*TraceBuilder) Process ¶
func (b *TraceBuilder) Process(evt StreamEvent)
Process ingests one stream event and emits any corresponding trace events.
type TraceEvent ¶
type TraceEvent struct {
Kind TraceKind
Provider string
Model string
ToolName string
Preview string
ProviderEvent string
RawPreview string
FinishReason string
Usage Usage
// CallID groups every event belonging to one LLM request. Two log paths
// record LLM activity — the agent session re-emits trace events as agent
// llm_* events, and the client-level writer catches calls no session sees
// (the autopilot interviewer, for one) — so one call can legitimately
// appear twice. Both paths are kept because their coverage differs; this
// is what lets a reader collapse the overlap rather than double-count
// usage when aggregating.
CallID string
// RequestRaw is the verbatim wire body, set on TraceRequestStart. The
// normalized Request says what tracker asked for; this says what actually
// went out after provider translation and ProviderOptions merging.
RequestRaw json.RawMessage
// ToolArguments is the untruncated tool-call arguments, set on
// TraceToolPrepare. Preview holds the same value clipped to 80 chars,
// which is enough to render but not enough to reconstruct the call.
ToolArguments json.RawMessage
// ProviderRaw is the untruncated provider chunk, set on TraceProviderRaw
// (verbose only). RawPreview holds the clipped form.
ProviderRaw json.RawMessage
// SessionOwned is true when the originating request carried
// request-level TraceObservers — in tracker only the agent session
// registers those, and it re-emits every trace event as an agent
// llm_* event. Activity-log writers listening at the client level
// skip SessionOwned events to avoid logging the same stream twice
// (issue #354); non-session calls (e.g. the autopilot interviewer)
// stay SessionOwned=false so the trace path remains their only,
// and therefore kept, log surface.
SessionOwned bool
}
TraceEvent is a normalized event for rendering live LLM activity.
Preview and RawPreview are display fields: previewText collapses newlines and clips to tracePreviewLimit (80 chars) so a TUI line stays a TUI line. The Raw* / Arguments fields alongside them carry the same content untruncated, for the activity log. Text and reasoning deltas are not clipped in either place — preserveSpacingText keeps them whole because coalescing clipped deltas would corrupt the reconstructed text.
type TraceLogger ¶
type TraceLogger struct {
// contains filtered or unexported fields
}
TraceLogger writes trace events to an io.Writer. Sequential text and reasoning deltas are accumulated and flushed as a single summary line when the stream finishes or a different event kind arrives.
func NewTraceLogger ¶
func NewTraceLogger(w io.Writer, opts TraceLoggerOptions) *TraceLogger
NewTraceLogger creates a console trace logger.
func (*TraceLogger) Flush ¶
func (l *TraceLogger) Flush()
Flush forces any pending batched output to be written. Callers should invoke this when a stream terminates without a normal finish event (e.g. on error) to avoid losing buffered trace output.
func (*TraceLogger) HandleTraceEvent ¶
func (l *TraceLogger) HandleTraceEvent(evt TraceEvent)
HandleTraceEvent implements TraceObserver.
type TraceLoggerOptions ¶
type TraceLoggerOptions struct {
Verbose bool
}
TraceLoggerOptions configure console trace logging.
type TraceObserver ¶
type TraceObserver interface {
HandleTraceEvent(evt TraceEvent)
}
TraceObserver receives normalized LLM trace events.
type TraceObserverFunc ¶
type TraceObserverFunc func(evt TraceEvent)
TraceObserverFunc adapts a function into a TraceObserver.
func (TraceObserverFunc) HandleTraceEvent ¶
func (f TraceObserverFunc) HandleTraceEvent(evt TraceEvent)
HandleTraceEvent implements TraceObserver.
type TraceOptions ¶
type TraceOptions struct {
Provider string
Model string
Verbose bool
// CallID identifies the one request this builder traces. It is stamped
// onto every event the builder emits so a reader can group them, and —
// more importantly — collapse the overlap between the two log paths that
// both record LLM activity (see TraceEvent.CallID).
CallID string
}
TraceOptions configure trace building behavior.
type Usage ¶
type Usage struct {
InputTokens int `json:"input_tokens"`
OutputTokens int `json:"output_tokens"`
TotalTokens int `json:"total_tokens"`
ReasoningTokens *int `json:"reasoning_tokens,omitempty"`
CacheReadTokens *int `json:"cache_read_tokens,omitempty"`
CacheWriteTokens *int `json:"cache_write_tokens,omitempty"`
EstimatedCost float64 `json:"estimated_cost,omitempty"`
Raw any `json:"raw,omitempty"`
}
Usage tracks token consumption.
The field names read as provider-neutral but the accounting rules are not obvious, and providers disagree about what nests inside what. Adapters are responsible for normalizing into the two invariants below; EstimateCost depends on them, so getting one wrong misprices every run on that provider.
InputTokens EXCLUDES cached tokens. CacheReadTokens and CacheWriteTokens are separate additive buckets, so InputTokens + CacheReadTokens + CacheWriteTokens is the full prompt. Anthropic reports this shape natively. OpenAI and Gemini do not — their prompt counts INCLUDE the cached portion, so those adapters must subtract it out. Wiring their cached count straight into CacheReadTokens double-counts the prompt. OutputTokens INCLUDES reasoning tokens. ReasoningTokens is an informational subset, never priced separately — pricing reasoning on top of OutputTokens would bill it twice. OpenAI reports this shape natively. Gemini does not: its candidate count excludes thinking tokens that it nonetheless bills, so that adapter must add them in.
func (Usage) Finalize ¶ added in v0.69.0
Finalize derives TotalTokens as the priced aggregate — fresh input plus output — and returns the normalized Usage. Cache-read tokens are deliberately excluded, so identical normalized usage yields identical budget behavior regardless of provider or cache state (SIFT-SUB-09-01). Reasoning already lives inside OutputTokens, so it is not added again. Every provider translator funnels through this helper instead of trusting the provider-reported total, which variously folds cached and reasoning tokens into its count.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
ABOUTME: Anthropic Messages API adapter implementing the ProviderAdapter interface.
|
ABOUTME: Anthropic Messages API adapter implementing the ProviderAdapter interface. |
|
ABOUTME: Google Gemini API adapter implementing the ProviderAdapter interface.
|
ABOUTME: Google Gemini API adapter implementing the ProviderAdapter interface. |
|
ABOUTME: OpenAI Responses API adapter implementing the ProviderAdapter interface.
|
ABOUTME: OpenAI Responses API adapter implementing the ProviderAdapter interface. |
|
ABOUTME: OpenAI Chat Completions compatible adapter implementing the ProviderAdapter interface.
|
ABOUTME: OpenAI Chat Completions compatible adapter implementing the ProviderAdapter interface. |