providers

package
v1.6.5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 29, 2026 License: Apache-2.0 Imports: 18 Imported by: 0

Documentation

Index

Constants

View Source
const (
	FinishReasonStop     = "stop"
	FinishReasonLength   = "length"
	FinishReasonToolCall = "tool_calls"
	FinishReasonError    = "error"
	FinishReasonSafety   = "safety"
)

Variables

This section is empty.

Functions

func ConvertMessagesToOpenAI added in v1.5.1

func ConvertMessagesToOpenAI(messages []Message) []openaiMessage

ConvertMessagesToOpenAI converts generic Message into native OpenAI format (string content or parts) Used internally by OpenAI provider

func ConvertTools added in v1.5.1

func ConvertTools(tools []Tool) []openaiTool

ConvertTools converts generic Tool into OpenAI format; shared by all OpenAI-compatible providers

func GetDefaultURL added in v1.5.2

func GetDefaultURL(name string) string

GetDefaultURL returns the default base URL for a provider. Returns empty string if the provider is not registered.

func GetRetryAfter added in v1.5.3

func GetRetryAfter(err error) int

func IsAuthError added in v1.5.3

func IsAuthError(err error) bool

func IsBuiltinRegistered added in v1.5.2

func IsBuiltinRegistered(name string) bool

IsBuiltinRegistered checks if a provider is registered as a builtin.

func IsContextOverflowError added in v1.6.0

func IsContextOverflowError(err error) bool

IsContextOverflowError reports whether the error indicates the input exceeded the model's context window. Callers can use this to decide whether to truncate messages and retry.

func IsModelError added in v1.5.3

func IsModelError(err error) bool

func IsNetworkError added in v1.5.3

func IsNetworkError(err error) bool

func IsOverloadedError added in v1.6.3

func IsOverloadedError(err error) bool

func IsRateLimitError added in v1.5.3

func IsRateLimitError(err error) bool

func IsRetryableError added in v1.5.3

func IsRetryableError(err error) bool

func IsValidationError added in v1.5.3

func IsValidationError(err error) bool

func LevelToBudget added in v1.6.0

func LevelToBudget(level string) int

LevelToBudget returns the default thinking token budget for a reasoning level. Returns 0 if the level is unknown or empty.

func ListBuiltins added in v1.5.2

func ListBuiltins() []string

ListBuiltins returns a list of all registered builtin provider names.

func NormalizeFinishReason added in v1.5.7

func NormalizeFinishReason(raw string) string

NormalizeFinishReason maps provider-specific stop reasons to canonical constants. Unknown values pass through unchanged.

func NormalizeToolCallID added in v1.6.0

func NormalizeToolCallID(id string) string

NormalizeToolCallID sanitizes a tool call ID for cross-provider compatibility. Keeps only [a-zA-Z0-9_-] and truncates to 64 characters (Anthropic limit). Already-compliant IDs pass through unchanged.

func RegisterBuiltin added in v1.5.2

func RegisterBuiltin(name string, factory BuiltinFactory, defaultURL string)

RegisterBuiltin registers a builtin provider with its factory and default URL. This should be called in each provider's init() function.

func ResolveBudgetTokens added in v1.6.0

func ResolveBudgetTokens(thinking *ThinkingConfig) *int

ResolveBudgetTokens returns a thinking budget: explicit BudgetTokens if set, otherwise derived from Level via LevelToBudget. Returns nil if neither is set or the level is unknown.

func WrapError added in v1.5.3

func WrapError(err error, provider string) error

WrapError wraps any error into LiteLLMError (and fills Provider if already wrapped).

Types

type AnthropicProvider

type AnthropicProvider struct {
	*BaseProvider
}

AnthropicProvider implements Anthropic Claude API integration

func NewAnthropic

func NewAnthropic(config ProviderConfig) *AnthropicProvider

NewAnthropic creates a new Anthropic provider

func (*AnthropicProvider) Chat

func (p *AnthropicProvider) Chat(ctx context.Context, req *Request) (*Response, error)

func (*AnthropicProvider) ListModels added in v1.5.5

func (p *AnthropicProvider) ListModels(ctx context.Context) ([]ModelInfo, error)

ListModels returns available models for Anthropic.

func (*AnthropicProvider) Stream

func (p *AnthropicProvider) Stream(ctx context.Context, req *Request) (StreamReader, error)

type BaseProvider

type BaseProvider struct {
	// contains filtered or unexported fields
}

BaseProvider provides common functionality for all providers

func NewBaseProvider

func NewBaseProvider(name string, config ProviderConfig) *BaseProvider

NewBaseProvider creates a new base provider with resilience

func (*BaseProvider) Config

func (p *BaseProvider) Config() ProviderConfig

func (*BaseProvider) HTTPClient

func (p *BaseProvider) HTTPClient() HTTPDoer

func (*BaseProvider) Name

func (p *BaseProvider) Name() string

func (*BaseProvider) NotifyPayload added in v1.6.0

func (p *BaseProvider) NotifyPayload(req *Request, payload []byte)

NotifyPayload calls the request's OnPayload hook if set.

func (*BaseProvider) ResilienceConfig

func (p *BaseProvider) ResilienceConfig() ResilienceConfig

func (*BaseProvider) ResolveAPIKey added in v1.5.8

func (p *BaseProvider) ResolveAPIKey(req *Request) string

ResolveAPIKey returns the per-request API key if set, otherwise the provider's default.

func (*BaseProvider) Validate

func (p *BaseProvider) Validate() error

func (*BaseProvider) ValidateExtra added in v1.5.4

func (p *BaseProvider) ValidateExtra(extra map[string]any, allowedKeys []string) error

ValidateExtra validates request-level extra parameters for a provider.

func (*BaseProvider) ValidateRequest

func (p *BaseProvider) ValidateRequest(req *Request) error

ValidateRequest validates common request parameters This should be called by all provider implementations before processing requests

type BedrockProvider added in v1.5.2

type BedrockProvider struct {
	*BaseProvider
	// contains filtered or unexported fields
}

func NewBedrock added in v1.5.2

func NewBedrock(config ProviderConfig) *BedrockProvider

func (*BedrockProvider) Chat added in v1.5.2

func (p *BedrockProvider) Chat(ctx context.Context, req *Request) (*Response, error)

func (*BedrockProvider) ListModels added in v1.5.5

func (p *BedrockProvider) ListModels(ctx context.Context) ([]ModelInfo, error)

ListModels returns available foundation models for Bedrock.

func (*BedrockProvider) Stream added in v1.5.2

func (p *BedrockProvider) Stream(ctx context.Context, req *Request) (StreamReader, error)

func (*BedrockProvider) Validate added in v1.5.2

func (p *BedrockProvider) Validate() error

type BuiltinFactory added in v1.5.2

type BuiltinFactory func(ProviderConfig) Provider

BuiltinFactory is a function that creates a Provider instance

func GetBuiltin added in v1.5.2

func GetBuiltin(name string) (BuiltinFactory, bool)

GetBuiltin returns the factory function for a builtin provider. Returns nil and false if the provider is not registered.

type CacheControl

type CacheControl struct {
	Type string `json:"type"`          // "ephemeral" or "persistent"
	TTL  *int   `json:"ttl,omitempty"` // TTL in seconds (optional)
}

CacheControl defines prompt caching behavior for providers.

type Compat added in v1.6.0

type Compat struct {
	// ProviderName is used in errors, Response.Provider, etc.
	ProviderName string

	// DefaultBaseURL is the fallback when ProviderConfig.BaseURL is empty.
	DefaultBaseURL string

	// EndpointPath is appended to BaseURL for chat completions.
	// Default: "/chat/completions"
	EndpointPath string

	// ExtraHeaders are sent with every request (e.g. OpenRouter: HTTP-Referer).
	ExtraHeaders map[string]string

	// StreamHeaders are sent only for stream requests.
	StreamHeaders map[string]string

	// MaxTokensField overrides the JSON key for max tokens.  Default: "max_tokens".
	MaxTokensField string

	// MaxStopSequences limits stop sequences sent.  0 = unlimited.
	MaxStopSequences int

	// OmitStop suppresses stop sequences entirely.
	OmitStop bool

	// SupportsJSONSchema enables full json_schema in response_format.
	SupportsJSONSchema bool

	// JSONSchemaToPrompt injects the JSON schema into the last user message
	// instead of using response_format.  Used by GLM.
	JSONSchemaToPrompt bool

	// ThinkingMapper converts ThinkingConfig into provider-specific request
	// body fields.  Returns nil to skip.  If nil, uses default
	// {"thinking":{"type":"enabled/disabled"}} format.
	ThinkingMapper func(thinking *ThinkingConfig, model string) map[string]any

	// ResponseFormatMapper converts ResponseFormat to the provider-specific
	// value for the "response_format" key.  Return nil to omit.
	// If the function itself is nil, defaults to json_object-only support.
	ResponseFormatMapper func(rf *ResponseFormat) any

	// CustomMessageConverter replaces the default ConvertMessages.
	// Must return a JSON-serializable slice.
	CustomMessageConverter func(messages []Message) any

	// CustomToolConverter replaces the default ConvertTools.
	CustomToolConverter func(tools []Tool) any

	// CleanSchema recursively cleans JSON schemas for strict mode.
	CleanSchema func(schema any) any

	// ReasoningField is the preferred JSON key for reasoning in message/delta.
	// When set, it is probed first; remaining default fields are tried as fallback.
	// Default probe order: "reasoning_content", "reasoning", "reasoning_text".
	ReasoningField string

	// ReasoningCondition controls when reasoning is extracted.
	//   ""/"always"                    — whenever the field is non-empty
	//   "model_contains:<substring>"   — only when model name contains substring
	ReasoningCondition string

	// ContentAsInterface parses Message.Content as interface{} (string or array)
	// instead of plain string.  Used by OpenRouter.
	ContentAsInterface bool

	// ModelFromResponse takes the model name from response JSON.
	// When false, uses the request model.
	ModelFromResponse bool

	// HasCompletionTokenDetails looks for completion_tokens_details.reasoning_tokens.
	HasCompletionTokenDetails bool

	// HasCacheTokens looks for prompt_cache_hit_tokens / prompt_cache_miss_tokens.
	HasCacheTokens bool

	// DataPrefix is the SSE data line prefix.  Default: "data: ".
	DataPrefix string

	// SkipAPIKeyValidation skips the API key check.  Used by local
	// providers like Ollama that don't require authentication.
	SkipAPIKeyValidation bool
}

Compat configures how an OpenAI-compatible provider differs from the standard OpenAI Chat Completions API. Zero values select the most common defaults so that a minimal config works for simple providers.

type ErrorType added in v1.5.3

type ErrorType string

ErrorType categorizes errors.

const (
	ErrorTypeAuth            ErrorType = "auth"             // Auth/authorization errors
	ErrorTypeRateLimit       ErrorType = "rate_limit"       // Rate limit errors
	ErrorTypeNetwork         ErrorType = "network"          // Network connectivity errors
	ErrorTypeValidation      ErrorType = "validation"       // Request validation errors
	ErrorTypeProvider        ErrorType = "provider"         // Upstream provider errors
	ErrorTypeTimeout         ErrorType = "timeout"          // Timeout errors
	ErrorTypeQuota           ErrorType = "quota"            // Quota/billing errors
	ErrorTypeModel           ErrorType = "model"            // Model not found/unsupported errors
	ErrorTypeInternal        ErrorType = "internal"         // Internal library errors
	ErrorTypeContextOverflow ErrorType = "context_overflow" // Input exceeds model's context window
	ErrorTypeOverloaded      ErrorType = "overloaded"       // Service overloaded (e.g. Anthropic 529)
)

type FunctionCall

type FunctionCall struct {
	Name      string `json:"name"`
	Arguments string `json:"arguments"`
}

type FunctionDef

type FunctionDef struct {
	Name        string `json:"name"`
	Description string `json:"description"`
	Parameters  any    `json:"parameters"`
}

type GeminiProvider

type GeminiProvider struct {
	*BaseProvider
}

GeminiProvider implements Google Gemini API integration

func NewGemini

func NewGemini(config ProviderConfig) *GeminiProvider

NewGemini creates a new Gemini provider

func (*GeminiProvider) Chat

func (p *GeminiProvider) Chat(ctx context.Context, req *Request) (*Response, error)

func (*GeminiProvider) ListModels added in v1.5.5

func (p *GeminiProvider) ListModels(ctx context.Context) ([]ModelInfo, error)

ListModels returns available models for Gemini.

func (*GeminiProvider) Stream

func (p *GeminiProvider) Stream(ctx context.Context, req *Request) (StreamReader, error)

type HTTPDoer

type HTTPDoer interface {
	Do(req *http.Request) (*http.Response, error)
}

type JSONSchema

type JSONSchema struct {
	Name        string `json:"name"`
	Description string `json:"description,omitempty"`
	Schema      any    `json:"schema"`
	Strict      *bool  `json:"strict,omitempty"`
}

type LiteLLMError added in v1.5.3

type LiteLLMError struct {
	Type     ErrorType `json:"type"`
	Code     string    `json:"code,omitempty"`
	Message  string    `json:"message"`
	Provider string    `json:"provider,omitempty"`
	Model    string    `json:"model,omitempty"`
	Cause    error     `json:"-"` // Original error, not serialized.

	// HTTP details (if applicable).
	StatusCode int               `json:"status_code,omitempty"`
	Headers    map[string]string `json:"headers,omitempty"`

	// Retry hints.
	Retryable  bool `json:"retryable"`
	RetryAfter int  `json:"retry_after,omitempty"` // seconds
}

LiteLLMError is a structured error with categorization and retry hints.

func NewAuthError added in v1.5.3

func NewAuthError(provider, message string) *LiteLLMError

func NewError added in v1.5.3

func NewError(errorType ErrorType, message string) *LiteLLMError

func NewErrorWithCause added in v1.5.3

func NewErrorWithCause(errorType ErrorType, message string, cause error) *LiteLLMError

func NewHTTPError added in v1.5.3

func NewHTTPError(provider string, statusCode int, message string) *LiteLLMError

func NewModelError added in v1.5.3

func NewModelError(provider, model, message string) *LiteLLMError

func NewNetworkError added in v1.5.3

func NewNetworkError(provider, message string, cause error) *LiteLLMError

func NewProviderError added in v1.5.3

func NewProviderError(provider string, errorType ErrorType, message string) *LiteLLMError

func NewRateLimitError added in v1.5.3

func NewRateLimitError(provider, message string, retryAfter int) *LiteLLMError

func NewTimeoutError added in v1.5.3

func NewTimeoutError(provider, message string) *LiteLLMError

func NewValidationError added in v1.5.3

func NewValidationError(provider, message string) *LiteLLMError

func (*LiteLLMError) Error added in v1.5.3

func (e *LiteLLMError) Error() string

func (*LiteLLMError) Is added in v1.5.3

func (e *LiteLLMError) Is(target error) bool

func (*LiteLLMError) IsRetryable added in v1.5.3

func (e *LiteLLMError) IsRetryable() bool

func (*LiteLLMError) Unwrap added in v1.5.3

func (e *LiteLLMError) Unwrap() error

type Message

type Message struct {
	Role         string           `json:"role"`
	Content      string           `json:"content"`
	Contents     []MessageContent `json:"contents,omitempty"`
	ToolCalls    []ToolCall       `json:"tool_calls,omitempty"`
	ToolCallID   string           `json:"tool_call_id,omitempty"`
	IsError      bool             `json:"is_error,omitempty"` // tool result error flag (Anthropic)
	CacheControl *CacheControl    `json:"cache_control,omitempty"`
}

func PrepareMessages added in v1.6.0

func PrepareMessages(messages []Message) []Message

PrepareMessages preprocesses a message slice for API submission:

  • Sanitizes invalid UTF-8 (including surrogate codepoints) in content
  • Skips error assistant messages and their associated tool results
  • Normalizes tool call IDs for cross-provider compatibility
  • Inserts synthetic error tool results for orphaned tool calls (assistant tool_calls with no matching tool result before next turn)

The original slice is not modified; a new slice is returned.

type MessageContent added in v1.5.1

type MessageContent struct {
	Type        string           `json:"type"`
	Text        string           `json:"text,omitempty"`
	ImageURL    *MessageImageURL `json:"image_url,omitempty"`
	ToolName    string           `json:"tool_name,omitempty"` // tool_reference blocks
	Annotations []map[string]any `json:"annotations,omitempty"`
	Logprobs    []map[string]any `json:"logprobs,omitempty"`
}

type MessageImageURL added in v1.5.1

type MessageImageURL struct {
	URL    string `json:"url"`
	Detail string `json:"detail,omitempty"` // "auto", "low", or "high"
}

type ModelInfo

type ModelInfo struct {
	ID               string `json:"id"`
	Name             string `json:"name,omitempty"`
	Provider         string `json:"provider,omitempty"`
	Description      string `json:"description,omitempty"`
	ContextLength    int    `json:"context_length,omitempty"`
	InputTokenLimit  int    `json:"input_token_limit,omitempty"`
	OutputTokenLimit int    `json:"output_token_limit,omitempty"`
	Created          int64  `json:"created,omitempty"`
	OwnedBy          string `json:"owned_by,omitempty"`

	// Capability flags (best-effort, may be zero-valued for unknown models)
	SupportsTools    bool `json:"supports_tools,omitempty"`
	SupportsVision   bool `json:"supports_vision,omitempty"`
	SupportsThinking bool `json:"supports_thinking,omitempty"`
}

ModelInfo provides a minimal, provider-agnostic model descriptor. Fields are best-effort and may be empty if a provider does not supply them.

type ModelLister added in v1.5.5

type ModelLister interface {
	ListModels(ctx context.Context) ([]ModelInfo, error)
}

ModelLister is an optional interface implemented by providers that support listing models.

type OpenAICompatMessage added in v1.5.1

type OpenAICompatMessage struct {
	Role             string           `json:"role"`
	Content          interface{}      `json:"content,omitempty"` // string or array (needed by OpenRouter)
	ToolCalls        []openaiToolCall `json:"tool_calls,omitempty"`
	ToolCallID       string           `json:"tool_call_id,omitempty"`
	ReasoningContent string           `json:"reasoning_content,omitempty"` // used by GLM/Qwen
}

OpenAICompatMessage is a generic OpenAI-compatible message format Used by DeepSeek/GLM/Qwen etc.; Content stays interface{} to allow flexible formats

func ConvertMessages added in v1.5.1

func ConvertMessages(messages []Message) []OpenAICompatMessage

ConvertMessages converts generic Message into OpenAI-compatible format (Content as interface{}) For DeepSeek/GLM/Qwen/OpenRouter etc.

type OpenAICompatProvider added in v1.6.0

type OpenAICompatProvider struct {
	*BaseProvider
	// contains filtered or unexported fields
}

OpenAICompatProvider implements Provider for any OpenAI-compatible API.

func NewDeepSeek

func NewDeepSeek(config ProviderConfig) *OpenAICompatProvider

NewDeepSeek creates a new DeepSeek provider.

func NewGLM

func NewGLM(config ProviderConfig) *OpenAICompatProvider

NewGLM creates a new GLM/ZhiPu AI provider.

func NewGrok added in v1.6.1

func NewGrok(config ProviderConfig) *OpenAICompatProvider

NewGrok creates a new Grok (xAI) provider.

func NewOllama added in v1.6.3

func NewOllama(config ProviderConfig) *OpenAICompatProvider

NewOllama creates a new Ollama provider using the OpenAI-compatible endpoint.

func NewOpenAICompat added in v1.6.0

func NewOpenAICompat(config ProviderConfig, compat Compat) *OpenAICompatProvider

NewOpenAICompat creates a new OpenAI-compatible provider.

func NewOpenRouter

func NewOpenRouter(config ProviderConfig) *OpenAICompatProvider

NewOpenRouter creates a new OpenRouter provider.

func NewQwen

func NewQwen(config ProviderConfig) *OpenAICompatProvider

NewQwen creates a new Qwen provider using the OpenAI-compatible endpoint.

func (*OpenAICompatProvider) Chat added in v1.6.0

func (p *OpenAICompatProvider) Chat(ctx context.Context, req *Request) (*Response, error)

Chat sends a non-streaming chat completion request.

func (*OpenAICompatProvider) ListModels added in v1.6.0

func (p *OpenAICompatProvider) ListModels(ctx context.Context) ([]ModelInfo, error)

ListModels returns available models from the provider.

func (*OpenAICompatProvider) Stream added in v1.6.0

Stream sends a streaming chat completion request.

type OpenAIProvider

type OpenAIProvider struct {
	*BaseProvider
}

OpenAIProvider implements OpenAI API integration

func NewOpenAI

func NewOpenAI(config ProviderConfig) *OpenAIProvider

NewOpenAI creates a new OpenAI provider

func (*OpenAIProvider) Chat

func (p *OpenAIProvider) Chat(ctx context.Context, req *Request) (*Response, error)

func (*OpenAIProvider) ListModels added in v1.5.5

func (p *OpenAIProvider) ListModels(ctx context.Context) ([]ModelInfo, error)

ListModels returns available models for OpenAI.

func (*OpenAIProvider) Responses added in v1.5.4

func (p *OpenAIProvider) Responses(ctx context.Context, req *OpenAIResponsesRequest) (*Response, error)

Responses executes a Responses API request for OpenAI.

func (*OpenAIProvider) ResponsesStream added in v1.5.4

func (p *OpenAIProvider) ResponsesStream(ctx context.Context, req *OpenAIResponsesRequest) (StreamReader, error)

ResponsesStream executes a streaming Responses API request for OpenAI.

func (*OpenAIProvider) Stream

func (p *OpenAIProvider) Stream(ctx context.Context, req *Request) (StreamReader, error)

type OpenAIResponsesRequest added in v1.5.4

type OpenAIResponsesRequest struct {
	Model string `json:"model"`

	// Use standard messages to build input items for Responses API.
	Messages []Message `json:"messages"`

	// Output configuration
	MaxOutputTokens *int `json:"max_output_tokens,omitempty"`

	// Sampling parameters
	Temperature *float64 `json:"temperature,omitempty"`
	TopP        *float64 `json:"top_p,omitempty"`

	// Text format configuration (replaces response_format in Responses API)
	ResponseFormat *ResponseFormat `json:"response_format,omitempty"`

	// Tool configuration
	Tools      []Tool `json:"tools,omitempty"`
	ToolChoice any    `json:"tool_choice,omitempty"`

	// Reasoning configuration (for gpt-5 and o-series models)
	ReasoningEffort  string `json:"reasoning_effort,omitempty"`  // none, low, medium, high
	ReasoningSummary string `json:"reasoning_summary,omitempty"` // auto, concise, detailed

	// Unified thinking control (optional). If set, it maps to reasoning fields.
	Thinking *ThinkingConfig `json:"thinking,omitempty"`

	// APIKey overrides the provider-level API key for this single request.
	APIKey string `json:"-"`

	// OnPayload is called with the serialized JSON body before sending
	// the HTTP request. Useful for debugging and logging API calls.
	OnPayload func(provider string, payload []byte) `json:"-"`
}

OpenAIResponsesRequest is a dedicated request type for OpenAI Responses API. It avoids polluting the generic Request with OpenAI-only fields.

type Provider

type Provider interface {
	Name() string
	Validate() error
	Chat(ctx context.Context, req *Request) (*Response, error)
	Stream(ctx context.Context, req *Request) (StreamReader, error)
}

Provider is the core interface that all LLM providers implement.

type ProviderConfig

type ProviderConfig struct {
	APIKey     string           `json:"api_key"`
	BaseURL    string           `json:"base_url,omitempty"`
	Timeout    time.Duration    `json:"timeout,omitempty"`
	Extra      map[string]any   `json:"extra,omitempty"`
	Resilience ResilienceConfig `json:"resilience,omitempty"`
	HTTPClient HTTPDoer         `json:"-"`
}

ProviderConfig holds configuration for a provider

type Request

type Request struct {
	Model          string          `json:"model"`
	Messages       []Message       `json:"messages"`
	MaxTokens      *int            `json:"max_tokens,omitempty"`
	Temperature    *float64        `json:"temperature,omitempty"`
	TopP           *float64        `json:"top_p,omitempty"`
	Tools          []Tool          `json:"tools,omitempty"`
	ToolChoice     any             `json:"tool_choice,omitempty"`
	ResponseFormat *ResponseFormat `json:"response_format,omitempty"`
	Stop           []string        `json:"stop,omitempty"`
	Thinking       *ThinkingConfig `json:"thinking,omitempty"`

	// APIKey overrides the provider-level API key for this single request.
	// When empty, the provider's default key is used.
	// Enables key rotation, OAuth short-lived tokens, and multi-tenant scenarios.
	APIKey string `json:"-"`

	// Provider-specific extensions
	Extra map[string]any `json:"extra,omitempty"`

	// OnPayload is called with the serialized JSON body before sending
	// the HTTP request. Useful for debugging and logging API calls.
	OnPayload func(provider string, payload []byte) `json:"-"`
}

type ResilienceConfig

type ResilienceConfig struct {
	MaxRetries     int           `json:"max_retries"`
	InitialDelay   time.Duration `json:"initial_delay"`
	MaxDelay       time.Duration `json:"max_delay"`
	Multiplier     float64       `json:"multiplier"`
	Jitter         bool          `json:"jitter"`
	RequestTimeout time.Duration `json:"request_timeout"`
	ConnectTimeout time.Duration `json:"connect_timeout"`
}

ResilienceConfig holds network resilience configuration for providers

func DefaultResilienceConfig

func DefaultResilienceConfig() ResilienceConfig

DefaultResilienceConfig returns default resilience configuration for providers

func ResolveResilienceConfig added in v1.5.6

func ResolveResilienceConfig(config ResilienceConfig) ResilienceConfig

ResolveResilienceConfig applies defaults when config is empty.

type Response

type Response struct {
	Content          string           `json:"content"`
	Contents         []MessageContent `json:"contents,omitempty"`
	ToolCalls        []ToolCall       `json:"tool_calls,omitempty"`
	Usage            Usage            `json:"usage"`
	Model            string           `json:"model"`
	Provider         string           `json:"provider"`
	FinishReason     string           `json:"finish_reason,omitempty"`
	ReasoningContent string           `json:"reasoning_content,omitempty"`

	// Extra captures provider-specific response fields not covered by the
	// standard schema (e.g. logprobs, annotations, system_fingerprint).
	Extra map[string]any `json:"extra,omitempty"`
}

type ResponseFormat

type ResponseFormat struct {
	Type       string      `json:"type"`
	JSONSchema *JSONSchema `json:"json_schema,omitempty"`
}

type StreamChunk

type StreamChunk struct {
	Type             string         `json:"type"`
	Content          string         `json:"content,omitempty"`
	ContentIndex     *int           `json:"content_index,omitempty"`
	OutputIndex      *int           `json:"output_index,omitempty"`
	ItemID           string         `json:"item_id,omitempty"`
	ToolCallDelta    *ToolCallDelta `json:"tool_call_delta,omitempty"`
	FinishReason     string         `json:"finish_reason,omitempty"`
	Model            string         `json:"model,omitempty"`
	Provider         string         `json:"provider"`
	Done             bool           `json:"done"`
	ReasoningContent string         `json:"reasoning_content,omitempty"`
	ReasoningDone    bool           `json:"reasoning_done,omitempty"`
	Usage            *Usage         `json:"usage,omitempty"`
}

type StreamReader

type StreamReader interface {
	Next() (*StreamChunk, error)
	Close() error
}

StreamReader reads streaming chunks from a provider.

type ThinkingConfig added in v1.5.4

type ThinkingConfig struct {
	Type         string `json:"type"`            // "enabled" or "disabled"
	Level        string `json:"level,omitempty"` // "low", "medium", "high" — provider translates to API-specific param
	BudgetTokens *int   `json:"budget_tokens,omitempty"`
}

type Tool

type Tool struct {
	Type         string      `json:"type"`
	Function     FunctionDef `json:"function"`
	DeferLoading bool        `json:"defer_loading,omitempty"`
}

type ToolCall

type ToolCall struct {
	ID       string       `json:"id"`
	Type     string       `json:"type"`
	Function FunctionCall `json:"function"`
}

type ToolCallDelta

type ToolCallDelta struct {
	Index          int    `json:"index"`
	ID             string `json:"id,omitempty"`
	Type           string `json:"type,omitempty"`
	FunctionName   string `json:"function_name,omitempty"`
	ArgumentsDelta string `json:"arguments_delta,omitempty"`
	OutputIndex    *int   `json:"output_index,omitempty"`
	ItemID         string `json:"item_id,omitempty"`
}

type Usage

type Usage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`
	ReasoningTokens  int `json:"reasoning_tokens,omitempty"`

	// Cache-related token statistics
	CacheCreationInputTokens int `json:"cache_creation_input_tokens,omitempty"` // Tokens written to cache
	CacheReadInputTokens     int `json:"cache_read_input_tokens,omitempty"`     // Tokens read from cache
}

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL