Documentation
¶
Overview ¶
Package llm provides a generic abstraction layer for interacting with Large Language Model providers.
Index ¶
- Variables
- func ApplyOptions(cfg *Config, opts ...Option)
- func ApplyRequestOptions(req *ChatRequest, opts ...RequestOption)
- func DefaultAPIKeyEnvVar(providerType ProviderType) string
- func DefaultBaseURL(providerType ProviderType) string
- func GetAPIKeyFromEnv(providerType ProviderType) string
- func IsAuthError(err error) bool
- func IsRateLimitError(err error) bool
- func IsRetryable(err error) bool
- func RegisterProvider(providerType ProviderType, factory ProviderFactory)
- func WrapError(provider string, err error) error
- type APIError
- type ChatRequest
- type ChatResponse
- type Config
- type HTTPClient
- type Message
- type Option
- type Provider
- type ProviderFactory
- type ProviderType
- type RequestOption
- type Role
- type StreamEvent
- type ThinkingEffort
- type ThinkingRequest
- type Usage
Constants ¶
This section is empty.
Variables ¶
var ( // ErrNoAPIKey indicates that no API key was provided. ErrNoAPIKey = errors.New("no API key provided") // ErrInvalidProvider indicates an unknown or invalid provider type. ErrInvalidProvider = errors.New("invalid provider") // ErrRateLimited indicates the API rate limit has been exceeded. ErrRateLimited = errors.New("rate limited") // ErrContextTooLong indicates the input exceeds the model's context limit. ErrContextTooLong = errors.New("context length exceeded") // ErrModelNotFound indicates the requested model does not exist. ErrModelNotFound = errors.New("model not found") // ErrInvalidRequest indicates a malformed request. ErrInvalidRequest = errors.New("invalid request") ErrUnauthorized = errors.New("unauthorized") // ErrServerError indicates a server-side error. ErrServerError = errors.New("server error") // ErrTimeout indicates the request timed out. ErrTimeout = errors.New("request timeout") // ErrStreamClosed indicates the stream was unexpectedly closed. ErrStreamClosed = errors.New("stream closed unexpectedly") )
Sentinel errors for common LLM error conditions.
Functions ¶
func ApplyOptions ¶
ApplyOptions applies the given options to a Config.
func ApplyRequestOptions ¶
func ApplyRequestOptions(req *ChatRequest, opts ...RequestOption)
ApplyRequestOptions applies the given options to a ChatRequest.
func DefaultAPIKeyEnvVar ¶
func DefaultAPIKeyEnvVar(providerType ProviderType) string
DefaultAPIKeyEnvVar returns the standard environment variable name for the API key of a given provider.
func DefaultBaseURL ¶
func DefaultBaseURL(providerType ProviderType) string
DefaultBaseURL returns the default API endpoint URL for a given provider.
func GetAPIKeyFromEnv ¶
func GetAPIKeyFromEnv(providerType ProviderType) string
GetAPIKeyFromEnv attempts to retrieve the API key for a provider from environment variables.
func IsAuthError ¶
IsAuthError returns true if the error is an authentication error.
func IsRateLimitError ¶
IsRateLimitError returns true if the error is a rate limit error.
func IsRetryable ¶
IsRetryable returns true if the error is retryable.
func RegisterProvider ¶
func RegisterProvider(providerType ProviderType, factory ProviderFactory)
RegisterProvider registers a provider factory for a given type. This is typically called in the init() function of each provider package.
Types ¶
type APIError ¶
type APIError struct {
// Provider is the name of the provider that returned the error.
Provider string
// StatusCode is the HTTP status code returned.
StatusCode int
// Message is the error message from the API.
Message string
// Retryable indicates whether the request can be retried.
Retryable bool
// Err is the underlying error, if any.
Err error
}
APIError represents an error response from an LLM API.
func NewAPIError ¶
NewAPIError creates a new APIError with the given parameters.
type ChatRequest ¶
type ChatRequest struct {
// Model is the identifier of the model to use.
Model string
// Messages is the conversation history to send to the model.
Messages []Message
// Temperature controls randomness in the response (0.0 to 2.0).
// Lower values make output more deterministic.
Temperature *float64
// MaxTokens is the maximum number of tokens to generate.
MaxTokens *int
// TopP is the nucleus sampling parameter (0.0 to 1.0).
TopP *float64
// Stop is a list of sequences where the model will stop generating.
Stop []string
// Thinking enables extended thinking/reasoning mode.
// Provider-specific handling in each provider implementation.
Thinking *ThinkingRequest
}
ChatRequest contains the input for a chat completion request.
func NewChatRequest ¶
func NewChatRequest(model string, messages []Message, opts ...RequestOption) *ChatRequest
NewChatRequest creates a new ChatRequest with the given model, messages, and options.
type ChatResponse ¶
type ChatResponse struct {
// Content is the generated text content.
Content string
// FinishReason indicates why the model stopped generating.
// Common values: "stop", "length", "content_filter".
FinishReason string
// Usage contains token usage statistics.
Usage Usage
}
ChatResponse contains the output from a chat completion request.
type Config ¶
type Config struct {
// APIKey is the authentication key for the provider.
APIKey string
// BaseURL is the API endpoint URL. If empty, the provider's default is used.
BaseURL string
// Timeout is the maximum time to wait for a response.
// Default is 60 seconds if not specified.
Timeout time.Duration
// Retry configuration
// MaxRetries is the maximum number of retry attempts for transient errors.
// Default is 3 if not specified.
MaxRetries int
// InitialInterval is the initial backoff interval before the first retry.
// Default is 1 second if not specified.
InitialInterval time.Duration
// MaxInterval is the maximum backoff interval between retries.
// Default is 30 seconds if not specified.
MaxInterval time.Duration
// Multiplier is the factor by which the interval increases after each retry.
// Default is 2.0 if not specified.
Multiplier float64
}
Config contains configuration for an LLM provider.
func DefaultConfig ¶
func DefaultConfig() Config
DefaultConfig returns a Config with sensible defaults.
type HTTPClient ¶
type HTTPClient struct {
// contains filtered or unexported fields
}
HTTPClient performs HTTP requests with retry logic using resty.
func NewHTTPClient ¶
func NewHTTPClient(cfg Config) *HTTPClient
NewHTTPClient creates a new HTTP client with the given configuration.
type Message ¶
type Message struct {
// Role identifies who sent the message (system, user, assistant, or tool).
Role Role `json:"role"`
// Content is the text content of the message.
Content string `json:"content"`
// Name is an optional identifier for the message sender.
// Useful in multi-agent scenarios or for tool messages.
Name string `json:"name,omitempty"`
// ToolCallID is the ID of the tool call this message is responding to.
// Required when Role is "tool".
ToolCallID string `json:"tool_call_id,omitempty"`
}
Message represents a single message in a conversation.
type Option ¶
type Option func(*Config)
Option is a functional option for configuring an LLM provider.
func WithAPIKey ¶
WithAPIKey sets the API key for the provider.
func WithBackoff ¶
WithBackoff sets the backoff configuration for retries.
func WithBaseURL ¶
WithBaseURL sets the base URL for the provider.
func WithMaxRetries ¶
WithMaxRetries sets the maximum number of retry attempts.
func WithTimeout ¶
WithTimeout sets the request timeout.
type Provider ¶
type Provider interface {
// Chat sends messages and returns the complete response.
Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)
// ChatStream sends messages and streams the response.
// Returns a channel that receives StreamEvents until Done is true or an error occurs.
ChatStream(ctx context.Context, req *ChatRequest) (<-chan StreamEvent, error)
// Name returns the provider name for logging and error messages.
Name() string
}
Provider is the interface that all LLM providers must implement.
func NewProvider ¶
func NewProvider(providerType ProviderType, cfg Config) (Provider, error)
NewProvider creates a new Provider instance based on the provider type and configuration.
func NewProviderFromEnv ¶
func NewProviderFromEnv(providerType ProviderType) (Provider, error)
NewProviderFromEnv creates a new Provider using API key from environment. This is a convenience function for simple use cases.
func NewProviderWithAPIKey ¶
func NewProviderWithAPIKey(providerType ProviderType, apiKey string) (Provider, error)
NewProviderWithAPIKey creates a new Provider with the given API key. This is a convenience function for simple use cases.
type ProviderFactory ¶
ProviderFactory is a function type that creates a new Provider instance.
type ProviderType ¶
type ProviderType string
ProviderType identifies the LLM provider.
const ( // ProviderOpenAI is the OpenAI provider (GPT models). ProviderOpenAI ProviderType = "openai" // ProviderAnthropic is the Anthropic provider (Claude models). ProviderAnthropic ProviderType = "anthropic" // ProviderGemini is the Google Gemini provider. ProviderGemini ProviderType = "gemini" // ProviderOpenRouter is the OpenRouter provider (multi-model gateway). ProviderOpenRouter ProviderType = "openrouter" // ProviderLocal is for local OpenAI-compatible servers (Ollama, vLLM, etc). ProviderLocal ProviderType = "local" )
func ParseProviderType ¶
func ParseProviderType(s string) (ProviderType, error)
ParseProviderType converts a string to a ProviderType.
type RequestOption ¶
type RequestOption func(*ChatRequest)
RequestOption is a functional option for configuring a ChatRequest.
func WithMaxTokens ¶
func WithMaxTokens(tokens int) RequestOption
WithMaxTokens sets the maximum tokens for the request.
func WithStop ¶
func WithStop(stop ...string) RequestOption
WithStop sets the stop sequences for the request.
func WithTemperature ¶
func WithTemperature(temp float64) RequestOption
WithTemperature sets the temperature for the request.
func WithTopP ¶
func WithTopP(topP float64) RequestOption
WithTopP sets the top_p (nucleus sampling) parameter.
type Role ¶
type Role string
Role represents the role of a message sender in a conversation.
const ( // RoleSystem represents a system message that sets behavior/context. RoleSystem Role = "system" // RoleUser represents a message from the user. RoleUser Role = "user" // RoleAssistant represents a message from the AI assistant. RoleAssistant Role = "assistant" // RoleTool represents a tool/function call result. RoleTool Role = "tool" )
type StreamEvent ¶
type StreamEvent struct {
// Delta is the incremental content received in this event.
Delta string
// Done indicates whether the stream has completed.
Done bool
// Error contains any error that occurred during streaming.
Error error
// Usage contains token usage statistics (only set when Done is true).
Usage *Usage
}
StreamEvent represents a single event in a streaming response.
type ThinkingEffort ¶
type ThinkingEffort string
ThinkingEffort represents the reasoning depth level for thinking mode.
const ( // ThinkingEffortLow provides quick reasoning with minimal token budget. ThinkingEffortLow ThinkingEffort = "low" // ThinkingEffortMedium provides balanced reasoning (default). ThinkingEffortMedium ThinkingEffort = "medium" // ThinkingEffortHigh provides thorough analysis with larger token budget. ThinkingEffortHigh ThinkingEffort = "high" // ThinkingEffortXHigh provides maximum reasoning depth. // Note: Not all providers support this level. ThinkingEffortXHigh ThinkingEffort = "xhigh" )
type ThinkingRequest ¶
type ThinkingRequest struct {
// Enabled activates thinking mode.
Enabled bool
// Effort controls reasoning depth: low, medium, high, xhigh.
Effort ThinkingEffort
// BudgetTokens sets explicit token budget (provider-specific).
BudgetTokens *int
// IncludeInOutput includes thinking blocks in the response content.
IncludeInOutput bool
}
ThinkingRequest contains configuration for extended thinking/reasoning mode. Each provider maps this to their native format.
type Usage ¶
type Usage struct {
// PromptTokens is the number of tokens in the prompt.
PromptTokens int `json:"prompt_tokens"`
// CompletionTokens is the number of tokens in the completion.
CompletionTokens int `json:"completion_tokens"`
// TotalTokens is the sum of prompt and completion tokens.
TotalTokens int `json:"total_tokens"`
}
Usage contains token usage information from an LLM API call.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package allproviders imports all LLM providers to register them.
|
Package allproviders imports all LLM providers to register them. |
|
providers
|
|
|
anthropic
Package anthropic provides an LLM provider implementation for Anthropic's Claude API.
|
Package anthropic provides an LLM provider implementation for Anthropic's Claude API. |
|
gemini
Package gemini provides an LLM provider implementation for Google's Gemini API.
|
Package gemini provides an LLM provider implementation for Google's Gemini API. |
|
local
Package local provides an LLM provider implementation for local OpenAI-compatible servers.
|
Package local provides an LLM provider implementation for local OpenAI-compatible servers. |
|
openai
Package openai provides an LLM provider implementation for OpenAI's API.
|
Package openai provides an LLM provider implementation for OpenAI's API. |
|
openrouter
Package openrouter provides an LLM provider implementation for OpenRouter's API.
|
Package openrouter provides an LLM provider implementation for OpenRouter's API. |