Documentation
¶
Overview ¶
Package model defines pluggable LLM provider interfaces.
Index ¶
- Constants
- Variables
- func ContextLimit(modelName string, fallback int) int
- func EstimateTokens(messages []Message) int
- func IsRetryable(err error) bool
- func IsTerminal(err error) bool
- func RegisterProviderFactory(name string, factory ProviderFactory)
- type APIError
- type Anthropic
- type AudioContent
- type AudioProvider
- type AzureConfig
- type AzureOpenAI
- type AzureOpenAIEmbeddings
- type BPECounter
- type Bedrock
- type CachedEmbeddings
- type ChatRequest
- type ChatResponse
- type Cohere
- type CohereEmbeddings
- type ContentPart
- type EmbeddingRequest
- type EmbeddingResponse
- type EmbeddingsProvider
- type EstimatingCounter
- type FallbackProvider
- type FunctionDef
- type Gemini
- type GoogleEmbeddings
- type Message
- type Mistral
- type ModelRef
- type Ollama
- type OllamaEmbeddings
- type OpenAI
- type OpenAIAudio
- type OpenAIAudioConfig
- type OpenAICompatible
- func NewAnyscale(apiKey, modelID string) *OpenAICompatible
- func NewDeepSeek(apiKey, modelID string) *OpenAICompatible
- func NewFireworks(apiKey, modelID string) *OpenAICompatible
- func NewGroq(apiKey, modelID string) *OpenAICompatible
- func NewOpenAICompatible(name, baseURL, apiKey, modelID string) *OpenAICompatible
- func NewOpenAICompatibleWithConfig(name string, cfg ProviderConfig) *OpenAICompatible
- func NewOpenRouter(apiKey, modelID string) *OpenAICompatible
- func NewPerplexity(apiKey, modelID string) *OpenAICompatible
- func NewTogether(apiKey, modelID string) *OpenAICompatible
- type OpenAIEmbeddings
- type Provider
- type ProviderConfig
- type ProviderFactory
- type ReasoningConfig
- type StopReason
- type SummarizationConfig
- type SummarizationResult
- type Summarizer
- type TokenCounter
- type ToolCall
- type ToolDefinition
- type Usage
Constants ¶
const ( RoleSystem = "system" RoleUser = "user" RoleAssistant = "assistant" RoleTool = "tool" )
Role constants for chat messages.
Variables ¶
var ErrCircuitOpen = errors.New("circuit breaker open")
ErrCircuitOpen is returned when a provider's circuit breaker is open and requests are short-circuited to give the upstream time to recover.
Functions ¶
func ContextLimit ¶
ContextLimit returns the maximum context window (in tokens) for a model. If the model is unknown, it returns the provided fallback value.
func EstimateTokens ¶ added in v0.3.0
EstimateTokens provides a rough token estimate for a list of messages.
func IsRetryable ¶ added in v0.4.0
IsRetryable reports whether err represents a transient failure worth retrying (network/transport failure, HTTP 408, 429 or 5xx). Context cancellation, deadline expiry and an open circuit are not retryable.
func IsTerminal ¶ added in v0.4.0
IsTerminal reports whether err is a permanent failure — a 4xx client error other than 408/429 — that will not succeed on retry or on a different provider (e.g. malformed request, authentication failure, not found).
func RegisterProviderFactory ¶ added in v0.3.0
func RegisterProviderFactory(name string, factory ProviderFactory)
RegisterProviderFactory registers a factory for a provider name.
Types ¶
type APIError ¶ added in v0.4.0
type APIError struct {
StatusCode int `json:"status_code"`
Status string `json:"status"`
Body string `json:"body"`
RetryAfter time.Duration `json:"retry_after"`
}
APIError represents a non-2xx HTTP response from a model provider API. It carries the status code, body snippet and any server-requested retry delay so callers (retry logic, fallback routing, circuit breakers) can classify the failure as retryable or terminal.
type Anthropic ¶
type Anthropic struct {
// contains filtered or unexported fields
}
Anthropic implements Provider for Claude models via the Anthropic Messages API.
func NewAnthropic ¶
NewAnthropic creates a new Anthropic provider with the given API key.
func NewAnthropicWithConfig ¶
func NewAnthropicWithConfig(cfg ProviderConfig) *Anthropic
NewAnthropicWithConfig creates an Anthropic provider with full configuration.
func (*Anthropic) Chat ¶
func (a *Anthropic) Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)
func (*Anthropic) StreamChat ¶
func (a *Anthropic) StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)
type AudioContent ¶ added in v0.3.0
type AudioContent struct {
Data []byte `json:"-"`
Format string `json:"format"` // wav, mp3, ogg, etc.
Transcript string `json:"transcript,omitempty"` // transcribed text (for input) or source text (for output)
}
AudioContent holds audio data for input transcription or output TTS.
type AudioProvider ¶ added in v0.3.0
type AudioProvider interface {
// Transcribe converts audio data to text using a speech-to-text model (e.g., Whisper).
Transcribe(ctx context.Context, audio AudioContent) (string, error)
// Synthesize converts text to speech using a TTS model.
// voice is the voice ID (e.g., "alloy", "echo", "fable", "onyx", "nova", "shimmer").
Synthesize(ctx context.Context, text, voice string) (*AudioContent, error)
}
AudioProvider is the interface for audio transcription and synthesis.
type AzureConfig ¶
type AzureConfig struct {
ProviderConfig
Deployment string `json:"deployment"`
APIVersion string `json:"api_version"`
}
AzureConfig extends ProviderConfig with Azure-specific fields.
type AzureOpenAI ¶
type AzureOpenAI struct {
// contains filtered or unexported fields
}
AzureOpenAI implements Provider for Azure-hosted OpenAI models. Standard requests use the deployment-scoped Chat Completions endpoint; native reasoning requests use /openai/v1/responses so reasoning and tools can be combined without the Chat Completions API restriction.
func NewAzureOpenAI ¶
func NewAzureOpenAI(endpoint, apiKey, deployment string) *AzureOpenAI
NewAzureOpenAI creates a new Azure OpenAI provider. endpoint is the Azure resource endpoint (e.g., "https://myresource.openai.azure.com"). apiKey is the Azure API key. deployment is the model deployment name.
func NewAzureOpenAIWithConfig ¶
func NewAzureOpenAIWithConfig(cfg AzureConfig) *AzureOpenAI
NewAzureOpenAIWithConfig creates an Azure OpenAI provider with full configuration.
func (*AzureOpenAI) Chat ¶
func (a *AzureOpenAI) Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)
func (*AzureOpenAI) Model ¶
func (a *AzureOpenAI) Model() string
func (*AzureOpenAI) Name ¶
func (a *AzureOpenAI) Name() string
func (*AzureOpenAI) StreamChat ¶
func (a *AzureOpenAI) StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)
type AzureOpenAIEmbeddings ¶ added in v0.3.0
type AzureOpenAIEmbeddings struct {
// contains filtered or unexported fields
}
AzureOpenAIEmbeddings implements EmbeddingsProvider using Azure OpenAI's embeddings API.
func NewAzureOpenAIEmbeddings ¶ added in v0.3.0
func NewAzureOpenAIEmbeddings(endpoint, apiKey, deployment string) *AzureOpenAIEmbeddings
NewAzureOpenAIEmbeddings creates an Azure OpenAI embeddings provider. endpoint is the Azure OpenAI resource endpoint (e.g., "https://myresource.openai.azure.com"). apiKey is the Azure API key. deployment is the deployment name (e.g., "text-embedding-3-large").
func NewAzureOpenAIEmbeddingsWithConfig ¶ added in v0.3.0
func NewAzureOpenAIEmbeddingsWithConfig(cfg ProviderConfig, deployment, apiVersion string) *AzureOpenAIEmbeddings
NewAzureOpenAIEmbeddingsWithConfig creates an Azure OpenAI embeddings provider with full config.
func (*AzureOpenAIEmbeddings) Embed ¶ added in v0.3.0
func (a *AzureOpenAIEmbeddings) Embed(ctx context.Context, req *EmbeddingRequest) (*EmbeddingResponse, error)
type BPECounter ¶ added in v0.4.0
type BPECounter struct {
// contains filtered or unexported fields
}
BPECounter counts tokens using a real byte-pair-encoding (BPE) tokenizer (tiktoken encodings), giving exact counts for OpenAI models and a close approximation for other providers. It replaces the character-ratio heuristic for accuracy in budgeting and context-window management.
func NewBPECounter ¶ added in v0.4.0
func NewBPECounter(modelName string) (*BPECounter, error)
NewBPECounter builds a BPE token counter for the given model. Known OpenAI models map to their exact encoding; unknown models (including Anthropic, Gemini, etc.) fall back to the newest general-purpose encoding (o200k_base), which yields a close approximation.
func (*BPECounter) CountString ¶ added in v0.4.0
func (c *BPECounter) CountString(s string) int
CountString returns the exact BPE token count for s.
func (*BPECounter) CountTokens ¶ added in v0.4.0
func (c *BPECounter) CountTokens(messages []Message) int
CountTokens returns the token count for a full set of chat messages, including per-message and conversation framing overhead.
type Bedrock ¶ added in v0.3.0
type Bedrock struct {
// contains filtered or unexported fields
}
Bedrock implements Provider for AWS Bedrock's InvokeModel API. Supports Claude, Titan, Llama, and other models hosted on Bedrock.
func NewBedrock ¶ added in v0.3.0
NewBedrock creates a Bedrock provider. region is the AWS region (e.g., "us-east-1"). accessKey and secretKey are AWS credentials. modelID is the Bedrock model identifier (e.g., "anthropic.claude-3-sonnet-20240229-v1:0").
func NewBedrockWithConfig ¶ added in v0.3.0
func NewBedrockWithConfig(region string, cfg ProviderConfig, secretKey string) *Bedrock
NewBedrockWithConfig creates a Bedrock provider with full configuration.
func (*Bedrock) Chat ¶ added in v0.3.0
func (b *Bedrock) Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)
func (*Bedrock) StreamChat ¶ added in v0.3.0
func (b *Bedrock) StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)
type CachedEmbeddings ¶
type CachedEmbeddings struct {
// contains filtered or unexported fields
}
CachedEmbeddings wraps an EmbeddingsProvider with an in-memory cache.
func NewCachedEmbeddings ¶
func NewCachedEmbeddings(inner EmbeddingsProvider) *CachedEmbeddings
func (*CachedEmbeddings) Embed ¶
func (c *CachedEmbeddings) Embed(ctx context.Context, req *EmbeddingRequest) (*EmbeddingResponse, error)
type ChatRequest ¶
type ChatRequest struct {
Model string `json:"model"`
Messages []Message `json:"messages"`
MaxTokens int `json:"max_tokens,omitempty"`
Temperature float64 `json:"temperature,omitempty"`
TopP float64 `json:"top_p,omitempty"`
Stream bool `json:"stream,omitempty"`
Tools []ToolDefinition `json:"tools,omitempty"`
Stop []string `json:"stop,omitempty"`
// ResponseFormat controls output format. Values:
// "json_object" — model returns valid JSON (no schema enforcement)
// "json_schema" — model returns JSON conforming to Metadata["json_schema"],
// a plain JSON Schema map (properties/required/type/etc.).
// Supported by OpenAI, Azure OpenAI, OpenAI-compatible
// providers (incl. Ollama, Mistral), Gemini, and the
// Responses API — passed to the provider's native
// structured-output parameter, not just requested and
// hoped for. Anthropic has no equivalent API parameter,
// so this mode has no effect there; callers needing
// schema-constrained output from Anthropic must use
// tool-forcing instead.
ResponseFormat string `json:"response_format,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"`
Reasoning *ReasoningConfig `json:"reasoning,omitempty"`
}
ChatRequest is the input to a chat completion.
type ChatResponse ¶
type ChatResponse struct {
ID string `json:"id"`
Content string `json:"content"`
Role string `json:"role"`
Usage Usage `json:"usage"`
ToolCalls []ToolCall `json:"tool_calls,omitempty"`
StopReason StopReason `json:"stop_reason,omitempty"`
// Reasoning carries provider-approved thinking/reasoning output separately
// from the final answer. Callers should not display it unless explicitly
// requested because providers may treat internal reasoning as sensitive.
Reasoning string `json:"reasoning,omitempty"`
// ProviderState carries opaque provider output required to continue a
// stateless tool-calling conversation. It is never exposed in JSON.
ProviderState any `json:"-"`
// Delta is true when this is a partial streaming response.
Delta bool `json:"delta,omitempty"`
// Err carries a streaming transport/parse error surfaced mid-stream. It is
// non-nil only on the final delta emitted when a stream fails to read
// cleanly (e.g. bufio.Scanner error). Not serialized.
Err error `json:"-"`
}
ChatResponse is the output of a chat completion.
func AggregateStream ¶ added in v0.4.0
func AggregateStream(ctx context.Context, ch <-chan *ChatResponse) (*ChatResponse, error)
AggregateStream consumes a StreamChat delta channel and merges the partial responses into a single complete ChatResponse: it concatenates text content, reassembles streamed tool calls, accumulates token usage and captures the final stop reason. It is the recommended way for non-streaming callers (and SDK-level orchestration) to obtain a full response from Provider.StreamChat.
If ctx is canceled it returns the context error. If any delta carried a streaming error (ChatResponse.Err) that error is returned once the stream drains.
type Cohere ¶ added in v0.3.0
type Cohere struct {
// contains filtered or unexported fields
}
Cohere implements Provider for the Cohere Chat API. Supports Command, Command-R, and Command-R+ models.
func NewCohere ¶ added in v0.3.0
NewCohere creates a Cohere provider. apiKey is the Cohere API key. modelID is the model identifier (e.g., "command-r-plus", "command-r", "command").
func NewCohereWithConfig ¶ added in v0.3.0
func NewCohereWithConfig(cfg ProviderConfig) *Cohere
NewCohereWithConfig creates a Cohere provider with full configuration.
func (*Cohere) Chat ¶ added in v0.3.0
func (c *Cohere) Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)
func (*Cohere) StreamChat ¶ added in v0.3.0
func (c *Cohere) StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)
type CohereEmbeddings ¶ added in v0.3.0
type CohereEmbeddings struct {
// contains filtered or unexported fields
}
CohereEmbeddings implements EmbeddingsProvider using the Cohere Embed API.
func NewCohereEmbeddings ¶ added in v0.3.0
func NewCohereEmbeddings(apiKey, modelID string) *CohereEmbeddings
NewCohereEmbeddings creates a Cohere embeddings provider. apiKey is the Cohere API key. modelID is the model identifier (e.g., "embed-english-v3.0", "embed-multilingual-v3.0").
func NewCohereEmbeddingsWithConfig ¶ added in v0.3.0
func NewCohereEmbeddingsWithConfig(cfg ProviderConfig) *CohereEmbeddings
NewCohereEmbeddingsWithConfig creates a Cohere embeddings provider with full config.
func (*CohereEmbeddings) Embed ¶ added in v0.3.0
func (c *CohereEmbeddings) Embed(ctx context.Context, req *EmbeddingRequest) (*EmbeddingResponse, error)
type ContentPart ¶ added in v0.3.0
type ContentPart struct {
Type string `json:"type"` // "text", "image_url", "file", "audio"
Text string `json:"text,omitempty"` // for type "text"
ImageURL string `json:"image_url,omitempty"` // for type "image_url" — URL or base64 data URI
MimeType string `json:"mime_type,omitempty"` // MIME type for image or file
FileName string `json:"file_name,omitempty"` // original filename for attachments
Data []byte `json:"-"` // raw binary data (not serialized to JSON)
}
ContentPart represents a multi-modal content part within a message.
type EmbeddingRequest ¶
EmbeddingRequest is input for embedding generation.
type EmbeddingResponse ¶
type EmbeddingResponse struct {
Embeddings [][]float32 `json:"embeddings"`
Usage Usage `json:"usage"`
}
EmbeddingResponse contains the generated embeddings.
type EmbeddingsProvider ¶
type EmbeddingsProvider interface {
Embed(ctx context.Context, req *EmbeddingRequest) (*EmbeddingResponse, error)
}
EmbeddingsProvider generates vector embeddings from text.
type EstimatingCounter ¶
type EstimatingCounter struct {
CharsPerToken float64
}
EstimatingCounter uses a character-ratio heuristic (1 token ~ 4 chars) to estimate token counts without external dependencies.
func NewEstimatingCounter ¶
func NewEstimatingCounter() *EstimatingCounter
NewEstimatingCounter returns a counter using the default 4-chars-per-token ratio.
func (*EstimatingCounter) CountString ¶
func (c *EstimatingCounter) CountString(s string) int
func (*EstimatingCounter) CountTokens ¶
func (c *EstimatingCounter) CountTokens(messages []Message) int
type FallbackProvider ¶
type FallbackProvider struct {
// OnFallback is called when a provider fails and the next one is tried.
// It receives the failed provider index, its name, and the error.
OnFallback func(index int, name string, err error)
// contains filtered or unexported fields
}
FallbackProvider wraps multiple providers and tries them in order. If the primary provider fails, it automatically falls back to the next one. This is useful for scenarios like primary-cloud -> cheaper-model or cloud-provider -> local-ollama.
func NewFallbackProvider ¶
func NewFallbackProvider(providers ...Provider) (*FallbackProvider, error)
NewFallbackProvider creates a fallback provider from the given providers. At least one provider is required.
func (*FallbackProvider) Chat ¶
func (f *FallbackProvider) Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)
Chat tries each provider in order until one succeeds. It does not fall through on terminal (4xx client) errors, which will not succeed on a different provider — those are returned immediately.
func (*FallbackProvider) Model ¶
func (f *FallbackProvider) Model() string
func (*FallbackProvider) Name ¶
func (f *FallbackProvider) Name() string
func (*FallbackProvider) StreamChat ¶
func (f *FallbackProvider) StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)
StreamChat tries each provider in order until one succeeds. Like Chat, it aborts immediately on terminal (4xx) errors rather than trying every provider.
type FunctionDef ¶
type FunctionDef struct {
Name string `json:"name"`
Description string `json:"description"`
Parameters map[string]any `json:"parameters"` // JSON Schema
}
FunctionDef describes a callable function for the model.
type Gemini ¶
type Gemini struct {
// contains filtered or unexported fields
}
Gemini implements Provider for Google's Gemini API.
func NewGeminiWithConfig ¶
func NewGeminiWithConfig(cfg ProviderConfig) *Gemini
NewGeminiWithConfig creates a Gemini provider with full configuration.
func (*Gemini) Chat ¶
func (g *Gemini) Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)
func (*Gemini) StreamChat ¶
func (g *Gemini) StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)
type GoogleEmbeddings ¶ added in v0.3.0
type GoogleEmbeddings struct {
// contains filtered or unexported fields
}
GoogleEmbeddings implements EmbeddingsProvider using the Google AI Gemini embeddings API.
func NewGoogleEmbeddings ¶ added in v0.3.0
func NewGoogleEmbeddings(apiKey, modelID string) *GoogleEmbeddings
NewGoogleEmbeddings creates a Google AI embeddings provider. apiKey is the Google AI API key. modelID is the model identifier (e.g., "text-embedding-004", "embedding-001").
func NewGoogleEmbeddingsWithConfig ¶ added in v0.3.0
func NewGoogleEmbeddingsWithConfig(cfg ProviderConfig) *GoogleEmbeddings
NewGoogleEmbeddingsWithConfig creates a Google AI embeddings provider with full config.
func (*GoogleEmbeddings) Embed ¶ added in v0.3.0
func (g *GoogleEmbeddings) Embed(ctx context.Context, req *EmbeddingRequest) (*EmbeddingResponse, error)
type Message ¶
type Message struct {
Role string `json:"role"` // system, user, assistant, tool
Content string `json:"content"`
Parts []ContentPart `json:"parts,omitempty"` // multi-modal content parts
Audio []AudioContent `json:"audio,omitempty"` // audio input/output attachments
Name string `json:"name,omitempty"`
ToolCallID string `json:"tool_call_id,omitempty"`
ToolCalls []ToolCall `json:"tool_calls,omitempty"`
// ProviderState carries opaque, provider-owned conversation items needed on
// a later tool round (for example encrypted Responses API reasoning items).
// The agent passes it back unchanged and never serializes or displays it.
ProviderState any `json:"-"`
}
Message represents a chat message.
func (*Message) AddAudio ¶ added in v0.3.0
AddAudio adds an audio content part to the message. format is the audio format (e.g., "wav", "mp3", "ogg").
func (*Message) AddFile ¶ added in v0.3.0
AddFile adds a file attachment content part to the message.
func (*Message) AddImageURL ¶ added in v0.3.0
AddImageURL adds an image URL content part to the message.
type Mistral ¶
type Mistral struct {
// contains filtered or unexported fields
}
Mistral implements Provider for the Mistral AI API. Mistral uses an OpenAI-compatible chat completions format.
func NewMistral ¶
NewMistral creates a new Mistral provider with the given API key.
func NewMistralWithConfig ¶
func NewMistralWithConfig(cfg ProviderConfig) *Mistral
NewMistralWithConfig creates a Mistral provider with full configuration.
func (*Mistral) Chat ¶
func (m *Mistral) Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)
func (*Mistral) StreamChat ¶
func (m *Mistral) StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)
type ModelRef ¶ added in v0.3.0
ModelRef represents a parsed "provider:model_id" string.
func ParseModelString ¶ added in v0.3.0
ParseModelString parses "provider:model_id" into its components. Examples: "openai:gpt-4o", "anthropic:claude-3-5-sonnet", "azure:gpt-4o-mini"
type Ollama ¶
type Ollama struct {
// contains filtered or unexported fields
}
Ollama implements Provider for locally running Ollama models. Ollama exposes an OpenAI-compatible API at /v1/chat/completions.
func NewOllama ¶
NewOllama creates a new Ollama provider pointing at a local instance. host is the Ollama server address (e.g., "http://localhost:11434"). modelName is the model tag (e.g., "llama3.2", "mistral", "codellama").
func NewOllamaWithConfig ¶
func NewOllamaWithConfig(cfg ProviderConfig) *Ollama
NewOllamaWithConfig creates an Ollama provider with full configuration.
func (*Ollama) Chat ¶
func (o *Ollama) Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)
func (*Ollama) StreamChat ¶
func (o *Ollama) StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)
type OllamaEmbeddings ¶
type OllamaEmbeddings struct {
// contains filtered or unexported fields
}
OllamaEmbeddings implements EmbeddingsProvider using a local Ollama server.
func NewOllamaEmbeddings ¶
func NewOllamaEmbeddings(baseURL, modelID string) *OllamaEmbeddings
NewOllamaEmbeddings creates an Ollama embeddings provider.
func (*OllamaEmbeddings) Embed ¶
func (o *OllamaEmbeddings) Embed(ctx context.Context, req *EmbeddingRequest) (*EmbeddingResponse, error)
type OpenAI ¶
type OpenAI struct {
// contains filtered or unexported fields
}
OpenAI implements Provider for OpenAI chat completion endpoints. Works with GPT-4, GPT-4o, GPT-3.5-turbo, o1, o3, and any OpenAI-compatible API.
func NewOpenAIWithConfig ¶
func NewOpenAIWithConfig(cfg ProviderConfig) *OpenAI
NewOpenAIWithConfig creates an OpenAI provider with full configuration.
func (*OpenAI) Chat ¶
func (o *OpenAI) Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)
func (*OpenAI) StreamChat ¶
func (o *OpenAI) StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)
type OpenAIAudio ¶ added in v0.3.0
type OpenAIAudio struct {
// contains filtered or unexported fields
}
OpenAIAudio implements AudioProvider for OpenAI's Whisper transcription and TTS endpoints.
func NewOpenAIAudio ¶ added in v0.3.0
func NewOpenAIAudio(apiKey string) *OpenAIAudio
NewOpenAIAudio creates an OpenAIAudio provider with the given API key and default models.
func NewOpenAIAudioWithConfig ¶ added in v0.3.0
func NewOpenAIAudioWithConfig(cfg OpenAIAudioConfig) *OpenAIAudio
NewOpenAIAudioWithConfig creates an OpenAIAudio provider with full configuration.
func (*OpenAIAudio) Synthesize ¶ added in v0.3.0
func (o *OpenAIAudio) Synthesize(ctx context.Context, text, voice string) (*AudioContent, error)
Synthesize sends text to the OpenAI TTS endpoint and returns an AudioContent with the synthesized audio bytes. voice should be one of: "alloy", "echo", "fable", "onyx", "nova", "shimmer". An empty voice defaults to "alloy".
func (*OpenAIAudio) Transcribe ¶ added in v0.3.0
func (o *OpenAIAudio) Transcribe(ctx context.Context, audio AudioContent) (string, error)
Transcribe sends audio data to the OpenAI Whisper transcription endpoint and returns the transcribed text. audio.Format is used as the filename extension (e.g., "wav", "mp3"). If audio.Transcript is already set it is returned immediately without an API call.
type OpenAIAudioConfig ¶ added in v0.3.0
type OpenAIAudioConfig struct {
// APIKey is the OpenAI API key.
APIKey string `json:"api_key"`
// BaseURL overrides the default OpenAI base URL.
BaseURL string `json:"base_url,omitempty"`
// OrgID is the optional OpenAI organization ID.
OrgID string `json:"org_id,omitempty"`
// TranscribeModel is the Whisper model to use for transcription (default: "whisper-1").
TranscribeModel string `json:"transcribe_model,omitempty"`
// TTSModel is the TTS model to use for speech synthesis (default: "tts-1").
TTSModel string `json:"tts_model,omitempty"`
// TTSFormat is the output audio format for TTS (default: "mp3").
// Supported: "mp3", "opus", "aac", "flac", "wav", "pcm".
TTSFormat string `json:"tts_format,omitempty"`
// TimeoutSec is the HTTP request timeout in seconds (default: 120).
TimeoutSec int `json:"timeout_sec,omitempty"`
}
OpenAIAudioConfig holds configuration for the OpenAI audio provider.
type OpenAICompatible ¶
type OpenAICompatible struct {
// contains filtered or unexported fields
}
OpenAICompatible implements Provider for any OpenAI-compatible API endpoint. Use this for self-hosted models (vLLM, TGI, LiteLLM), cloud providers that expose an OpenAI-compatible interface (Together, Anyscale, Fireworks, Groq, DeepSeek, OpenRouter, Perplexity), or any custom LLM server.
func NewAnyscale ¶
func NewAnyscale(apiKey, modelID string) *OpenAICompatible
NewAnyscale creates a provider for Anyscale Endpoints.
func NewDeepSeek ¶
func NewDeepSeek(apiKey, modelID string) *OpenAICompatible
NewDeepSeek creates a provider for DeepSeek.
func NewFireworks ¶
func NewFireworks(apiKey, modelID string) *OpenAICompatible
NewFireworks creates a provider for Fireworks AI.
func NewGroq ¶
func NewGroq(apiKey, modelID string) *OpenAICompatible
NewGroq creates a provider for Groq.
func NewOpenAICompatible ¶
func NewOpenAICompatible(name, baseURL, apiKey, modelID string) *OpenAICompatible
NewOpenAICompatible creates a provider for any OpenAI-compatible endpoint. name is a human-readable identifier (e.g., "vllm", "together", "deepseek"). baseURL is the API base (e.g., "https://api.together.xyz/v1"). apiKey is the authentication key. modelID is the model identifier (e.g., "meta-llama/Llama-3.1-70B-Instruct").
func NewOpenAICompatibleWithConfig ¶
func NewOpenAICompatibleWithConfig(name string, cfg ProviderConfig) *OpenAICompatible
NewOpenAICompatibleWithConfig creates an OpenAI-compatible provider with full config.
func NewOpenRouter ¶
func NewOpenRouter(apiKey, modelID string) *OpenAICompatible
NewOpenRouter creates a provider for OpenRouter.
func NewPerplexity ¶
func NewPerplexity(apiKey, modelID string) *OpenAICompatible
NewPerplexity creates a provider for Perplexity.
func NewTogether ¶
func NewTogether(apiKey, modelID string) *OpenAICompatible
NewTogether creates a provider for Together AI.
func (*OpenAICompatible) Chat ¶
func (c *OpenAICompatible) Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)
func (*OpenAICompatible) Model ¶
func (c *OpenAICompatible) Model() string
func (*OpenAICompatible) Name ¶
func (c *OpenAICompatible) Name() string
func (*OpenAICompatible) StreamChat ¶
func (c *OpenAICompatible) StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)
type OpenAIEmbeddings ¶
type OpenAIEmbeddings struct {
// contains filtered or unexported fields
}
OpenAIEmbeddings implements EmbeddingsProvider using the OpenAI embeddings API.
func NewOpenAIEmbeddings ¶
func NewOpenAIEmbeddings(apiKey string) *OpenAIEmbeddings
NewOpenAIEmbeddings creates an OpenAI embeddings provider with the given API key.
func NewOpenAIEmbeddingsWithConfig ¶
func NewOpenAIEmbeddingsWithConfig(cfg ProviderConfig) *OpenAIEmbeddings
NewOpenAIEmbeddingsWithConfig creates an OpenAI embeddings provider with full config.
func (*OpenAIEmbeddings) Embed ¶
func (o *OpenAIEmbeddings) Embed(ctx context.Context, req *EmbeddingRequest) (*EmbeddingResponse, error)
type Provider ¶
type Provider interface {
// Chat sends a request and returns a complete response.
Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)
// StreamChat returns a channel of partial responses for streaming.
// The channel is closed when the response is complete.
StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)
// Name returns a human-readable name for this provider.
Name() string
// Model returns the default model ID for this provider.
Model() string
}
Provider is the interface all LLM backends must implement.
func ProviderFromString ¶ added in v0.3.0
ProviderFromString creates a Provider from a "provider:model_id" string.
type ProviderConfig ¶
type ProviderConfig struct {
APIKey string `json:"api_key"`
BaseURL string `json:"base_url,omitempty"`
Model string `json:"model"`
MaxRetries int `json:"max_retries,omitempty"`
TimeoutSec int `json:"timeout_sec,omitempty"`
OrgID string `json:"org_id,omitempty"`
ContextWindow int `json:"context_window,omitempty"` // override default context window size for the model
}
ProviderConfig holds common configuration shared by all providers.
type ProviderFactory ¶ added in v0.3.0
ProviderFactory creates a Provider from a model reference. Register provider constructors here for model-as-string support.
type ReasoningConfig ¶ added in v0.11.0
type ReasoningConfig struct {
Enabled bool `json:"enabled" yaml:"enabled"`
Effort string `json:"effort,omitempty" yaml:"effort,omitempty"` // low, medium, high
BudgetTokens int `json:"budget_tokens,omitempty" yaml:"budget_tokens,omitempty"`
Summary bool `json:"summary,omitempty" yaml:"summary,omitempty"`
}
ReasoningConfig controls provider-native reasoning/thinking when supported. Providers ignore fields they cannot represent rather than leaking a provider-specific request shape into the agent SDK.
type StopReason ¶
type StopReason string
StopReason indicates why the model stopped generating.
const ( StopReasonEnd StopReason = "end" StopReasonMaxTokens StopReason = "max_tokens" StopReasonToolCall StopReason = "tool_call" StopReasonFilter StopReason = "content_filter" )
type SummarizationConfig ¶
type SummarizationConfig struct {
Threshold float64 // fraction of context window that triggers summarization (default 0.8)
PreserveRecentTurns int // number of recent user/assistant pairs to keep verbatim
}
SummarizationConfig controls when and how auto-summarization triggers.
type SummarizationResult ¶ added in v0.3.0
SummarizationResult holds the output of a summarization pass.
type Summarizer ¶
type Summarizer struct {
// contains filtered or unexported fields
}
Summarizer compresses conversation history to fit within context windows.
func NewSummarizer ¶
func NewSummarizer(p Provider, counter TokenCounter, cfg SummarizationConfig) *Summarizer
NewSummarizer creates a summarizer with the given provider, counter, and config.
func (*Summarizer) NeedsSummarization ¶
func (s *Summarizer) NeedsSummarization(systemTokens int, history []Message, contextLimit int) bool
NeedsSummarization returns true if the total estimated tokens (system + history) exceed the threshold fraction of the context limit.
func (*Summarizer) Summarize ¶
func (s *Summarizer) Summarize(ctx context.Context, existingSummary string, messages []Message) (SummarizationResult, error)
Summarize compresses older messages into a running summary. If an existing summary is provided, it is incorporated into the new summary.
type TokenCounter ¶
TokenCounter estimates the token count for a set of messages.
func NewTokenCounter ¶ added in v0.4.0
func NewTokenCounter(modelName string) TokenCounter
NewTokenCounter returns the best available token counter for a model: a real BPE counter when its encoding can be loaded, otherwise the character-ratio estimator. It never returns an error, so callers can use it inline.
type ToolCall ¶
type ToolCall struct {
ID string `json:"id"`
Name string `json:"name"`
Arguments string `json:"arguments"` // JSON string
}
ToolCall represents a model-requested tool invocation.
type ToolDefinition ¶
type ToolDefinition struct {
Type string `json:"type"` // "function"
Function FunctionDef `json:"function"`
}
ToolDefinition describes a tool the model can call (OpenAI function-calling format).
Source Files
¶
- anthropic.go
- azure.go
- azure_embeddings.go
- bedrock.go
- circuitbreaker.go
- cohere.go
- cohere_embeddings.go
- compatible.go
- embeddings.go
- errors.go
- fallback.go
- gemini.go
- google_embeddings.go
- httpclient.go
- mistral.go
- ollama.go
- ollama_embeddings.go
- openai.go
- openai_audio.go
- openai_embeddings.go
- parse.go
- provider.go
- responses.go
- stream.go
- summarizer.go
- tokenizer.go