model

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 25, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Overview

Package model defines pluggable LLM provider interfaces.

Index

Constants

View Source
const (
	RoleSystem    = "system"
	RoleUser      = "user"
	RoleAssistant = "assistant"
	RoleTool      = "tool"
)

Role constants for chat messages.

Variables

This section is empty.

Functions

func ContextLimit

func ContextLimit(modelName string, fallback int) int

ContextLimit returns the maximum context window (in tokens) for a model. If the model is unknown, it returns the provided fallback value.

func EstimateTokens added in v0.3.0

func EstimateTokens(messages []Message) int

EstimateTokens provides a rough token estimate for a list of messages.

func RegisterProviderFactory added in v0.3.0

func RegisterProviderFactory(name string, factory ProviderFactory)

RegisterProviderFactory registers a factory for a provider name.

Types

type Anthropic

type Anthropic struct {
	// contains filtered or unexported fields
}

Anthropic implements Provider for Claude models via the Anthropic Messages API.

func NewAnthropic

func NewAnthropic(apiKey string) *Anthropic

NewAnthropic creates a new Anthropic provider with the given API key.

func NewAnthropicWithConfig

func NewAnthropicWithConfig(cfg ProviderConfig) *Anthropic

NewAnthropicWithConfig creates an Anthropic provider with full configuration.

func (*Anthropic) Chat

func (a *Anthropic) Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)

func (*Anthropic) Model

func (a *Anthropic) Model() string

func (*Anthropic) Name

func (a *Anthropic) Name() string

func (*Anthropic) StreamChat

func (a *Anthropic) StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)

type AudioContent added in v0.3.0

type AudioContent struct {
	Data       []byte `json:"-"`
	Format     string `json:"format"`               // wav, mp3, ogg, etc.
	Transcript string `json:"transcript,omitempty"` // transcribed text (for input) or source text (for output)
}

AudioContent holds audio data for input transcription or output TTS.

type AudioProvider added in v0.3.0

type AudioProvider interface {
	// Transcribe converts audio data to text using a speech-to-text model (e.g., Whisper).
	Transcribe(ctx context.Context, audio AudioContent) (string, error)
	// Synthesize converts text to speech using a TTS model.
	// voice is the voice ID (e.g., "alloy", "echo", "fable", "onyx", "nova", "shimmer").
	Synthesize(ctx context.Context, text, voice string) (*AudioContent, error)
}

AudioProvider is the interface for audio transcription and synthesis.

type AzureConfig

type AzureConfig struct {
	ProviderConfig
	Deployment string `json:"deployment"`
	APIVersion string `json:"api_version"`
}

AzureConfig extends ProviderConfig with Azure-specific fields.

type AzureOpenAI

type AzureOpenAI struct {
	// contains filtered or unexported fields
}

AzureOpenAI implements Provider for Azure-hosted OpenAI models. Uses the Azure-specific endpoint format: {base}/openai/deployments/{deployment}/chat/completions?api-version={version}

func NewAzureOpenAI

func NewAzureOpenAI(endpoint, apiKey, deployment string) *AzureOpenAI

NewAzureOpenAI creates a new Azure OpenAI provider. endpoint is the Azure resource endpoint (e.g., "https://myresource.openai.azure.com"). apiKey is the Azure API key. deployment is the model deployment name.

func NewAzureOpenAIWithConfig

func NewAzureOpenAIWithConfig(cfg AzureConfig) *AzureOpenAI

NewAzureOpenAIWithConfig creates an Azure OpenAI provider with full configuration.

func (*AzureOpenAI) Chat

func (a *AzureOpenAI) Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)

func (*AzureOpenAI) Model

func (a *AzureOpenAI) Model() string

func (*AzureOpenAI) Name

func (a *AzureOpenAI) Name() string

func (*AzureOpenAI) StreamChat

func (a *AzureOpenAI) StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)

type AzureOpenAIEmbeddings added in v0.3.0

type AzureOpenAIEmbeddings struct {
	// contains filtered or unexported fields
}

AzureOpenAIEmbeddings implements EmbeddingsProvider using Azure OpenAI's embeddings API.

func NewAzureOpenAIEmbeddings added in v0.3.0

func NewAzureOpenAIEmbeddings(endpoint, apiKey, deployment string) *AzureOpenAIEmbeddings

NewAzureOpenAIEmbeddings creates an Azure OpenAI embeddings provider. endpoint is the Azure OpenAI resource endpoint (e.g., "https://myresource.openai.azure.com"). apiKey is the Azure API key. deployment is the deployment name (e.g., "text-embedding-3-large").

func NewAzureOpenAIEmbeddingsWithConfig added in v0.3.0

func NewAzureOpenAIEmbeddingsWithConfig(cfg ProviderConfig, deployment, apiVersion string) *AzureOpenAIEmbeddings

NewAzureOpenAIEmbeddingsWithConfig creates an Azure OpenAI embeddings provider with full config.

func (*AzureOpenAIEmbeddings) Embed added in v0.3.0

type Bedrock added in v0.3.0

type Bedrock struct {
	// contains filtered or unexported fields
}

Bedrock implements Provider for AWS Bedrock's InvokeModel API. Supports Claude, Titan, Llama, and other models hosted on Bedrock.

func NewBedrock added in v0.3.0

func NewBedrock(region, accessKey, secretKey, modelID string) *Bedrock

NewBedrock creates a Bedrock provider. region is the AWS region (e.g., "us-east-1"). accessKey and secretKey are AWS credentials. modelID is the Bedrock model identifier (e.g., "anthropic.claude-3-sonnet-20240229-v1:0").

func NewBedrockWithConfig added in v0.3.0

func NewBedrockWithConfig(region string, cfg ProviderConfig, secretKey string) *Bedrock

NewBedrockWithConfig creates a Bedrock provider with full configuration.

func (*Bedrock) Chat added in v0.3.0

func (b *Bedrock) Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)

func (*Bedrock) Model added in v0.3.0

func (b *Bedrock) Model() string

func (*Bedrock) Name added in v0.3.0

func (b *Bedrock) Name() string

func (*Bedrock) StreamChat added in v0.3.0

func (b *Bedrock) StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)

type CachedEmbeddings

type CachedEmbeddings struct {
	// contains filtered or unexported fields
}

CachedEmbeddings wraps an EmbeddingsProvider with an in-memory cache.

func NewCachedEmbeddings

func NewCachedEmbeddings(inner EmbeddingsProvider) *CachedEmbeddings

func (*CachedEmbeddings) Embed

type ChatRequest

type ChatRequest struct {
	Model       string           `json:"model"`
	Messages    []Message        `json:"messages"`
	MaxTokens   int              `json:"max_tokens,omitempty"`
	Temperature float64          `json:"temperature,omitempty"`
	TopP        float64          `json:"top_p,omitempty"`
	Stream      bool             `json:"stream,omitempty"`
	Tools       []ToolDefinition `json:"tools,omitempty"`
	Stop        []string         `json:"stop,omitempty"`
	// ResponseFormat controls output format. Values:
	//   "json_object"  — model returns valid JSON (no schema enforcement)
	//   "json_schema"  — model returns JSON conforming to Metadata["json_schema"]
	ResponseFormat string         `json:"response_format,omitempty"`
	Metadata       map[string]any `json:"metadata,omitempty"`
}

ChatRequest is the input to a chat completion.

type ChatResponse

type ChatResponse struct {
	ID         string     `json:"id"`
	Content    string     `json:"content"`
	Role       string     `json:"role"`
	Usage      Usage      `json:"usage"`
	ToolCalls  []ToolCall `json:"tool_calls,omitempty"`
	StopReason StopReason `json:"stop_reason,omitempty"`
	// Delta is true when this is a partial streaming response.
	Delta bool `json:"delta,omitempty"`
}

ChatResponse is the output of a chat completion.

type Cohere added in v0.3.0

type Cohere struct {
	// contains filtered or unexported fields
}

Cohere implements Provider for the Cohere Chat API. Supports Command, Command-R, and Command-R+ models.

func NewCohere added in v0.3.0

func NewCohere(apiKey, modelID string) *Cohere

NewCohere creates a Cohere provider. apiKey is the Cohere API key. modelID is the model identifier (e.g., "command-r-plus", "command-r", "command").

func NewCohereWithConfig added in v0.3.0

func NewCohereWithConfig(cfg ProviderConfig) *Cohere

NewCohereWithConfig creates a Cohere provider with full configuration.

func (*Cohere) Chat added in v0.3.0

func (c *Cohere) Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)

func (*Cohere) Model added in v0.3.0

func (c *Cohere) Model() string

func (*Cohere) Name added in v0.3.0

func (c *Cohere) Name() string

func (*Cohere) StreamChat added in v0.3.0

func (c *Cohere) StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)

type CohereEmbeddings added in v0.3.0

type CohereEmbeddings struct {
	// contains filtered or unexported fields
}

CohereEmbeddings implements EmbeddingsProvider using the Cohere Embed API.

func NewCohereEmbeddings added in v0.3.0

func NewCohereEmbeddings(apiKey, modelID string) *CohereEmbeddings

NewCohereEmbeddings creates a Cohere embeddings provider. apiKey is the Cohere API key. modelID is the model identifier (e.g., "embed-english-v3.0", "embed-multilingual-v3.0").

func NewCohereEmbeddingsWithConfig added in v0.3.0

func NewCohereEmbeddingsWithConfig(cfg ProviderConfig) *CohereEmbeddings

NewCohereEmbeddingsWithConfig creates a Cohere embeddings provider with full config.

func (*CohereEmbeddings) Embed added in v0.3.0

type ContentPart added in v0.3.0

type ContentPart struct {
	Type     string `json:"type"`                // "text", "image_url", "file", "audio"
	Text     string `json:"text,omitempty"`      // for type "text"
	ImageURL string `json:"image_url,omitempty"` // for type "image_url" — URL or base64 data URI
	MimeType string `json:"mime_type,omitempty"` // MIME type for image or file
	FileName string `json:"file_name,omitempty"` // original filename for attachments
	Data     []byte `json:"-"`                   // raw binary data (not serialized to JSON)
}

ContentPart represents a multi-modal content part within a message.

type EmbeddingRequest

type EmbeddingRequest struct {
	Model string   `json:"model"`
	Input []string `json:"input"`
}

EmbeddingRequest is input for embedding generation.

type EmbeddingResponse

type EmbeddingResponse struct {
	Embeddings [][]float32 `json:"embeddings"`
	Usage      Usage       `json:"usage"`
}

EmbeddingResponse contains the generated embeddings.

type EmbeddingsProvider

type EmbeddingsProvider interface {
	Embed(ctx context.Context, req *EmbeddingRequest) (*EmbeddingResponse, error)
}

EmbeddingsProvider generates vector embeddings from text.

type EstimatingCounter

type EstimatingCounter struct {
	CharsPerToken float64
}

EstimatingCounter uses a character-ratio heuristic (1 token ~ 4 chars) to estimate token counts without external dependencies.

func NewEstimatingCounter

func NewEstimatingCounter() *EstimatingCounter

NewEstimatingCounter returns a counter using the default 4-chars-per-token ratio.

func (*EstimatingCounter) CountString

func (c *EstimatingCounter) CountString(s string) int

func (*EstimatingCounter) CountTokens

func (c *EstimatingCounter) CountTokens(messages []Message) int

type FallbackProvider

type FallbackProvider struct {

	// OnFallback is called when a provider fails and the next one is tried.
	// It receives the failed provider index, its name, and the error.
	OnFallback func(index int, name string, err error)
	// contains filtered or unexported fields
}

FallbackProvider wraps multiple providers and tries them in order. If the primary provider fails, it automatically falls back to the next one. This is useful for scenarios like primary-cloud -> cheaper-model or cloud-provider -> local-ollama.

func NewFallbackProvider

func NewFallbackProvider(providers ...Provider) (*FallbackProvider, error)

NewFallbackProvider creates a fallback provider from the given providers. At least one provider is required.

func (*FallbackProvider) Chat

Chat tries each provider in order until one succeeds.

func (*FallbackProvider) Model

func (f *FallbackProvider) Model() string

func (*FallbackProvider) Name

func (f *FallbackProvider) Name() string

func (*FallbackProvider) StreamChat

func (f *FallbackProvider) StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)

StreamChat tries each provider in order until one succeeds.

type FunctionDef

type FunctionDef struct {
	Name        string         `json:"name"`
	Description string         `json:"description"`
	Parameters  map[string]any `json:"parameters"` // JSON Schema
}

FunctionDef describes a callable function for the model.

type Gemini

type Gemini struct {
	// contains filtered or unexported fields
}

Gemini implements Provider for Google's Gemini API.

func NewGemini

func NewGemini(apiKey string) *Gemini

NewGemini creates a new Gemini provider with the given API key.

func NewGeminiWithConfig

func NewGeminiWithConfig(cfg ProviderConfig) *Gemini

NewGeminiWithConfig creates a Gemini provider with full configuration.

func (*Gemini) Chat

func (g *Gemini) Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)

func (*Gemini) Model

func (g *Gemini) Model() string

func (*Gemini) Name

func (g *Gemini) Name() string

func (*Gemini) StreamChat

func (g *Gemini) StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)

type GoogleEmbeddings added in v0.3.0

type GoogleEmbeddings struct {
	// contains filtered or unexported fields
}

GoogleEmbeddings implements EmbeddingsProvider using the Google AI Gemini embeddings API.

func NewGoogleEmbeddings added in v0.3.0

func NewGoogleEmbeddings(apiKey, modelID string) *GoogleEmbeddings

NewGoogleEmbeddings creates a Google AI embeddings provider. apiKey is the Google AI API key. modelID is the model identifier (e.g., "text-embedding-004", "embedding-001").

func NewGoogleEmbeddingsWithConfig added in v0.3.0

func NewGoogleEmbeddingsWithConfig(cfg ProviderConfig) *GoogleEmbeddings

NewGoogleEmbeddingsWithConfig creates a Google AI embeddings provider with full config.

func (*GoogleEmbeddings) Embed added in v0.3.0

type Message

type Message struct {
	Role       string         `json:"role"` // system, user, assistant, tool
	Content    string         `json:"content"`
	Parts      []ContentPart  `json:"parts,omitempty"` // multi-modal content parts
	Audio      []AudioContent `json:"audio,omitempty"` // audio input/output attachments
	Name       string         `json:"name,omitempty"`
	ToolCallID string         `json:"tool_call_id,omitempty"`
	ToolCalls  []ToolCall     `json:"tool_calls,omitempty"`
}

Message represents a chat message.

func (*Message) AddAudio added in v0.3.0

func (m *Message) AddAudio(data []byte, format string)

AddAudio adds an audio content part to the message. format is the audio format (e.g., "wav", "mp3", "ogg").

func (*Message) AddFile added in v0.3.0

func (m *Message) AddFile(filename, mimeType string, data []byte)

AddFile adds a file attachment content part to the message.

func (*Message) AddImageURL added in v0.3.0

func (m *Message) AddImageURL(url, mimeType string)

AddImageURL adds an image URL content part to the message.

type Mistral

type Mistral struct {
	// contains filtered or unexported fields
}

Mistral implements Provider for the Mistral AI API. Mistral uses an OpenAI-compatible chat completions format.

func NewMistral

func NewMistral(apiKey string) *Mistral

NewMistral creates a new Mistral provider with the given API key.

func NewMistralWithConfig

func NewMistralWithConfig(cfg ProviderConfig) *Mistral

NewMistralWithConfig creates a Mistral provider with full configuration.

func (*Mistral) Chat

func (m *Mistral) Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)

func (*Mistral) Model

func (m *Mistral) Model() string

func (*Mistral) Name

func (m *Mistral) Name() string

func (*Mistral) StreamChat

func (m *Mistral) StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)

type ModelRef added in v0.3.0

type ModelRef struct {
	Provider string
	ModelID  string
}

ModelRef represents a parsed "provider:model_id" string.

func ParseModelString added in v0.3.0

func ParseModelString(s string) (ModelRef, error)

ParseModelString parses "provider:model_id" into its components. Examples: "openai:gpt-4o", "anthropic:claude-3-5-sonnet", "azure:gpt-4o-mini"

func (ModelRef) String added in v0.3.0

func (r ModelRef) String() string

String returns the "provider:model_id" format.

type Ollama

type Ollama struct {
	// contains filtered or unexported fields
}

Ollama implements Provider for locally running Ollama models. Ollama exposes an OpenAI-compatible API at /v1/chat/completions.

func NewOllama

func NewOllama(host, modelName string) *Ollama

NewOllama creates a new Ollama provider pointing at a local instance. host is the Ollama server address (e.g., "http://localhost:11434"). modelName is the model tag (e.g., "llama3.2", "mistral", "codellama").

func NewOllamaWithConfig

func NewOllamaWithConfig(cfg ProviderConfig) *Ollama

NewOllamaWithConfig creates an Ollama provider with full configuration.

func (*Ollama) Chat

func (o *Ollama) Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)

func (*Ollama) Model

func (o *Ollama) Model() string

func (*Ollama) Name

func (o *Ollama) Name() string

func (*Ollama) StreamChat

func (o *Ollama) StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)

type OllamaEmbeddings

type OllamaEmbeddings struct {
	// contains filtered or unexported fields
}

OllamaEmbeddings implements EmbeddingsProvider using a local Ollama server.

func NewOllamaEmbeddings

func NewOllamaEmbeddings(baseURL, modelID string) *OllamaEmbeddings

NewOllamaEmbeddings creates an Ollama embeddings provider.

func (*OllamaEmbeddings) Embed

type OpenAI

type OpenAI struct {
	// contains filtered or unexported fields
}

OpenAI implements Provider for OpenAI chat completion endpoints. Works with GPT-4, GPT-4o, GPT-3.5-turbo, o1, o3, and any OpenAI-compatible API.

func NewOpenAI

func NewOpenAI(apiKey string) *OpenAI

NewOpenAI creates a new OpenAI provider with the given API key.

func NewOpenAIWithConfig

func NewOpenAIWithConfig(cfg ProviderConfig) *OpenAI

NewOpenAIWithConfig creates an OpenAI provider with full configuration.

func (*OpenAI) Chat

func (o *OpenAI) Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)

func (*OpenAI) Model

func (o *OpenAI) Model() string

func (*OpenAI) Name

func (o *OpenAI) Name() string

func (*OpenAI) StreamChat

func (o *OpenAI) StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)

type OpenAIAudio added in v0.3.0

type OpenAIAudio struct {
	// contains filtered or unexported fields
}

OpenAIAudio implements AudioProvider for OpenAI's Whisper transcription and TTS endpoints.

func NewOpenAIAudio added in v0.3.0

func NewOpenAIAudio(apiKey string) *OpenAIAudio

NewOpenAIAudio creates an OpenAIAudio provider with the given API key and default models.

func NewOpenAIAudioWithConfig added in v0.3.0

func NewOpenAIAudioWithConfig(cfg OpenAIAudioConfig) *OpenAIAudio

NewOpenAIAudioWithConfig creates an OpenAIAudio provider with full configuration.

func (*OpenAIAudio) Synthesize added in v0.3.0

func (o *OpenAIAudio) Synthesize(ctx context.Context, text, voice string) (*AudioContent, error)

Synthesize sends text to the OpenAI TTS endpoint and returns an AudioContent with the synthesized audio bytes. voice should be one of: "alloy", "echo", "fable", "onyx", "nova", "shimmer". An empty voice defaults to "alloy".

func (*OpenAIAudio) Transcribe added in v0.3.0

func (o *OpenAIAudio) Transcribe(ctx context.Context, audio AudioContent) (string, error)

Transcribe sends audio data to the OpenAI Whisper transcription endpoint and returns the transcribed text. audio.Format is used as the filename extension (e.g., "wav", "mp3"). If audio.Transcript is already set it is returned immediately without an API call.

type OpenAIAudioConfig added in v0.3.0

type OpenAIAudioConfig struct {
	// APIKey is the OpenAI API key.
	APIKey string `json:"api_key"`
	// BaseURL overrides the default OpenAI base URL.
	BaseURL string `json:"base_url,omitempty"`
	// OrgID is the optional OpenAI organization ID.
	OrgID string `json:"org_id,omitempty"`
	// TranscribeModel is the Whisper model to use for transcription (default: "whisper-1").
	TranscribeModel string `json:"transcribe_model,omitempty"`
	// TTSModel is the TTS model to use for speech synthesis (default: "tts-1").
	TTSModel string `json:"tts_model,omitempty"`
	// TTSFormat is the output audio format for TTS (default: "mp3").
	// Supported: "mp3", "opus", "aac", "flac", "wav", "pcm".
	TTSFormat string `json:"tts_format,omitempty"`
	// TimeoutSec is the HTTP request timeout in seconds (default: 120).
	TimeoutSec int `json:"timeout_sec,omitempty"`
}

OpenAIAudioConfig holds configuration for the OpenAI audio provider.

type OpenAICompatible

type OpenAICompatible struct {
	// contains filtered or unexported fields
}

OpenAICompatible implements Provider for any OpenAI-compatible API endpoint. Use this for self-hosted models (vLLM, TGI, LiteLLM), cloud providers that expose an OpenAI-compatible interface (Together, Anyscale, Fireworks, Groq, DeepSeek, OpenRouter, Perplexity), or any custom LLM server.

func NewAnyscale

func NewAnyscale(apiKey, modelID string) *OpenAICompatible

NewAnyscale creates a provider for Anyscale Endpoints.

func NewDeepSeek

func NewDeepSeek(apiKey, modelID string) *OpenAICompatible

NewDeepSeek creates a provider for DeepSeek.

func NewFireworks

func NewFireworks(apiKey, modelID string) *OpenAICompatible

NewFireworks creates a provider for Fireworks AI.

func NewGroq

func NewGroq(apiKey, modelID string) *OpenAICompatible

NewGroq creates a provider for Groq.

func NewOpenAICompatible

func NewOpenAICompatible(name, baseURL, apiKey, modelID string) *OpenAICompatible

NewOpenAICompatible creates a provider for any OpenAI-compatible endpoint. name is a human-readable identifier (e.g., "vllm", "together", "deepseek"). baseURL is the API base (e.g., "https://api.together.xyz/v1"). apiKey is the authentication key. modelID is the model identifier (e.g., "meta-llama/Llama-3.1-70B-Instruct").

func NewOpenAICompatibleWithConfig

func NewOpenAICompatibleWithConfig(name string, cfg ProviderConfig) *OpenAICompatible

NewOpenAICompatibleWithConfig creates an OpenAI-compatible provider with full config.

func NewOpenRouter

func NewOpenRouter(apiKey, modelID string) *OpenAICompatible

NewOpenRouter creates a provider for OpenRouter.

func NewPerplexity

func NewPerplexity(apiKey, modelID string) *OpenAICompatible

NewPerplexity creates a provider for Perplexity.

func NewTogether

func NewTogether(apiKey, modelID string) *OpenAICompatible

NewTogether creates a provider for Together AI.

func (*OpenAICompatible) Chat

func (*OpenAICompatible) Model

func (c *OpenAICompatible) Model() string

func (*OpenAICompatible) Name

func (c *OpenAICompatible) Name() string

func (*OpenAICompatible) StreamChat

func (c *OpenAICompatible) StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)

type OpenAIEmbeddings

type OpenAIEmbeddings struct {
	// contains filtered or unexported fields
}

OpenAIEmbeddings implements EmbeddingsProvider using the OpenAI embeddings API.

func NewOpenAIEmbeddings

func NewOpenAIEmbeddings(apiKey string) *OpenAIEmbeddings

NewOpenAIEmbeddings creates an OpenAI embeddings provider with the given API key.

func NewOpenAIEmbeddingsWithConfig

func NewOpenAIEmbeddingsWithConfig(cfg ProviderConfig) *OpenAIEmbeddings

NewOpenAIEmbeddingsWithConfig creates an OpenAI embeddings provider with full config.

func (*OpenAIEmbeddings) Embed

type Provider

type Provider interface {
	// Chat sends a request and returns a complete response.
	Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)
	// StreamChat returns a channel of partial responses for streaming.
	// The channel is closed when the response is complete.
	StreamChat(ctx context.Context, req *ChatRequest) (<-chan *ChatResponse, error)
	// Name returns a human-readable name for this provider.
	Name() string
	// Model returns the default model ID for this provider.
	Model() string
}

Provider is the interface all LLM backends must implement.

func ProviderFromString added in v0.3.0

func ProviderFromString(s string) (Provider, error)

ProviderFromString creates a Provider from a "provider:model_id" string.

type ProviderConfig

type ProviderConfig struct {
	APIKey        string `json:"api_key"`
	BaseURL       string `json:"base_url,omitempty"`
	Model         string `json:"model"`
	MaxRetries    int    `json:"max_retries,omitempty"`
	TimeoutSec    int    `json:"timeout_sec,omitempty"`
	OrgID         string `json:"org_id,omitempty"`
	ContextWindow int    `json:"context_window,omitempty"` // override default context window size for the model
}

ProviderConfig holds common configuration shared by all providers.

type ProviderFactory added in v0.3.0

type ProviderFactory func(ref ModelRef) (Provider, error)

ProviderFactory creates a Provider from a model reference. Register provider constructors here for model-as-string support.

type StopReason

type StopReason string

StopReason indicates why the model stopped generating.

const (
	StopReasonEnd       StopReason = "end"
	StopReasonMaxTokens StopReason = "max_tokens"
	StopReasonToolCall  StopReason = "tool_call"
	StopReasonFilter    StopReason = "content_filter"
)

type SummarizationConfig

type SummarizationConfig struct {
	Threshold           float64 // fraction of context window that triggers summarization (default 0.8)
	PreserveRecentTurns int     // number of recent user/assistant pairs to keep verbatim
}

SummarizationConfig controls when and how auto-summarization triggers.

type SummarizationResult added in v0.3.0

type SummarizationResult struct {
	Summary           string
	PreservedMessages []Message
	SummarizedCount   int
}

SummarizationResult holds the output of a summarization pass.

type Summarizer

type Summarizer struct {
	// contains filtered or unexported fields
}

Summarizer compresses conversation history to fit within context windows.

func NewSummarizer

func NewSummarizer(p Provider, counter TokenCounter, cfg SummarizationConfig) *Summarizer

NewSummarizer creates a summarizer with the given provider, counter, and config.

func (*Summarizer) NeedsSummarization

func (s *Summarizer) NeedsSummarization(systemTokens int, history []Message, contextLimit int) bool

NeedsSummarization returns true if the total estimated tokens (system + history) exceed the threshold fraction of the context limit.

func (*Summarizer) Summarize

func (s *Summarizer) Summarize(ctx context.Context, existingSummary string, messages []Message) (SummarizationResult, error)

Summarize compresses older messages into a running summary. If an existing summary is provided, it is incorporated into the new summary.

type TokenCounter

type TokenCounter interface {
	CountTokens(messages []Message) int
	CountString(s string) int
}

TokenCounter estimates the token count for a set of messages.

type ToolCall

type ToolCall struct {
	ID        string `json:"id"`
	Name      string `json:"name"`
	Arguments string `json:"arguments"` // JSON string
}

ToolCall represents a model-requested tool invocation.

type ToolDefinition

type ToolDefinition struct {
	Type     string      `json:"type"` // "function"
	Function FunctionDef `json:"function"`
}

ToolDefinition describes a tool the model can call (OpenAI function-calling format).

type Usage

type Usage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
}

Usage tracks token consumption.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL