Documentation
¶
Overview ¶
Package provider defines the interface for LLM providers.
Index ¶
- Variables
- func BuildJSONInstruction(prompt string, schema json.RawMessage) string
- func InjectJSONInstruction(messages []message.Message, schema json.RawMessage) []message.Message
- func ParseProviderOptions[T any](m map[string]any) (*T, error)
- type Capabilities
- type EmbeddingProvider
- type ImageProvider
- type LanguageProvider
- type ModelInfo
- type Options
- type Provider
- type Registry
- type RerankingProvider
- type SpeechProvider
- type StreamInput
- type TranscriptionProvider
Constants ¶
This section is empty.
Variables ¶
var ErrResponseFormatUnsupported = errors.New("provider does not support requested ResponseFormat")
ErrResponseFormatUnsupported is returned by a provider that cannot honor a requested stream.ResponseFormat. Wrap with fmt.Errorf("%w: details", ...) to add provider context. Callers match with errors.Is.
Functions ¶
func BuildJSONInstruction ¶
func BuildJSONInstruction(prompt string, schema json.RawMessage) string
BuildJSONInstruction returns the instruction text that is injected into the system message. If prompt is non-empty it is kept as the first line. If schema is non-nil, the schema and schema-aware suffix are appended; otherwise a generic "answer with JSON" suffix is appended.
func InjectJSONInstruction ¶
InjectJSONInstruction returns a new message slice with JSON-formatting instructions merged into (or prepended as) a system message.
Mirrors ai-sdk's injectJsonInstructionIntoMessages. If the first message is a system message, its text is augmented; otherwise a new system message is prepended. The input slice is not mutated.
func ParseProviderOptions ¶
ParseProviderOptions parses a map[string]any into a typed options struct. This is a Go-idiomatic way to handle provider-specific options. Usage:
opts, err := provider.ParseProviderOptions[ResponsesOptions](options.ProviderOptions)
if err != nil {
return err
}
if opts.ReasoningEffort != "" {
req.Reasoning = &reasoningConfig{Effort: opts.ReasoningEffort}
}
Types ¶
type Capabilities ¶
type Capabilities struct {
Tools bool // Supports tool/function calling
Vision bool // Supports image input
Streaming bool // Supports streaming responses
Reasoning bool // Supports extended thinking/reasoning
Temperature bool // Supports temperature parameter
}
Capabilities describes what a model supports.
type EmbeddingProvider ¶
type EmbeddingProvider interface {
EmbeddingModel(modelID string) model.EmbeddingModel
}
EmbeddingProvider is a provider that supports embeddings.
type ImageProvider ¶
type ImageProvider interface {
ImageModel(modelID string) model.ImageModel
}
ImageProvider is a provider that supports image generation.
type LanguageProvider ¶
type LanguageProvider interface {
LanguageModel(modelID string) model.LanguageModel
}
LanguageProvider is a provider that supports language models. All providers must implement this.
type ModelInfo ¶
type ModelInfo struct {
ID string
Name string
Provider string
ContextLimit int
OutputLimit int
Capabilities Capabilities
}
ModelInfo contains metadata about a model.
type Options ¶
type Options struct {
// APIKey is the API key for authentication.
APIKey string
// BaseURL overrides the default API endpoint.
BaseURL string
// Headers are additional HTTP headers to send.
Headers map[string]string
// Organization is the organization ID (for providers that support it).
Organization string
// Project is the project ID (for providers that support it, e.g., OpenAI).
Project string
}
Options contains common provider configuration.
type Provider ¶
type Provider interface {
// ID returns the provider identifier (e.g., "openai", "anthropic").
ID() string
// Model returns a language model instance for the given model ID.
// This is the primary method for getting a model.
Model(modelID string) stream.Model
// LanguageModel returns a language model instance.
// Returns nil if the model is not supported.
LanguageModel(modelID string) model.LanguageModel
// ImageModel returns an image generation model instance.
// Returns nil if the provider doesn't support image generation.
ImageModel(modelID string) model.ImageModel
// EmbeddingModel returns an embedding model instance.
// Returns nil if the provider doesn't support embeddings.
EmbeddingModel(modelID string) model.EmbeddingModel
// SpeechModel returns a text-to-speech model instance.
// Returns nil if the provider doesn't support speech generation.
SpeechModel(modelID string) model.SpeechModel
// TranscriptionModel returns a speech-to-text model instance.
// Returns nil if the provider doesn't support transcription.
TranscriptionModel(modelID string) model.TranscriptionModel
// RerankingModel returns a reranking model instance.
// Returns nil if the provider doesn't support reranking.
RerankingModel(modelID string) model.RerankingModel
// Models returns a list of available model IDs.
Models() []string
}
Provider is the interface that all LLM providers must implement. At minimum, providers must implement LanguageModel support. Other model types are optional and return nil if not supported.
type Registry ¶
type Registry struct {
// contains filtered or unexported fields
}
Registry holds registered providers.
type RerankingProvider ¶
type RerankingProvider interface {
RerankingModel(modelID string) model.RerankingModel
}
RerankingProvider is a provider that supports document reranking.
type SpeechProvider ¶
type SpeechProvider interface {
SpeechModel(modelID string) model.SpeechModel
}
SpeechProvider is a provider that supports text-to-speech.
type StreamInput ¶
StreamInput wraps the input for provider streaming.
type TranscriptionProvider ¶
type TranscriptionProvider interface {
TranscriptionModel(modelID string) model.TranscriptionModel
}
TranscriptionProvider is a provider that supports speech-to-text.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package anthropic provides an Anthropic provider implementation.
|
Package anthropic provides an Anthropic provider implementation. |
|
Package assemblyai provides an AssemblyAI provider implementation for transcription.
|
Package assemblyai provides an AssemblyAI provider implementation for transcription. |
|
Package azure provides an Azure OpenAI provider implementation.
|
Package azure provides an Azure OpenAI provider implementation. |
|
Package baseten provides a Baseten provider implementation.
|
Package baseten provides a Baseten provider implementation. |
|
Package bedrock provides an Amazon Bedrock provider implementation.
|
Package bedrock provides an Amazon Bedrock provider implementation. |
|
Package blackforestlabs provides a Black Forest Labs (Flux) provider implementation.
|
Package blackforestlabs provides a Black Forest Labs (Flux) provider implementation. |
|
Package cerebras provides a Cerebras provider implementation.
|
Package cerebras provides a Cerebras provider implementation. |
|
Package cohere provides a Cohere provider implementation.
|
Package cohere provides a Cohere provider implementation. |
|
Package deepgram provides a Deepgram provider implementation for speech and transcription.
|
Package deepgram provides a Deepgram provider implementation for speech and transcription. |
|
Package deepinfra provides a DeepInfra provider implementation.
|
Package deepinfra provides a DeepInfra provider implementation. |
|
Package deepseek provides a DeepSeek provider implementation.
|
Package deepseek provides a DeepSeek provider implementation. |
|
Package elevenlabs provides an ElevenLabs provider implementation for speech generation.
|
Package elevenlabs provides an ElevenLabs provider implementation for speech generation. |
|
Package fal provides a fal.ai provider implementation for image generation.
|
Package fal provides a fal.ai provider implementation for image generation. |
|
Package fireworks provides a Fireworks AI provider implementation.
|
Package fireworks provides a Fireworks AI provider implementation. |
|
Package gladia provides a Gladia provider implementation for transcription.
|
Package gladia provides a Gladia provider implementation for transcription. |
|
Package google provides a Google AI (Gemini) provider implementation.
|
Package google provides a Google AI (Gemini) provider implementation. |
|
Package groq provides a Groq provider implementation.
|
Package groq provides a Groq provider implementation. |
|
Package huggingface provides a Hugging Face Inference API provider implementation.
|
Package huggingface provides a Hugging Face Inference API provider implementation. |
|
Package hume provides a Hume AI provider implementation for expressive speech.
|
Package hume provides a Hume AI provider implementation for expressive speech. |
|
Package lmnt provides an LMNT provider implementation for speech generation.
|
Package lmnt provides an LMNT provider implementation for speech generation. |
|
Package luma provides a Luma Labs provider implementation for video/image generation.
|
Package luma provides a Luma Labs provider implementation for video/image generation. |
|
Package mistral provides a Mistral AI provider implementation.
|
Package mistral provides a Mistral AI provider implementation. |
|
Package openai provides an OpenAI provider implementation.
|
Package openai provides an OpenAI provider implementation. |
|
Package openaicompat provides an OpenAI-compatible provider base.
|
Package openaicompat provides an OpenAI-compatible provider base. |
|
Package perplexity provides a Perplexity provider implementation.
|
Package perplexity provides a Perplexity provider implementation. |
|
Package prodia provides a Prodia provider implementation for image generation.
|
Package prodia provides a Prodia provider implementation for image generation. |
|
Package proxy implements a stream.Model that proxies LLM calls through an Airlock-compatible NDJSON endpoint (POST /api/agent/llm/stream).
|
Package proxy implements a stream.Model that proxies LLM calls through an Airlock-compatible NDJSON endpoint (POST /api/agent/llm/stream). |
|
Package replicate provides a Replicate provider implementation.
|
Package replicate provides a Replicate provider implementation. |
|
Package revai provides a Rev.ai provider implementation for transcription.
|
Package revai provides a Rev.ai provider implementation for transcription. |
|
Package togetherai provides a Together AI provider implementation.
|
Package togetherai provides a Together AI provider implementation. |
|
Package vertex provides a Google Vertex AI provider implementation.
|
Package vertex provides a Google Vertex AI provider implementation. |
|
Package vertexanthropic provides a Google Vertex AI Anthropic provider.
|
Package vertexanthropic provides a Google Vertex AI Anthropic provider. |
|
Package vertexmaas provides a Google Vertex AI MaaS (Model as a Service) provider.
|
Package vertexmaas provides a Google Vertex AI MaaS (Model as a Service) provider. |
|
Package xai provides an xAI (Grok) provider implementation.
|
Package xai provides an xAI (Grok) provider implementation. |