Documentation
¶
Overview ¶
Package provider builds and dispatches to LLM provider clients.
The package is organised across four files:
- provider.go (this file): the public Provider interfaces and the entry points New and NewWithModels that callers use to construct a provider from a model config.
- aliases.go: the built-in provider alias table (OpenAI-compatible gateways such as ollama, mistral, xai, ...) and the helpers that expose it to other packages without leaking the underlying map.
- defaults.go: pure config-merging logic that fills in defaults from custom providers, built-in aliases, and model-specific rules (thinking budget, interleaved thinking, ...).
- factory.go: dispatch from a resolved provider type to the concrete client constructor (openai, anthropic, google, dmr, amazon-bedrock, vertex AI), plus the rule-based router.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var Aliases = map[string]Alias{
"requesty": {
APIType: "openai",
BaseURL: "https://router.requesty.ai/v1",
TokenEnvVar: "REQUESTY_API_KEY",
},
"azure": {
APIType: "openai",
TokenEnvVar: "AZURE_API_KEY",
},
"xai": {
APIType: "openai",
BaseURL: "https://api.x.ai/v1",
TokenEnvVar: "XAI_API_KEY",
},
"nebius": {
APIType: "openai",
BaseURL: "https://api.studio.nebius.com/v1",
TokenEnvVar: "NEBIUS_API_KEY",
},
"mistral": {
APIType: "openai",
BaseURL: "https://api.mistral.ai/v1",
TokenEnvVar: "MISTRAL_API_KEY",
},
"ollama": {
APIType: "openai",
BaseURL: "http://localhost:11434/v1",
},
"minimax": {
APIType: "openai",
BaseURL: "https://api.minimax.io/v1",
TokenEnvVar: "MINIMAX_API_KEY",
},
"github-copilot": {
APIType: "openai",
BaseURL: "https://api.githubcopilot.com",
TokenEnvVar: "GITHUB_TOKEN",
},
}
Aliases maps provider names to their corresponding configurations.
Most consumers should call LookupAlias for a single lookup or EachAlias to iterate, both of which keep the rest of the codebase decoupled from this concrete map. Direct mutation of Aliases is not supported.
var CoreProviders = []string{
"openai",
"anthropic",
"google",
"dmr",
"amazon-bedrock",
}
CoreProviders lists all natively implemented provider types. These are the provider types that have direct implementations (not aliases).
Functions ¶
func AllProviders ¶
func AllProviders() []string
AllProviders returns all known provider names (core providers + aliases), sorted for deterministic output.
func CatalogProviders ¶
func CatalogProviders() []string
CatalogProviders returns the list of provider names that should be shown in the model catalog. This includes core providers and aliases that have a defined BaseURL (self-contained endpoints). Aliases without a BaseURL (like azure) require user configuration and are excluded.
func EachAlias ¶ added in v1.53.0
EachAlias returns an iterator over every registered (name, Alias) pair. Iteration order is not guaranteed; callers that need a deterministic order should sort by name.
func IsCatalogProvider ¶
IsCatalogProvider returns true if the provider name is valid for the model catalog.
func IsKnownProvider ¶
IsKnownProvider returns true if the provider name is a core provider or an alias.
Types ¶
type Alias ¶
type Alias struct {
APIType string // The actual API type to use (openai, anthropic, etc.)
BaseURL string // Default base URL for the provider
TokenEnvVar string // Environment variable name for the API token
}
Alias defines the configuration for a provider alias.
func LookupAlias ¶ added in v1.53.0
LookupAlias returns the Alias registered for the given name (if any). Lookup is case-sensitive; callers that need case-insensitive matching should normalise the name first (e.g. strings.ToLower).
type BatchEmbeddingProvider ¶
type BatchEmbeddingProvider interface {
EmbeddingProvider
// CreateBatchEmbedding generates embedding vectors for multiple texts with usage tracking.
// Returns embeddings in the same order as input texts.
CreateBatchEmbedding(ctx context.Context, texts []string) (*base.BatchEmbeddingResult, error)
}
BatchEmbeddingProvider defines the interface for providers that support batch embeddings.
type EmbeddingProvider ¶
type EmbeddingProvider interface {
Provider
// CreateEmbedding generates an embedding vector for the given text with usage tracking.
CreateEmbedding(ctx context.Context, text string) (*base.EmbeddingResult, error)
}
EmbeddingProvider defines the interface for providers that support embeddings.
type Provider ¶
type Provider interface {
// ID returns the model provider ID
ID() string
// CreateChatCompletionStream creates a streaming chat completion request.
// It returns a stream that can be iterated over to get completion chunks.
CreateChatCompletionStream(
ctx context.Context,
messages []chat.Message,
tools []tools.Tool,
) (chat.MessageStream, error)
// BaseConfig returns the base configuration of this provider.
BaseConfig() base.Config
}
Provider defines the interface for model providers.
func CloneWithOptions ¶
CloneWithOptions returns a new Provider instance using the same provider/model as the base provider, applying the provided options. If cloning fails, the original base provider is returned.
func New ¶
func New(ctx context.Context, cfg *latest.ModelConfig, env environment.Provider, opts ...options.Opt) (Provider, error)
New creates a new provider from a model config. This is a convenience wrapper for NewWithModels with no models map.
func NewWithModels ¶
func NewWithModels(ctx context.Context, cfg *latest.ModelConfig, models map[string]latest.ModelConfig, env environment.Provider, opts ...options.Opt) (Provider, error)
NewWithModels creates a new provider from a model config with access to the full models map. The models map is used to resolve model references in routing rules.
type RerankingProvider ¶
type RerankingProvider interface {
Provider
// Rerank scores documents by relevance to the query.
// Returns relevance scores in the same order as input documents.
// Scores are typically in [0, 1] range where higher means more relevant.
// criteria: Optional domain-specific guidance for relevance scoring (appended to base prompt)
// documents: Array of types.Document with content and metadata
Rerank(ctx context.Context, query string, documents []types.Document, criteria string) ([]float64, error)
}
RerankingProvider defines the interface for providers that support reranking. Reranking models score query-document pairs to assess relevance.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package rulebased provides a rule-based model router that selects the appropriate model based on text similarity using Bleve full-text search.
|
Package rulebased provides a rule-based model router that selects the appropriate model based on text similarity using Bleve full-text search. |
|
Package vertexai provides support for non-Gemini models hosted on Google Cloud's Vertex AI Model Garden.
|
Package vertexai provides support for non-Gemini models hosted on Google Cloud's Vertex AI Model Garden. |