Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var Aliases = map[string]Alias{
"requesty": {
APIType: "openai",
BaseURL: "https://router.requesty.ai/v1",
TokenEnvVar: "REQUESTY_API_KEY",
},
"azure": {
APIType: "openai",
TokenEnvVar: "AZURE_API_KEY",
},
"xai": {
APIType: "openai",
BaseURL: "https://api.x.ai/v1",
TokenEnvVar: "XAI_API_KEY",
},
"nebius": {
APIType: "openai",
BaseURL: "https://api.studio.nebius.com/v1",
TokenEnvVar: "NEBIUS_API_KEY",
},
"mistral": {
APIType: "openai",
BaseURL: "https://api.mistral.ai/v1",
TokenEnvVar: "MISTRAL_API_KEY",
},
"ollama": {
APIType: "openai",
BaseURL: "http://localhost:11434/v1",
},
"minimax": {
APIType: "openai",
BaseURL: "https://api.minimax.io/v1",
TokenEnvVar: "MINIMAX_API_KEY",
},
}
Aliases maps provider names to their corresponding configurations
var CoreProviders = []string{
"openai",
"anthropic",
"google",
"dmr",
"amazon-bedrock",
}
CoreProviders lists all natively implemented provider types. These are the provider types that have direct implementations (not aliases).
Functions ¶
func AllProviders ¶
func AllProviders() []string
AllProviders returns all known provider names (core providers + aliases), sorted for deterministic output.
func CatalogProviders ¶
func CatalogProviders() []string
CatalogProviders returns the list of provider names that should be shown in the model catalog. This includes core providers and aliases that have a defined BaseURL (self-contained endpoints). Aliases without a BaseURL (like azure) require user configuration and are excluded.
func IsCatalogProvider ¶
IsCatalogProvider returns true if the provider name is valid for the model catalog.
func IsKnownProvider ¶
IsKnownProvider returns true if the provider name is a core provider or an alias.
Types ¶
type Alias ¶
type Alias struct {
APIType string // The actual API type to use (openai, anthropic, etc.)
BaseURL string // Default base URL for the provider
TokenEnvVar string // Environment variable name for the API token
}
Alias defines the configuration for a provider alias
type BatchEmbeddingProvider ¶
type BatchEmbeddingProvider interface {
EmbeddingProvider
// CreateBatchEmbedding generates embedding vectors for multiple texts with usage tracking.
// Returns embeddings in the same order as input texts.
CreateBatchEmbedding(ctx context.Context, texts []string) (*base.BatchEmbeddingResult, error)
}
BatchEmbeddingProvider defines the interface for providers that support batch embeddings.
type EmbeddingProvider ¶
type EmbeddingProvider interface {
Provider
// CreateEmbedding generates an embedding vector for the given text with usage tracking.
CreateEmbedding(ctx context.Context, text string) (*base.EmbeddingResult, error)
}
EmbeddingProvider defines the interface for providers that support embeddings.
type Provider ¶
type Provider interface {
// ID returns the model provider ID
ID() string
// CreateChatCompletionStream creates a streaming chat completion request
// It returns a stream that can be iterated over to get completion chunks
CreateChatCompletionStream(
ctx context.Context,
messages []chat.Message,
tools []tools.Tool,
) (chat.MessageStream, error)
// BaseConfig returns the base configuration of this provider
BaseConfig() base.Config
}
Provider defines the interface for model providers
func CloneWithOptions ¶
CloneWithOptions returns a new Provider instance using the same provider/model as the base provider, applying the provided options. If cloning fails, the original base provider is returned.
func New ¶
func New(ctx context.Context, cfg *latest.ModelConfig, env environment.Provider, opts ...options.Opt) (Provider, error)
New creates a new provider from a model config. This is a convenience wrapper for NewWithModels with no models map.
func NewWithModels ¶
func NewWithModels(ctx context.Context, cfg *latest.ModelConfig, models map[string]latest.ModelConfig, env environment.Provider, opts ...options.Opt) (Provider, error)
NewWithModels creates a new provider from a model config with access to the full models map. The models map is used to resolve model references in routing rules.
type RerankingProvider ¶
type RerankingProvider interface {
Provider
// Rerank scores documents by relevance to the query.
// Returns relevance scores in the same order as input documents.
// Scores are typically in [0, 1] range where higher means more relevant.
// criteria: Optional domain-specific guidance for relevance scoring (appended to base prompt)
// documents: Array of types.Document with content and metadata
Rerank(ctx context.Context, query string, documents []types.Document, criteria string) ([]float64, error)
}
RerankingProvider defines the interface for providers that support reranking. Reranking models score query-document pairs to assess relevance.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package rulebased provides a rule-based model router that selects the appropriate model based on text similarity using Bleve full-text search.
|
Package rulebased provides a rule-based model router that selects the appropriate model based on text similarity using Bleve full-text search. |