provider

package
v1.19.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 20, 2026 License: Apache-2.0 Imports: 17 Imported by: 1

Documentation

Index

Constants

This section is empty.

Variables

View Source
var Aliases = map[string]Alias{
	"requesty": {
		APIType:     "openai",
		BaseURL:     "https://router.requesty.ai/v1",
		TokenEnvVar: "REQUESTY_API_KEY",
	},
	"azure": {
		APIType:     "openai",
		TokenEnvVar: "AZURE_API_KEY",
	},
	"xai": {
		APIType:     "openai",
		BaseURL:     "https://api.x.ai/v1",
		TokenEnvVar: "XAI_API_KEY",
	},
	"nebius": {
		APIType:     "openai",
		BaseURL:     "https://api.studio.nebius.com/v1",
		TokenEnvVar: "NEBIUS_API_KEY",
	},
	"mistral": {
		APIType:     "openai",
		BaseURL:     "https://api.mistral.ai/v1",
		TokenEnvVar: "MISTRAL_API_KEY",
	},
	"ollama": {
		APIType: "openai",
		BaseURL: "http://localhost:11434/v1",
	},
}

Aliases maps provider names to their corresponding configurations

View Source
var CoreProviders = []string{
	"openai",
	"anthropic",
	"google",
	"dmr",
	"amazon-bedrock",
}

CoreProviders lists all natively implemented provider types. These are the provider types that have direct implementations (not aliases).

Functions

func CatalogProviders added in v1.19.3

func CatalogProviders() []string

CatalogProviders returns the list of provider names that should be shown in the model catalog. This includes core providers and aliases that have a defined BaseURL (self-contained endpoints). Aliases without a BaseURL (like azure) require user configuration and are excluded.

func IsCatalogProvider added in v1.19.3

func IsCatalogProvider(name string) bool

IsCatalogProvider returns true if the provider name is valid for the model catalog.

Types

type Alias added in v1.0.10

type Alias struct {
	APIType     string // The actual API type to use (openai, anthropic, etc.)
	BaseURL     string // Default base URL for the provider
	TokenEnvVar string // Environment variable name for the API token
}

Alias defines the configuration for a provider alias

type BatchEmbeddingProvider added in v1.8.2

type BatchEmbeddingProvider interface {
	EmbeddingProvider
	// CreateBatchEmbedding generates embedding vectors for multiple texts with usage tracking.
	// Returns embeddings in the same order as input texts.
	CreateBatchEmbedding(ctx context.Context, texts []string) (*base.BatchEmbeddingResult, error)
}

BatchEmbeddingProvider defines the interface for providers that support batch embeddings.

type EmbeddingProvider added in v1.8.2

type EmbeddingProvider interface {
	Provider
	// CreateEmbedding generates an embedding vector for the given text with usage tracking.
	CreateEmbedding(ctx context.Context, text string) (*base.EmbeddingResult, error)
}

EmbeddingProvider defines the interface for providers that support embeddings.

type Provider

type Provider interface {
	// ID returns the model provider ID
	ID() string
	// CreateChatCompletionStream creates a streaming chat completion request
	// It returns a stream that can be iterated over to get completion chunks
	CreateChatCompletionStream(
		ctx context.Context,
		messages []chat.Message,
		tools []tools.Tool,
	) (chat.MessageStream, error)
	// BaseConfig returns the base configuration of this provider
	BaseConfig() base.Config
}

Provider defines the interface for model providers

func CloneWithOptions added in v1.7.1

func CloneWithOptions(ctx context.Context, base Provider, opts ...options.Opt) Provider

CloneWithOptions returns a new Provider instance using the same provider/model as the base provider, applying the provided options. If cloning fails, the original base provider is returned.

func New

New creates a new provider from a model config. This is a convenience wrapper for NewWithModels with no models map.

func NewWithModels added in v1.8.2

func NewWithModels(ctx context.Context, cfg *latest.ModelConfig, models map[string]latest.ModelConfig, env environment.Provider, opts ...options.Opt) (Provider, error)

NewWithModels creates a new provider from a model config with access to the full models map. The models map is used to resolve model references in routing rules.

type RerankingProvider added in v1.8.2

type RerankingProvider interface {
	Provider
	// Rerank scores documents by relevance to the query.
	// Returns relevance scores in the same order as input documents.
	// Scores are typically in [0, 1] range where higher means more relevant.
	// criteria: Optional domain-specific guidance for relevance scoring (appended to base prompt)
	// documents: Array of types.Document with content and metadata
	Rerank(ctx context.Context, query string, documents []types.Document, criteria string) ([]float64, error)
}

RerankingProvider defines the interface for providers that support reranking. Reranking models score query-document pairs to assess relevance.

Directories

Path Synopsis
Package rulebased provides a rule-based model router that selects the appropriate model based on NLP analysis of the input using Bleve.
Package rulebased provides a rule-based model router that selects the appropriate model based on NLP analysis of the input using Bleve.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL