provider

package
v1.53.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 28, 2026 License: Apache-2.0 Imports: 22 Imported by: 0

Documentation

Overview

Package provider builds and dispatches to LLM provider clients.

The package is organised across four files:

  • provider.go (this file): the public Provider interfaces and the entry points New and NewWithModels that callers use to construct a provider from a model config.
  • aliases.go: the built-in provider alias table (OpenAI-compatible gateways such as ollama, mistral, xai, ...) and the helpers that expose it to other packages without leaking the underlying map.
  • defaults.go: pure config-merging logic that fills in defaults from custom providers, built-in aliases, and model-specific rules (thinking budget, interleaved thinking, ...).
  • factory.go: dispatch from a resolved provider type to the concrete client constructor (openai, anthropic, google, dmr, amazon-bedrock, vertex AI), plus the rule-based router.

Index

Constants

This section is empty.

Variables

View Source
var Aliases = map[string]Alias{
	"requesty": {
		APIType:     "openai",
		BaseURL:     "https://router.requesty.ai/v1",
		TokenEnvVar: "REQUESTY_API_KEY",
	},
	"azure": {
		APIType:     "openai",
		TokenEnvVar: "AZURE_API_KEY",
	},
	"xai": {
		APIType:     "openai",
		BaseURL:     "https://api.x.ai/v1",
		TokenEnvVar: "XAI_API_KEY",
	},
	"nebius": {
		APIType:     "openai",
		BaseURL:     "https://api.studio.nebius.com/v1",
		TokenEnvVar: "NEBIUS_API_KEY",
	},
	"mistral": {
		APIType:     "openai",
		BaseURL:     "https://api.mistral.ai/v1",
		TokenEnvVar: "MISTRAL_API_KEY",
	},
	"ollama": {
		APIType: "openai",
		BaseURL: "http://localhost:11434/v1",
	},
	"minimax": {
		APIType:     "openai",
		BaseURL:     "https://api.minimax.io/v1",
		TokenEnvVar: "MINIMAX_API_KEY",
	},
	"github-copilot": {
		APIType:     "openai",
		BaseURL:     "https://api.githubcopilot.com",
		TokenEnvVar: "GITHUB_TOKEN",
	},
}

Aliases maps provider names to their corresponding configurations.

Most consumers should call LookupAlias for a single lookup or EachAlias to iterate, both of which keep the rest of the codebase decoupled from this concrete map. Direct mutation of Aliases is not supported.

View Source
var CoreProviders = []string{
	"openai",
	"anthropic",
	"google",
	"dmr",
	"amazon-bedrock",
}

CoreProviders lists all natively implemented provider types. These are the provider types that have direct implementations (not aliases).

Functions

func AllProviders

func AllProviders() []string

AllProviders returns all known provider names (core providers + aliases), sorted for deterministic output.

func CatalogProviders

func CatalogProviders() []string

CatalogProviders returns the list of provider names that should be shown in the model catalog. This includes core providers and aliases that have a defined BaseURL (self-contained endpoints). Aliases without a BaseURL (like azure) require user configuration and are excluded.

func EachAlias added in v1.53.0

func EachAlias() iter.Seq2[string, Alias]

EachAlias returns an iterator over every registered (name, Alias) pair. Iteration order is not guaranteed; callers that need a deterministic order should sort by name.

func IsCatalogProvider

func IsCatalogProvider(name string) bool

IsCatalogProvider returns true if the provider name is valid for the model catalog.

func IsKnownProvider

func IsKnownProvider(name string) bool

IsKnownProvider returns true if the provider name is a core provider or an alias.

Types

type Alias

type Alias struct {
	APIType     string // The actual API type to use (openai, anthropic, etc.)
	BaseURL     string // Default base URL for the provider
	TokenEnvVar string // Environment variable name for the API token
}

Alias defines the configuration for a provider alias.

func LookupAlias added in v1.53.0

func LookupAlias(name string) (Alias, bool)

LookupAlias returns the Alias registered for the given name (if any). Lookup is case-sensitive; callers that need case-insensitive matching should normalise the name first (e.g. strings.ToLower).

type BatchEmbeddingProvider

type BatchEmbeddingProvider interface {
	EmbeddingProvider
	// CreateBatchEmbedding generates embedding vectors for multiple texts with usage tracking.
	// Returns embeddings in the same order as input texts.
	CreateBatchEmbedding(ctx context.Context, texts []string) (*base.BatchEmbeddingResult, error)
}

BatchEmbeddingProvider defines the interface for providers that support batch embeddings.

type EmbeddingProvider

type EmbeddingProvider interface {
	Provider
	// CreateEmbedding generates an embedding vector for the given text with usage tracking.
	CreateEmbedding(ctx context.Context, text string) (*base.EmbeddingResult, error)
}

EmbeddingProvider defines the interface for providers that support embeddings.

type Provider

type Provider interface {
	// ID returns the model provider ID
	ID() string
	// CreateChatCompletionStream creates a streaming chat completion request.
	// It returns a stream that can be iterated over to get completion chunks.
	CreateChatCompletionStream(
		ctx context.Context,
		messages []chat.Message,
		tools []tools.Tool,
	) (chat.MessageStream, error)
	// BaseConfig returns the base configuration of this provider.
	BaseConfig() base.Config
}

Provider defines the interface for model providers.

func CloneWithOptions

func CloneWithOptions(ctx context.Context, baseProvider Provider, opts ...options.Opt) Provider

CloneWithOptions returns a new Provider instance using the same provider/model as the base provider, applying the provided options. If cloning fails, the original base provider is returned.

func New

New creates a new provider from a model config. This is a convenience wrapper for NewWithModels with no models map.

func NewWithModels

func NewWithModels(ctx context.Context, cfg *latest.ModelConfig, models map[string]latest.ModelConfig, env environment.Provider, opts ...options.Opt) (Provider, error)

NewWithModels creates a new provider from a model config with access to the full models map. The models map is used to resolve model references in routing rules.

type RerankingProvider

type RerankingProvider interface {
	Provider
	// Rerank scores documents by relevance to the query.
	// Returns relevance scores in the same order as input documents.
	// Scores are typically in [0, 1] range where higher means more relevant.
	// criteria: Optional domain-specific guidance for relevance scoring (appended to base prompt)
	// documents: Array of types.Document with content and metadata
	Rerank(ctx context.Context, query string, documents []types.Document, criteria string) ([]float64, error)
}

RerankingProvider defines the interface for providers that support reranking. Reranking models score query-document pairs to assess relevance.

Directories

Path Synopsis
Package rulebased provides a rule-based model router that selects the appropriate model based on text similarity using Bleve full-text search.
Package rulebased provides a rule-based model router that selects the appropriate model based on text similarity using Bleve full-text search.
Package vertexai provides support for non-Gemini models hosted on Google Cloud's Vertex AI Model Garden.
Package vertexai provides support for non-Gemini models hosted on Google Cloud's Vertex AI Model Garden.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL