providers

package
v0.16.7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 9, 2026 License: MIT Imports: 24 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	CerebrasClientType    = "cerebras"
	ChutesClientType      = "chutes"
	DeepinfraClientType   = "deepinfra"
	DeepseekClientType    = "deepseek"
	LmstudioClientType    = "lmstudio"
	MinimaxClientType     = "minimax"
	MistralClientType     = "mistral"
	OllamaCloudClientType = "ollama-cloud"
	OpenaiClientType      = "openai"
	OpenrouterClientType  = "openrouter"
	ZaiClientType         = "zai"
)

ClientType constants for all providers These are auto-generated from provider configs (as strings to avoid import cycles)

Variables

This section is empty.

Functions

func AllProviderNames 

func AllProviderNames() []string

AllProviderNames returns all provider names as strings

func BuildOpenAIChatMessages 

func BuildOpenAIChatMessages(messages []api.Message, opts MessageConversionOptions) []map[string]interface{}

BuildOpenAIChatMessages converts agent messages into OpenAI/OpenRouter style chat message payloads, including multimodal content where necessary.

func BuildOpenAIStreamingMessages 

func BuildOpenAIStreamingMessages(messages []api.Message, opts MessageConversionOptions) []interface{}

BuildOpenAIStreamingMessages converts messages for streaming endpoints. The content is identical to the chat payload, but represented as []interface{} to match the JSON marshalling performed by providers.

func BuildOpenAIToolsPayload 

func BuildOpenAIToolsPayload(tools []api.Tool) []map[string]interface{}

BuildOpenAIToolsPayload normalises internal tool definitions to the OpenAI function-calling schema used by OpenRouter, DeepInfra, and other compatible providers.

func CalculateMaxTokens 

func CalculateMaxTokens(contextLimit int, messages []api.Message, tools []api.Tool) int

CalculateMaxTokens returns an appropriate max_tokens value given the context window and prompt size. The caller passes the effective context limit, making it easy to reuse across providers with custom limit lookups.

func CalculateMaxTokensWithLimits 

func CalculateMaxTokensWithLimits(contextLimit int, completionLimit int, messages []api.Message, tools []api.Tool) int

CalculateMaxTokensWithLimits computes a token budget from context and optional completion caps. Uses centralized token estimation for consistency across all providers.

func ClientTypeToString 

func ClientTypeToString(ct string) string

ClientTypeToString converts ClientType (as string) to string This replaces the hardcoded mapClientTypeToString function

func EstimateInputTokens 

func EstimateInputTokens(messages []api.Message, tools []api.Tool) int

EstimateInputTokens provides a quick upper bound for prompt tokens based on message lengths and attached tool metadata. Delegates to the centralized implementation in agent_api for consistency.

func KnownProviders 

func KnownProviders() []string

KnownProviders returns the list of known built-in provider names This replaces the hardcoded knownProviderNames list in api_keys.go

func ProviderDisplayNames 

func ProviderDisplayNames() map[string]string

ProviderDisplayNames returns a map of provider names to display names This replaces the hardcoded knownProviderDisplayNames map in api_keys.go

func ProviderEnvVar 

func ProviderEnvVar(name string) string

ProviderEnvVar returns the environment variable name for a provider's API key

func ProviderRequiresAPIKey 

func ProviderRequiresAPIKey(name string) bool

ProviderRequiresAPIKey returns whether a provider requires an API key

func StringToClientType 

func StringToClientType(name string) (string, error)

StringToClientType converts a string to ClientType (as string) This replaces the hardcoded ParseProviderName function

Types

type AuthConfig 

type AuthConfig struct {
	Type   string `json:"type"`    // "bearer", "api_key", "basic", "oauth"
	EnvVar string `json:"env_var"` // Environment variable containing the auth token
	Key    string `json:"-"`       // Runtime-only API key; injected at startup, never persisted
}

AuthConfig defines authentication configuration

type CostConfig 

type CostConfig struct {
	InputTokenCost  float64 `json:"input_token_cost"`
	OutputTokenCost float64 `json:"output_token_cost"`
	Currency        string  `json:"currency"`
}

CostConfig defines cost tracking configuration

type GenericProvider 

type GenericProvider struct {
	// contains filtered or unexported fields
}

GenericProvider implements ClientInterface using JSON configuration

func NewGenericProvider 

func NewGenericProvider(config *ProviderConfig) (*GenericProvider, error)

NewGenericProvider creates a new generic provider from configuration

func (*GenericProvider) CheckConnection 

func (p *GenericProvider) CheckConnection() error

CheckConnection tests provider connection with current model

func (*GenericProvider) GetAverageTPS 

func (p *GenericProvider) GetAverageTPS() float64

func (*GenericProvider) GetHTTPClient 

func (p *GenericProvider) GetHTTPClient() *http.Client

GetHTTPClient returns the current HTTP client used for non-streaming requests. Useful for WASM environments that need to verify client injection.

func (*GenericProvider) GetLastTPS 

func (p *GenericProvider) GetLastTPS() float64

TPS tracking methods - simplified for now

func (*GenericProvider) GetModel 

func (p *GenericProvider) GetModel() string

GetModel returns the current model

func (*GenericProvider) GetModelContextLimit 

func (p *GenericProvider) GetModelContextLimit() (int, error)

GetModelContextLimit returns the context limit for the current model

func (*GenericProvider) GetProvider 

func (p *GenericProvider) GetProvider() string

GetProvider returns the provider name

func (*GenericProvider) GetStreamingClient 

func (p *GenericProvider) GetStreamingClient() *http.Client

GetStreamingClient returns the current HTTP client used for streaming requests. Useful for WASM environments that need to verify client injection.

func (*GenericProvider) GetTPSStats 

func (p *GenericProvider) GetTPSStats() map[string]float64

func (*GenericProvider) GetVisionModel 

func (p *GenericProvider) GetVisionModel() string

GetVisionModel returns the vision model

func (*GenericProvider) ListModels 

func (p *GenericProvider) ListModels(ctx context.Context) ([]api.ModelInfo, error)

ListModels returns available models Priority: 1. Fetch from provider API models endpoint (primary source of truth) 2. Enrich endpoint data with config (context_length, tags, name) 3. Fall back to config model_info if endpoint fails 4. Final fallback: return just current model

func (*GenericProvider) RefreshAPIKey 

func (p *GenericProvider) RefreshAPIKey() error

RefreshAPIKey re-resolves the provider's API key from the credential store, updating the cached key in p.config.Auth.Key. This is called after a rate-limit rotation advances the key pool counter, so subsequent requests use the new key.

func (*GenericProvider) ResetTPSStats 

func (p *GenericProvider) ResetTPSStats()

func (*GenericProvider) SendChatRequest 

func (p *GenericProvider) SendChatRequest(ctx context.Context, messages []api.Message, tools []api.Tool, reasoning string, disableThinking bool) (*api.ChatResponse, error)

SendChatRequest sends a non-streaming chat request

func (*GenericProvider) SendChatRequestStream 

func (p *GenericProvider) SendChatRequestStream(ctx context.Context, messages []api.Message, tools []api.Tool, reasoning string, disableThinking bool, callback api.StreamCallback) (*api.ChatResponse, error)

SendChatRequestStream sends a streaming chat request

func (*GenericProvider) SendVisionRequest 

func (p *GenericProvider) SendVisionRequest(ctx context.Context, messages []api.Message, tools []api.Tool, reasoning string, disableThinking bool) (*api.ChatResponse, error)

SendVisionRequest sends a vision request (for providers that support it)

func (*GenericProvider) SetDebug 

func (p *GenericProvider) SetDebug(debug bool)

SetDebug enables or disables debug mode

func (*GenericProvider) SetHTTPClient 

func (p *GenericProvider) SetHTTPClient(c *http.Client)

SetHTTPClient sets the HTTP client used for non-streaming requests.

func (*GenericProvider) SetModel 

func (p *GenericProvider) SetModel(model string) error

SetModel sets the current model

func (*GenericProvider) SetStreamingClient 

func (p *GenericProvider) SetStreamingClient(c *http.Client)

SetStreamingClient sets the HTTP client used for streaming requests.

func (*GenericProvider) SupportsVision 

func (p *GenericProvider) SupportsVision() bool

SupportsVision returns whether the provider supports vision

type MessageConversion 

type MessageConversion struct {
	IncludeToolCallID        bool   `json:"include_tool_call_id"`
	ConvertToolRoleToUser    bool   `json:"convert_tool_role_to_user"`
	ReasoningContentField    string `json:"reasoning_content_field"`
	ArgumentsAsJSON          bool   `json:"arguments_as_json"`
	SkipToolExecutionSummary bool   `json:"skip_tool_execution_summary"` // For providers with strict role alternation
	ForceToolCallType        string `json:"force_tool_call_type"`        // Force tool call type to specific value (e.g., "function" for Mistral)
}

MessageConversion defines how messages should be converted

type MessageConversionOptions 

type MessageConversionOptions struct {
	// Convert tool-role messages to user messages with a labeled prefix. Some
	// providers (DeepInfra) reject the "tool" role entirely.
	ConvertToolRoleToUser bool
	// Include tool_call_id when present. Required for OpenRouter when sending
	// tool execution results back to the model.
	IncludeToolCallID bool
	// Force tool call type to specific value (e.g., "function" for Mistral)
	ForceToolCallType string
}

MessageConversionOptions controls how agent messages are transformed into OpenAI-compatible payloads.

type ModelConfig 

type ModelConfig struct {
	DefaultContextLimit        int               `json:"default_context_limit"`
	DefaultMaxCompletionTokens int               `json:"default_max_completion_tokens,omitempty"`
	ModelOverrides             map[string]int    `json:"model_overrides"`
	MaxCompletionOverrides     map[string]int    `json:"max_completion_overrides,omitempty"`
	PatternOverrides           []PatternOverride `json:"pattern_overrides"`
	CompletionPatternOverrides []PatternOverride `json:"completion_pattern_overrides,omitempty"`
	// Config-based model definitions (fallback when endpoint fetch fails or lacks details)
	ModelInfo []ModelInfo `json:"model_info,omitempty"`
	// Legacy fields for backward compatibility
	ContextLimit    int      `json:"context_limit,omitempty"`
	SupportsVision  bool     `json:"supports_vision"`
	VisionModel     string   `json:"vision_model"`
	DefaultModel    string   `json:"default_model"`
	AvailableModels []string `json:"available_models"`
}

ModelConfig defines model-related configuration

type ModelInfo 

type ModelInfo struct {
	ID            string   `json:"id"`
	Name          string   `json:"name,omitempty"`
	Description   string   `json:"description,omitempty"`
	ContextLength int      `json:"context_length"`
	Tags          []string `json:"tags,omitempty"`
}

ModelInfo represents information about a model (simplified version for config)

type PatternOverride 

type PatternOverride struct {
	Pattern      string `json:"pattern"`
	ContextLimit int    `json:"context_limit"`
}

PatternOverride defines context limit overrides for model patterns

type ProviderConfig 

type ProviderConfig struct {
	Name       string            `json:"name"`
	Endpoint   string            `json:"endpoint"`
	Auth       AuthConfig        `json:"auth"`
	Headers    map[string]string `json:"headers"`
	Defaults   RequestDefaults   `json:"defaults"`
	Conversion MessageConversion `json:"message_conversion"`
	Streaming  StreamingConfig   `json:"streaming"`
	Models     ModelConfig       `json:"models"`
	Retry      RetryConfig       `json:"retry"`
	Cost       CostConfig        `json:"cost"`
}

ProviderConfig defines the configuration for a generic provider

func LoadProviderConfig 

func LoadProviderConfig(configPath string) (*ProviderConfig, error)

LoadProviderConfig loads a provider configuration from a JSON file

func (*ProviderConfig) GetAuthToken 

func (c *ProviderConfig) GetAuthToken() (string, error)

GetAuthToken retrieves the authentication token based on the auth configuration.

For "bearer" and "api_key" auth types, token resolution follows this precedence:

  1. Auth.Key — runtime-resolved key injected by the provider factory via the unified credential path (credentials.ResolveProvider). This is the primary source because it checks env vars, keyring, and the encrypted file store.
  2. Auth.EnvVar — direct os.Getenv lookup as a fallback (only used when the factory did not pre-resolve credentials, e.g., in unit tests).

Auth.Key is runtime-only and must never be persisted to disk.

func (*ProviderConfig) GetContextLimit 

func (c *ProviderConfig) GetContextLimit(model string) int

GetContextLimit returns the context limit for a given model based on configuration Uses the following priority: 1. Exact model match in model_overrides 2. Pattern match in pattern_overrides 3. Provider default_context_limit 4. Legacy context_limit field (for backward compatibility) 5. Conservative fallback (32000)

func (*ProviderConfig) GetMaxCompletionLimit 

func (c *ProviderConfig) GetMaxCompletionLimit(model string) int

GetMaxCompletionLimit returns the completion-token limit for a given model. Uses the following priority: 1. Exact model match in max_completion_overrides 2. Pattern match in completion_pattern_overrides 3. Provider default_max_completion_tokens 4. 0 (unknown/unset)

func (*ProviderConfig) GetModelInfo 

func (c *ProviderConfig) GetModelInfo(modelID string) *ModelInfo

GetModelInfo returns model information from config if available

func (*ProviderConfig) GetStreamingTimeout 

func (c *ProviderConfig) GetStreamingTimeout() time.Duration

GetStreamingTimeout returns the configured streaming timeout duration

func (*ProviderConfig) GetTimeout 

func (c *ProviderConfig) GetTimeout() time.Duration

GetTimeout returns the configured timeout duration

func (*ProviderConfig) Validate 

func (c *ProviderConfig) Validate() error

Validate validates the provider configuration

type ProviderFactory 

type ProviderFactory struct {
	// contains filtered or unexported fields
}

ProviderFactory creates provider instances from JSON configurations

func GlobalFactory added in v0.16.2 

func GlobalFactory() *ProviderFactory

GlobalFactory returns the singleton ProviderFactory instance. It is initialized with embedded configs during package init.

func NewProviderFactory 

func NewProviderFactory() *ProviderFactory

NewProviderFactory creates a new provider factory

func (*ProviderFactory) CreateProvider 

func (f *ProviderFactory) CreateProvider(name string) (api.ClientInterface, error)

CreateProvider creates a provider instance by name

func (*ProviderFactory) CreateProviderWithModel 

func (f *ProviderFactory) CreateProviderWithModel(name, model string) (api.ClientInterface, error)

CreateProviderWithModel creates a provider instance with a specific model

func (*ProviderFactory) GetAvailableProviders 

func (f *ProviderFactory) GetAvailableProviders() []string

GetAvailableProviders returns a list of available provider names

func (*ProviderFactory) GetDefaultProvider 

func (f *ProviderFactory) GetDefaultProvider() string

GetDefaultProvider returns the default provider name

func (*ProviderFactory) GetProviderConfig 

func (f *ProviderFactory) GetProviderConfig(name string) (*ProviderConfig, error)

GetProviderConfig returns a copy of the configuration for a provider. A copy is returned (rather than a pointer to internal state) so that callers cannot mutate the factory's stored config after the RLock is released.

func (*ProviderFactory) GetRegistry 

func (f *ProviderFactory) GetRegistry() *ProviderRegistry

GetRegistry returns a deep copy of the provider registry. A copy is returned (rather than a pointer to internal state) so that callers cannot mutate the factory's stored registry after the RLock is released.

func (*ProviderFactory) ListProvidersWithModels 

func (f *ProviderFactory) ListProvidersWithModels() map[string][]string

ListProvidersWithModels returns all providers with their available models

func (*ProviderFactory) LoadConfigFromBytes 

func (f *ProviderFactory) LoadConfigFromBytes(data []byte) error

LoadConfigFromBytes loads a provider configuration from byte data

func (*ProviderFactory) LoadConfigFromFile 

func (f *ProviderFactory) LoadConfigFromFile(filename string) error

LoadConfigFromFile loads a single provider configuration from file

func (*ProviderFactory) LoadConfigsFromDirectory 

func (f *ProviderFactory) LoadConfigsFromDirectory(configDir string) error

LoadConfigsFromDirectory loads all provider configurations from a directory

func (*ProviderFactory) LoadEmbeddedConfigs 

func (f *ProviderFactory) LoadEmbeddedConfigs() error

LoadEmbeddedConfigs loads all provider configurations from the embedded filesystem

func (*ProviderFactory) ReloadConfig 

func (f *ProviderFactory) ReloadConfig(filename string) error

ReloadConfig reloads a provider configuration from file

func (*ProviderFactory) UpsertConfig added in v0.16.2 

func (f *ProviderFactory) UpsertConfig(name string, cfg *ProviderConfig) error

UpsertConfig inserts or updates a provider configuration in the factory. The provided config is deep-copied so external mutations have no effect. If cfg.Name differs from name, cfg.Name is overwritten to match name for consistency. The config is validated before insertion.

func (*ProviderFactory) ValidateProvider 

func (f *ProviderFactory) ValidateProvider(providerName, modelName string) error

ValidateProvider checks if a provider and model combination is valid

type ProviderRegistry 

type ProviderRegistry struct {
	DefaultProvider  string                    `json:"default_provider"`
	EnabledProviders []string                  `json:"enabled_providers"`
	ProviderConfigs  map[string]ProviderConfig `json:"provider_configs"`
}

ProviderRegistry holds all provider configurations

func LoadProviderRegistry 

func LoadProviderRegistry(registryPath string) (*ProviderRegistry, error)

LoadProviderRegistry loads the provider registry from a JSON file

type RequestDefaults 

type RequestDefaults struct {
	Model       string                 `json:"model"`
	Temperature *float64               `json:"temperature"`
	MaxTokens   *int                   `json:"max_tokens"`
	TopP        *float64               `json:"top_p"`
	Parameters  map[string]interface{} `json:"parameters,omitempty"` // Provider-specific parameters
}

RequestDefaults defines default request parameters

type RetryConfig 

type RetryConfig struct {
	MaxAttempts       int      `json:"max_attempts"`
	BaseDelayMs       int      `json:"base_delay_ms"`
	BackoffMultiplier float64  `json:"backoff_multiplier"`
	MaxDelayMs        int      `json:"max_delay_ms"`
	RetryableErrors   []string `json:"retryable_errors"`
}

RetryConfig defines retry behavior

type StreamingConfig 

type StreamingConfig struct {
	Format         string `json:"format"` // "sse", "json_lines", "raw"
	ChunkTimeoutMs int    `json:"chunk_timeout_ms"`
	DoneMarker     string `json:"done_marker"`
}

StreamingConfig defines streaming behavior

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.