providers

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 11, 2026 License: AGPL-3.0 Imports: 16 Imported by: 0

Documentation

Overview

Package providers defines provider-agnostic interfaces and factories for LLM backends used by the Trace agent. It enables per-role provider configuration (Main, Router, ParamExtractor) so each role can use a different provider (Ollama, Anthropic, OpenAI, Gemini).

Thread Safety: 

All interfaces in this package must be implemented as safe for concurrent use.

Index

Constants

View Source 
const (
	ProviderOllama    = "ollama"
	ProviderAnthropic = "anthropic"
	ProviderOpenAI    = "openai"
	ProviderGemini    = "gemini"
)

Provider constants for supported LLM providers.

View Source 
const (
	RoleMain           = "MAIN"
	RoleRouter         = "ROUTER"
	RoleParamExtractor = "PARAM"
)

Role constants for LLM roles in the Trace agent.

Variables

View Source 
var ValidProviders = []string{ProviderOllama, ProviderAnthropic, ProviderOpenAI, ProviderGemini}

ValidProviders contains the set of valid provider names.

Functions

func InferProvider 

func InferProvider(model string) string

InferProvider infers the provider from a model name prefix.

Description: 

Maps known model name prefixes to provider names:
  - "claude-*" -> "anthropic"
  - "gpt-*" -> "openai"
  - "gemini-*" -> "gemini"
  - anything else -> "" (unknown)

This is a utility function for display/inference purposes.
It does not auto-apply; the default Ollama fallback is unchanged.

Inputs:

  • model: The model name to infer from.

Outputs:

  • string: The inferred provider name, or empty string if unknown.

func ResolveOllamaURL 

func ResolveOllamaURL() string

ResolveOllamaURL resolves the Ollama server URL from environment variables.

Description: 

Resolution order:
  1. OLLAMA_BASE_URL (preferred)
  2. OLLAMA_URL (deprecated, emits warning)
  3. http://localhost:11434 (default)

Outputs:

  • string: The resolved Ollama URL.

Types

type AnthropicChatAdapter 

type AnthropicChatAdapter struct {
	// contains filtered or unexported fields
}

AnthropicChatAdapter wraps AnthropicClient to implement ChatClient.

Description: 

Delegates chat requests to the Anthropic Claude API via the existing
AnthropicClient. Ollama-specific options (KeepAlive, NumCtx) are ignored.

Thread Safety: AnthropicChatAdapter is safe for concurrent use.

func NewAnthropicChatAdapter 

func NewAnthropicChatAdapter(client *llm.AnthropicClient) *AnthropicChatAdapter

NewAnthropicChatAdapter creates a new AnthropicChatAdapter.

Inputs:

  • client: The AnthropicClient to wrap. Must not be nil.

Outputs:

  • *AnthropicChatAdapter: The configured adapter.

func (*AnthropicChatAdapter) Chat 

func (a *AnthropicChatAdapter) Chat(ctx context.Context, messages []datatypes.Message, opts ChatOptions) (string, error)

Chat implements ChatClient by delegating to AnthropicClient.Chat.

type ChatClient 

type ChatClient interface {
	// Chat sends messages and returns the assistant's response text.
	//
	// Inputs:
	//   - ctx: Context for cancellation and timeout.
	//   - messages: Conversation messages (system, user, assistant).
	//   - opts: Provider-agnostic chat options.
	//
	// Outputs:
	//   - string: The assistant's response text.
	//   - error: Non-nil on failure.
	Chat(ctx context.Context, messages []datatypes.Message, opts ChatOptions) (string, error)
}

ChatClient is the minimal interface used by Router and ParamExtractor.

Description: 

Router and ParamExtractor only need simple chat (no tool calls, no streaming).
This minimal interface makes adapters trivial for any provider.

Thread Safety: Implementations must be safe for concurrent use.

type ChatOptions 

type ChatOptions struct {
	// Temperature controls randomness (0.0-1.0). Set to 0.0 for most
	// deterministic output. Set to a negative value (e.g., -1) to omit
	// from the request and use the provider's default. The Go zero value
	// (0.0) is treated as an explicit "most deterministic" setting.
	Temperature float64

	// MaxTokens limits the response length.
	MaxTokens int

	// KeepAlive controls model VRAM lifetime (Ollama-specific, ignored by cloud).
	KeepAlive string

	// NumCtx sets the context window size (Ollama-specific, ignored by cloud).
	NumCtx int

	// Model specifies the model for this request. For OllamaChatAdapter, if empty,
	// falls back to the defaultModel set at adapter construction time.
	// For cloud providers, this is typically ignored (model set at client creation).
	Model string
}

ChatOptions holds provider-agnostic options for a chat request.

Description: 

Contains options common to all providers plus Ollama-specific fields
that are ignored by cloud providers.

type CloudLifecycleAdapter 

type CloudLifecycleAdapter struct {
	// contains filtered or unexported fields
}

CloudLifecycleAdapter is a no-op lifecycle manager for cloud providers.

Description: 

Cloud providers (Anthropic, OpenAI, Gemini) don't need explicit model
loading/unloading. WarmModel logs a confirmation, UnloadModel is a no-op.
IsLocal returns false so callers skip the re-warm dance.

Thread Safety: CloudLifecycleAdapter is safe for concurrent use.

func NewCloudLifecycleAdapter 

func NewCloudLifecycleAdapter(provider string) *CloudLifecycleAdapter

NewCloudLifecycleAdapter creates a new CloudLifecycleAdapter.

Inputs:

  • provider: The provider name (for logging).

Outputs:

  • *CloudLifecycleAdapter: The configured adapter.

func (*CloudLifecycleAdapter) IsLocal 

func (a *CloudLifecycleAdapter) IsLocal() bool

IsLocal returns false because cloud providers don't manage local GPU resources.

func (*CloudLifecycleAdapter) UnloadModel 

func (a *CloudLifecycleAdapter) UnloadModel(ctx context.Context, model string) error

UnloadModel is a no-op for cloud providers.

func (*CloudLifecycleAdapter) WarmModel 

func (a *CloudLifecycleAdapter) WarmModel(ctx context.Context, model string, opts WarmupOptions) error

WarmModel is a no-op for cloud providers. Logs the action for visibility.

type FactoryOption 

type FactoryOption func(*ProviderFactory)

FactoryOption configures a ProviderFactory.

func WithEgressGuard 

func WithEgressGuard(builder *egress.EgressGuardBuilder) FactoryOption

WithEgressGuard configures the factory to wrap all created cloud clients with egress guard decorators for data egress control and audit trail.

Inputs:

  • builder: The egress guard builder with shared components.

Outputs:

  • FactoryOption: Option to pass to NewProviderFactory.

type GeminiChatAdapter 

type GeminiChatAdapter struct {
	// contains filtered or unexported fields
}

GeminiChatAdapter wraps GeminiClient to implement ChatClient.

Description: 

Delegates chat requests to the Google Gemini API via GeminiClient.
Ollama-specific options (KeepAlive, NumCtx) are ignored.

Thread Safety: GeminiChatAdapter is safe for concurrent use.

func NewGeminiChatAdapter 

func NewGeminiChatAdapter(client *llm.GeminiClient) *GeminiChatAdapter

NewGeminiChatAdapter creates a new GeminiChatAdapter.

Inputs:

  • client: The GeminiClient to wrap. Must not be nil.

Outputs:

  • *GeminiChatAdapter: The configured adapter.

func (*GeminiChatAdapter) Chat 

func (a *GeminiChatAdapter) Chat(ctx context.Context, messages []datatypes.Message, opts ChatOptions) (string, error)

Chat implements ChatClient by delegating to GeminiClient.Chat.

type ModelLifecycleManager 

type ModelLifecycleManager interface {
	// WarmModel pre-loads or validates a model.
	//
	// For Ollama: loads model into VRAM with keep_alive.
	// For cloud: validates API key and connectivity.
	//
	// Inputs:
	//   - ctx: Context for cancellation.
	//   - model: Provider-specific model identifier.
	//   - opts: Warmup options.
	//
	// Outputs:
	//   - error: Non-nil if warmup/validation fails.
	WarmModel(ctx context.Context, model string, opts WarmupOptions) error

	// UnloadModel releases model resources.
	//
	// For Ollama: unloads model from VRAM.
	// For cloud: no-op.
	//
	// Inputs:
	//   - ctx: Context for cancellation.
	//   - model: Provider-specific model identifier.
	//
	// Outputs:
	//   - error: Non-nil if unload fails.
	UnloadModel(ctx context.Context, model string) error

	// IsLocal returns true if the provider manages local GPU resources.
	//
	// When true, the caller should perform the re-warm dance after loading
	// a new model (re-warm main model after loading router model).
	// When false (cloud providers), no re-warm is needed.
	IsLocal() bool
}

ModelLifecycleManager handles provider-specific model lifecycle operations.

Description: 

Ollama needs explicit warmup (loading model into VRAM) and unload operations.
Cloud providers only need an auth check on warmup. The IsLocal() method
allows callers to skip the re-warm dance for cloud providers.

Thread Safety: Implementations must be safe for concurrent use.

type OllamaChatAdapter 

type OllamaChatAdapter struct {
	// contains filtered or unexported fields
}

OllamaChatAdapter wraps MultiModelManager to implement ChatClient.

Description: 

Delegates chat requests to the shared MultiModelManager, which coordinates
multiple Ollama models and prevents VRAM thrashing.

Thread Safety: OllamaChatAdapter is safe for concurrent use.

func NewOllamaChatAdapter 

func NewOllamaChatAdapter(manager *llm.MultiModelManager, defaultModel string) *OllamaChatAdapter

NewOllamaChatAdapter creates a new OllamaChatAdapter.

Description: 

Creates an adapter that delegates to the MultiModelManager for chat.
The defaultModel is used as a fallback when ChatOptions.Model is empty.

Inputs:

  • manager: The MultiModelManager to delegate to. Must not be nil.
  • defaultModel: Fallback model when ChatOptions.Model is empty. May be empty if the caller always provides a model in ChatOptions.

Outputs:

  • *OllamaChatAdapter: The configured adapter.

func (*OllamaChatAdapter) Chat 

func (a *OllamaChatAdapter) Chat(ctx context.Context, messages []datatypes.Message, opts ChatOptions) (string, error)

Chat implements ChatClient by delegating to MultiModelManager.Chat.

type OllamaLifecycleAdapter 

type OllamaLifecycleAdapter struct {
	// contains filtered or unexported fields
}

OllamaLifecycleAdapter wraps MultiModelManager for lifecycle operations.

Thread Safety: OllamaLifecycleAdapter is safe for concurrent use.

func NewOllamaLifecycleAdapter 

func NewOllamaLifecycleAdapter(manager *llm.MultiModelManager) *OllamaLifecycleAdapter

NewOllamaLifecycleAdapter creates a new OllamaLifecycleAdapter.

Inputs:

  • manager: The MultiModelManager to delegate to. Must not be nil.

Outputs:

  • *OllamaLifecycleAdapter: The configured adapter.

func (*OllamaLifecycleAdapter) IsLocal 

func (a *OllamaLifecycleAdapter) IsLocal() bool

IsLocal returns true because Ollama manages local GPU resources.

func (*OllamaLifecycleAdapter) UnloadModel 

func (a *OllamaLifecycleAdapter) UnloadModel(ctx context.Context, model string) error

UnloadModel unloads a model from VRAM via MultiModelManager.

func (*OllamaLifecycleAdapter) WarmModel 

func (a *OllamaLifecycleAdapter) WarmModel(ctx context.Context, model string, opts WarmupOptions) error

WarmModel loads a model into VRAM via MultiModelManager.

type OpenAIChatAdapter 

type OpenAIChatAdapter struct {
	// contains filtered or unexported fields
}

OpenAIChatAdapter wraps OpenAIClient to implement ChatClient.

Description: 

Delegates chat requests to the OpenAI API via the existing OpenAIClient.
Ollama-specific options (KeepAlive, NumCtx) are ignored.

Thread Safety: OpenAIChatAdapter is safe for concurrent use.

func NewOpenAIChatAdapter 

func NewOpenAIChatAdapter(client *llm.OpenAIClient) *OpenAIChatAdapter

NewOpenAIChatAdapter creates a new OpenAIChatAdapter.

Inputs:

  • client: The OpenAIClient to wrap. Must not be nil.

Outputs:

  • *OpenAIChatAdapter: The configured adapter.

func (*OpenAIChatAdapter) Chat 

func (a *OpenAIChatAdapter) Chat(ctx context.Context, messages []datatypes.Message, opts ChatOptions) (string, error)

Chat implements ChatClient by delegating to OpenAIClient.Chat.

type ProviderConfig 

type ProviderConfig struct {
	// Provider is the backend to use: "ollama", "anthropic", "openai", "gemini".
	Provider string

	// Model is the provider-specific model identifier.
	// Examples: "granite4:micro-h" (Ollama), "claude-sonnet-4-20250514" (Anthropic).
	Model string

	// BaseURL is an optional endpoint override.
	// For Ollama: defaults to OLLAMA_BASE_URL or http://localhost:11434.
	// For cloud providers: uses the provider's default API URL.
	BaseURL string

	// APIKey is the authentication key for cloud providers.
	// Loaded from environment: ANTHROPIC_API_KEY, OPENAI_API_KEY, GEMINI_API_KEY.
	APIKey string

	// KeepAlive controls model VRAM lifetime (Ollama-specific).
	KeepAlive string

	// NumCtx sets the context window size (Ollama-specific).
	NumCtx int
}

ProviderConfig holds the configuration for a single LLM provider instance.

Description: 

Specifies which provider to use, which model, and any provider-specific
settings. Used by ProviderFactory to create the right adapter.

type ProviderFactory 

type ProviderFactory struct {
	// contains filtered or unexported fields
}

ProviderFactory creates the right LLM adapters based on provider configuration.

Description: 

ProviderFactory is the central creation point for all LLM adapters.
It creates ChatClient adapters (for Router/ParamExtractor) and agent
Client adapters (for Main model) based on per-role ProviderConfig.

Thread Safety: ProviderFactory is safe for concurrent use after construction.

func NewProviderFactory 

func NewProviderFactory(ollamaModelManager *llm.MultiModelManager, opts ...FactoryOption) *ProviderFactory

NewProviderFactory creates a new ProviderFactory.

Description: 

Creates a factory that can create adapters for any provider.
The ollamaModelManager is optional and only needed if any role uses Ollama.

Inputs:

  • ollamaModelManager: Shared MultiModelManager for Ollama (may be nil).
  • opts: Optional factory configuration (e.g., WithEgressGuard).

Outputs:

  • *ProviderFactory: Configured factory.

func (*ProviderFactory) CreateAgentClient 

func (f *ProviderFactory) CreateAgentClient(cfg ProviderConfig) (agentllm.Client, error)

CreateAgentClient creates an agent/llm.Client adapter for the given provider config.

Description: 

Creates the appropriate agent Client adapter based on the provider type.
Used for the Main model role which needs tool calling support.

Inputs:

  • cfg: Provider configuration specifying provider type and model.

Outputs:

  • agentllm.Client: The agent client adapter for the specified provider.
  • error: Non-nil if the provider is unsupported or construction fails.

Example: 

client, err := factory.CreateAgentClient(ProviderConfig{
    Provider: "anthropic",
    Model:    "claude-sonnet-4-20250514",
    APIKey:   "sk-ant-...",
})

func (*ProviderFactory) CreateChatClient 

func (f *ProviderFactory) CreateChatClient(cfg ProviderConfig) (ChatClient, error)

CreateChatClient creates a ChatClient adapter for the given provider config.

Description: 

Creates the appropriate ChatClient adapter based on the provider type.
Used for Router and ParamExtractor roles which only need simple chat.

Inputs:

  • cfg: Provider configuration specifying provider type and model.

Outputs:

  • ChatClient: The chat adapter for the specified provider.
  • error: Non-nil if the provider is unsupported or construction fails.

Example: 

client, err := factory.CreateChatClient(ProviderConfig{
    Provider: "anthropic",
    Model:    "claude-haiku-4-5-20251001",
    APIKey:   "sk-ant-...",
})

func (*ProviderFactory) CreateLifecycleManager 

func (f *ProviderFactory) CreateLifecycleManager(cfg ProviderConfig) (ModelLifecycleManager, error)

CreateLifecycleManager creates a ModelLifecycleManager for the given provider.

Description: 

Creates the appropriate lifecycle manager based on provider type.
Ollama gets a real lifecycle manager; cloud providers get a no-op manager.

Inputs:

  • cfg: Provider configuration specifying provider type.

Outputs:

  • ModelLifecycleManager: The lifecycle manager.
  • error: Non-nil if construction fails.

func (*ProviderFactory) EgressBuilder 

func (f *ProviderFactory) EgressBuilder() *egress.EgressGuardBuilder

EgressBuilder returns the egress guard builder, if configured.

Outputs:

  • *egress.EgressGuardBuilder: The builder, or nil if not configured.

func (*ProviderFactory) WrapChatClientForEgress 

func (f *ProviderFactory) WrapChatClientForEgress(
	client ChatClient,
	provider, model, sessionID, role string,
	tokenLimit int,
) ChatClient

WrapChatClientForEgress wraps a providers.ChatClient with egress guard checks.

Description: 

Bridges the type gap between providers.ChatClient (which uses providers.ChatOptions)
and egress.ChatClient (which uses egress.ChatOptions) by using a thin adapter.
This enables per-session egress wrapping for router and param extractor clients
at the handler layer where the actual session ID is available.

Returns the original client unwrapped if no egress builder is configured.

Inputs:

  • client: The providers.ChatClient to wrap.
  • provider: The provider name (e.g., "anthropic").
  • model: The model name.
  • sessionID: The agent session ID.
  • role: The role name ("ROUTER" or "PARAM").
  • tokenLimit: Token budget for this session/role. 0 means unlimited.

Outputs:

  • ChatClient: The guarded client (or the original if no egress builder or provider is "ollama").

type RoleConfig 

type RoleConfig struct {
	Main           ProviderConfig
	Router         ProviderConfig
	ParamExtractor ProviderConfig
}

RoleConfig holds per-role provider configurations.

Description: 

Contains the provider configuration for each of the three LLM roles
in the Trace agent: Main (synthesizer), Router, and ParamExtractor.

func LoadRoleConfig 

func LoadRoleConfig(mainModelFallback, routerModelFallback, paramModelFallback string) (*RoleConfig, error)

LoadRoleConfig reads per-role provider configuration from environment variables.

Description: 

Reads TRACE_<ROLE>_PROVIDER and TRACE_<ROLE>_MODEL environment variables
for each role. Falls back to Ollama with existing env vars for backward
compatibility when the new vars are not set.

Resolution order:

  1. TRACE_<ROLE>_PROVIDER -> explicit provider
  2. Fallback: "ollama" (backward compatible)
  3. TRACE_<ROLE>_MODEL -> explicit model
  4. Fallback: existing env vars (OLLAMA_MODEL for main, session config for router/param)

Inputs:

  • mainModelFallback: Fallback model for the main role (e.g., from OLLAMA_MODEL).
  • routerModelFallback: Fallback model for the router role (e.g., from SessionConfig).
  • paramModelFallback: Fallback model for the param extractor role.

Outputs:

  • *RoleConfig: Per-role configurations.
  • error: Non-nil if an invalid provider is specified.

Example: 

cfg, err := LoadRoleConfig("glm-4.7-flash", "granite4:micro-h", "ministral-3:3b")

func MergeSessionOverrides 

func MergeSessionOverrides(base *RoleConfig, mainModel, routerModel, paramModel string) *RoleConfig

MergeSessionOverrides returns a copy of base with per-session model overrides applied.

Description: 

Creates a shallow copy of the base RoleConfig and overrides model fields
when the override values are non-empty. CB-62: mainModel override allows
the user's OpenWebUI model selection to control the main reasoning LLM.

Inputs:

  • base: The base RoleConfig loaded at startup. Must not be nil.
  • mainModel: Override for Main.Model. Empty string means keep base.
  • routerModel: Override for Router.Model. Empty string means keep base.
  • paramModel: Override for ParamExtractor.Model. Empty string means keep base.

Outputs:

  • *RoleConfig: A new RoleConfig with overrides applied.

Thread Safety: Safe for concurrent use (returns a new copy).

type WarmupOptions 

type WarmupOptions struct {
	// KeepAlive controls how long the model stays loaded (Ollama-specific).
	KeepAlive string

	// NumCtx sets the context window size (Ollama-specific).
	NumCtx int
}

WarmupOptions configures model warmup behavior.

Source Files

View all Source files

Directories

Path Synopsis
Package egress provides data egress control, audit trail, and compliance enforcement for LLM provider calls.
 Package egress provides data egress control, audit trail, and compliance enforcement for LLM provider calls.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.