providers

package
v0.2.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 22, 2026 License: Apache-2.0 Imports: 15 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	ProviderOpenAI           = "openai"
	ProviderOpenAICompatible = "openai_compatible"
	ProviderGoogle           = "google"
	ProviderVertex           = "vertex"
	ProviderAnthropic        = "anthropic"
	ProviderBedrock          = "bedrock"
	ProviderAzure            = "azure"
	ProviderMistral          = "mistral"
	ProviderGroq             = "groq"
	ProviderDeepSeek         = "deepseek"
)

Provider name constants. Use these as keys for HTTPClientPool.Get() and anywhere a provider needs to be identified by name.

View Source 
const (
	OpenAIAPICompletions = "completions"
	OpenAIAPIResponses   = "responses"
)
View Source 
const ProbeHTTPTimeout = 10 * time.Second
View Source 
const StreamTimeout = 5 * time.Minute

StreamTimeout bounds the total duration of a single streamed response.

Variables

View Source 
var DefaultHTTPTimeout = 120 * time.Second

DefaultHTTPTimeout is the timeout used by all provider HTTP clients. Override with SetDefaultHTTPTimeout during initialization.

Functions

func DrainBody 

func DrainBody(r io.ReadCloser)

DrainBody reads and discards up to 64 KB of remaining data from r, then closes it. This ensures the underlying TCP connection is returned cleanly to the transport's pool and is never reused with stale data. Callers should use this in error paths where the body was only partially read (e.g. after io.CopyN for an error preview).

func IsValidProvider 

func IsValidProvider(name string) bool

IsValidProvider reports whether name is a supported provider.

func SetDefaultHTTPTimeout 

func SetDefaultHTTPTimeout(d time.Duration)

func StreamResponse 

func StreamResponse(ctx context.Context, rc io.ReadCloser) iter.Seq2[[]byte, error]

func StreamSSE 

func StreamSSE(ctx context.Context, rc io.ReadCloser) iter.Seq2[[]byte, error]

func SupportedProviders 

func SupportedProviders() []string

SupportedProviders returns every provider name the gateway can route to.

func ValidateProviderOptions 

func ValidateProviderOptions(provider string, options map[string]any) error

Types

type AwsBedrock 

type AwsBedrock struct {
	Region       string `json:"region"`
	AccessKey    string `json:"access_key"`    // #nosec G117 -- DTO field for AWS access key configuration
	SecretKey    string `json:"secret_key"`    // #nosec G117 -- DTO field for AWS secret key configuration
	SessionToken string `json:"session_token"` // #nosec G117 -- DTO field for AWS session token configuration
	UseRole      bool   `json:"use_role"`
	RoleARN      string `json:"role_arn"`
}

type Azure 

type Azure struct {
	Endpoint     string        `json:"endpoint"`
	ApiVersion   string        `json:"api_version"`
	AuthMode     AzureAuthMode `json:"auth_mode"`
	UseIdentity  bool          `json:"use_identity"`
	TenantID     string        `json:"tenant_id"`
	ClientID     string        `json:"client_id"`
	ClientSecret string        `json:"client_secret"` // #nosec G117 -- Azure client secret credential
}

type AzureAuthMode 

type AzureAuthMode string
const (
	AzureAuthModeAPIKey                 AzureAuthMode = "api_key"
	AzureAuthModeServicePrincipal       AzureAuthMode = "service_principal"
	AzureAuthModeDefaultAzureCredential AzureAuthMode = "default_azure_credential" // #nosec G101 -- auth mode identifier, not a credential value
)

type Client 

type Client interface {
	Completions(
		ctx context.Context,
		config *Config,
		reqBody []byte,
	) ([]byte, error)
	// CompletionsStream performs the streaming request. The outer error carries
	// pre-stream failures (e.g. a registry.BackendError on a non-2xx response, for
	// verbatim passthrough). The returned sequence yields raw SSE lines and
	// surfaces mid-stream read errors as the second value.
	CompletionsStream(
		ctx context.Context,
		config *Config,
		reqBody []byte,
	) (iter.Seq2[[]byte, error], error)
}

type Config 

type Config struct {
	Credentials   Credentials    `json:"credentials"`
	AllowedModels []string       `json:"allowed_models"`
	DefaultModel  string         `json:"default_model"`
	Model         string         `json:"model"`
	MaxTokens     int            `json:"max_tokens,omitempty"`
	Temperature   float64        `json:"temperature,omitempty"`
	SystemPrompt  string         `json:"system_prompt,omitempty"`
	Instructions  []string       `json:"instructions,omitempty"`
	Options       map[string]any `json:"options,omitempty"`
}

type ConnectionTester 

type ConnectionTester interface {
	TestConnection(ctx context.Context, config *Config) ProbeResult
}

type Credentials 

type Credentials struct {
	ApiKey     string      `json:"api_key"` // #nosec G117 -- DTO field for provider API key configuration
	AwsBedrock *AwsBedrock `json:"aws,omitempty"`
	Azure      *Azure      `json:"azure,omitempty"`
}

type HTTPClientPool 

type HTTPClientPool struct {
	// contains filtered or unexported fields
}

HTTPClientPool manages a pool of *http.Client instances keyed by provider name. It uses singleflight to ensure only one client is created per key even under concurrent access.

Each *http.Client returned is safe for concurrent use: Go's http.Client.Do() creates a completely independent HTTP transaction per call — there is no request/response caching between calls. The underlying Transport reuses TCP connections (keep-alive) for performance, but each request gets its own HTTP round-trip with its own headers, body, and response.

func NewHTTPClientPool 

func NewHTTPClientPool() *HTTPClientPool

NewHTTPClientPool returns a ready-to-use pool.

func (*HTTPClientPool) Get 

func (p *HTTPClientPool) Get(key string, timeout time.Duration) *http.Client

Get returns (or lazily creates) an *http.Client for the given key with the specified timeout. Typical keys are provider names ("openai", "anthropic", etc.) so each provider gets its own client and transport instance.

func (*HTTPClientPool) GetStream 

func (p *HTTPClientPool) GetStream(key string) *http.Client

GetStream returns an *http.Client suitable for SSE streaming. Unlike Get, it has no overall client timeout (a streamed response can outlive DefaultHTTPTimeout); callers bound the stream with a context deadline instead. Stream clients are pooled under a distinct key so they never share the non-streaming client's timeout.

type OpenAICompatibleOptions 

type OpenAICompatibleOptions struct {
	BaseURL string            `mapstructure:"base_url"`
	Headers map[string]string `mapstructure:"headers"`
}

func DecodeOpenAICompatibleOptions 

func DecodeOpenAICompatibleOptions(options map[string]any) (OpenAICompatibleOptions, error)

type OpenAIOptions 

type OpenAIOptions struct {
	API     string `mapstructure:"api"`
	BaseURL string `mapstructure:"base_url"`
}

func DecodeOpenAIOptions 

func DecodeOpenAIOptions(options map[string]any) (OpenAIOptions, error)

type ProbeResult 

type ProbeResult struct {
	OK         bool
	Stage      ProbeStage
	StatusCode int
	Message    string
}

func ClassifyProbeStatus 

func ClassifyProbeStatus(statusCode int) ProbeResult

func ClassifyProbeStatusForProvider 

func ClassifyProbeStatusForProvider(providerKey string, statusCode int) ProbeResult

func RunAPIKeyGETProbe 

func RunAPIKeyGETProbe(
	ctx context.Context,
	providerKey string,
	url string,
	apiKey string,
	applyHeaders func(*http.Request, string),
) ProbeResult

func RunBearerGETProbe 

func RunBearerGETProbe(ctx context.Context, providerKey, url, apiKey string) ProbeResult

func RunHTTPProbe 

func RunHTTPProbe(providerKey string, req *http.Request) ProbeResult

type ProbeStage 

type ProbeStage string
const (
	StageConnectivity   ProbeStage = "connectivity"
	StageAuthentication ProbeStage = "authentication"
	StageProvider       ProbeStage = "provider"
	StageUnsupported    ProbeStage = "unsupported"
)

type VertexOptions 

type VertexOptions struct {
	Project  string `mapstructure:"project"`
	Location string `mapstructure:"location"`
	Version  string `mapstructure:"version"`
}

func DecodeVertexOptions 

func DecodeVertexOptions(options map[string]any) (VertexOptions, error)

Source Files

View all Source files

Directories

Path Synopsis
mocks
Package openaicompat implements a provider client for arbitrary OpenAI-compatible Chat Completions endpoints (Together, Fireworks, vLLM, Ollama, self-hosted gateways, ...).
 Package openaicompat implements a provider client for arbitrary OpenAI-compatible Chat Completions endpoints (Together, Fireworks, vLLM, Ollama, self-hosted gateways, ...).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.