provider

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 22, 2026 License: Apache-2.0 Imports: 16 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var ErrProviderTypeDisabled = errors.New("provider type is disabled")

ErrProviderTypeDisabled is returned when a registered provider type is disabled.

Functions

func APIKeyFromContextOrConfig

func APIKeyFromContextOrConfig(ctx context.Context, configAPIKey string) string

func CheckResponse

func CheckResponse(resp *http.Response) error

CheckResponse returns a StatusError if the HTTP response status is not 2xx. It reads up to 4 KB of the body to include in the error message.

func CredentialFromContext

func CredentialFromContext(ctx context.Context) (*credentialmgr.Credential, bool)

func DecodeStoredProviderConfig

func DecodeStoredProviderConfig(data []byte) (any, error)

DecodeStoredProviderConfig converts a config-store provider payload into ProviderConfig.

func DisableProviderType

func DisableProviderType(name string) error

DisableProviderType disables a registered provider type.

func EnableProviderType

func EnableProviderType(name string) error

EnableProviderType enables a registered provider type.

func FinishReason

func FinishReason(msg *schema.Message) string

func IsProviderTypeEnabled

func IsProviderTypeEnabled(name string) (bool, bool)

IsProviderTypeEnabled reports whether a registered provider type is enabled.

func ListProviderTypes

func ListProviderTypes() []string

ListProviderTypes returns the names of all registered providers.

func RegisterProviderFactory

func RegisterProviderFactory(name string, factory ProviderFactory)

RegisterProviderFactory registers a provider factory by name.

func RetryProviderCall

func RetryProviderCall[T any](config NetworkConfig, fn func() (T, error)) (T, error)

RetryProviderCall retries fn up to NetworkConfig.MaxRetries times on retryable errors (429, 5xx). Non-retryable 4xx errors are returned immediately. Do NOT use this for streaming; retry semantics are undefined once a stream starts.

func StreamResponsesViaChat

func StreamResponsesViaChat(ctx context.Context, prov Provider, req *ResponsesRequest) (*schema.StreamReader[*ResponsesStreamEvent], error)

StreamResponsesViaChat adapts a streaming Responses API request onto the provider Chat stream API. Providers opt into this compatibility path explicitly by calling this helper.

func UpstreamErrorFields

func UpstreamErrorFields(err error) []zap.Field

UpstreamErrorFields extracts structured log fields from a provider UpstreamError.

func WithCredential

func WithCredential(ctx context.Context, cred *credentialmgr.Credential) context.Context

Types

type ChatRequest

type ChatRequest struct {
	Model    string
	Messages []*schema.Message
	Options  []einomodel.Option
}

ChatRequest is the unified internal chat request format passed to providers.

func ResponsesToChatRequest

func ResponsesToChatRequest(req *ResponsesRequest) (*ChatRequest, error)

ResponsesToChatRequest converts a minimal Responses API request into ChatRequest.

type ChatRequestState

type ChatRequestState struct {
	ModelName     string
	Messages      []*schema.Message
	Options       []einomodel.Option
	CommonOptions *einomodel.Options
}

func ResolveChatRequest

func ResolveChatRequest(_ context.Context, config ProviderConfig, req *ChatRequest) (*ChatRequestState, error)

type ChatResponse

type ChatResponse struct {
	Message *schema.Message
}

ChatResponse is the unified internal chat response format returned by providers.

func ChatResponseFromEinoMessage

func ChatResponseFromEinoMessage(msg *schema.Message) *ChatResponse

type EmbeddingProvider

type EmbeddingProvider interface {
	Provider
	Embedding(ctx context.Context, req *EmbeddingRequest) (*EmbeddingResponse, error)
}

EmbeddingProvider is an optional interface for providers that support embeddings. The memory module uses this to generate vectors for storage and search.

type EmbeddingRequest

type EmbeddingRequest struct {
	// Model is the embedding model to use. Leave empty to use provider default.
	Model string
	// Texts are the strings to embed.
	Texts []string
}

EmbeddingRequest is the request to generate vector embeddings.

type EmbeddingResponse

type EmbeddingResponse struct {
	Embeddings [][]float64
	Model      string
	Usage      Usage
}

EmbeddingResponse contains the generated embeddings.

type LLMApiRequestType

type LLMApiRequestType string

LLMApiRequestType identifies the prepared provider-facing request kind.

const (
	LLMApiRequestTypeChat      LLMApiRequestType = "chat"
	LLMApiRequestTypeEmbedding LLMApiRequestType = "embedding"
	LLMApiRequestTypeResponses LLMApiRequestType = "responses"
)

type ModelCapabilities

type ModelCapabilities struct {
	Streaming       bool `json:"streaming,omitempty"`
	Tools           bool `json:"tools,omitempty"`
	Vision          bool `json:"vision,omitempty"`
	Embeddings      bool `json:"embeddings,omitempty"`
	ContextWindow   int  `json:"context_window,omitempty"`
	MaxOutputTokens int  `json:"max_output_tokens,omitempty"`
}

ModelCapabilities describes what one specific upstream model supports. It is the routing authority for model-catalog decisions.

func ModelCapabilitiesFromProviderSummary

func ModelCapabilitiesFromProviderSummary(c ProviderCapabilities) ModelCapabilities

ModelCapabilitiesFromProviderSummary converts coarse provider-level capability metadata into a per-model fallback shape.

type ModelInfo

type ModelInfo struct {
	ID           string
	Name         string
	DisplayName  string
	Description  string
	Capabilities ModelCapabilities
}

ModelInfo describes a model available from a provider.

type NetworkConfig

type NetworkConfig = httpclient.NetworkConfig

NetworkConfig re-exports the shared HTTP network config type for provider configs.

type Provider

type Provider interface {
	// Chat performs a non-streaming chat completion and returns the full response.
	Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)

	// StreamChat performs a streaming chat completion and returns an Eino message stream.
	StreamChat(ctx context.Context, req *ChatRequest) (*schema.StreamReader[*schema.Message], error)

	// ListModels returns the list of models available from this provider.
	ListModels(ctx context.Context) ([]ModelInfo, error)

	// Capabilities returns what this provider instance supports.
	Capabilities() ProviderCapabilities

	// Config returns the provider instance configuration view used at runtime.
	Config() ProviderConfig
}

Provider is the core interface implemented by all LLM providers. Additional capabilities are exposed through optional interfaces such as EmbeddingProvider and ResponsesProvider.

func NewProvider

func NewProvider(config ProviderConfig) (Provider, error)

NewProvider creates a provider by name using registered factories.

type ProviderCapabilities

type ProviderCapabilities struct {
	Streaming       bool
	Tools           bool
	Vision          bool
	Embeddings      bool
	ContextWindow   int
	MaxOutputTokens int
}

ProviderCapabilities describes what a provider instance supports.

type ProviderConfig

type ProviderConfig struct {
	// Id is the unique provider config ID.
	Id string `json:"id"`
	// ProviderType is the registered provider type (e.g. "openai", "anthropic").
	ProviderType string `json:"provider_type"`
	// Disabled prevents the provider from being selected at runtime.
	Disabled bool `json:"disabled"`
	// APIKey is the provider API key. May be empty for local providers (Ollama).
	APIKey string `json:"api_key,omitempty"`
	// BaseURL overrides the provider's default API base URL.
	BaseURL string `json:"base_url,omitempty"`
	// DefaultModel is used when the request does not specify a model.
	DefaultModel string `json:"default_model,omitempty"`
	// Network contains HTTP client configuration (timeout, retry, proxy).
	Network NetworkConfig `json:"network"`
	// Options holds provider-specific extra configuration.
	Options map[string]any `json:"options,omitempty"`
}

ProviderConfig contains configuration for a provider instance.

func NormalizeConfig

func NormalizeConfig(cfg ProviderConfig, fallbackId string, fallbackName string) ProviderConfig

NormalizeConfig returns a runtime-ready provider config without mutating the source value. If ProviderType is empty, fallbackName is applied before defaults.

func NormalizeStoredProviderConfig

func NormalizeStoredProviderConfig(fallbackId string, fallbackName string, obj any) (ProviderConfig, error)

NormalizeStoredProviderConfig converts a decoded config-store object into a runtime-ready ProviderConfig without mutating the decoded object.

func (*ProviderConfig) Defaults

func (c *ProviderConfig) Defaults()

Defaults fills in zero values with sensible defaults.

type ProviderFactory

type ProviderFactory func(config ProviderConfig) (Provider, error)

ProviderFactory creates a Provider instance from config.

type ResponsesProvider

type ResponsesProvider interface {
	Provider
	CreateResponses(ctx context.Context, req *ResponsesRequest) (*ResponsesResponse, error)
	StreamResponses(ctx context.Context, req *ResponsesRequest) (*schema.StreamReader[*ResponsesStreamEvent], error)
}

ResponsesProvider is an optional interface for providers that expose OpenAI-compatible Responses API semantics directly.

type ResponsesRequest

type ResponsesRequest struct {
	Model              string         `json:"model"`
	Input              any            `json:"input"`
	MaxOutputTokens    int            `json:"max_output_tokens,omitempty"`
	Temperature        float64        `json:"temperature,omitempty"`
	TopP               float64        `json:"top_p,omitempty"`
	Stream             bool           `json:"stream,omitempty"`
	Instructions       string         `json:"instructions,omitempty"`
	PreviousResponseID string         `json:"previous_response_id,omitempty"`
	Store              *bool          `json:"store,omitempty"`
	Text               map[string]any `json:"text,omitempty"`
}

ResponsesRequest is the minimal provider-level request model for the OpenAI Responses API. Input is intentionally preserved as structured JSON to avoid collapsing it into the chat-only ChatRequest abstraction.

type ResponsesResponse

type ResponsesResponse struct {
	ID        string                    `json:"id"`
	Object    string                    `json:"object"`
	CreatedAt int64                     `json:"created_at"`
	Model     string                    `json:"model"`
	Output    []ResponsesResponseOutput `json:"output"`
	Usage     *ResponsesResponseUsage   `json:"usage,omitempty"`
}

ResponsesResponse mirrors the OpenAI-compatible Responses API envelope.

func CreateResponsesViaChat

func CreateResponsesViaChat(ctx context.Context, prov Provider, req *ResponsesRequest) (*ResponsesResponse, error)

CreateResponsesViaChat adapts a Responses API request onto the provider Chat API. Providers opt into this compatibility path explicitly by calling this helper.

func ResponsesFromChatResponse

func ResponsesFromChatResponse(resp *ChatResponse, model string) *ResponsesResponse

ResponsesFromChatResponse converts a Chat response into a minimal Responses envelope.

type ResponsesResponseContentPart

type ResponsesResponseContentPart struct {
	Type        string `json:"type"`
	Text        string `json:"text"`
	Annotations []any  `json:"annotations"`
}

type ResponsesResponseOutput

type ResponsesResponseOutput struct {
	ID      string                         `json:"id,omitempty"`
	Type    string                         `json:"type"`
	Role    string                         `json:"role"`
	Status  string                         `json:"status,omitempty"`
	Content []ResponsesResponseContentPart `json:"content"`
}

type ResponsesResponseUsage

type ResponsesResponseUsage struct {
	InputTokens  int `json:"input_tokens"`
	OutputTokens int `json:"output_tokens"`
	TotalTokens  int `json:"total_tokens"`
}

type ResponsesStreamEvent

type ResponsesStreamEvent struct {
	Type         string             `json:"type"`
	Response     *ResponsesResponse `json:"response,omitempty"`
	Delta        string             `json:"delta,omitempty"`
	ItemID       string             `json:"item_id,omitempty"`
	OutputIndex  int                `json:"output_index,omitempty"`
	ContentIndex int                `json:"content_index,omitempty"`
}

ResponsesStreamEvent is the minimal event model currently required by the gateway for OpenAI-compatible Responses API streaming.

func ResponsesCompletedEvent

func ResponsesCompletedEvent(resp *ResponsesResponse) *ResponsesStreamEvent

func ResponsesCreatedEvent

func ResponsesCreatedEvent(resp *ResponsesResponse) *ResponsesStreamEvent

func ResponsesDeltaEvent

func ResponsesDeltaEvent(itemID string, delta string) *ResponsesStreamEvent

type UpstreamError

type UpstreamError struct {
	Status     int
	StatusText string
	Body       string
}

UpstreamError describes an HTTP error returned by an upstream provider API.

func (*UpstreamError) Error

func (e *UpstreamError) Error() string

func (*UpstreamError) StatusCode

func (e *UpstreamError) StatusCode() int

type Usage

type Usage struct {
	InputTokens  int
	OutputTokens int
}

Usage contains token consumption information.

func UsageFromMessage

func UsageFromMessage(msg *schema.Message) Usage

Directories

Path Synopsis
Package anthropic implements the Anthropic provider (Claude models).
Package anthropic implements the Anthropic provider (Claude models).
Package deepseek implements the DeepSeek provider.
Package deepseek implements the DeepSeek provider.
Package gemini implements the Google Gemini provider.
Package gemini implements the Google Gemini provider.
Package ollama implements the Ollama provider (local deployment, OpenAI-compatible).
Package ollama implements the Ollama provider (local deployment, OpenAI-compatible).
Package openai implements the OpenAI provider.
Package openai implements the OpenAI provider.
Package openaibase provides shared OpenAI-compatible wire types still used for model listing and embeddings.
Package openaibase provides shared OpenAI-compatible wire types still used for model listing and embeddings.
Package openrouter implements the OpenRouter provider (OpenAI-compatible API).
Package openrouter implements the OpenRouter provider (OpenAI-compatible API).
Package zhipu implements the Zhipu BigModel provider.
Package zhipu implements the Zhipu BigModel provider.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL