provider

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 19, 2026 License: Apache-2.0 Imports: 18 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var ErrProviderTypeDisabled = errors.New("provider type is disabled")

ErrProviderTypeDisabled is returned when a registered provider type is disabled.

Functions

func APIKeyFromContextOrConfig

func APIKeyFromContextOrConfig(ctx context.Context, configAPIKey string) string

func ChatCompletionsExtraFieldsFromOptions added in v0.2.0

func ChatCompletionsExtraFieldsFromOptions(reasoning ReasoningFieldStyle, opts ...einomodel.Option) map[string]any

ChatCompletionsExtraFieldsFromOptions maps preserved request context onto chat-completions-style request fields shared by OpenAI-compatible upstreams. It folds in both the Responses API request context (when a Responses request was bridged onto chat) and the inbound chat extra fields, emitting only fields expressible on the chat wire shape. The reasoning style selects how reasoning is encoded for the target dialect.

func CheckResponse

func CheckResponse(resp *http.Response) error

CheckResponse returns a StatusError if the HTTP response status is not 2xx. It reads up to 4 KB of the body to include in the error message.

func ConfigureProviderTypes added in v0.2.0

func ConfigureProviderTypes(settings []ProviderTypeSetting, exclusive bool) error

ConfigureProviderTypes applies startup-only provider type availability. If exclusive is true, every registered provider type not listed is disabled.

func CredentialFromContext

func CredentialFromContext(ctx context.Context) (*credentialmgr.Credential, bool)

func DecodeStoredProviderConfig

func DecodeStoredProviderConfig(data []byte) (any, error)

DecodeStoredProviderConfig converts a config-store provider payload into ProviderConfig.

func EnableAllProviderTypes added in v0.2.0

func EnableAllProviderTypes()

EnableAllProviderTypes clears the disabled set so every registered provider type is enabled. It is used when no startup provider_types policy is set.

func FinishReason

func FinishReason(msg *schema.Message) string

func IsProviderTypeEnabled

func IsProviderTypeEnabled(name string) (bool, bool)

IsProviderTypeEnabled reports whether a registered provider type is enabled.

func ListProviderTypes

func ListProviderTypes() []string

ListProviderTypes returns the names of all registered providers.

func MergeExtraFields added in v0.2.0

func MergeExtraFields(base map[string]any, overlays ...map[string]any) map[string]any

MergeExtraFields shallow-merges one or more request-body extension maps. Later maps override earlier keys.

func RegisterProviderFactory

func RegisterProviderFactory(name string, factory ProviderFactory)

RegisterProviderFactory registers a provider factory by name.

func RetryProviderCall

func RetryProviderCall[T any](config NetworkConfig, fn func() (T, error)) (T, error)

RetryProviderCall retries fn up to NetworkConfig.MaxRetries times on retryable errors (429, 5xx). Non-retryable 4xx errors are returned immediately. Do NOT use this for streaming; retry semantics are undefined once a stream starts.

func StreamResponsesViaChat

func StreamResponsesViaChat(ctx context.Context, prov Provider, req *ResponsesRequest) (*schema.StreamReader[*ResponsesStreamEvent], error)

StreamResponsesViaChat adapts a streaming Responses API request onto the provider Chat stream API. Providers opt into this compatibility path explicitly by calling this helper.

func StripCCUnsupportedChatFields added in v0.2.0

func StripCCUnsupportedChatFields(fields map[string]any)

StripCCUnsupportedChatFields removes the OpenAI-style `metadata` and `user` chat-completions request fields. Some OpenAI-compatible upstreams (e.g. GLM) reject these with a generic 400, while Claude Code always populates `metadata.user_id`. Providers running in cc-compat mode call this to drop the unsupported fields before the upstream request. It is a no-op on a nil map.

func UpstreamErrorFields

func UpstreamErrorFields(err error) []zap.Field

UpstreamErrorFields extracts structured log fields from a provider UpstreamError.

func WithChatExtraFields added in v0.2.0

func WithChatExtraFields(fields *ChatExtraFields) einomodel.Option

WithChatExtraFields stores extra chat-completions request fields inside ChatRequest.Options so they survive generic chat-provider compatibility.

func WithCredential

func WithCredential(ctx context.Context, cred *credentialmgr.Credential) context.Context

func WithResponsesRequestContext added in v0.2.0

func WithResponsesRequestContext(ctx *ResponsesRequestContext) einomodel.Option

WithResponsesRequestContext stores extra Responses API request fields inside ChatRequest.Options so they survive generic chat-provider compatibility.

func WithTopK added in v0.2.0

func WithTopK(topK int) einomodel.Option

WithTopK adds a top-k sampling option. It is encoded as an impl-specific option so it travels inside ChatRequest.Options alongside standard options (temperature, max_tokens, etc.) and any provider can read it via GetChatOptions.

Types

type ChatExtraFields added in v0.2.0

type ChatExtraFields struct {
	ResponseFormat    any
	Reasoning         map[string]any
	ReasoningEffort   string
	User              string
	Metadata          map[string]any
	ParallelToolCalls *bool
	Store             *bool
}

ChatExtraFields carries chat-completions request fields that have no eino common-option equivalent but should still reach OpenAI-compatible providers. Values are already in chat wire shape so they pass through unchanged.

func ChatExtraFieldsFromOptions added in v0.2.0

func ChatExtraFieldsFromOptions(opts ...einomodel.Option) *ChatExtraFields

ChatExtraFieldsFromOptions extracts any stored chat-completions extra fields from a chat option list.

type ChatOptions added in v0.2.0

type ChatOptions struct {
	TopK      int
	Responses *ResponsesRequestContext
	ChatExtra *ChatExtraFields
}

ChatOptions carries additional chat request options that extend the standard eino model options. Use WithTopK / GetChatOptions to set and read these.

func GetChatOptions added in v0.2.0

func GetChatOptions(opts ...einomodel.Option) *ChatOptions

GetChatOptions extracts ChatOptions from an option list.

type ChatRequest

type ChatRequest struct {
	Model    string
	Messages []*schema.Message
	Options  []einomodel.Option
}

ChatRequest is the unified internal chat request format passed to providers.

func ResponsesToChatRequest

func ResponsesToChatRequest(req *ResponsesRequest) (*ChatRequest, error)

ResponsesToChatRequest converts a minimal Responses API request into ChatRequest.

type ChatRequestState

type ChatRequestState struct {
	ModelName     string
	Messages      []*schema.Message
	Options       []einomodel.Option
	CommonOptions *einomodel.Options
}

func ResolveChatRequest

func ResolveChatRequest(_ context.Context, config ProviderConfig, req *ChatRequest) (*ChatRequestState, error)

type ChatResponse

type ChatResponse struct {
	Message *schema.Message
}

ChatResponse is the unified internal chat response format returned by providers.

func ChatResponseFromEinoMessage

func ChatResponseFromEinoMessage(msg *schema.Message) *ChatResponse

type CompactMode added in v0.2.0

type CompactMode string
const (
	CompactModeNone  CompactMode = "none"
	CompactModeCC    CompactMode = "cc"
	CompactModeCodex CompactMode = "codex"
)

func CompactModeFromOptions added in v0.2.0

func CompactModeFromOptions(opts map[string]any) (CompactMode, error)

CompactModeFromOptions reads options.compact. Supported values are "cc", "codex", and "none"; missing or empty values resolve to "none".

type EmbeddingProvider

type EmbeddingProvider interface {
	Provider
	Embedding(ctx context.Context, req *EmbeddingRequest) (*EmbeddingResponse, error)
}

EmbeddingProvider is an optional interface for providers that support embeddings. The memory module uses this to generate vectors for storage and search.

type EmbeddingRequest

type EmbeddingRequest struct {
	// Model is the embedding model to use. Leave empty to use provider default.
	Model string
	// Texts are the strings to embed.
	Texts []string
}

EmbeddingRequest is the request to generate vector embeddings.

type EmbeddingResponse

type EmbeddingResponse struct {
	Embeddings [][]float64
	Model      string
	Usage      Usage
}

EmbeddingResponse contains the generated embeddings.

type LLMApiRequestType

type LLMApiRequestType string

LLMApiRequestType identifies the prepared provider-facing request kind.

const (
	LLMApiRequestTypeChat      LLMApiRequestType = "chat"
	LLMApiRequestTypeEmbedding LLMApiRequestType = "embedding"
	LLMApiRequestTypeResponses LLMApiRequestType = "responses"
	LLMApiRequestTypeModels    LLMApiRequestType = "models"
)

type ModelCapabilities

type ModelCapabilities struct {
	Streaming       bool `json:"streaming,omitempty"`
	Tools           bool `json:"tools,omitempty"`
	Vision          bool `json:"vision,omitempty"`
	Embeddings      bool `json:"embeddings,omitempty"`
	ContextWindow   int  `json:"context_window,omitempty"`
	MaxOutputTokens int  `json:"max_output_tokens,omitempty"`
}

ModelCapabilities describes what one specific upstream model supports. It is the routing authority for model-catalog decisions.

func ModelCapabilitiesFromProviderSummary

func ModelCapabilitiesFromProviderSummary(c ProviderCapabilities) ModelCapabilities

ModelCapabilitiesFromProviderSummary converts coarse provider-level capability metadata into a per-model fallback shape.

type ModelInfo

type ModelInfo struct {
	ID           string
	Name         string
	DisplayName  string
	Description  string
	Capabilities ModelCapabilities
}

ModelInfo describes a model available from a provider.

type NetworkConfig

type NetworkConfig = httpclient.NetworkConfig

NetworkConfig re-exports the shared HTTP network config type for provider configs.

type Provider

type Provider interface {
	// Chat performs a non-streaming chat completion and returns the full response.
	Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)

	// StreamChat performs a streaming chat completion and returns an Eino message stream.
	StreamChat(ctx context.Context, req *ChatRequest) (*schema.StreamReader[*schema.Message], error)

	// ListModels returns the list of models available from this provider.
	ListModels(ctx context.Context) ([]ModelInfo, error)

	// Capabilities returns what this provider instance supports.
	Capabilities() ProviderCapabilities

	// Config returns the provider instance configuration view used at runtime.
	Config() ProviderConfig
}

Provider is the core interface implemented by all LLM providers. Additional capabilities are exposed through optional interfaces such as EmbeddingProvider and ResponsesProvider.

func NewProvider

func NewProvider(config ProviderConfig) (Provider, error)

NewProvider creates a provider by name using registered factories.

type ProviderCapabilities

type ProviderCapabilities struct {
	Streaming       bool
	Tools           bool
	Vision          bool
	Embeddings      bool
	ContextWindow   int
	MaxOutputTokens int
}

ProviderCapabilities describes what a provider instance supports.

type ProviderConfig

type ProviderConfig struct {
	// Id is the unique provider config ID.
	Id string `json:"id"`
	// ProviderType is the registered provider type (e.g. "openai", "anthropic").
	ProviderType string `json:"provider_type"`
	// Disabled prevents the provider from being selected at runtime.
	Disabled bool `json:"disabled"`
	// APIKey is the provider API key. May be empty for local providers (Ollama).
	APIKey string `json:"api_key,omitempty"`
	// BaseURL overrides the provider's default API base URL.
	BaseURL string `json:"base_url,omitempty"`
	// DefaultModel is used when the request does not specify a model.
	DefaultModel string `json:"default_model,omitempty"`
	// Network contains HTTP client configuration (timeout, retry, proxy).
	Network NetworkConfig `json:"network"`
	// Options holds provider-specific extra configuration.
	Options map[string]any `json:"options,omitempty"`
	// CreatedAt is set when the dynamic provider config is first persisted.
	CreatedAt time.Time `json:"created_at"`
	// UpdatedAt is bumped on every persisted create/update. It is used as the
	// runtime resolver's cache fingerprint so a config change rebuilds the
	// provider instance even if explicit invalidation was missed.
	UpdatedAt time.Time `json:"updated_at"`
}

ProviderConfig contains configuration for a provider instance.

func NormalizeConfig

func NormalizeConfig(cfg ProviderConfig, fallbackId string, fallbackName string) ProviderConfig

NormalizeConfig returns a runtime-ready provider config without mutating the source value. If ProviderType is empty, fallbackName is applied before defaults.

func NormalizeStoredProviderConfig

func NormalizeStoredProviderConfig(fallbackId string, fallbackName string, obj any) (ProviderConfig, error)

NormalizeStoredProviderConfig converts a decoded config-store object into a runtime-ready ProviderConfig without mutating the decoded object.

func (ProviderConfig) CompactMode added in v0.2.0

func (c ProviderConfig) CompactMode() CompactMode

CompactMode reads options.compact and treats invalid values as "none". Use CompactModeFromOptions in provider constructors that should reject bad config.

func (*ProviderConfig) Defaults

func (c *ProviderConfig) Defaults()

Defaults fills in zero values with sensible defaults.

func (ProviderConfig) Fingerprint added in v0.3.0

func (c ProviderConfig) Fingerprint() string

Fingerprint returns a cheap version string for the provider config used as the runtime resolver materializer key. UpdatedAt is bumped on every persisted create/update, so it changes whenever the stored config changes. Static providers have a zero UpdatedAt, which is a stable per-id constant and matches their immutable-at-runtime nature. Avoid marshaling the whole config here; this runs on every resolved request.

type ProviderFactory

type ProviderFactory func(config ProviderConfig) (Provider, error)

ProviderFactory creates a Provider instance from config.

type ProviderTypeSetting added in v0.2.0

type ProviderTypeSetting struct {
	ProviderType string `json:"provider_type"`
	Enabled      bool   `json:"enabled"`
}

type ReasoningFieldStyle added in v0.2.0

type ReasoningFieldStyle int

ReasoningFieldStyle selects how reasoning is expressed on an OpenAI-compatible chat-completions wire shape.

const (
	// ReasoningEffortField emits reasoning as a "reasoning_effort" string, the
	// standard OpenAI chat-completions shape.
	ReasoningEffortField ReasoningFieldStyle = iota
	// ReasoningObjectField emits reasoning as a "reasoning" object, the shape
	// used by OpenAI-compatible upstreams that accept a structured reasoning
	// config (e.g. OpenRouter).
	ReasoningObjectField
)

type ResponsesProvider

type ResponsesProvider interface {
	Provider
	CreateResponses(ctx context.Context, req *ResponsesRequest) (*ResponsesResponse, error)
	StreamResponses(ctx context.Context, req *ResponsesRequest) (*schema.StreamReader[*ResponsesStreamEvent], error)
}

ResponsesProvider is an optional interface for providers that expose OpenAI-compatible Responses API semantics directly.

type ResponsesRequest

type ResponsesRequest struct {
	Model              string                    `json:"model"`
	Input              any                       `json:"input"`
	Tools              []ResponsesToolDefinition `json:"tools,omitempty"`
	ToolChoice         json.RawMessage           `json:"tool_choice,omitempty"`
	MaxOutputTokens    int                       `json:"max_output_tokens,omitempty"`
	Temperature        float64                   `json:"temperature,omitempty"`
	TopP               float64                   `json:"top_p,omitempty"`
	Stream             bool                      `json:"stream,omitempty"`
	Instructions       string                    `json:"instructions,omitempty"`
	PreviousResponseID string                    `json:"previous_response_id,omitempty"`
	Store              *bool                     `json:"store,omitempty"`
	Text               map[string]any            `json:"text,omitempty"`
	Metadata           map[string]any            `json:"metadata,omitempty"`
	User               string                    `json:"user,omitempty"`
	Reasoning          map[string]any            `json:"reasoning,omitempty"`
	ParallelToolCalls  *bool                     `json:"parallel_tool_calls,omitempty"`
	Truncation         any                       `json:"truncation,omitempty"`
}

ResponsesRequest is the minimal provider-level request model for the OpenAI Responses API. Input is intentionally preserved as structured JSON to avoid collapsing it into the chat-only ChatRequest abstraction.

func ResponsesRequestFromChatState added in v0.2.0

func ResponsesRequestFromChatState(state *ChatRequestState, stream bool) *ResponsesRequest

ResponsesRequestFromChatState rebuilds a provider-level Responses request from a resolved ChatRequestState. This is the generic inverse of the ResponsesToChatRequest compatibility path and is intended for providers that bridge chat calls onto an upstream Responses API.

type ResponsesRequestContext added in v0.2.0

type ResponsesRequestContext struct {
	PreviousResponseID string
	Store              *bool
	Text               map[string]any
	Metadata           map[string]any
	User               string
	Reasoning          map[string]any
	ParallelToolCalls  *bool
	Truncation         any
}

ResponsesRequestContext carries Responses API request fields that do not map directly onto the shared chat abstraction but still need to survive the compatibility path so any provider can inspect or reuse them.

func ResponsesRequestContextFromOptions added in v0.2.0

func ResponsesRequestContextFromOptions(opts ...einomodel.Option) *ResponsesRequestContext

ResponsesRequestContextFromOptions extracts any stored Responses API request context from a chat option list.

type ResponsesResponse

type ResponsesResponse struct {
	ID        string                    `json:"id"`
	Object    string                    `json:"object"`
	CreatedAt int64                     `json:"created_at"`
	Model     string                    `json:"model"`
	Output    []ResponsesResponseOutput `json:"output"`
	Usage     *ResponsesResponseUsage   `json:"usage,omitempty"`
	RawJSON   json.RawMessage           `json:"-"`
}

ResponsesResponse mirrors the OpenAI-compatible Responses API envelope.

func CreateResponsesViaChat

func CreateResponsesViaChat(ctx context.Context, prov Provider, req *ResponsesRequest) (*ResponsesResponse, error)

CreateResponsesViaChat adapts a Responses API request onto the provider Chat API. Providers opt into this compatibility path explicitly by calling this helper.

func ResponsesFromChatResponse

func ResponsesFromChatResponse(resp *ChatResponse, model string) *ResponsesResponse

ResponsesFromChatResponse converts a Chat response into a minimal Responses envelope.

type ResponsesResponseContentPart

type ResponsesResponseContentPart struct {
	Type        string          `json:"type"`
	Text        string          `json:"text,omitempty"`
	Annotations []any           `json:"annotations,omitempty"`
	Refusal     string          `json:"refusal,omitempty"`
	Summary     []any           `json:"summary,omitempty"`
	RawJSON     json.RawMessage `json:"-"`
}

type ResponsesResponseOutput

type ResponsesResponseOutput struct {
	ID        string                         `json:"id,omitempty"`
	Type      string                         `json:"type"`
	Role      string                         `json:"role,omitempty"`
	Status    string                         `json:"status,omitempty"`
	Content   []ResponsesResponseContentPart `json:"content,omitempty"`
	CallID    string                         `json:"call_id,omitempty"`
	Name      string                         `json:"name,omitempty"`
	Arguments string                         `json:"arguments,omitempty"`
}

type ResponsesResponseUsage

type ResponsesResponseUsage struct {
	InputTokens  int `json:"input_tokens"`
	OutputTokens int `json:"output_tokens"`
	TotalTokens  int `json:"total_tokens"`
}

type ResponsesStreamEvent

type ResponsesStreamEvent struct {
	Type         string                   `json:"type"`
	Response     *ResponsesResponse       `json:"response,omitempty"`
	Item         *ResponsesResponseOutput `json:"item,omitempty"`
	Delta        string                   `json:"delta,omitempty"`
	ItemID       string                   `json:"item_id,omitempty"`
	OutputIndex  int                      `json:"output_index,omitempty"`
	ContentIndex int                      `json:"content_index,omitempty"`
	RawJSON      json.RawMessage          `json:"-"`
}

ResponsesStreamEvent is the minimal event model currently required by the gateway for OpenAI-compatible Responses API streaming.

func ResponsesCompletedEvent

func ResponsesCompletedEvent(resp *ResponsesResponse) *ResponsesStreamEvent

func ResponsesCreatedEvent

func ResponsesCreatedEvent(resp *ResponsesResponse) *ResponsesStreamEvent

func ResponsesDeltaEvent

func ResponsesDeltaEvent(itemID string, outputIndex int, delta string) *ResponsesStreamEvent

func ResponsesFunctionCallArgumentsDeltaEvent added in v0.2.0

func ResponsesFunctionCallArgumentsDeltaEvent(itemID string, outputIndex int, delta string) *ResponsesStreamEvent

func ResponsesOutputItemAddedEvent added in v0.2.0

func ResponsesOutputItemAddedEvent(outputIndex int, item *ResponsesResponseOutput) *ResponsesStreamEvent

func ResponsesOutputItemDoneEvent added in v0.2.0

func ResponsesOutputItemDoneEvent(outputIndex int, item *ResponsesResponseOutput) *ResponsesStreamEvent

type ResponsesToolDefinition added in v0.2.0

type ResponsesToolDefinition struct {
	Type        string                 `json:"type"`
	Name        string                 `json:"name,omitempty"`
	Description string                 `json:"description,omitempty"`
	Parameters  json.RawMessage        `json:"parameters,omitempty"`
	Function    *ResponsesToolFunction `json:"function,omitempty"`
}

type ResponsesToolFunction added in v0.2.0

type ResponsesToolFunction struct {
	Name        string          `json:"name"`
	Description string          `json:"description,omitempty"`
	Parameters  json.RawMessage `json:"parameters,omitempty"`
}

type UpstreamError

type UpstreamError struct {
	Status     int
	StatusText string
	Body       string
}

UpstreamError describes an HTTP error returned by an upstream provider API.

func (*UpstreamError) Error

func (e *UpstreamError) Error() string

func (*UpstreamError) StatusCode

func (e *UpstreamError) StatusCode() int

type Usage

type Usage struct {
	InputTokens  int
	OutputTokens int
}

Usage contains token consumption information.

func UsageFromMessage

func UsageFromMessage(msg *schema.Message) Usage

Directories

Path Synopsis
Package anthropic implements the Anthropic provider (Claude models).
Package anthropic implements the Anthropic provider (Claude models).
Package anthropicbase provides shared Anthropic Messages API wire helpers.
Package anthropicbase provides shared Anthropic Messages API wire helpers.
Package deepseek implements the DeepSeek provider.
Package deepseek implements the DeepSeek provider.
Package gemini implements the Google Gemini provider.
Package gemini implements the Google Gemini provider.
Package ollama implements the Ollama provider (local deployment, OpenAI-compatible).
Package ollama implements the Ollama provider (local deployment, OpenAI-compatible).
Package openai implements the OpenAI provider.
Package openai implements the OpenAI provider.
Package openaibase provides shared OpenAI-compatible wire types still used for model listing and embeddings.
Package openaibase provides shared OpenAI-compatible wire types still used for model listing and embeddings.
Package openrouter implements the OpenRouter provider (OpenAI-compatible API).
Package openrouter implements the OpenRouter provider (OpenAI-compatible API).
Package zhipu implements the Zhipu BigModel provider.
Package zhipu implements the Zhipu BigModel provider.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL