Documentation
¶
Index ¶
- Variables
- func APIKeyFromContextOrConfig(ctx context.Context, configAPIKey string) string
- func ChatCompletionsExtraFieldsFromOptions(reasoning ReasoningFieldStyle, opts ...einomodel.Option) map[string]any
- func CheckResponse(resp *http.Response) error
- func ConfigureProviderTypes(settings []ProviderTypeSetting, exclusive bool) error
- func CredentialFromContext(ctx context.Context) (*credentialmgr.Credential, bool)
- func DecodeStoredProviderConfig(data []byte) (any, error)
- func EnableAllProviderTypes()
- func FinishReason(msg *schema.Message) string
- func IsProviderTypeEnabled(name string) (bool, bool)
- func ListProviderTypes() []string
- func MergeExtraFields(base map[string]any, overlays ...map[string]any) map[string]any
- func RegisterProviderFactory(name string, factory ProviderFactory)
- func RetryProviderCall[T any](config NetworkConfig, fn func() (T, error)) (T, error)
- func StreamResponsesViaChat(ctx context.Context, prov Provider, req *ResponsesRequest) (*schema.StreamReader[*ResponsesStreamEvent], error)
- func StripCCUnsupportedChatFields(fields map[string]any)
- func UpstreamErrorFields(err error) []zap.Field
- func WithChatExtraFields(fields *ChatExtraFields) einomodel.Option
- func WithCredential(ctx context.Context, cred *credentialmgr.Credential) context.Context
- func WithResponsesRequestContext(ctx *ResponsesRequestContext) einomodel.Option
- func WithTopK(topK int) einomodel.Option
- type ChatExtraFields
- type ChatOptions
- type ChatRequest
- type ChatRequestState
- type ChatResponse
- type CompactMode
- type EmbeddingProvider
- type EmbeddingRequest
- type EmbeddingResponse
- type LLMApiRequestType
- type ModelCapabilities
- type ModelInfo
- type NetworkConfig
- type Provider
- type ProviderCapabilities
- type ProviderConfig
- type ProviderFactory
- type ProviderTypeSetting
- type ReasoningFieldStyle
- type ResponsesProvider
- type ResponsesRequest
- type ResponsesRequestContext
- type ResponsesResponse
- type ResponsesResponseContentPart
- type ResponsesResponseOutput
- type ResponsesResponseUsage
- type ResponsesStreamEvent
- func ResponsesCompletedEvent(resp *ResponsesResponse) *ResponsesStreamEvent
- func ResponsesCreatedEvent(resp *ResponsesResponse) *ResponsesStreamEvent
- func ResponsesDeltaEvent(itemID string, outputIndex int, delta string) *ResponsesStreamEvent
- func ResponsesFunctionCallArgumentsDeltaEvent(itemID string, outputIndex int, delta string) *ResponsesStreamEvent
- func ResponsesOutputItemAddedEvent(outputIndex int, item *ResponsesResponseOutput) *ResponsesStreamEvent
- func ResponsesOutputItemDoneEvent(outputIndex int, item *ResponsesResponseOutput) *ResponsesStreamEvent
- type ResponsesToolDefinition
- type ResponsesToolFunction
- type UpstreamError
- type Usage
Constants ¶
This section is empty.
Variables ¶
var ErrProviderTypeDisabled = errors.New("provider type is disabled")
ErrProviderTypeDisabled is returned when a registered provider type is disabled.
Functions ¶
func ChatCompletionsExtraFieldsFromOptions ¶ added in v0.2.0
func ChatCompletionsExtraFieldsFromOptions(reasoning ReasoningFieldStyle, opts ...einomodel.Option) map[string]any
ChatCompletionsExtraFieldsFromOptions maps preserved request context onto chat-completions-style request fields shared by OpenAI-compatible upstreams. It folds in both the Responses API request context (when a Responses request was bridged onto chat) and the inbound chat extra fields, emitting only fields expressible on the chat wire shape. The reasoning style selects how reasoning is encoded for the target dialect.
func CheckResponse ¶
CheckResponse returns a StatusError if the HTTP response status is not 2xx. It reads up to 4 KB of the body to include in the error message.
func ConfigureProviderTypes ¶ added in v0.2.0
func ConfigureProviderTypes(settings []ProviderTypeSetting, exclusive bool) error
ConfigureProviderTypes applies startup-only provider type availability. If exclusive is true, every registered provider type not listed is disabled.
func CredentialFromContext ¶
func CredentialFromContext(ctx context.Context) (*credentialmgr.Credential, bool)
func DecodeStoredProviderConfig ¶
DecodeStoredProviderConfig converts a config-store provider payload into ProviderConfig.
func EnableAllProviderTypes ¶ added in v0.2.0
func EnableAllProviderTypes()
EnableAllProviderTypes clears the disabled set so every registered provider type is enabled. It is used when no startup provider_types policy is set.
func FinishReason ¶
func IsProviderTypeEnabled ¶
IsProviderTypeEnabled reports whether a registered provider type is enabled.
func ListProviderTypes ¶
func ListProviderTypes() []string
ListProviderTypes returns the names of all registered providers.
func MergeExtraFields ¶ added in v0.2.0
MergeExtraFields shallow-merges one or more request-body extension maps. Later maps override earlier keys.
func RegisterProviderFactory ¶
func RegisterProviderFactory(name string, factory ProviderFactory)
RegisterProviderFactory registers a provider factory by name.
func RetryProviderCall ¶
func RetryProviderCall[T any](config NetworkConfig, fn func() (T, error)) (T, error)
RetryProviderCall retries fn up to NetworkConfig.MaxRetries times on retryable errors (429, 5xx). Non-retryable 4xx errors are returned immediately. Do NOT use this for streaming; retry semantics are undefined once a stream starts.
func StreamResponsesViaChat ¶
func StreamResponsesViaChat(ctx context.Context, prov Provider, req *ResponsesRequest) (*schema.StreamReader[*ResponsesStreamEvent], error)
StreamResponsesViaChat adapts a streaming Responses API request onto the provider Chat stream API. Providers opt into this compatibility path explicitly by calling this helper.
func StripCCUnsupportedChatFields ¶ added in v0.2.0
StripCCUnsupportedChatFields removes the OpenAI-style `metadata` and `user` chat-completions request fields. Some OpenAI-compatible upstreams (e.g. GLM) reject these with a generic 400, while Claude Code always populates `metadata.user_id`. Providers running in cc-compat mode call this to drop the unsupported fields before the upstream request. It is a no-op on a nil map.
func UpstreamErrorFields ¶
UpstreamErrorFields extracts structured log fields from a provider UpstreamError.
func WithChatExtraFields ¶ added in v0.2.0
func WithChatExtraFields(fields *ChatExtraFields) einomodel.Option
WithChatExtraFields stores extra chat-completions request fields inside ChatRequest.Options so they survive generic chat-provider compatibility.
func WithCredential ¶
func WithCredential(ctx context.Context, cred *credentialmgr.Credential) context.Context
func WithResponsesRequestContext ¶ added in v0.2.0
func WithResponsesRequestContext(ctx *ResponsesRequestContext) einomodel.Option
WithResponsesRequestContext stores extra Responses API request fields inside ChatRequest.Options so they survive generic chat-provider compatibility.
Types ¶
type ChatExtraFields ¶ added in v0.2.0
type ChatExtraFields struct {
ResponseFormat any
Reasoning map[string]any
ReasoningEffort string
User string
Metadata map[string]any
ParallelToolCalls *bool
Store *bool
}
ChatExtraFields carries chat-completions request fields that have no eino common-option equivalent but should still reach OpenAI-compatible providers. Values are already in chat wire shape so they pass through unchanged.
func ChatExtraFieldsFromOptions ¶ added in v0.2.0
func ChatExtraFieldsFromOptions(opts ...einomodel.Option) *ChatExtraFields
ChatExtraFieldsFromOptions extracts any stored chat-completions extra fields from a chat option list.
type ChatOptions ¶ added in v0.2.0
type ChatOptions struct {
TopK int
Responses *ResponsesRequestContext
ChatExtra *ChatExtraFields
}
ChatOptions carries additional chat request options that extend the standard eino model options. Use WithTopK / GetChatOptions to set and read these.
func GetChatOptions ¶ added in v0.2.0
func GetChatOptions(opts ...einomodel.Option) *ChatOptions
GetChatOptions extracts ChatOptions from an option list.
type ChatRequest ¶
ChatRequest is the unified internal chat request format passed to providers.
func ResponsesToChatRequest ¶
func ResponsesToChatRequest(req *ResponsesRequest) (*ChatRequest, error)
ResponsesToChatRequest converts a minimal Responses API request into ChatRequest.
type ChatRequestState ¶
type ChatRequestState struct {
ModelName string
Messages []*schema.Message
Options []einomodel.Option
CommonOptions *einomodel.Options
}
func ResolveChatRequest ¶
func ResolveChatRequest(_ context.Context, config ProviderConfig, req *ChatRequest) (*ChatRequestState, error)
type ChatResponse ¶
ChatResponse is the unified internal chat response format returned by providers.
func ChatResponseFromEinoMessage ¶
func ChatResponseFromEinoMessage(msg *schema.Message) *ChatResponse
type CompactMode ¶ added in v0.2.0
type CompactMode string
const ( CompactModeNone CompactMode = "none" CompactModeCC CompactMode = "cc" CompactModeCodex CompactMode = "codex" )
func CompactModeFromOptions ¶ added in v0.2.0
func CompactModeFromOptions(opts map[string]any) (CompactMode, error)
CompactModeFromOptions reads options.compact. Supported values are "cc", "codex", and "none"; missing or empty values resolve to "none".
type EmbeddingProvider ¶
type EmbeddingProvider interface {
Provider
Embedding(ctx context.Context, req *EmbeddingRequest) (*EmbeddingResponse, error)
}
EmbeddingProvider is an optional interface for providers that support embeddings. The memory module uses this to generate vectors for storage and search.
type EmbeddingRequest ¶
type EmbeddingRequest struct {
// Model is the embedding model to use. Leave empty to use provider default.
Model string
// Texts are the strings to embed.
Texts []string
}
EmbeddingRequest is the request to generate vector embeddings.
type EmbeddingResponse ¶
EmbeddingResponse contains the generated embeddings.
type LLMApiRequestType ¶
type LLMApiRequestType string
LLMApiRequestType identifies the prepared provider-facing request kind.
const ( LLMApiRequestTypeChat LLMApiRequestType = "chat" LLMApiRequestTypeEmbedding LLMApiRequestType = "embedding" LLMApiRequestTypeResponses LLMApiRequestType = "responses" LLMApiRequestTypeModels LLMApiRequestType = "models" )
type ModelCapabilities ¶
type ModelCapabilities struct {
Streaming bool `json:"streaming,omitempty"`
Tools bool `json:"tools,omitempty"`
Vision bool `json:"vision,omitempty"`
Embeddings bool `json:"embeddings,omitempty"`
ContextWindow int `json:"context_window,omitempty"`
MaxOutputTokens int `json:"max_output_tokens,omitempty"`
}
ModelCapabilities describes what one specific upstream model supports. It is the routing authority for model-catalog decisions.
func ModelCapabilitiesFromProviderSummary ¶
func ModelCapabilitiesFromProviderSummary(c ProviderCapabilities) ModelCapabilities
ModelCapabilitiesFromProviderSummary converts coarse provider-level capability metadata into a per-model fallback shape.
type ModelInfo ¶
type ModelInfo struct {
ID string
Name string
DisplayName string
Description string
Capabilities ModelCapabilities
}
ModelInfo describes a model available from a provider.
type NetworkConfig ¶
type NetworkConfig = httpclient.NetworkConfig
NetworkConfig re-exports the shared HTTP network config type for provider configs.
type Provider ¶
type Provider interface {
// Chat performs a non-streaming chat completion and returns the full response.
Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)
// StreamChat performs a streaming chat completion and returns an Eino message stream.
StreamChat(ctx context.Context, req *ChatRequest) (*schema.StreamReader[*schema.Message], error)
// ListModels returns the list of models available from this provider.
ListModels(ctx context.Context) ([]ModelInfo, error)
// Capabilities returns what this provider instance supports.
Capabilities() ProviderCapabilities
// Config returns the provider instance configuration view used at runtime.
Config() ProviderConfig
}
Provider is the core interface implemented by all LLM providers. Additional capabilities are exposed through optional interfaces such as EmbeddingProvider and ResponsesProvider.
func NewProvider ¶
func NewProvider(config ProviderConfig) (Provider, error)
NewProvider creates a provider by name using registered factories.
type ProviderCapabilities ¶
type ProviderCapabilities struct {
Streaming bool
Tools bool
Vision bool
Embeddings bool
ContextWindow int
MaxOutputTokens int
}
ProviderCapabilities describes what a provider instance supports.
type ProviderConfig ¶
type ProviderConfig struct {
// Id is the unique provider config ID.
Id string `json:"id"`
// ProviderType is the registered provider type (e.g. "openai", "anthropic").
ProviderType string `json:"provider_type"`
// Disabled prevents the provider from being selected at runtime.
Disabled bool `json:"disabled"`
// APIKey is the provider API key. May be empty for local providers (Ollama).
APIKey string `json:"api_key,omitempty"`
// BaseURL overrides the provider's default API base URL.
BaseURL string `json:"base_url,omitempty"`
// DefaultModel is used when the request does not specify a model.
DefaultModel string `json:"default_model,omitempty"`
// Network contains HTTP client configuration (timeout, retry, proxy).
Network NetworkConfig `json:"network"`
// Options holds provider-specific extra configuration.
Options map[string]any `json:"options,omitempty"`
}
ProviderConfig contains configuration for a provider instance.
func NormalizeConfig ¶
func NormalizeConfig(cfg ProviderConfig, fallbackId string, fallbackName string) ProviderConfig
NormalizeConfig returns a runtime-ready provider config without mutating the source value. If ProviderType is empty, fallbackName is applied before defaults.
func NormalizeStoredProviderConfig ¶
func NormalizeStoredProviderConfig(fallbackId string, fallbackName string, obj any) (ProviderConfig, error)
NormalizeStoredProviderConfig converts a decoded config-store object into a runtime-ready ProviderConfig without mutating the decoded object.
func (ProviderConfig) CompactMode ¶ added in v0.2.0
func (c ProviderConfig) CompactMode() CompactMode
CompactMode reads options.compact and treats invalid values as "none". Use CompactModeFromOptions in provider constructors that should reject bad config.
func (*ProviderConfig) Defaults ¶
func (c *ProviderConfig) Defaults()
Defaults fills in zero values with sensible defaults.
type ProviderFactory ¶
type ProviderFactory func(config ProviderConfig) (Provider, error)
ProviderFactory creates a Provider instance from config.
type ProviderTypeSetting ¶ added in v0.2.0
type ReasoningFieldStyle ¶ added in v0.2.0
type ReasoningFieldStyle int
ReasoningFieldStyle selects how reasoning is expressed on an OpenAI-compatible chat-completions wire shape.
const ( // ReasoningEffortField emits reasoning as a "reasoning_effort" string, the // standard OpenAI chat-completions shape. ReasoningEffortField ReasoningFieldStyle = iota // ReasoningObjectField emits reasoning as a "reasoning" object, the shape // used by OpenAI-compatible upstreams that accept a structured reasoning // config (e.g. OpenRouter). ReasoningObjectField )
type ResponsesProvider ¶
type ResponsesProvider interface {
Provider
CreateResponses(ctx context.Context, req *ResponsesRequest) (*ResponsesResponse, error)
StreamResponses(ctx context.Context, req *ResponsesRequest) (*schema.StreamReader[*ResponsesStreamEvent], error)
}
ResponsesProvider is an optional interface for providers that expose OpenAI-compatible Responses API semantics directly.
type ResponsesRequest ¶
type ResponsesRequest struct {
Model string `json:"model"`
Input any `json:"input"`
Tools []ResponsesToolDefinition `json:"tools,omitempty"`
ToolChoice json.RawMessage `json:"tool_choice,omitempty"`
MaxOutputTokens int `json:"max_output_tokens,omitempty"`
Temperature float64 `json:"temperature,omitempty"`
TopP float64 `json:"top_p,omitempty"`
Stream bool `json:"stream,omitempty"`
Instructions string `json:"instructions,omitempty"`
PreviousResponseID string `json:"previous_response_id,omitempty"`
Store *bool `json:"store,omitempty"`
Text map[string]any `json:"text,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"`
User string `json:"user,omitempty"`
Reasoning map[string]any `json:"reasoning,omitempty"`
ParallelToolCalls *bool `json:"parallel_tool_calls,omitempty"`
Truncation any `json:"truncation,omitempty"`
}
ResponsesRequest is the minimal provider-level request model for the OpenAI Responses API. Input is intentionally preserved as structured JSON to avoid collapsing it into the chat-only ChatRequest abstraction.
func ResponsesRequestFromChatState ¶ added in v0.2.0
func ResponsesRequestFromChatState(state *ChatRequestState, stream bool) *ResponsesRequest
ResponsesRequestFromChatState rebuilds a provider-level Responses request from a resolved ChatRequestState. This is the generic inverse of the ResponsesToChatRequest compatibility path and is intended for providers that bridge chat calls onto an upstream Responses API.
type ResponsesRequestContext ¶ added in v0.2.0
type ResponsesRequestContext struct {
PreviousResponseID string
Store *bool
Text map[string]any
Metadata map[string]any
User string
Reasoning map[string]any
ParallelToolCalls *bool
Truncation any
}
ResponsesRequestContext carries Responses API request fields that do not map directly onto the shared chat abstraction but still need to survive the compatibility path so any provider can inspect or reuse them.
func ResponsesRequestContextFromOptions ¶ added in v0.2.0
func ResponsesRequestContextFromOptions(opts ...einomodel.Option) *ResponsesRequestContext
ResponsesRequestContextFromOptions extracts any stored Responses API request context from a chat option list.
type ResponsesResponse ¶
type ResponsesResponse struct {
ID string `json:"id"`
Object string `json:"object"`
CreatedAt int64 `json:"created_at"`
Model string `json:"model"`
Output []ResponsesResponseOutput `json:"output"`
Usage *ResponsesResponseUsage `json:"usage,omitempty"`
RawJSON json.RawMessage `json:"-"`
}
ResponsesResponse mirrors the OpenAI-compatible Responses API envelope.
func CreateResponsesViaChat ¶
func CreateResponsesViaChat(ctx context.Context, prov Provider, req *ResponsesRequest) (*ResponsesResponse, error)
CreateResponsesViaChat adapts a Responses API request onto the provider Chat API. Providers opt into this compatibility path explicitly by calling this helper.
func ResponsesFromChatResponse ¶
func ResponsesFromChatResponse(resp *ChatResponse, model string) *ResponsesResponse
ResponsesFromChatResponse converts a Chat response into a minimal Responses envelope.
type ResponsesResponseOutput ¶
type ResponsesResponseOutput struct {
ID string `json:"id,omitempty"`
Type string `json:"type"`
Role string `json:"role,omitempty"`
Status string `json:"status,omitempty"`
Content []ResponsesResponseContentPart `json:"content,omitempty"`
CallID string `json:"call_id,omitempty"`
Name string `json:"name,omitempty"`
Arguments string `json:"arguments,omitempty"`
}
type ResponsesResponseUsage ¶
type ResponsesStreamEvent ¶
type ResponsesStreamEvent struct {
Type string `json:"type"`
Response *ResponsesResponse `json:"response,omitempty"`
Item *ResponsesResponseOutput `json:"item,omitempty"`
Delta string `json:"delta,omitempty"`
ItemID string `json:"item_id,omitempty"`
OutputIndex int `json:"output_index,omitempty"`
ContentIndex int `json:"content_index,omitempty"`
RawJSON json.RawMessage `json:"-"`
}
ResponsesStreamEvent is the minimal event model currently required by the gateway for OpenAI-compatible Responses API streaming.
func ResponsesCompletedEvent ¶
func ResponsesCompletedEvent(resp *ResponsesResponse) *ResponsesStreamEvent
func ResponsesCreatedEvent ¶
func ResponsesCreatedEvent(resp *ResponsesResponse) *ResponsesStreamEvent
func ResponsesDeltaEvent ¶
func ResponsesDeltaEvent(itemID string, outputIndex int, delta string) *ResponsesStreamEvent
func ResponsesFunctionCallArgumentsDeltaEvent ¶ added in v0.2.0
func ResponsesFunctionCallArgumentsDeltaEvent(itemID string, outputIndex int, delta string) *ResponsesStreamEvent
func ResponsesOutputItemAddedEvent ¶ added in v0.2.0
func ResponsesOutputItemAddedEvent(outputIndex int, item *ResponsesResponseOutput) *ResponsesStreamEvent
func ResponsesOutputItemDoneEvent ¶ added in v0.2.0
func ResponsesOutputItemDoneEvent(outputIndex int, item *ResponsesResponseOutput) *ResponsesStreamEvent
type ResponsesToolDefinition ¶ added in v0.2.0
type ResponsesToolDefinition struct {
Type string `json:"type"`
Name string `json:"name,omitempty"`
Description string `json:"description,omitempty"`
Parameters json.RawMessage `json:"parameters,omitempty"`
Function *ResponsesToolFunction `json:"function,omitempty"`
}
type ResponsesToolFunction ¶ added in v0.2.0
type ResponsesToolFunction struct {
Name string `json:"name"`
Description string `json:"description,omitempty"`
Parameters json.RawMessage `json:"parameters,omitempty"`
}
type UpstreamError ¶
UpstreamError describes an HTTP error returned by an upstream provider API.
func (*UpstreamError) Error ¶
func (e *UpstreamError) Error() string
func (*UpstreamError) StatusCode ¶
func (e *UpstreamError) StatusCode() int
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package anthropic implements the Anthropic provider (Claude models).
|
Package anthropic implements the Anthropic provider (Claude models). |
|
Package anthropicbase provides shared Anthropic Messages API wire helpers.
|
Package anthropicbase provides shared Anthropic Messages API wire helpers. |
|
Package deepseek implements the DeepSeek provider.
|
Package deepseek implements the DeepSeek provider. |
|
Package gemini implements the Google Gemini provider.
|
Package gemini implements the Google Gemini provider. |
|
Package ollama implements the Ollama provider (local deployment, OpenAI-compatible).
|
Package ollama implements the Ollama provider (local deployment, OpenAI-compatible). |
|
Package openai implements the OpenAI provider.
|
Package openai implements the OpenAI provider. |
|
Package openaibase provides shared OpenAI-compatible wire types still used for model listing and embeddings.
|
Package openaibase provides shared OpenAI-compatible wire types still used for model listing and embeddings. |
|
Package openrouter implements the OpenRouter provider (OpenAI-compatible API).
|
Package openrouter implements the OpenRouter provider (OpenAI-compatible API). |
|
Package zhipu implements the Zhipu BigModel provider.
|
Package zhipu implements the Zhipu BigModel provider. |