provider

package
v0.5.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 28, 2026 License: Apache-2.0 Imports: 23 Imported by: 0

Documentation

Index

Constants

View Source
const (
	ProtocolFeatureClassModeSelection ProtocolFeatureClass = "mode_selection"
	ProtocolFeatureClassRequirement   ProtocolFeatureClass = "requirement"

	FeatureAnthropicServerToolRequest   ProtocolFeature = "anthropic.server_tool_request"
	FeatureAnthropicNativeResponse      ProtocolFeature = "anthropic.native_response"
	FeatureAnthropicNativeHistoryReplay ProtocolFeature = "anthropic.native_history_replay"
	FeatureAnthropicReasoningReplay     ProtocolFeature = "anthropic.reasoning_replay"
	FeatureAnthropicStreamRelay         ProtocolFeature = "anthropic.native_stream_relay"
	FeatureAnthropicBodyRelay           ProtocolFeature = "anthropic.native_body_relay"
)
View Source
const (
	// ChatMessagePartTypeEncryptedReasoning carries opaque reasoning state that
	// must be replayed unchanged but is not safe to expose as readable text.
	// Anthropic redacted_thinking blocks are represented with this neutral
	// internal type so compatible providers can preserve them across tool turns.
	ChatMessagePartTypeEncryptedReasoning schema.ChatMessagePartType = "encrypted_reasoning"
	// ChatMessagePartTypeReasoningEnd marks the end of one streamed reasoning
	// block. It lets protocol adapters preserve multiple consecutive reasoning
	// blocks without folding their signatures together.
	ChatMessagePartTypeReasoningEnd schema.ChatMessagePartType = "reasoning_end"
)

Variables

View Source
var (
	OpenAIChatCompletionsFields = ChatCompletionsFieldPolicy{
		ReasoningStyle: ReasoningEffortField,
		AllowedEfforts: []string{"none", "minimal", "low", "medium", "high", "xhigh"},
		EffortAliases:  map[string]string{"max": "xhigh"},
	}
	OpenRouterChatCompletionsFields = ChatCompletionsFieldPolicy{
		ReasoningStyle:      ReasoningObjectField,
		ThinkingBudgetField: "max_tokens",
	}
	ZhipuChatCompletionsFields = ChatCompletionsFieldPolicy{
		ReasoningStyle: ReasoningEffortField,
		AllowThinking:  true,
		AllowedEfforts: []string{"none", "minimal", "low", "medium", "high", "xhigh", "max"},
	}
)
View Source
var ErrProviderTypeDisabled = errors.New("provider type is disabled")

ErrProviderTypeDisabled is returned when a registered provider type is disabled.

Functions

func APIKeyFromContextOrConfig

func APIKeyFromContextOrConfig(ctx context.Context, configAPIKey string) string

func AttachMessageProtocolState added in v0.5.1

func AttachMessageProtocolState(msg *schema.Message, state *ProtocolState) *schema.Message

func AttachReasoningParts added in v0.5.0

func AttachReasoningParts(msg *schema.Message, parts ...schema.MessageOutputPart) *schema.Message

AttachReasoningParts appends gateway-owned structured reasoning metadata to msg without populating eino's provider-defined multimodal output field.

func BoolOption added in v0.5.0

func BoolOption(options map[string]any, name string, fallback bool) (bool, error)

BoolOption reads a boolean provider option from bool or string input.

func ChatCompletionsExtraFieldsFromOptions added in v0.2.0

func ChatCompletionsExtraFieldsFromOptions(policy ChatCompletionsFieldPolicy, opts ...einomodel.Option) map[string]any

ChatCompletionsExtraFieldsFromOptions maps preserved request context onto chat-completions-style request fields shared by OpenAI-compatible upstreams. It folds in both the Responses API request context (when a Responses request was bridged onto chat) and the inbound chat extra fields, emitting only fields expressible on the chat wire shape. The reasoning style selects how reasoning is encoded for the target dialect.

func CheckResponse

func CheckResponse(resp *http.Response) error

CheckResponse returns a StatusError if the HTTP response status is not 2xx. It reads up to 4 KB of the body to include in the error message.

func ConfigureProviderTypes added in v0.2.0

func ConfigureProviderTypes(settings []ProviderTypeSetting, exclusive bool) error

ConfigureProviderTypes applies startup-only provider type availability. If exclusive is true, every registered provider type not listed is disabled.

func CredentialFromContext

func CredentialFromContext(ctx context.Context) (*credential.Credential, bool)

func DecodeStoredProviderConfig

func DecodeStoredProviderConfig(data []byte) (any, error)

DecodeStoredProviderConfig converts a config-store provider payload into ProviderConfig.

func EnableAllProviderTypes added in v0.2.0

func EnableAllProviderTypes()

EnableAllProviderTypes clears the disabled set so every registered provider type is enabled. It is used when no startup provider_types policy is set.

func EncryptedReasoningData added in v0.5.0

func EncryptedReasoningData(part schema.MessageOutputPart) string

EncryptedReasoningData returns the opaque payload from an encrypted reasoning part.

func FinishReason

func FinishReason(msg *schema.Message) string

func FoldMessageProtocolState added in v0.5.1

func FoldMessageProtocolState(msg *schema.Message) error

func GatewayThinkingSignature added in v0.5.0

func GatewayThinkingSignature(reasoning string) string

GatewayThinkingSignature returns a deterministic placeholder signature for reasoning produced by providers that do not expose an upstream signature. It is suitable for Anthropic client-facing responses, but must never be sent to an Anthropic upstream as though it were an authentic opaque signature.

func IsGatewayThinkingSignature added in v0.5.0

func IsGatewayThinkingSignature(signature string) bool

IsGatewayThinkingSignature reports whether a signature was synthesized by GatewayThinkingSignature rather than supplied by an upstream model.

func IsProviderTypeEnabled

func IsProviderTypeEnabled(name string) (bool, bool)

IsProviderTypeEnabled reports whether a registered provider type is enabled.

func ListProviderTypes

func ListProviderTypes() []string

ListProviderTypes returns the names of all registered providers.

func MergeExtraFields added in v0.2.0

func MergeExtraFields(base map[string]any, overlays ...map[string]any) map[string]any

MergeExtraFields shallow-merges one or more request-body extension maps. Later maps override earlier keys.

func NewEncryptedReasoningOutputPart added in v0.5.0

func NewEncryptedReasoningOutputPart(data string, index *int) schema.MessageOutputPart

NewEncryptedReasoningOutputPart builds an opaque reasoning part. The data is deliberately not stored in ReasoningContent because callers must never edit, summarize, or expose it as model-authored text.

func NewReasoningEndOutputPart added in v0.5.0

func NewReasoningEndOutputPart(index int) schema.MessageOutputPart

NewReasoningEndOutputPart marks a streamed reasoning content block complete.

func NewReasoningOutputPart added in v0.5.0

func NewReasoningOutputPart(text, signature string, index *int) schema.MessageOutputPart

NewReasoningOutputPart builds a structured reasoning part. index identifies the source stream content-block index when the part came from streaming.

func NormalizeObjectToolInputSchema added in v0.5.1

func NormalizeObjectToolInputSchema(raw json.RawMessage) (json.RawMessage, error)

NormalizeObjectToolInputSchema converts an absent parameter schema into the canonical no-argument object schema and validates an explicitly supplied schema's top-level shape. An empty JSON Schema object remains valid.

func OnChatEnd added in v0.5.0

func OnChatEnd(ctx context.Context, modelName string, msg *schema.Message)

OnChatEnd fires the OnEnd timing for a non-streaming chat result.

func OnChatError added in v0.5.0

func OnChatError(ctx context.Context, err error)

OnChatError fires the OnError timing.

func OnChatStart added in v0.5.0

func OnChatStart(ctx context.Context, providerType, modelName string, messages []*schema.Message) context.Context

OnChatStart ensures chat-model run info on the context and fires the OnStart timing. The returned context must be passed to the matching OnChatEnd / OnChatError / OnChatStreamEnd call.

func OnChatStreamEnd added in v0.5.0

func OnChatStreamEnd(ctx context.Context, modelName string, stream *schema.StreamReader[*schema.Message]) *schema.StreamReader[*schema.Message]

OnChatStreamEnd fires the OnEndWithStreamOutput timing and returns the stream the provider must hand to its caller. The framework only copies the stream for handlers whose TimingChecker asks for the stream timing; with none registered this is a cheap lazy conversion.

func PositiveIntOption added in v0.5.0

func PositiveIntOption(options map[string]any, name string, fallback int) (int, error)

PositiveIntOption reads a positive integer provider option. String values support Caddyfile configuration, while integer and integral float values support JSON and YAML configuration.

func PositiveIntValue added in v0.5.0

func PositiveIntValue(raw any) (int, bool)

PositiveIntValue converts a supported configuration value to a positive int.

func RegisterDialectCodec added in v0.5.1

func RegisterDialectCodec(dialect ProtocolDialect, codec DialectCodec)

func RegisterProtocolFeature added in v0.5.1

func RegisterProtocolFeature(def ProtocolFeatureDefinition)

func RegisterProviderFactory

func RegisterProviderFactory(name string, factory ProviderFactory)

RegisterProviderFactory registers a provider factory by name.

func RegisterProviderTypeCapabilities added in v0.5.1

func RegisterProviderTypeCapabilities(name string, capabilities ProviderTypeCapabilities)

RegisterProviderTypeCapabilities declares protocol-level capabilities used before a provider instance is selected or a credential is charged.

func RetryProviderCall

func RetryProviderCall[T any](config NetworkConfig, fn func() (T, error)) (T, error)

RetryProviderCall retries fn up to NetworkConfig.MaxRetries times on retryable errors (429, 5xx). Non-retryable 4xx errors are returned immediately. Do NOT use this for streaming; retry semantics are undefined once a stream starts.

func StreamResponsesViaChat

func StreamResponsesViaChat(ctx context.Context, prov Provider, req *ResponsesRequest) (*schema.StreamReader[*ResponsesStreamEvent], error)

StreamResponsesViaChat adapts a streaming Responses API request onto the provider Chat stream API. Providers opt into this compatibility path explicitly by calling this helper.

func StripCCUnsupportedChatFields added in v0.2.0

func StripCCUnsupportedChatFields(fields map[string]any)

StripCCUnsupportedChatFields removes the OpenAI-style `metadata` and `user` chat-completions request fields. Some OpenAI-compatible upstreams (e.g. GLM) reject these with a generic 400, while Claude Code always populates `metadata.user_id`. Providers running in cc-compat mode call this to drop the unsupported fields before the upstream request. It is a no-op on a nil map.

func UpstreamErrorFields

func UpstreamErrorFields(err error) []zap.Field

UpstreamErrorFields extracts structured log fields from a provider UpstreamError.

func WithChatExtraFields added in v0.2.0

func WithChatExtraFields(fields *ChatExtraFields) einomodel.Option

WithChatExtraFields stores extra chat-completions request fields inside ChatRequest.Options so they survive generic chat-provider compatibility.

func WithCredential

func WithCredential(ctx context.Context, cred *credential.Credential) context.Context

func WithProtocolState added in v0.5.1

func WithProtocolState(state *ProtocolState) einomodel.Option

WithProtocolState carries request-scoped protocol state across the eino ToolCallingChatModel option boundary.

func WithResponsesRequestContext added in v0.2.0

func WithResponsesRequestContext(ctx *ResponsesRequestContext) einomodel.Option

WithResponsesRequestContext stores extra Responses API request fields inside ChatRequest.Options so they survive generic chat-provider compatibility.

func WithTopK added in v0.2.0

func WithTopK(topK int) einomodel.Option

WithTopK adds a top-k sampling option. It is encoded as an impl-specific option so it travels inside ChatRequest.Options alongside standard options (temperature, max_tokens, etc.) and any provider can read it via GetChatOptions.

Types

type AttemptAttribution added in v0.5.1

type AttemptAttribution struct {
	CredentialID     string
	CredentialSource string
}

type CandidateIdentity added in v0.5.1

type CandidateIdentity struct {
	ProviderID    string
	ProviderType  string
	UpstreamModel string
}

type CapabilityOption added in v0.5.0

type CapabilityOption string

CapabilityOption identifies one provider capability that may be overridden through generic provider options.

const (
	CapabilityContextWindow   CapabilityOption = "context_window"
	CapabilityMaxOutputTokens CapabilityOption = "max_output_tokens"
	CapabilityVision          CapabilityOption = "vision"
	CapabilityEmbeddings      CapabilityOption = "embeddings"
)

type ChatCompletionsFieldPolicy added in v0.5.0

type ChatCompletionsFieldPolicy struct {
	ReasoningStyle ReasoningFieldStyle
	AllowThinking  bool
	AllowedEfforts []string
	EffortAliases  map[string]string
	// ThinkingBudgetField maps Anthropic thinking.budget_tokens into a field
	// on a structured reasoning object. It is ignored for effort-style dialects.
	ThinkingBudgetField string
}

ChatCompletionsFieldPolicy describes the reasoning controls accepted by one chat-completions wire dialect. Anthropic-native `thinking` must be opted into explicitly instead of leaking into every OpenAI-compatible provider.

type ChatExecution added in v0.5.1

type ChatExecution struct {
	Response *ChatResponse
	Resolved ResolvedExecution
}

type ChatExtraFields added in v0.2.0

type ChatExtraFields struct {
	ResponseFormat    any
	Reasoning         map[string]any
	Thinking          map[string]any
	ReasoningEffort   string
	ToolStream        *bool
	StreamOptions     map[string]any
	User              string
	Metadata          map[string]any
	ParallelToolCalls *bool
	Store             *bool
}

ChatExtraFields carries protocol request fields that have no eino common-option equivalent but should still reach compatible providers.

func ChatExtraFieldsFromOptions added in v0.2.0

func ChatExtraFieldsFromOptions(opts ...einomodel.Option) *ChatExtraFields

ChatExtraFieldsFromOptions extracts any stored chat-completions extra fields from a chat option list.

type ChatOptions added in v0.2.0

type ChatOptions struct {
	TopK          int
	Responses     *ResponsesRequestContext
	ChatExtra     *ChatExtraFields
	ProtocolState *ProtocolState
}

ChatOptions carries additional chat request options that extend the standard eino model options. Use WithTopK / GetChatOptions to set and read these.

func GetChatOptions added in v0.2.0

func GetChatOptions(opts ...einomodel.Option) *ChatOptions

GetChatOptions extracts ChatOptions from an option list.

type ChatRequest

type ChatRequest struct {
	Model         string
	Messages      []*schema.Message
	Options       []einomodel.Option
	ProtocolState *ProtocolState
}

ChatRequest is the unified internal chat request format passed to providers.

func ResponsesToChatRequest

func ResponsesToChatRequest(req *ResponsesRequest) (*ChatRequest, error)

ResponsesToChatRequest converts a minimal Responses API request into ChatRequest.

type ChatRequestState

type ChatRequestState struct {
	ModelName     string
	Messages      []*schema.Message
	Options       []einomodel.Option
	CommonOptions *einomodel.Options
	ProtocolState *ProtocolState
}

func ResolveChatRequest

func ResolveChatRequest(_ context.Context, config ProviderConfig, req *ChatRequest) (*ChatRequestState, error)

type ChatResponse

type ChatResponse struct {
	Message *schema.Message
}

ChatResponse is the unified internal chat response format returned by providers.

func ChatResponseFromEinoMessage

func ChatResponseFromEinoMessage(msg *schema.Message) *ChatResponse

type CompactMode added in v0.2.0

type CompactMode string
const (
	CompactModeNone  CompactMode = "none"
	CompactModeCC    CompactMode = "cc"
	CompactModeCodex CompactMode = "codex"
)

func CompactModeFromOptions added in v0.2.0

func CompactModeFromOptions(opts map[string]any) (CompactMode, error)

CompactModeFromOptions reads options.compact. Supported values are "cc", "codex", and "none"; missing or empty values resolve to "none".

type DialectCodec added in v0.5.1

type DialectCodec interface {
	Capture(NativeCaptureInput) (NativeEnvelope, error)
	Overlay(NativeOverlayInput) (json.RawMessage, error)
	FoldResponse([]NativeEnvelope) ([]NativeEnvelope, error)
	FoldStreamEvents([]NativeEnvelope) ([]NativeEnvelope, error)
	ValidateFragments(NativeStateKind, []NativeEnvelope) error
	ValidateOrder([]NativeEnvelope) error
}

func DialectCodecFor added in v0.5.1

func DialectCodecFor(dialect ProtocolDialect) (DialectCodec, error)

type EmbeddingProvider

type EmbeddingProvider interface {
	Provider
	Embedding(ctx context.Context, req *EmbeddingRequest) (*EmbeddingResponse, error)
}

EmbeddingProvider is an optional interface for providers that support embeddings. The memory module uses this to generate vectors for storage and search.

type EmbeddingRequest

type EmbeddingRequest struct {
	// Model is the embedding model to use. Leave empty to use provider default.
	Model string
	// Texts are the strings to embed.
	Texts []string
}

EmbeddingRequest is the request to generate vector embeddings.

type EmbeddingResponse

type EmbeddingResponse struct {
	Embeddings [][]float64
	Model      string
	Usage      Usage
}

EmbeddingResponse contains the generated embeddings.

type LLMApiRequestType

type LLMApiRequestType string

LLMApiRequestType identifies the prepared provider-facing request kind.

const (
	LLMApiRequestTypeChat      LLMApiRequestType = "chat"
	LLMApiRequestTypeEmbedding LLMApiRequestType = "embedding"
	LLMApiRequestTypeResponses LLMApiRequestType = "responses"
	LLMApiRequestTypeModels    LLMApiRequestType = "models"
)

type MessageProtocolState added in v0.5.1

type MessageProtocolState ProtocolState

type ModelCapabilities

type ModelCapabilities struct {
	Streaming       bool `json:"streaming,omitempty"`
	Tools           bool `json:"tools,omitempty"`
	Vision          bool `json:"vision,omitempty"`
	Embeddings      bool `json:"embeddings,omitempty"`
	ContextWindow   int  `json:"context_window,omitempty"`
	MaxOutputTokens int  `json:"max_output_tokens,omitempty"`
}

ModelCapabilities describes what one specific upstream model supports. It is the routing authority for model-catalog decisions.

func ModelCapabilitiesFromProviderSummary

func ModelCapabilitiesFromProviderSummary(c ProviderCapabilities) ModelCapabilities

ModelCapabilitiesFromProviderSummary converts coarse provider-level capability metadata into a per-model fallback shape.

type ModelInfo

type ModelInfo struct {
	ID           string
	Name         string
	DisplayName  string
	Description  string
	Capabilities ModelCapabilities
}

ModelInfo describes a model available from a provider.

type ModeledFieldBaseline added in v0.5.1

type ModeledFieldBaseline struct {
	Path    string   `json:"path"`
	Present bool     `json:"present"`
	Digest  [32]byte `json:"digest"`
}

func DigestModeledField added in v0.5.1

func DigestModeledField(path string, present bool, value any) (ModeledFieldBaseline, error)

type ModeledProjection added in v0.5.1

type ModeledProjection any

type NativeCaptureInput added in v0.5.1

type NativeCaptureInput struct {
	Scope    NativeStateScope
	Kind     NativeStateKind
	Location NativeLocation
	Raw      json.RawMessage
	Modeled  ModeledProjection
}

type NativeEnvelope added in v0.5.1

type NativeEnvelope struct {
	Dialect   ProtocolDialect        `json:"dialect"`
	Scope     NativeStateScope       `json:"scope"`
	Kind      NativeStateKind        `json:"kind"`
	Location  NativeLocation         `json:"location"`
	Raw       json.RawMessage        `json:"raw"`
	Baselines []ModeledFieldBaseline `json:"baselines,omitempty"`
}

type NativeLocation added in v0.5.1

type NativeLocation struct {
	MessageIndex int    `json:"message_index,omitempty"`
	ContentIndex int    `json:"content_index,omitempty"`
	ToolIndex    int    `json:"tool_index,omitempty"`
	SourceIndex  int    `json:"source_index,omitempty"`
	Event        string `json:"event,omitempty"`
}

type NativeOverlayInput added in v0.5.1

type NativeOverlayInput struct {
	Envelope NativeEnvelope
	Current  ModeledProjection
}

type NativeStateKind added in v0.5.1

type NativeStateKind string
const (
	NativeKindToolDefinition   NativeStateKind = "tool_definition"
	NativeKindToolChoice       NativeStateKind = "tool_choice"
	NativeKindContentBlock     NativeStateKind = "content_block"
	NativeKindResponseBody     NativeStateKind = "response_body"
	NativeKindStreamEvent      NativeStateKind = "stream_event"
	NativeKindStreamProjection NativeStateKind = "stream_projection"
)

type NativeStateScope added in v0.5.1

type NativeStateScope string
const (
	NativeScopeRequest           NativeStateScope = "request"
	NativeScopeMessageHistory    NativeStateScope = "message_history"
	NativeScopeResponseEphemeral NativeStateScope = "response_ephemeral"
	NativeScopeStreamEvent       NativeStateScope = "stream_event"
)

type NetworkConfig

type NetworkConfig = httpclient.NetworkConfig

NetworkConfig re-exports the shared HTTP network config type for provider configs.

type ProtocolDialect added in v0.5.1

type ProtocolDialect string

ProtocolDialect identifies wire-level state that is only portable between providers capable of preserving the same upstream protocol dialect.

const ProtocolDialectAnthropic ProtocolDialect = "anthropic"

type ProtocolFeature added in v0.5.1

type ProtocolFeature string

func MissingProtocolFeatures added in v0.5.1

func MissingProtocolFeatures(required ProtocolRequirementSet, supported map[ProtocolFeature]struct{}) []ProtocolFeature

type ProtocolFeatureClass added in v0.5.1

type ProtocolFeatureClass string

type ProtocolFeatureDefinition added in v0.5.1

type ProtocolFeatureDefinition struct {
	ID        ProtocolFeature
	Dialect   ProtocolDialect
	Class     ProtocolFeatureClass
	DependsOn []ProtocolFeature
}

func ProtocolFeatureDefinitionFor added in v0.5.1

func ProtocolFeatureDefinitionFor(id ProtocolFeature) (ProtocolFeatureDefinition, error)

type ProtocolRequirementSet added in v0.5.1

type ProtocolRequirementSet struct {
	// contains filtered or unexported fields
}

func CloneProtocolRequirementSet added in v0.5.1

func CloneProtocolRequirementSet(s ProtocolRequirementSet) ProtocolRequirementSet

func NewProtocolRequirementSet added in v0.5.1

func NewProtocolRequirementSet(values map[ProtocolFeature][]RequirementReason) (ProtocolRequirementSet, error)

func (ProtocolRequirementSet) Empty added in v0.5.1

func (s ProtocolRequirementSet) Empty() bool

func (ProtocolRequirementSet) Equal added in v0.5.1

func (ProtocolRequirementSet) Features added in v0.5.1

func (s ProtocolRequirementSet) Features() []ProtocolFeature

func (ProtocolRequirementSet) Reasons added in v0.5.1

type ProtocolState added in v0.5.1

type ProtocolState struct {
	Envelopes    []NativeEnvelope       `json:"envelopes,omitempty"`
	Requirements ProtocolRequirementSet `json:"requirements,omitempty"`
}

func CloneProtocolState added in v0.5.1

func CloneProtocolState(state *ProtocolState) *ProtocolState

func MergeMessageProtocolStates added in v0.5.1

func MergeMessageProtocolStates(states ...*ProtocolState) (*ProtocolState, error)

func ProtocolStateFromMessage added in v0.5.1

func ProtocolStateFromMessage(msg *schema.Message) *ProtocolState

type Provider

type Provider interface {
	// Chat performs a non-streaming chat completion and returns the full response.
	Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error)

	// StreamChat performs a streaming chat completion and returns an Eino message stream.
	StreamChat(ctx context.Context, req *ChatRequest) (*schema.StreamReader[*schema.Message], error)

	// ListModels returns the list of models available from this provider.
	ListModels(ctx context.Context) ([]ModelInfo, error)

	// Capabilities returns what this provider instance supports.
	Capabilities() ProviderCapabilities

	// Config returns the provider instance configuration view used at runtime.
	Config() ProviderConfig
}

Provider is the core interface implemented by all LLM providers. Additional capabilities are exposed through optional interfaces such as EmbeddingProvider and ResponsesProvider.

func NewProvider

func NewProvider(config ProviderConfig) (Provider, error)

NewProvider creates a provider by name using registered factories.

type ProviderCapabilities

type ProviderCapabilities struct {
	Streaming       bool
	Tools           bool
	Vision          bool
	Embeddings      bool
	ContextWindow   int
	MaxOutputTokens int
}

ProviderCapabilities describes what a provider instance supports.

func CapabilitiesFromOptions added in v0.5.0

func CapabilitiesFromOptions(options map[string]any, defaults ProviderCapabilities, supported ...CapabilityOption) (ProviderCapabilities, error)

CapabilitiesFromOptions applies the selected generic capability overrides to defaults. Callers choose the supported fields so an unrelated option cannot silently advertise a capability the provider does not implement.

type ProviderConfig

type ProviderConfig struct {
	// Id is the unique provider config ID.
	Id string `json:"id"`
	// ProviderType is the registered provider type (e.g. "openai", "anthropic").
	ProviderType string `json:"provider_type"`
	// Disabled prevents the provider from being selected at runtime.
	Disabled bool `json:"disabled"`
	// APIKey is the provider API key. May be empty for local providers (Ollama).
	APIKey string `json:"api_key,omitempty"`
	// BaseURL overrides the provider's default API base URL.
	BaseURL string `json:"base_url,omitempty"`
	// DefaultModel is used when the request does not specify a model.
	DefaultModel string `json:"default_model,omitempty"`
	// Network contains HTTP client configuration (timeout, retry, proxy).
	Network NetworkConfig `json:"network"`
	// Options holds provider-specific extra configuration.
	Options map[string]any `json:"options,omitempty"`
	// CreatedAt is set when the dynamic provider config is first persisted.
	CreatedAt time.Time `json:"created_at"`
	// UpdatedAt is bumped on every persisted create/update. It is used as the
	// runtime resolver's cache fingerprint so a config change rebuilds the
	// provider instance even if explicit invalidation was missed.
	UpdatedAt time.Time `json:"updated_at"`
}

ProviderConfig contains configuration for a provider instance.

func NormalizeConfig

func NormalizeConfig(cfg ProviderConfig, fallbackId string, fallbackName string) ProviderConfig

NormalizeConfig returns a runtime-ready provider config without mutating the source value. If ProviderType is empty, fallbackName is applied before defaults.

func NormalizeStoredProviderConfig

func NormalizeStoredProviderConfig(fallbackId string, fallbackName string, obj any) (ProviderConfig, error)

NormalizeStoredProviderConfig converts a decoded config-store object into a runtime-ready ProviderConfig without mutating the decoded object.

func (ProviderConfig) CompactMode added in v0.2.0

func (c ProviderConfig) CompactMode() CompactMode

CompactMode reads options.compact and treats invalid values as "none". Use CompactModeFromOptions in provider constructors that should reject bad config.

func (*ProviderConfig) Defaults

func (c *ProviderConfig) Defaults()

Defaults fills in zero values with sensible defaults.

func (ProviderConfig) Fingerprint added in v0.3.0

func (c ProviderConfig) Fingerprint() string

Fingerprint returns a cheap version string for the provider config used as the runtime resolver materializer key. UpdatedAt is bumped on every persisted create/update, so it changes whenever the stored config changes. Static providers have a zero UpdatedAt, which is a stable per-id constant and matches their immutable-at-runtime nature. Avoid marshaling the whole config here; this runs on every resolved request.

type ProviderFactory

type ProviderFactory func(config ProviderConfig) (Provider, error)

ProviderFactory creates a Provider instance from config.

type ProviderTypeCapabilities added in v0.5.1

type ProviderTypeCapabilities struct {
	Dialect          ProtocolDialect
	ProtocolFeatures map[ProtocolFeature]struct{}
}

ProviderTypeCapabilities describes protocol-level fidelity that is stable for every instance of one registered provider type.

func CapabilitiesForProviderType added in v0.5.1

func CapabilitiesForProviderType(name string) ProviderTypeCapabilities

CapabilitiesForProviderType returns the registered protocol-level capabilities for name. Unknown and undeclared provider types fail closed.

func (ProviderTypeCapabilities) SupportsProtocolFeature added in v0.5.1

func (c ProviderTypeCapabilities) SupportsProtocolFeature(feature ProtocolFeature) bool

type ProviderTypeSetting added in v0.2.0

type ProviderTypeSetting struct {
	ProviderType string `json:"provider_type"`
	Enabled      bool   `json:"enabled"`
}

type ReasoningFieldStyle added in v0.2.0

type ReasoningFieldStyle int

ReasoningFieldStyle selects how reasoning is expressed on an OpenAI-compatible chat-completions wire shape.

const (
	// ReasoningEffortField emits reasoning as a "reasoning_effort" string, the
	// standard OpenAI chat-completions shape.
	ReasoningEffortField ReasoningFieldStyle = iota
	// ReasoningObjectField emits reasoning as a "reasoning" object, the shape
	// used by OpenAI-compatible upstreams that accept a structured reasoning
	// config (e.g. OpenRouter).
	ReasoningObjectField
)

type ReasoningParts added in v0.5.0

type ReasoningParts []schema.MessageOutputPart

ReasoningParts is the gateway-owned representation of structured reasoning carried in schema.Message.Extra. It intentionally does not use AssistantGenMultiContent, whose types and semantics belong to eino model components and differ between providers.

func ReasoningPartsFromMessage added in v0.5.0

func ReasoningPartsFromMessage(msg *schema.Message) ReasoningParts

ReasoningPartsFromMessage returns a copy of the gateway-owned structured reasoning metadata attached to msg.

type RequirementGap added in v0.5.1

type RequirementGap struct {
	Candidate CandidateIdentity
	Missing   []ProtocolFeature
}

func (*RequirementGap) Error added in v0.5.1

func (g *RequirementGap) Error() string

func (*RequirementGap) StatusCode added in v0.5.1

func (*RequirementGap) StatusCode() int

type RequirementGapsError added in v0.5.1

type RequirementGapsError struct {
	Target string
	Gaps   []RequirementGap
}

func (*RequirementGapsError) Error added in v0.5.1

func (e *RequirementGapsError) Error() string

func (*RequirementGapsError) StatusCode added in v0.5.1

func (*RequirementGapsError) StatusCode() int

type RequirementReason added in v0.5.1

type RequirementReason string
const (
	ReasonAnthropicServerTool      RequirementReason = "anthropic_server_tool"
	ReasonAnthropicNativeHistory   RequirementReason = "anthropic_native_history"
	ReasonAnthropicSignedReasoning RequirementReason = "anthropic_signed_reasoning"
	ReasonAnthropicOpaqueContent   RequirementReason = "anthropic_opaque_content"
)

type ResolvedExecution added in v0.5.1

type ResolvedExecution struct {
	Candidate   ServedCandidate
	Attribution AttemptAttribution
}

type ResponsesInputTokensUsage added in v0.5.0

type ResponsesInputTokensUsage struct {
	CachedTokens int `json:"cached_tokens,omitempty"`
}

type ResponsesOutputTokensUsage added in v0.5.0

type ResponsesOutputTokensUsage struct {
	ReasoningTokens int `json:"reasoning_tokens,omitempty"`
}

type ResponsesProvider

type ResponsesProvider interface {
	Provider
	CreateResponses(ctx context.Context, req *ResponsesRequest) (*ResponsesResponse, error)
	StreamResponses(ctx context.Context, req *ResponsesRequest) (*schema.StreamReader[*ResponsesStreamEvent], error)
}

ResponsesProvider is an optional interface for providers that expose OpenAI-compatible Responses API semantics directly.

type ResponsesReasoningSummaryPart added in v0.5.0

type ResponsesReasoningSummaryPart struct {
	Type string `json:"type"`
	Text string `json:"text"`
}

type ResponsesRequest

type ResponsesRequest struct {
	Model              string                    `json:"model"`
	Input              any                       `json:"input"`
	Tools              []ResponsesToolDefinition `json:"tools,omitempty"`
	ToolChoice         json.RawMessage           `json:"tool_choice,omitempty"`
	MaxOutputTokens    int                       `json:"max_output_tokens,omitempty"`
	Temperature        float64                   `json:"temperature,omitempty"`
	TopP               float64                   `json:"top_p,omitempty"`
	Stream             bool                      `json:"stream,omitempty"`
	Instructions       string                    `json:"instructions,omitempty"`
	PreviousResponseID string                    `json:"previous_response_id,omitempty"`
	Store              *bool                     `json:"store,omitempty"`
	Text               map[string]any            `json:"text,omitempty"`
	Metadata           map[string]any            `json:"metadata,omitempty"`
	User               string                    `json:"user,omitempty"`
	Reasoning          map[string]any            `json:"reasoning,omitempty"`
	ParallelToolCalls  *bool                     `json:"parallel_tool_calls,omitempty"`
	Truncation         any                       `json:"truncation,omitempty"`
}

ResponsesRequest is the minimal provider-level request model for the OpenAI Responses API. Input is intentionally preserved as structured JSON to avoid collapsing it into the chat-only ChatRequest abstraction.

func ResponsesRequestFromChatState added in v0.2.0

func ResponsesRequestFromChatState(state *ChatRequestState, stream bool) *ResponsesRequest

ResponsesRequestFromChatState rebuilds a provider-level Responses request from a resolved ChatRequestState. This is the generic inverse of the ResponsesToChatRequest compatibility path and is intended for providers that bridge chat calls onto an upstream Responses API.

type ResponsesRequestContext added in v0.2.0

type ResponsesRequestContext struct {
	PreviousResponseID string
	Store              *bool
	Text               map[string]any
	Metadata           map[string]any
	User               string
	Reasoning          map[string]any
	ParallelToolCalls  *bool
	Truncation         any
}

ResponsesRequestContext carries Responses API request fields that do not map directly onto the shared chat abstraction but still need to survive the compatibility path so any provider can inspect or reuse them.

func ResponsesRequestContextFromOptions added in v0.2.0

func ResponsesRequestContextFromOptions(opts ...einomodel.Option) *ResponsesRequestContext

ResponsesRequestContextFromOptions extracts any stored Responses API request context from a chat option list.

type ResponsesResponse

type ResponsesResponse struct {
	ID        string                    `json:"id"`
	Object    string                    `json:"object"`
	CreatedAt int64                     `json:"created_at"`
	Model     string                    `json:"model"`
	Output    []ResponsesResponseOutput `json:"output"`
	Usage     *ResponsesResponseUsage   `json:"usage,omitempty"`
	RawJSON   json.RawMessage           `json:"-"`
}

ResponsesResponse mirrors the OpenAI-compatible Responses API envelope.

func CreateResponsesViaChat

func CreateResponsesViaChat(ctx context.Context, prov Provider, req *ResponsesRequest) (*ResponsesResponse, error)

CreateResponsesViaChat adapts a Responses API request onto the provider Chat API. Providers opt into this compatibility path explicitly by calling this helper.

func ResponsesFromChatResponse

func ResponsesFromChatResponse(resp *ChatResponse, model string) *ResponsesResponse

ResponsesFromChatResponse converts a Chat response into a minimal Responses envelope.

type ResponsesResponseContentPart

type ResponsesResponseContentPart struct {
	Type        string          `json:"type"`
	Text        string          `json:"text,omitempty"`
	Annotations []any           `json:"annotations,omitempty"`
	Refusal     string          `json:"refusal,omitempty"`
	Summary     []any           `json:"summary,omitempty"`
	RawJSON     json.RawMessage `json:"-"`
}

type ResponsesResponseOutput

type ResponsesResponseOutput struct {
	ID        string                          `json:"id,omitempty"`
	Type      string                          `json:"type"`
	Role      string                          `json:"role,omitempty"`
	Status    string                          `json:"status,omitempty"`
	Content   []ResponsesResponseContentPart  `json:"content,omitempty"`
	CallID    string                          `json:"call_id,omitempty"`
	Name      string                          `json:"name,omitempty"`
	Arguments string                          `json:"arguments,omitempty"`
	Summary   []ResponsesReasoningSummaryPart `json:"summary,omitempty"`
}

type ResponsesResponseUsage

type ResponsesResponseUsage struct {
	InputTokens         int                        `json:"input_tokens"`
	OutputTokens        int                        `json:"output_tokens"`
	TotalTokens         int                        `json:"total_tokens"`
	InputTokensDetails  ResponsesInputTokensUsage  `json:"input_tokens_details,omitzero"`
	OutputTokensDetails ResponsesOutputTokensUsage `json:"output_tokens_details,omitzero"`
}

type ResponsesStreamEvent

type ResponsesStreamEvent struct {
	Type         string                         `json:"type"`
	Response     *ResponsesResponse             `json:"response,omitempty"`
	Item         *ResponsesResponseOutput       `json:"item,omitempty"`
	Delta        string                         `json:"delta,omitempty"`
	ItemID       string                         `json:"item_id,omitempty"`
	OutputIndex  int                            `json:"output_index,omitempty"`
	ContentIndex int                            `json:"content_index,omitempty"`
	SummaryIndex int                            `json:"summary_index,omitempty"`
	Part         *ResponsesReasoningSummaryPart `json:"part,omitempty"`
	RawJSON      json.RawMessage                `json:"-"`
}

ResponsesStreamEvent is the minimal event model currently required by the gateway for OpenAI-compatible Responses API streaming.

func ResponsesCompletedEvent

func ResponsesCompletedEvent(resp *ResponsesResponse) *ResponsesStreamEvent

func ResponsesCreatedEvent

func ResponsesCreatedEvent(resp *ResponsesResponse) *ResponsesStreamEvent

func ResponsesDeltaEvent

func ResponsesDeltaEvent(itemID string, outputIndex int, delta string) *ResponsesStreamEvent

func ResponsesFunctionCallArgumentsDeltaEvent added in v0.2.0

func ResponsesFunctionCallArgumentsDeltaEvent(itemID string, outputIndex int, delta string) *ResponsesStreamEvent

func ResponsesOutputItemAddedEvent added in v0.2.0

func ResponsesOutputItemAddedEvent(outputIndex int, item *ResponsesResponseOutput) *ResponsesStreamEvent

func ResponsesOutputItemDoneEvent added in v0.2.0

func ResponsesOutputItemDoneEvent(outputIndex int, item *ResponsesResponseOutput) *ResponsesStreamEvent

func ResponsesReasoningSummaryDeltaEvent added in v0.5.0

func ResponsesReasoningSummaryDeltaEvent(itemID string, outputIndex int, delta string) *ResponsesStreamEvent

func ResponsesReasoningSummaryPartAddedEvent added in v0.5.0

func ResponsesReasoningSummaryPartAddedEvent(itemID string, outputIndex int, part *ResponsesReasoningSummaryPart) *ResponsesStreamEvent

func ResponsesReasoningSummaryPartDoneEvent added in v0.5.0

func ResponsesReasoningSummaryPartDoneEvent(itemID string, outputIndex int, part *ResponsesReasoningSummaryPart) *ResponsesStreamEvent

type ResponsesToolDefinition added in v0.2.0

type ResponsesToolDefinition struct {
	Type        string                 `json:"type"`
	Name        string                 `json:"name,omitempty"`
	Description string                 `json:"description,omitempty"`
	Parameters  json.RawMessage        `json:"parameters,omitempty"`
	Function    *ResponsesToolFunction `json:"function,omitempty"`
}

type ResponsesToolFunction added in v0.2.0

type ResponsesToolFunction struct {
	Name        string          `json:"name"`
	Description string          `json:"description,omitempty"`
	Parameters  json.RawMessage `json:"parameters,omitempty"`
}

type RoutedChatExecutor added in v0.5.1

type RoutedChatExecutor interface {
	ExecuteChat(context.Context, *ChatRequest) (*ChatExecution, error)
	ExecuteStreamChat(context.Context, *ChatRequest) (*StreamExecution, error)
}

type ServedCandidate added in v0.5.1

type ServedCandidate struct {
	Dialect       ProtocolDialect
	ProviderType  string
	ProviderID    string
	LogicalModel  string
	ClientModel   string
	UpstreamModel string
	Features      map[ProtocolFeature]struct{}
}

func (ServedCandidate) Supports added in v0.5.1

func (c ServedCandidate) Supports(feature ProtocolFeature) bool

type StreamExecution added in v0.5.1

type StreamExecution struct {
	Stream   *schema.StreamReader[*schema.Message]
	Resolved ResolvedExecution
}

type UpstreamError

type UpstreamError struct {
	Status     int
	StatusText string
	Body       string
}

UpstreamError describes an HTTP error returned by an upstream provider API.

func (*UpstreamError) Error

func (e *UpstreamError) Error() string

func (*UpstreamError) StatusCode

func (e *UpstreamError) StatusCode() int

type Usage

type Usage struct {
	InputTokens     int
	OutputTokens    int
	TotalTokens     int
	CachedTokens    int
	ReasoningTokens int
}

Usage contains token consumption information. TotalTokens is the upstream-reported total when available and InputTokens+OutputTokens otherwise. CachedTokens counts the prompt tokens served from a provider-side cache (a subset of InputTokens); ReasoningTokens counts completion tokens spent on reasoning (a subset of OutputTokens). The detail fields are zero when the upstream does not report them.

func UsageFromMessage

func UsageFromMessage(msg *schema.Message) Usage

Directories

Path Synopsis
Package anthropic implements the Anthropic provider (Claude models).
Package anthropic implements the Anthropic provider (Claude models).
Package anthropicbase provides shared Anthropic Messages API wire helpers.
Package anthropicbase provides shared Anthropic Messages API wire helpers.
Package deepseek implements the DeepSeek provider.
Package deepseek implements the DeepSeek provider.
Package einomodel presents a gateway provider as an eino model.ToolCallingChatModel, so eino agents, ADK runners, and compose graphs can consume gateway-routed models directly.
Package einomodel presents a gateway provider as an eino model.ToolCallingChatModel, so eino agents, ADK runners, and compose graphs can consume gateway-routed models directly.
Package gemini implements the Google Gemini provider.
Package gemini implements the Google Gemini provider.
Package ollama implements the Ollama provider (local deployment, OpenAI-compatible).
Package ollama implements the Ollama provider (local deployment, OpenAI-compatible).
Package openai implements the OpenAI provider.
Package openai implements the OpenAI provider.
Package openaibase provides shared OpenAI-compatible wire types still used for model listing and embeddings.
Package openaibase provides shared OpenAI-compatible wire types still used for model listing and embeddings.
Package openrouter implements the OpenRouter provider (OpenAI-compatible API).
Package openrouter implements the OpenRouter provider (OpenAI-compatible API).
Package qwen implements the Alibaba Qwen provider (DashScope OpenAI-compatible mode) on top of the eino-ext qwen component.
Package qwen implements the Alibaba Qwen provider (DashScope OpenAI-compatible mode) on top of the eino-ext qwen component.
Package zhipu implements the Zhipu BigModel provider.
Package zhipu implements the Zhipu BigModel provider.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL