providers

package
v1.7.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 8, 2026 License: MIT Imports: 29 Imported by: 0

Documentation

Index

Constants

View Source
const (
	ProviderAvailable  = "available"
	ProviderPlanned    = "planned"
	ProviderClientOnly = "client_only"

	ConnectionScopeSystemOrPersonal = "system_or_personal"
	ConnectionScopePersonal         = "personal"
	ConnectionScopeGatewayClient    = "gateway_client"
)
View Source
const (
	AuthAPIKey            = "api_key"
	AuthGatewayClient     = "gateway_client"
	AuthNone              = "none"
	AuthOAuthBrowser      = "oauth_browser"
	AuthOAuthDevice       = "oauth_device"
	AuthTokenImport       = "token_import"
	AuthGCPServiceAccount = "gcp_service_account"
	AuthSetupToken        = "setup_token"
)

Auth methods a registry entry may advertise.

View Source
const (
	RiskGreen  = "green"
	RiskYellow = "yellow"
	RiskRed    = "red"
)

Risk levels. Every level but RiskGreen requires a risk notice.

View Source
const (
	// CatalogCodeHTTPError is a status other than success, 401 or 403.
	CatalogCodeHTTPError = "http_error"
	// CatalogCodeAuthenticationFailed is the upstream rejecting the
	// credential with 401 or 403.
	CatalogCodeAuthenticationFailed = "authentication_failed"
	// CatalogCodeNotDiscoverable is a catalog that cannot be listed, such
	// as one over the size limit.
	CatalogCodeNotDiscoverable = "not_discoverable"
	// CatalogCodeTransportError is a catalog request that got no complete
	// answer.
	CatalogCodeTransportError = "transport_error"
	// CatalogCodeInvalidJSON is an answer that is not JSON.
	CatalogCodeInvalidJSON = "invalid_json"
	// CatalogCodeInvalidShape is JSON that is not a catalog: not an object,
	// without its model array, or with a row that names no model.
	CatalogCodeInvalidShape = "invalid_shape"
)

Catalog error codes. A catalog failure is a *core.ProviderError whose cause is a *CatalogError with one of these codes: the gateway's catalog failure codes without their catalog_ prefix, so a product maps a failure without reading its message.

View Source
const BedrockDefaultRegion = "us-east-1"

BedrockDefaultRegion is the region NewBedrock serves when given none, as the gateway does.

View Source
const CatalogCodeConfigurationIncomplete = "configuration_incomplete"

CatalogCodeConfigurationIncomplete is an instance configured too little to list a catalog, such as Vertex AI without a project.

View Source
const OllamaDefaultBaseURL = "http://127.0.0.1:11434"

OllamaDefaultBaseURL is the daemon root Ollama talks to when its config names none, as the gateway does.

Variables

This section is empty.

Functions

func AnonymousVerificationModel added in v0.2.0

func AnonymousVerificationModel(registryID string, models []core.ModelInfo) string

AnonymousVerificationModel returns the probe model among discovered free models; see SelectVerificationModel.

func AzureChatField added in v1.5.0

func AzureChatField(field string) bool

AzureChatField reports a Chat field Azure OpenAI's v1 Chat Completions operation accepts, besides the model, messages and stream flag: the request properties of the published v1 API definition, and service_tier, which the service documents for Chat Completions ahead of that definition. A field outside the list, such as thinking or web_search_options, is not part of that operation and is never sent. A product can weigh a target with it before sending a request, as the gateway's router does.

func CanonicalRegistryID added in v0.2.0

func CanonicalRegistryID(id string) string

CanonicalRegistryID resolves a default registry alias while preserving unknown custom identifiers in normalized form.

func DiscoverAnonymousModels added in v0.2.0

func DiscoverAnonymousModels(ctx context.Context, profile AnonymousProviderProfile, client *http.Client) ([]core.ModelInfo, error)

DiscoverAnonymousModels queries an anonymous provider's catalog and returns only the models AdmitAnonymousModel admits.

func InvocationCircuitFailure added in v0.4.0

func InvocationCircuitFailure(err error) bool

InvocationCircuitFailure reports failures that should count against a target circuit.

func InvocationFailoverEligible added in v0.2.0

func InvocationFailoverEligible(err error) bool

InvocationFailoverEligible selects provider failures that an ordered endpoint may advance past.

func InvocationRetryable added in v0.2.0

func InvocationRetryable(err error) bool

InvocationRetryable returns true if an invocation error is safe to retry on the same target.

func IsThrottle added in v0.2.0

func IsThrottle(err error) bool

IsThrottle inspects an error message or status for rate limit signals.

func NewAnonymousOpenAICompatibleAdapter deprecated added in v0.3.0

func NewAnonymousOpenAICompatibleAdapter(baseURL string, client *http.Client) core.ProviderAdapter

NewAnonymousOpenAICompatibleAdapter adapts a reviewed, keyless OpenAI-compatible endpoint to the shared provider connector.

Deprecated: use an anonymous.Orchestrator, whose adapter reads the catalog and sends probes through the product's Catalog and Invoker, so each provider's vertical admits its models and shapes its requests. This adapter reads every catalog row unadmitted and posts to /chat/completions with its own client, which Pollinations and Zen do not serve as the gateway does.

func OllamaBaseURLIssue added in v0.14.0

func OllamaBaseURLIssue(base string) string

OllamaBaseURLIssue describes, in the gateway's words, what keeps base from being an Ollama daemon's native root, or returns "" when it is one: an http or https URL with a host and no path. The OpenAI-compatible /v1 URL gets its own advice.

func SelectVerificationModel added in v0.8.0

func SelectVerificationModel(registryID string, freeModelIDs []string) string

SelectVerificationModel returns the model to probe an anonymous provider with: the first reviewed default among the free model ids, otherwise the lexically first free id, otherwise "". Matching ignores case.

Types

type AnonymousAdmission added in v0.8.0

type AnonymousAdmission struct {
	// Free reports a free model that is usable without credentials.
	Free bool
	// ContextWindow is the declared context length, or 0.
	ContextWindow int
	// Reasoning and ToolCalls report capabilities the row declares.
	Reasoning bool
	ToolCalls bool
}

AnonymousAdmission is the verdict on one raw catalog row of an anonymous provider.

func AdmitAnonymousModel added in v0.8.0

func AdmitAnonymousModel(registryID string, row map[string]any) AnonymousAdmission

AdmitAnonymousModel applies the reviewed free-model rules to one raw row of an anonymous provider's catalog. Admission fails closed: an unknown provider is never admitted, and neither is OpenCode Zen, whose admission needs its verified metadata rather than a raw row (see package zen).

type AnonymousProviderProfile added in v0.2.0

type AnonymousProviderProfile struct {
	RegistryID         string   `json:"registry_id"`
	ProviderID         string   `json:"provider_id"`
	RuntimeType        string   `json:"runtime_type"`
	BaseURL            string   `json:"base_url"`
	VerificationModels []string `json:"verification_models"`
}

AnonymousProviderProfile describes one provider curated for anonymous automation. Whether a product enrolls it remains the product's policy.

func AnonymousProviderProfiles added in v0.2.0

func AnonymousProviderProfiles() []AnonymousProviderProfile

AnonymousProviderProfiles returns the default registry's anonymous profiles.

type Anthropic added in v0.14.0

type Anthropic struct {
	// contains filtered or unexported fields
}

Anthropic implements core.Provider for the Anthropic Messages API, as the gateway's native Anthropic provider serves it: Messages passed through with its model and stream flag set, token counts, and the catalog with the capabilities the registry declares for its rows.

Messages is the only native surface, and it is preserved: the request reaches Anthropic as the client wrote it and the answer comes back as Anthropic sent it, so Anthropic implements core.WirePreserver. Serving Chat Completions means translating, which is left to a translation.Adapter in front.

Each operation authenticates with the credential it is given: an API key is sent as x-api-key, and a credential of any other kind is refused before anything is sent. Without a credential nothing authenticates, for an Anthropic-compatible endpoint that needs none.

Of the headers the client sent, in core.Request.Header, every Messages request and token count forwards anthropic-version, in place of the default, and each anthropic-beta value, such as one that enables extended context or interleaved thinking. A value that is blank or holds a control character is dropped, and no other header is read, so a client's own Authorization or x-api-key never reaches Anthropic.

func NewAnthropic added in v0.14.0

func NewAnthropic(config AnthropicConfig) (*Anthropic, error)

NewAnthropic returns an Anthropic provider. A base URL that is not an absolute http(s) URL without a query or fragment is refused, since every path is appended to it.

func (*Anthropic) CountTokens added in v0.14.0

func (p *Anthropic) CountTokens(ctx context.Context, request core.TokenCountRequest) (core.TokenCount, error)

CountTokens implements core.TokenCounter for Messages, through /v1/messages/count_tokens as the gateway counts: the body with its model set to the request's, authenticated as its Messages request would be, without a preamble. The client's protocol headers are forwarded as for a Messages request, from the count's Header, or from the request's own when the count's is nil. Any other surface fails with core.ErrTokenCountUnsupported.

func (*Anthropic) Invoke added in v0.14.0

func (p *Anthropic) Invoke(ctx context.Context, request core.Request) (core.Response, error)

Invoke performs one Messages request. The answer is returned as Anthropic sent it once it checks out as the gateway checks it: a JSON object with a content array.

func (*Anthropic) ListModels added in v0.14.0

func (p *Anthropic) ListModels(ctx context.Context, credential *core.Credential) ([]core.ModelInfo, error)

ListModels lists the models /v1/models returns for the credential, as the gateway lists them: the one page the endpoint answers, every row a nonblank string id. Each model serves Messages, with the display name Anthropic gives it and the capabilities the registry declares for Anthropic, fresh for an hour. A catalog failure's *CatalogError has the gateway's code.

func (*Anthropic) NativeSurfaces added in v0.14.0

func (p *Anthropic) NativeSurfaces(string) []core.ModelSurface

NativeSurfaces reports Messages for every model.

func (*Anthropic) PreservesWire added in v0.14.0

func (p *Anthropic) PreservesWire(_ string, surface core.ModelSurface) bool

PreservesWire implements core.WirePreserver: Messages is forwarded as written, as the gateway declares it.

func (*Anthropic) Stream added in v0.14.0

func (p *Anthropic) Stream(ctx context.Context, request core.Request) (core.StreamIter, error)

Stream performs one Messages request as a stream and relays Anthropic's records byte for byte; see anthropicStream for how it ends.

type AnthropicConfig added in v0.14.0

type AnthropicConfig struct {
	// BaseURL is where the API lives: requests go to <BaseURL>/v1/messages.
	// It may carry a path, such as a proxy's mount. Empty uses
	// https://api.anthropic.com.
	BaseURL string
	// Client performs Messages requests and token counts. Nil uses a
	// client that times out after 60 seconds, the gateway's default.
	Client *http.Client
	// CatalogClient lists models. Nil uses a client that times out after
	// 10 seconds, the gateway's bound on catalog requests.
	CatalogClient *http.Client
	// Headers are sent with every request, the catalog's included. None
	// replaces a header Anthropic sets itself, such as Content-Type or the
	// default anthropic-version, and a request with an API key sends no
	// configured x-api-key or Authorization. A configured anthropic-beta is
	// sent beside the client's. A name that is not an HTTP token, or a
	// value with a control character, is refused.
	Headers map[string]string
	// Preamble returns the text to put before the system prompt of a
	// Messages request, such as a gateway preamble, or "" for none. It runs
	// once per Invoke or Stream with that operation's context, so a product
	// can return "" for a request that must reach Anthropic as sent. When
	// it returns "", a preamble the body carries under _llmgw_preamble, as
	// the gateway sets it, is used instead. Nil adds none.
	Preamble func(ctx context.Context) string
	// Now stamps catalog discovery and reads Retry-After. Nil uses time.Now.
	Now func() time.Time
}

AnthropicConfig configures Anthropic. Every field is optional.

type AnthropicProvider

type AnthropicProvider struct {
	ID         string
	BaseURL    string
	DefaultKey string
	Client     *http.Client
	Version    string
}

AnthropicProvider is a client for the Anthropic Messages API.

func NewAnthropicProvider

func NewAnthropicProvider(id, baseURL, defaultKey string, client *http.Client) *AnthropicProvider

func (*AnthropicProvider) Complete

func (p *AnthropicProvider) Complete(ctx context.Context, model string, payload map[string]any, cred *core.Credential) (map[string]any, error)

func (*AnthropicProvider) ListModels

func (p *AnthropicProvider) ListModels(ctx context.Context, cred *core.Credential) ([]core.ModelInfo, error)

func (*AnthropicProvider) Stream

func (p *AnthropicProvider) Stream(ctx context.Context, model string, payload map[string]any, cred *core.Credential) (StreamIter, error)

type AutoConnectResult deprecated added in v0.2.0

type AutoConnectResult struct {
	RegistryID string   `json:"registry_id"`
	ProviderID string   `json:"provider_id"`
	Status     string   `json:"status"` // "verified", "connected", "failed"
	Models     []string `json:"models,omitempty"`
	ProbeModel string   `json:"probe_model,omitempty"`
	LatencyMs  int64    `json:"latency_ms,omitempty"`
	Error      string   `json:"error,omitempty"`
}

AutoConnectResult represents the outcome of self-adjudicating an anonymous provider.

Deprecated: use anonymous.Result, the outcome of an anonymous.Orchestrator check.

func AutoConnectAnonymousProviders deprecated added in v0.2.0

func AutoConnectAnonymousProviders(ctx context.Context, httpClient *http.Client) []AutoConnectResult

AutoConnectAnonymousProviders discovers models for all reviewed anonymous providers, self-adjudicates each with a minimal completion probe, and returns the live results.

Deprecated: use an anonymous.Orchestrator. This probes one model per provider through the legacy OpenAIProvider with its own HTTP client, so it sends none of the verticals' requests, such as Zen's anonymous identity or Pollinations' path, and loses a probe's status. The orchestrator reads catalogs and probes every model through the product's Catalog and Invoker, and keeps enrollment, claims and evidence in product hooks.

type AzureOpenAI added in v0.14.0

type AzureOpenAI struct {
	// contains filtered or unexported fields
}

AzureOpenAI implements core.Provider for one Azure OpenAI resource, as the gateway's azure_openai provider serves it. It diverges from other OpenAI-compatible providers where a live resource does:

  • The credential is an API key, sent as the api-key header, never as a bearer. A request without one is refused before anything is sent.
  • Chat Completions is the only native surface, sent to the resource's /openai/v1 route without an api-version. The model is the deployment name, and the answer comes back as Azure sent it, its version-stamped model and Azure-only fields included. A translation.Adapter in front serves Messages and Responses over Chat.
  • The catalog is the resource's own deployments; see ListModels.

A Chat body is rebuilt from the client's: the model, the messages, the stream flag and the fields Azure's v1 Chat Completions accepts, any other field dropped and reported as an advisory loss. It is not the body the client sent, so Azure OpenAI does not implement core.WirePreserver, and a product labels its answers translated, as the gateway labels Azure Chat.

Redirects that leave the resource's origin are not followed: Go would send the api-key header with them.

func NewAzureOpenAI added in v0.14.0

func NewAzureOpenAI(config AzureOpenAIConfig) (*AzureOpenAI, error)

NewAzureOpenAI returns an Azure OpenAI provider. A base URL that is not a resource endpoint is refused, as the gateway refuses it at configuration; see azureInferenceBaseURL.

func (*AzureOpenAI) Invoke added in v0.14.0

func (p *AzureOpenAI) Invoke(ctx context.Context, request core.Request) (core.Response, error)

Invoke performs one Chat Completions request. The answer is returned as Azure sent it once it checks out as the gateway checks it: a JSON object whose first choice is an object.

func (*AzureOpenAI) ListModels added in v0.14.0

func (p *AzureOpenAI) ListModels(ctx context.Context, credential *core.Credential) ([]core.ModelInfo, error)

ListModels lists the resource's own deployments, the only models it can serve, as the gateway lists them. GET /models is Azure's whole vendor catalogue, hundreds of rows against a resource's few deployments with no field that tells them apart, so it is never requested, not even when the deployments listing fails.

The route has not been seen to page, so the walk is defensive: a page with has_more continues after its last_id, or its last row's id, carried as a query value on a URL built from the configured origin and checked to stay there, so no answer can send the api-key elsewhere. A nextLink is ignored. A page with has_more but no new cursor, or a walk past 50 pages, fails rather than returning part of the catalog.

A deployment is listed only when it succeeded and its base model is chat-callable (see azureDeploymentIsChatCallable). Its id is the deployment name a request sends as its model, its display name the base model when that differs, and it declares chat over Chat Completions and Messages, with the capabilities the gateway infers from that.

A key the resource rejects fails with CatalogCodeAuthenticationFailed and its status, and any other failure with CatalogCodeNotDiscoverable, the gateway's codes, each classified by what failed.

func (*AzureOpenAI) NativeSurfaces added in v0.14.0

func (p *AzureOpenAI) NativeSurfaces(string) []core.ModelSurface

NativeSurfaces reports Chat Completions for every deployment.

func (*AzureOpenAI) Stream added in v0.14.0

func (p *AzureOpenAI) Stream(ctx context.Context, request core.Request) (core.StreamIter, error)

Stream performs one Chat Completions request as a stream and relays Azure's records byte for byte, [DONE] included. The stream reports the request's losses through core.LossReporter.

type AzureOpenAIConfig added in v0.14.0

type AzureOpenAIConfig struct {
	// BaseURL is the resource's endpoint: the origin the Azure portal
	// shows, optionally followed by /openai or /openai/v1. Chat Completions
	// are sent to <origin>/openai/v1/chat/completions and the deployments
	// are listed from the origin.
	BaseURL string
	// Client performs Chat requests. Nil uses a client that times out after
	// 300 seconds, the gateway's default for OpenAI-compatible providers.
	Client *http.Client
	// CatalogClient lists deployments. Nil uses a client that times out
	// after 10 seconds, the gateway's bound on catalog requests.
	CatalogClient *http.Client
	// Headers are sent with every request, the catalog's included. None
	// replaces a header Azure OpenAI sets itself, such as Content-Type or
	// api-key, and no configured Authorization is sent beside the key. A
	// name that is not an HTTP token, or a value with a control character,
	// is refused.
	Headers map[string]string
	// Now stamps catalog discovery and reads Retry-After. Nil uses time.Now.
	Now func() time.Time
}

AzureOpenAIConfig configures AzureOpenAI. BaseURL is required.

type ByteStreamIter

type ByteStreamIter struct {
	// contains filtered or unexported fields
}

ByteStreamIter frames an HTTP response body into complete, bounded SSE records.

func NewByteStreamIter

func NewByteStreamIter(r io.ReadCloser) *ByteStreamIter

func (*ByteStreamIter) Close

func (s *ByteStreamIter) Close() error

func (*ByteStreamIter) Next

func (s *ByteStreamIter) Next() ([]byte, error)

type CatalogError added in v0.2.0

type CatalogError struct {
	Code       string
	Detail     string
	Status     int
	RetryAfter time.Duration
	Cause      error
}

CatalogError represents a failure discovering or reading a provider catalog.

func (*CatalogError) Error added in v0.2.0

func (e *CatalogError) Error() string

func (*CatalogError) Unwrap added in v0.4.0

func (e *CatalogError) Unwrap() error

type ElevenLabs added in v1.2.0

type ElevenLabs struct {
	// contains filtered or unexported fields
}

ElevenLabs serves the OpenAI audio surfaces over ElevenLabs' native speech APIs. It has no chat surface.

func NewElevenLabs added in v1.2.0

func NewElevenLabs(config ElevenLabsConfig) (*ElevenLabs, error)

NewElevenLabs returns an ElevenLabs provider with explicit audio capability lists. Capability is never inferred from an arbitrary model name.

func (*ElevenLabs) Invoke added in v1.2.0

func (p *ElevenLabs) Invoke(ctx context.Context, request core.Request) (core.Response, error)

Invoke serves speech-to-text and text-to-speech and normalizes successful transcription answers to the OpenAI {"text": ...} response shape.

func (*ElevenLabs) ListModels added in v1.2.0

func (p *ElevenLabs) ListModels(context.Context, *core.Credential) ([]core.ModelInfo, error)

ListModels returns the reviewed model roster with explicit audio surfaces.

func (*ElevenLabs) NativeSurfaces added in v1.2.0

func (p *ElevenLabs) NativeSurfaces(model string) []core.ModelSurface

NativeSurfaces reports only the explicitly configured model capabilities.

func (*ElevenLabs) Stream added in v1.2.0

func (p *ElevenLabs) Stream(_ context.Context, request core.Request) (core.StreamIter, error)

type ElevenLabsConfig added in v1.2.0

type ElevenLabsConfig struct {
	BaseURL             string
	Client              *http.Client
	TranscriptionModels []string
	SpeechModels        []string
	// Headers are sent with every request. None replaces a header
	// ElevenLabs sets itself, such as Content-Type or xi-api-key, and no
	// configured Authorization is sent beside the key. A name that is not
	// an HTTP token, or a value with a control character, is refused.
	Headers map[string]string
}

ElevenLabsConfig configures the ElevenLabs speech provider. Nil model lists select the reviewed defaults; an explicit empty list disables that surface.

type EntryOverride added in v0.7.0

type EntryOverride struct {
	ID               string                     `json:"id"`
	Label            *string                    `json:"label,omitempty"`
	Description      *string                    `json:"description,omitempty"`
	Categories       []string                   `json:"categories,omitempty"`
	Icon             *string                    `json:"icon,omitempty"`
	Domain           *string                    `json:"domain,omitempty"`
	DocsURL          *string                    `json:"docs_url,omitempty"`
	Priority         *float64                   `json:"priority,omitempty"`
	Availability     *string                    `json:"availability,omitempty"`
	Aliases          []string                   `json:"aliases,omitempty"`
	CommonModels     []string                   `json:"common_models,omitempty"`
	OnboardingFields []string                   `json:"onboarding_fields,omitempty"`
	Extensions       map[string]json.RawMessage `json:"extensions,omitempty"`
}

EntryOverride changes presentation, priority and curation fields of one entry. A nil pointer or nil slice leaves the field unchanged, and an empty slice clears it. Aliases are added to the entry's own, so the aliases core resolves keep working; Extensions are merged by key.

type Google added in v0.14.0

type Google struct {
	// contains filtered or unexported fields
}

Google implements core.Provider for Google's Gemini models on AI Studio or Vertex AI, speaking their native generateContent grammar as the gateway does. It is the whole vertical: the Chat and embeddings transport, image generation, the catalog and what its rows mean, and the credential each deployment takes.

Chat Completions and embeddings are its surfaces. Google converts a Chat request to Gemini contents itself: the text of the messages, max_tokens and temperature, with the gateway's mapping, and function tools, tool_choice, an assistant's tool calls and tool results, as googleChatContents describes. Every other field is dropped and reported in Response.Losses, a material loss when it changes the answer. A Chat stream is Gemini's streamGenerateContent as Chat chunks, as googleStream renders them; embeddings do not stream. A translation.Adapter in front serves Messages and Responses over Chat, and Messages streams too. GenerateImages implements core.ImageGenerator. Video generation is not served yet.

Each operation authenticates with the credential it is given: an API key travels in x-goog-api-key and never in a URL, and a token is the bearer. On Vertex AI a credential of kind core.TokenTypeGCPServiceAccount is a service-account key, exchanged for cloud-platform access tokens that the instance caches until shortly before they expire. Vertex AI refuses to run without a credential and nothing is sent; AI Studio sends what it has, as the gateway does, for a proxy that holds the key.

func NewGoogle added in v0.14.0

func NewGoogle(config GoogleConfig) (*Google, error)

NewGoogle returns a Google provider for one deployment.

func (*Google) GenerateImages added in v0.14.0

func (p *Google) GenerateImages(ctx context.Context, request core.GenerateImagesRequest, credential *core.Credential) (core.GenerateImagesResult, error)

GenerateImages asks an image-capable Gemini model for inline images over generateContent, as the gateway does, and returns at most Count of them with Google's usage, whose modality-tagged token counts carry the image cost. A model that answers without image data, such as a text-only one, fails and permits failover.

func (*Google) Invoke added in v0.14.0

func (p *Google) Invoke(ctx context.Context, request core.Request) (core.Response, error)

Invoke performs one Chat Completions or embeddings request.

func (*Google) ListModels added in v0.14.0

func (p *Google) ListModels(ctx context.Context, credential *core.Credential) ([]core.ModelInfo, error)

ListModels returns the models the deployment lists for the credential, with the capabilities the gateway derives for them.

AI Studio's catalog is paged, and a model's supportedGenerationMethods say what it serves. Vertex AI lists publisher models only on the v1beta1 collection route, scoped to the instance's location and never merged across locations, and only to an OAuth principal: with an API key alone its catalog is not discoverable. Its rows carry no capability field, so a model's family, read from its ID, says what it serves, and a family the classifier cannot place is left out rather than listed as a chat model. Only models an operator can call directly are listed. What the catalog omits stays unknown.

func (*Google) NativeSurfaces added in v0.14.0

func (p *Google) NativeSurfaces(string) []core.ModelSurface

NativeSurfaces reports Chat Completions and embeddings for every model.

func (*Google) Stream added in v0.14.0

func (p *Google) Stream(ctx context.Context, request core.Request) (core.StreamIter, error)

Stream performs one Chat Completions request as a stream; see streamChat. Embeddings do not stream, so a stream of them is refused before anything is sent, permitting failover.

type GoogleConfig added in v0.14.0

type GoogleConfig struct {
	Deployment GoogleDeployment
	// BaseURL is the API root. Empty uses Google's: AI Studio's v1beta root,
	// or the v1 root of Location's Vertex AI endpoint. A Vertex AI BaseURL is
	// an inference root that ends in /v1, as the default does; model
	// discovery replaces that version with v1beta1.
	BaseURL string
	// Project is the Vertex AI project calls are billed to. Empty takes the
	// credential's, and a credential that names another project is refused.
	Project string
	// Location is the Vertex AI location. Empty is "global".
	Location string
	// RequestType is how Vertex AI accounts an invocation: empty or
	// "default" leaves it to Vertex AI, "paygo" asks for shared pay-as-you-go
	// capacity, and "dedicated" for Provisioned Throughput.
	RequestType string
	// Client performs every request but a service account's token
	// exchange. Nil uses a client that times out after 120 seconds.
	Client *http.Client
	// Headers are sent with every request but a service account's token
	// exchange, the catalog's included; x-goog-user-project is one Google
	// reads. None replaces a header Google sets itself, such as
	// Content-Type, and an authenticated request sends neither a configured
	// Authorization nor a configured x-goog-api-key, since Vertex AI refuses
	// a request that presents two credentials. A name that is not an HTTP
	// token, or a value with a control character, is refused.
	Headers map[string]string
	// Now reads Retry-After dates, stamps catalog discovery and dates the
	// assertions of service accounts. Nil uses time.Now.
	Now func() time.Time
}

GoogleConfig configures Google. Deployment is required; every other field is optional, and Project, Location and RequestType apply to Vertex AI only.

type GoogleDeployment added in v0.14.0

type GoogleDeployment string

GoogleDeployment selects the Google API a Google provider calls.

const (
	// GoogleAIStudio is the Gemini API of Google AI Studio.
	GoogleAIStudio GoogleDeployment = "ai_studio"
	// GoogleVertexAI is Vertex AI in a Google Cloud project.
	GoogleVertexAI GoogleDeployment = "vertex_ai"
)

type InvocationError added in v0.2.0

type InvocationError struct {
	Msg              string
	Status           int
	Retryable        bool
	FailoverEligible bool
	CircuitFailure   bool
	RetryAfter       time.Duration
	Cause            error
	// contains filtered or unexported fields
}

InvocationError represents an upstream provider call failure.

func (*InvocationError) Error added in v0.2.0

func (e *InvocationError) Error() string

func (*InvocationError) ProviderErrorClassification added in v0.4.0

func (e *InvocationError) ProviderErrorClassification() core.ProviderErrorClassification

func (*InvocationError) Unwrap added in v0.4.0

func (e *InvocationError) Unwrap() error

type MiMo added in v1.2.0

type MiMo struct {
	// contains filtered or unexported fields
}

MiMo serves OpenAI audio_speech by translating it to MiMo's documented Chat Completions audio request and decoding the returned base64 audio.

func NewMiMo added in v1.2.0

func NewMiMo(config MiMoConfig) (*MiMo, error)

func (*MiMo) Invoke added in v1.2.0

func (p *MiMo) Invoke(ctx context.Context, request core.Request) (core.Response, error)

func (*MiMo) ListModels added in v1.2.0

func (p *MiMo) ListModels(context.Context, *core.Credential) ([]core.ModelInfo, error)

func (*MiMo) NativeSurfaces added in v1.2.0

func (p *MiMo) NativeSurfaces(model string) []core.ModelSurface

func (*MiMo) Stream added in v1.2.0

func (p *MiMo) Stream(_ context.Context, request core.Request) (core.StreamIter, error)

type MiMoConfig added in v1.2.0

type MiMoConfig struct {
	BaseURL      string
	Client       *http.Client
	SpeechModels []string
	// Headers are sent with every request. None replaces a header MiMo
	// sets itself, such as Content-Type or Api-Key, and no configured
	// Authorization is sent beside the key. A name that is not an HTTP
	// token, or a value with a control character, is refused.
	Headers map[string]string
}

MiMoConfig configures Xiaomi MiMo's chat-completions speech bridge. A nil SpeechModels list selects the reviewed roster; an empty list disables it.

type Ollama added in v0.14.0

type Ollama struct {
	// contains filtered or unexported fields
}

Ollama implements core.Provider for an Ollama daemon over its native /api/chat, as the gateway's OllamaProvider does. Ollama is keyless, so every operation ignores its credential.

Chat Completions is its only surface, and Ollama serves it natively by converting each request to /api/chat and each answer back, byte for byte as the gateway does. A request sends its messages, temperature, top_p, tools, and max_tokens as num_predict; every other field is dropped and reported in Response.Losses. Content that is not a string reaches Ollama printed as Go prints it, as the gateway sends it, so an image never reaches it as an image; that is reported as a material loss at the content's path. A stream is Ollama's NDJSON as Chat chunks, ending in [DONE]. A translation.Adapter in front serves Messages and Responses over Chat.

func NewOllama added in v0.14.0

func NewOllama(config OllamaConfig) (*Ollama, error)

NewOllama returns an Ollama provider. A base URL that OllamaBaseURLIssue finds an issue with is a configuration error carrying that issue.

func (*Ollama) Invoke added in v0.14.0

func (p *Ollama) Invoke(ctx context.Context, request core.Request) (core.Response, error)

Invoke performs one Chat Completions request.

func (*Ollama) ListModels added in v0.14.0

func (p *Ollama) ListModels(ctx context.Context, _ *core.Credential) ([]core.ModelInfo, error)

ListModels returns the models the daemon holds, from /api/tags, as the gateway lists them. Every row is checked before any is used. A row is named by its name, or by its model when the name is blank, and made by its details.family, or by "ollama". Ollama reports no surfaces or capabilities for a row.

func (*Ollama) NativeSurfaces added in v0.14.0

func (p *Ollama) NativeSurfaces(string) []core.ModelSurface

NativeSurfaces reports Chat Completions for every model.

func (*Ollama) Stream added in v0.14.0

func (p *Ollama) Stream(ctx context.Context, request core.Request) (core.StreamIter, error)

Stream performs one Chat Completions request whose frames are Ollama's NDJSON events as Chat chunks, ending in [DONE]. The stream reports the request's losses through core.LossReporter.

type OllamaConfig added in v0.14.0

type OllamaConfig struct {
	// BaseURL is the daemon's native root, such as OllamaDefaultBaseURL,
	// the default. OllamaBaseURLIssue says what it may not be.
	BaseURL string
	// Client performs inference. Nil uses a client that times out after 30
	// seconds, the gateway's default for Ollama.
	Client *http.Client
	// CatalogClient lists models. Nil uses a client that times out after 10
	// seconds, the gateway's bound on catalog requests.
	CatalogClient *http.Client
	// Headers are sent with every request, the catalog's included, such as
	// the Authorization of a proxy in front of the daemon. None replaces
	// the Content-Type Ollama sets. A name that is not an HTTP token, or a
	// value with a control character, is refused.
	Headers map[string]string
	// Now reads a Retry-After date. Nil uses time.Now.
	Now func() time.Time
}

OllamaConfig configures Ollama. Every field is optional.

type OpenAICompatible added in v0.14.0

type OpenAICompatible struct {
	// contains filtered or unexported fields
}

OpenAICompatible implements core.Provider for an upstream that speaks the OpenAI wire: the gateway's openai_compatible, openai and litellm instances, the anonymous catalogs the registry curates, and, through NewBedrock, Amazon Bedrock. It is the gateway's OpenAI transport without its OpenCode Zen and GitHub Copilot branches, which Zen and Copilot own.

Chat Completions is native for every model, and Responses for a model whose catalog row lists it or that the openai registry entry serves. A translation.Adapter in front serves Messages over Chat, and Responses over Chat for the other models. The OpenAI audio surfaces are native for every model too, except on Bedrock: audio_transcriptions forwards the multipart upload unchanged to /audio/transcriptions, and audio_speech the speech JSON, with the request's model, to /audio/speech. Their answers are returned as sent, with the upstream's content type, and neither streams.

A credential's API key, or else its token, is the bearer, without any "Bearer " prefix. No key, "free" or "none" sends no Authorization, and to an anonymous registry entry it is anonymous access, as "public" is. A credential's headers are sent too.

Errors are *core.ProviderError. A refusal is classified by the gateway's status set with the upstream's Retry-After, and its message never quotes the upstream; a catalog failure's *CatalogError has a CatalogCode* code.

func NewBedrock added in v0.14.0

func NewBedrock(region, baseURL string, config OpenAICompatibleConfig) (*OpenAICompatible, error)

NewBedrock returns an OpenAICompatible provider for Amazon Bedrock's OpenAI-compatible endpoint, which replaces config.BaseURL. baseURL, when set, is the endpoint, such as a bedrock-mantle or VPC endpoint; otherwise it is https://bedrock-runtime.<region>.amazonaws.com/v1, in BedrockDefaultRegion when region is blank.

A Bedrock API key is a bearer token, so a credential authenticates as it does for any OpenAI-compatible upstream: nothing is signed and no AWS SDK is needed. A region that is not a region name is refused before any request, so a key never reaches a host the region did not name.

Bedrock's OpenAI-compatible endpoint has no audio API, so the provider serves neither audio surface: a request for one fails with a *core.SurfaceError before anything is sent.

func NewOpenAICompatible added in v0.14.0

func NewOpenAICompatible(config OpenAICompatibleConfig) (*OpenAICompatible, error)

NewOpenAICompatible returns an OpenAICompatible provider.

func (*OpenAICompatible) Invoke added in v0.14.0

func (p *OpenAICompatible) Invoke(ctx context.Context, request core.Request) (core.Response, error)

Invoke performs one Chat Completions, Responses or audio request and returns the upstream's answer as sent, or, for Chat that adaptation serves over Responses, the answer converted to Chat.

func (*OpenAICompatible) ListModels added in v0.14.0

func (p *OpenAICompatible) ListModels(ctx context.Context, credential *core.Credential) ([]core.ModelInfo, error)

ListModels returns the catalog the credential can use, read as the gateway reads an OpenAI-compatible catalog: every row of /models, with its vendor, display name, endpoints and the capabilities it reports. Without a key, an anonymous registry entry lists only the models anonymous access admits; see openAIAnonymousEntry. Each row's typed capabilities are what core.InferCapabilities derives from it, as the gateway derives them when it stores the row.

func (*OpenAICompatible) NativeSurfaces added in v0.14.0

func (p *OpenAICompatible) NativeSurfaces(model string) []core.ModelSurface

NativeSurfaces reports Chat Completions for every model, Responses for a model the openai registry entry serves or whose row lists it, and the OpenAI audio surfaces, audio_transcriptions and audio_speech, for every model but Bedrock's. Whether the upstream serves a model on the audio endpoints is the upstream's to say, as it is for Chat.

func (*OpenAICompatible) PreservesWire added in v0.14.0

func (p *OpenAICompatible) PreservesWire(model string, surface core.ModelSurface) bool

PreservesWire implements core.WirePreserver. Responses is forwarded as sent, and so is Chat on the Chat endpoint: the model and stream flag are set, and fields the transport does not forward are dropped, but nothing is converted. Chat that the configured adaptation serves over Responses is converted, so it is not preserved; neither is a request whose own force_api_support turns adaptation on, which this declaration cannot see. The audio surfaces are forwarded as sent too, speech with the request's model, and their answers returned as sent.

func (*OpenAICompatible) Stream added in v0.14.0

func (p *OpenAICompatible) Stream(ctx context.Context, request core.Request) (core.StreamIter, error)

Stream performs one streaming Chat Completions or Responses request. Its frames are the upstream's SSE records as sent, [DONE] included; Chat that adaptation serves over Responses streams the complete answer as Chat chunks ending in [DONE], as the gateway does. The stream reports the request's losses through core.LossReporter. An audio surface does not stream: see audioStreamUnsupported.

type OpenAICompatibleConfig added in v0.14.0

type OpenAICompatibleConfig struct {
	// BaseURL is the API base the operation paths follow, such as
	// https://api.openai.com/v1: /chat/completions, /responses and /models.
	BaseURL string
	// RegistryID is the registry entry the instance serves, as a Registry
	// names it. It selects the wire facts the gateway keys on an entry:
	// "openai" serves Responses for every model, and kilo_code, llm7,
	// ovh_ai_endpoints and pollinations list, without a key, only the
	// models anonymous access admits. Pollinations also takes Chat at
	// /v1/chat/completions and lists its catalog as a bare array.
	RegistryID string
	// Models returns the row a product's catalog holds for a model, as
	// ListModels returned it. A model whose row lists a Responses endpoint
	// serves Responses natively, and adaptation reads the row too. A model
	// the catalog lacks, or a nil Models, serves Chat Completions only,
	// unless RegistryID is "openai".
	Models func(model string) (core.ModelInfo, bool)
	// Headers are sent with every request, before a credential's own
	// headers. Neither replaces a header the transport sets, Authorization
	// included when the request has a key. A name that is not an HTTP
	// token, or a value with a control character, is refused, since Go's
	// transport would refuse to send it.
	Headers map[string]string
	// ForwardAllFields sends every Chat field a request sets to the Chat
	// endpoint. By default only the fields the gateway's transport forwards
	// are sent, and each other field is dropped and reported as an advisory
	// loss. Chat served over Responses carries only the fields the
	// gateway's Chat facade forwards, either way.
	ForwardAllFields bool
	// ForceAPISupport turns the gateway's API adaptation on for every Chat
	// request, as its provider-level force_api_support does: a model whose
	// row lists Responses but not Chat Completions is served over
	// Responses, and a model whose row lists reasoning efforts takes
	// max_tokens as max_completion_tokens. A request's boolean
	// force_api_support turns it on or off for that request.
	ForceAPISupport bool
	// Client performs inference. Nil uses a client that times out after
	// 300 seconds, the gateway's default.
	Client *http.Client
	// CatalogClient lists models. Nil uses a client that times out after
	// 10 seconds, the gateway's bound on catalog requests.
	CatalogClient *http.Client
	// Now stamps catalog discovery and reads Retry-After dates. Nil uses
	// time.Now.
	Now func() time.Time
}

OpenAICompatibleConfig configures OpenAICompatible. BaseURL is required.

type OpenAIProvider

type OpenAIProvider struct {
	ID         string
	BaseURL    string
	DefaultKey string
	Client     *http.Client
	Headers    map[string]string
}

OpenAIProvider is a universal client for OpenAI and OpenAI-compatible APIs.

func NewOpenAIProvider

func NewOpenAIProvider(id, baseURL, defaultKey string, client *http.Client) *OpenAIProvider

func (*OpenAIProvider) Complete

func (p *OpenAIProvider) Complete(ctx context.Context, model string, payload map[string]any, cred *core.Credential) (map[string]any, error)

func (*OpenAIProvider) ListModels

func (p *OpenAIProvider) ListModels(ctx context.Context, cred *core.Credential) ([]core.ModelInfo, error)

func (*OpenAIProvider) Stream

func (p *OpenAIProvider) Stream(ctx context.Context, model string, payload map[string]any, cred *core.Credential) (StreamIter, error)

type Overlay added in v0.7.0

type Overlay struct {
	// Add appends product-only entries. Their ids must not already exist in
	// the base registry, even when removed.
	Add []RegistryEntry `json:"add,omitempty"`
	// Override changes fields of existing entries.
	Override []EntryOverride `json:"override,omitempty"`
	// Remove drops base entries the product does not offer.
	Remove []string `json:"remove,omitempty"`
}

Overlay layers one product's curation onto a registry. Core owns each entry's wire and safety facts, so an overlay changes only presentation, priority and curation fields; a product that needs different wire behavior adds its own entry under its own id. The result is validated like a manifest.

func DecodeOverlay added in v0.7.0

func DecodeOverlay(payload []byte) (Overlay, error)

DecodeOverlay strictly decodes an overlay. An override naming a field the overlay may not change is an unknown field, and so an error.

type Provider

type Provider interface {
	// Complete executes a non-streaming chat completion.
	Complete(ctx context.Context, model string, payload map[string]any, cred *core.Credential) (map[string]any, error)

	// Stream starts a streaming chat completion, yielding complete SSE frames.
	Stream(ctx context.Context, model string, payload map[string]any, cred *core.Credential) (StreamIter, error)

	// ListModels queries the provider's active catalog.
	ListModels(ctx context.Context, cred *core.Credential) ([]core.ModelInfo, error)
}

Provider represents a backend LLM client.

type Registry added in v0.7.0

type Registry struct {
	// contains filtered or unexported fields
}

Registry is an immutable, validated set of provider integrations. It is safe for concurrent use, and every accessor returns copies.

func DefaultRegistry added in v0.7.0

func DefaultRegistry() *Registry

DefaultRegistry returns the reviewed default registry. Products layer their own curation on it with WithOverlay.

func NewRegistry added in v0.7.0

func NewRegistry(entries []RegistryEntry, options ValidationOptions) (*Registry, error)

NewRegistry validates and normalizes entries into a Registry. The input is not modified.

func (*Registry) AnonymousProfiles added in v0.7.0

func (r *Registry) AnonymousProfiles() []AnonymousProviderProfile

AnonymousProfiles returns a profile for every entry curated for anonymous automation, sorted by registry id.

func (*Registry) ByID added in v0.7.0

func (r *Registry) ByID(id string) (RegistryEntry, bool)

ByID resolves only canonical ids, never aliases.

func (*Registry) CanonicalID added in v0.7.0

func (r *Registry) CanonicalID(name string) string

CanonicalID resolves an alias to its entry id and returns unknown identifiers in normalized form.

func (*Registry) Entries added in v0.7.0

func (r *Registry) Entries() []RegistryEntry

Entries returns copies of every entry, highest priority first and otherwise in manifest order.

func (*Registry) Len added in v0.7.0

func (r *Registry) Len() int

Len reports the number of entries.

func (*Registry) Lookup added in v0.7.0

func (r *Registry) Lookup(name string) (RegistryEntry, bool)

Lookup resolves an entry by id or alias, ignoring case and surrounding space.

func (*Registry) WithOverlay added in v0.7.0

func (r *Registry) WithOverlay(overlay Overlay, options ValidationOptions) (*Registry, error)

WithOverlay returns a new Registry with the overlay applied and validated. The receiver is unchanged.

type RegistryEntry added in v0.2.0

type RegistryEntry struct {
	ID                     string   `json:"id"`
	Aliases                []string `json:"aliases,omitempty"`
	Label                  string   `json:"label"`
	Description            string   `json:"description"`
	Categories             []string `json:"categories,omitempty"`
	RuntimeType            string   `json:"runtime_type"`
	Protocol               string   `json:"protocol"`
	Availability           string   `json:"availability"`
	AuthMethods            []string `json:"auth_methods"`
	ConnectionScope        string   `json:"connection_scope"`
	DefaultProviderID      string   `json:"default_provider_id"`
	DefaultBaseURL         string   `json:"default_base_url,omitempty"`
	DefaultRegion          string   `json:"default_region,omitempty"`
	DefaultLocation        string   `json:"default_location,omitempty"`
	RequiresBaseURL        bool     `json:"requires_base_url,omitempty"`
	RequiresAPIKey         bool     `json:"requires_api_key,omitempty"`
	SupportsAPIAdaptation  bool     `json:"supports_api_adaptation,omitempty"`
	SupportsModelDiscovery bool     `json:"supports_model_discovery,omitempty"`
	AnonymousAutomation    bool     `json:"anonymous_automation,omitempty"`
	InferAudioCapabilities bool     `json:"infer_audio_capabilities,omitempty"`
	OnboardingFields       []string `json:"onboarding_fields,omitempty"`
	AuthAdapter            string   `json:"auth_adapter,omitempty"`
	QuotaAdapter           string   `json:"quota_adapter,omitempty"`
	RiskLevel              string   `json:"risk_level,omitempty"`
	RiskNotice             string   `json:"risk_notice,omitempty"`
	DocsURL                string   `json:"docs_url,omitempty"`
	ClientOnly             bool     `json:"client_only,omitempty"`

	// Priority orders entries: higher first, ties in manifest order.
	Priority float64 `json:"priority,omitempty"`
	// Icon is a presentation slug, such as a brand icon name.
	Icon string `json:"icon,omitempty"`
	// Domain is the provider's public host name, for presentation.
	Domain string `json:"domain,omitempty"`
	// CommonModels suggests model ids to offer first.
	CommonModels []string `json:"common_models,omitempty"`
	// Extensions carries product-private metadata keyed by product name.
	// Core validates only that each value is JSON.
	Extensions map[string]json.RawMessage `json:"extensions,omitempty"`
}

RegistryEntry describes a curated provider integration independently from a configured provider instance.

Core owns an entry's wire and safety facts. Priority, Icon, Domain, CommonModels and Extensions are presentation and curation fields that product overlays set; the default manifest leaves them empty.

func DecodeRegistryEntries added in v0.7.0

func DecodeRegistryEntries(payload []byte) ([]RegistryEntry, error)

DecodeRegistryEntries strictly decodes a manifest: unknown fields and trailing content are errors. Pass the result to NewRegistry to validate it.

func ProviderRegistry added in v0.2.0

func ProviderRegistry() []RegistryEntry

ProviderRegistry returns copies of the default registry's entries.

func RegistryProvider added in v0.2.0

func RegistryProvider(id string) (RegistryEntry, bool)

RegistryProvider resolves a default registry entry by id or alias.

func RegistryProviderByID added in v0.2.0

func RegistryProviderByID(id string) (RegistryEntry, bool)

RegistryProviderByID resolves only canonical default registry ids.

type ResponsesProvider added in v0.4.0

type ResponsesProvider interface {
	CompleteResponses(ctx context.Context, model string, payload map[string]any, cred *core.Credential) (map[string]any, error)
}

ResponsesProvider is the optional native, non-streaming Responses surface. Callers may use it after PlanTransport selects ModelSurfaceResponses without depending on a concrete provider implementation.

type ResponsesStreamProvider added in v0.4.0

type ResponsesStreamProvider interface {
	StreamResponses(ctx context.Context, model string, payload map[string]any, cred *core.Credential) (StreamIter, error)
}

ResponsesStreamProvider is the optional native streaming Responses surface. Returned chunks are complete SSE frames in the upstream wire format.

type StreamIter

type StreamIter = core.StreamIter

StreamIter is the shared core stream contract.

type ValidationOptions added in v0.7.0

type ValidationOptions struct {
	// RuntimeTypes lists the runtime types the product can execute. Every
	// available entry must use one of them. Nil skips the check.
	RuntimeTypes []string
}

ValidationOptions supplies the product facts a manifest cannot know.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL