Documentation
¶
Overview ¶
Package openai implements ai.Provider for the OpenAI Chat Completions, Images and Embeddings APIs and for compatible servers: provider types "openai" (api.openai.com or any OpenAI-compatible base_url such as vLLM, LiteLLM, LM Studio or OpenRouter), "ollama" (OpenAI-compatible endpoint of a local Ollama) and "azure-openai" (Azure OpenAI / Azure AI Foundry, API key or Entra ID authentication).
Cloudflare is supported through base_url: Workers AI's OpenAI-compatible endpoint (https://api.cloudflare.com/client/v4/accounts/<id>/ai/v1, API token as api_key) for chat and embeddings, with image generation routed to the native Workers AI run endpoint (…/ai/run/<model>, e.g. @cf/black-forest-labs/flux-1-schnell); and AI Gateway URLs (https://gateway.ai.cloudflare.com/v1/<account>/<gateway>/openai, …/workers-ai/v1 or …/compat), with gateway auth via headers (cf-aig-authorization).
Image URLs in chat messages are passed through to OpenAI and Azure OpenAI, which download them from the public internet. Every other upstream (Ollama, vLLM, LiteLLM, LM Studio, Workers AI, …) gets the image inline as a data: URL, downloaded by Backyard's SSRF-guarded fetcher: many of these servers cannot fetch URLs at all, and a self-hosted server that can would do so from inside its own network.
It uses plain net/http and encoding/json. Differences between compatible servers are handled adaptively: when an upstream rejects max_tokens vs max_completion_tokens, temperature, stream_options or response_format with a 400, the request is retried once with the parameter adjusted and the adjustment is remembered for that model.
Index ¶
- Constants
- type Options
- type Provider
- func (p *Provider) Chat(ctx context.Context, req ai.ChatRequest) (*ai.ChatResponse, error)
- func (p *Provider) ChatStream(ctx context.Context, req ai.ChatRequest, onDelta func(string) error) (*ai.ChatResponse, error)
- func (p *Provider) Embed(ctx context.Context, req ai.EmbedRequest) (*ai.EmbedResponse, error)
- func (p *Provider) Image(ctx context.Context, req ai.ImageRequest) (*ai.ImageResponse, error)
- func (p *Provider) Name() string
Constants ¶
const ( TypeOpenAI = "openai" TypeOllama = "ollama" TypeAzureOpenAI = "azure-openai" )
Provider types handled by this package.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Options ¶
type Options struct {
// HTTPClient performs upstream requests (default: httpx.DefaultClient()).
HTTPClient *http.Client
// Logger receives debug logs (default: slog.Default()).
Logger *slog.Logger
// FetchImage downloads URL images for upstreams that only accept inline
// images (ollama). Default: an SSRF-guarded fetcher.
FetchImage imagefetch.Func
// AzureCredential overrides the Entra ID credential used when
// use_azure_ad is set (default: azidentity.NewDefaultAzureCredential).
AzureCredential azcore.TokenCredential
}
Options are optional dependencies.
type Provider ¶
type Provider struct {
// contains filtered or unexported fields
}
Provider is an ai.Provider for OpenAI-compatible APIs. It is safe for concurrent use.
func New ¶
New creates a provider from its config. pc.Type must be openai, ollama or azure-openai (empty means openai).
func (*Provider) Chat ¶
func (p *Provider) Chat(ctx context.Context, req ai.ChatRequest) (*ai.ChatResponse, error)
Chat implements ai.Provider.
func (*Provider) ChatStream ¶
func (p *Provider) ChatStream(ctx context.Context, req ai.ChatRequest, onDelta func(string) error) (*ai.ChatResponse, error)
ChatStream implements ai.Provider.
func (*Provider) Embed ¶
func (p *Provider) Embed(ctx context.Context, req ai.EmbedRequest) (*ai.EmbedResponse, error)
Embed implements ai.Provider via POST /embeddings.
func (*Provider) Image ¶
func (p *Provider) Image(ctx context.Context, req ai.ImageRequest) (*ai.ImageResponse, error)
Image implements ai.Provider via POST /images/generations.
Source Files