openai

package
v0.2.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 6, 2026 License: MIT Imports: 24 Imported by: 0

Documentation

Overview

Package openai provides OpenAI API integration for the LLM gateway.

Index

Constants

View Source 
const DefaultSpeechVoice = "alloy"

DefaultSpeechVoice is the voice native OpenAI and Azure OpenAI speech requests use when the client sends none. OpenAI requires a voice; other OpenAI-compatible providers have their own default speaker and reject OpenAI's voice names, so only these two fill it in.

Variables

View Source 
var Registration = providers.Registration{
	Type:                        "openai",
	New:                         New,
	PassthroughSemanticEnricher: passthroughSemanticEnricher,
	Discovery: providers.DiscoveryConfig{
		DefaultBaseURL:   defaultBaseURL,
		WorkloadIdentity: workloadidentity.TokenExchange,
	},
}

Registration provides factory registration for the OpenAI provider.

Functions

func ChatViaResponses added in v0.1.108 

func ChatViaResponses(ctx context.Context, upstream ResponsesUpstream, req *core.ChatRequest, providerName string) (*core.ChatResponse, error)

ChatViaResponses serves a chat request on a model its upstream exposes only through the Responses API, rendering the result as a chat completion. It uses the same translation the OpenAI provider routes with, but has no Chat Completions endpoint to fall back on: a request the translation cannot carry exactly is rejected instead of being sent with fields dropped.

func New 

func New(cfg providers.ProviderConfig, opts providers.ProviderOptions) core.Provider

New creates a new OpenAI provider.

func StreamChatViaResponses added in v0.1.108 

func StreamChatViaResponses(ctx context.Context, upstream ResponsesUpstream, req *core.ChatRequest, providerName string) (io.ReadCloser, error)

StreamChatViaResponses is ChatViaResponses for streaming requests: the Responses stream is converted into chat completion chunks.

func WithDefaultSpeechVoice added in v0.1.108 

func WithDefaultSpeechVoice(req *core.AudioSpeechRequest) *core.AudioSpeechRequest

WithDefaultSpeechVoice returns req, or a copy naming DefaultSpeechVoice when req names no voice.

Types

type BatchSurface 

type BatchSurface struct {
	// contains filtered or unexported fields
}

BatchSurface is an embeddable facet exposing CompatibleProvider's native batch API. Partial-surface providers (see the CompatibleProvider doc) embed it to advertise exactly the core.NativeBatchProvider capability without inheriting the rest of the full OpenAI surface.

func NewBatchSurface 

func NewBatchSurface(compat *CompatibleProvider) *BatchSurface

NewBatchSurface wraps compat's batch endpoints as an embeddable facet.

func (*BatchSurface) CancelBatch 

func (s *BatchSurface) CancelBatch(ctx context.Context, id string) (*core.BatchResponse, error)

CancelBatch cancels a native batch job.

func (*BatchSurface) CreateBatch 

func (s *BatchSurface) CreateBatch(ctx context.Context, req *core.BatchRequest) (*core.BatchResponse, error)

CreateBatch creates a native batch job.

func (*BatchSurface) GetBatch 

func (s *BatchSurface) GetBatch(ctx context.Context, id string) (*core.BatchResponse, error)

GetBatch retrieves a native batch job.

func (*BatchSurface) GetBatchResults 

func (s *BatchSurface) GetBatchResults(ctx context.Context, id string) (*core.BatchResultsResponse, error)

GetBatchResults fetches batch results via the output file API.

func (*BatchSurface) ListBatches 

func (s *BatchSurface) ListBatches(ctx context.Context, limit int, after string) (*core.BatchListResponse, error)

ListBatches lists native batch jobs.

type ChatCompatible 

type ChatCompatible struct {
	// contains filtered or unexported fields
}

ChatCompatible is an embeddable adapter for providers that expose the chat-centric subset of an OpenAI-compatible API: chat completions, model listing, embeddings, and passthrough, with the Responses API translated through chat completions. When CompatibleProviderConfig.SetHeaders is nil, standard "Authorization: Bearer" headers are applied.

Providers embed *ChatCompatible and override only the methods where their API deviates. Responses and StreamResponses dispatch through this type's own ChatCompletion, so a provider that overrides ChatCompletion or StreamChatCompletion must also override Responses and StreamResponses (re-calling providers.ResponsesViaChat with itself) for the translation to pick up the override.

func NewChatCompatible 

func NewChatCompatible(apiKey string, opts providers.ProviderOptions, cfg CompatibleProviderConfig) *ChatCompatible

NewChatCompatible creates a chat-centric adapter using provider options from the factory (resilience, hooks).

func (*ChatCompatible) ChatCompletion 

func (c *ChatCompatible) ChatCompletion(ctx context.Context, req *core.ChatRequest) (*core.ChatResponse, error)

ChatCompletion sends a chat completion request to the provider.

func (*ChatCompatible) Compatible added in v0.1.99 

func (c *ChatCompatible) Compatible() *CompatibleProvider

Compatible exposes the underlying OpenAI-compatible adapter, so a provider that embeds ChatCompatible can serve a native Responses endpoint through the same instance instead of building a second adapter with the same configuration.

func (*ChatCompatible) Do added in v0.1.96 

func (c *ChatCompatible) Do(ctx context.Context, req llmclient.Request, out any) error

Do sends a request through the underlying compatible provider's client, for adapters that decode a provider-specific payload (a richer model listing) while keeping the shared transport, auth and resilience.

func (*ChatCompatible) Embeddings 

func (c *ChatCompatible) Embeddings(ctx context.Context, req *core.EmbeddingRequest) (*core.EmbeddingResponse, error)

Embeddings sends an embeddings request to the provider.

func (*ChatCompatible) GetBaseURL 

func (c *ChatCompatible) GetBaseURL() string

GetBaseURL returns the provider's current base URL (reads live from the client, so it reflects SetBaseURL overrides). Used to derive realtime websocket targets.

func (*ChatCompatible) HTTPClient added in v0.2.0 

func (c *ChatCompatible) HTTPClient() *http.Client

HTTPClient returns the client the provider's requests go through.

func (*ChatCompatible) ListModels 

func (c *ChatCompatible) ListModels(ctx context.Context) (*core.ModelsResponse, error)

ListModels retrieves the list of available models from the provider.

func (*ChatCompatible) Passthrough 

func (c *ChatCompatible) Passthrough(ctx context.Context, req *core.PassthroughRequest) (*core.PassthroughResponse, error)

Passthrough routes an opaque provider-native request to the provider.

func (*ChatCompatible) ProviderName added in v0.1.108 

func (c *ChatCompatible) ProviderName() string

ProviderName returns the name this provider reports to clients: the configured instance name when there is one, the provider type otherwise.

func (*ChatCompatible) Responses 

func (c *ChatCompatible) Responses(ctx context.Context, req *core.ResponsesRequest) (*core.ResponsesResponse, error)

Responses sends a Responses API request using chat-completions translation.

func (*ChatCompatible) SetBaseURL 

func (c *ChatCompatible) SetBaseURL(url string)

SetBaseURL allows configuring a custom base URL for the provider.

func (*ChatCompatible) StreamChatCompletion 

func (c *ChatCompatible) StreamChatCompletion(ctx context.Context, req *core.ChatRequest) (io.ReadCloser, error)

StreamChatCompletion returns a raw response body for streaming.

func (*ChatCompatible) StreamResponses 

func (c *ChatCompatible) StreamResponses(ctx context.Context, req *core.ResponsesRequest) (io.ReadCloser, error)

StreamResponses streams a Responses API request using chat-completions translation.

type CompatibleProvider 

type CompatibleProvider struct {
	// contains filtered or unexported fields
}

CompatibleProvider is the single transport engine for every OpenAI-compatible upstream. Provider packages must not hand-roll llmclient calls for OpenAI-shaped endpoints; they wrap this type in one of three ways, chosen by how much of the OpenAI surface the upstream actually implements:

  • Embed *ChatCompatible for chat-centric upstreams (chat, models, embeddings, passthrough; Responses translated via chat) — e.g. xiaomi, zai, fireworks, deepseek.
  • Embed *CompatibleProvider only when the upstream implements the full surface, including audio, files, batches, and native response lifecycle management — e.g. openrouter, azure.
  • Compose an unexported *CompatibleProvider field and delegate the supported methods explicitly when the upstream implements a partial surface — e.g. groq (no passthrough), xai (no audio/passthrough), ollama (native embeddings, no passthrough). Go embedding cannot subtract methods, and an accidentally inherited method advertises a capability the upstream lacks (the router discovers capabilities by interface assertion).

Provider quirks belong in CompatibleProviderConfig hooks (SetHeaders, AdaptChatRequest, ChatRequestHeaders, RequestMutator), not in copies of the transport methods.

func NewCompatibleProvider 

func NewCompatibleProvider(apiKey string, opts providers.ProviderOptions, cfg CompatibleProviderConfig) *CompatibleProvider

func (*CompatibleProvider) CancelBatch 

func (p *CompatibleProvider) CancelBatch(ctx context.Context, id string) (*core.BatchResponse, error)

func (*CompatibleProvider) CancelResponse 

func (p *CompatibleProvider) CancelResponse(ctx context.Context, id string) (*core.ResponsesResponse, error)

func (*CompatibleProvider) ChatCompletion 

func (p *CompatibleProvider) ChatCompletion(ctx context.Context, req *core.ChatRequest) (*core.ChatResponse, error)

func (*CompatibleProvider) CompactResponse 

func (p *CompatibleProvider) CompactResponse(ctx context.Context, req *core.ResponsesRequest) (*core.ResponseCompactResponse, error)

func (*CompatibleProvider) CountResponseInputTokens 

func (p *CompatibleProvider) CountResponseInputTokens(ctx context.Context, req *core.ResponsesRequest) (*core.ResponseInputTokensResponse, error)

func (*CompatibleProvider) CreateBatch 

func (p *CompatibleProvider) CreateBatch(ctx context.Context, req *core.BatchRequest) (*core.BatchResponse, error)

func (*CompatibleProvider) CreateFile 

func (p *CompatibleProvider) CreateFile(ctx context.Context, req *core.FileCreateRequest) (*core.FileObject, error)

func (*CompatibleProvider) CreateImage added in v0.1.81 

func (p *CompatibleProvider) CreateImage(ctx context.Context, req *core.ImageGenerationRequest) (*core.ImageGenerationResponse, error)

CreateImage implements OpenAI image generation (POST /images/generations). The typed request marshals its known fields plus any extra parameters the client sent, so model-specific options (style, background, output_format, moderation, ...) reach the upstream unchanged.

func (*CompatibleProvider) CreateImageEdit added in v0.1.81 

func (p *CompatibleProvider) CreateImageEdit(ctx context.Context, req *core.ImageEditRequest) (*core.ImageGenerationResponse, error)

CreateImageEdit implements OpenAI image editing (POST /images/edits). The request is re-encoded as multipart/form-data: the typed model and prompt, every extra form field the client sent, then the image(s) and optional mask. The body is buffered (the images are already in memory) so the client can retry transient upstream failures.

func (*CompatibleProvider) CreateSpeech 

func (p *CompatibleProvider) CreateSpeech(ctx context.Context, req *core.AudioSpeechRequest) (*core.AudioResponse, error)

CreateSpeech implements OpenAI text-to-speech (POST /audio/speech). The upstream produces the audio as it synthesizes it, so the body is relayed as it arrives: buffering it charges the client the whole generation before the first byte. The response is tagged with the upstream Content-Type so it describes the bytes actually returned (see speechResponseContentType), which usage relies on to price output-duration models.

func (*CompatibleProvider) CreateTranscription 

func (p *CompatibleProvider) CreateTranscription(ctx context.Context, req *core.AudioTranscriptionRequest) (*core.AudioResponse, error)

CreateTranscription implements OpenAI speech-to-text (POST /audio/transcriptions). The request is multipart/form-data; the response (JSON or text per response_format) is proxied verbatim.

func (*CompatibleProvider) CreateTranslation added in v0.1.68 

func (p *CompatibleProvider) CreateTranslation(ctx context.Context, req *core.AudioTranscriptionRequest) (*core.AudioResponse, error)

CreateTranslation implements OpenAI speech translation (POST /audio/translations). Translation accepts the transcription upload fields except language and timestamp granularities, and always returns English text.

func (*CompatibleProvider) DeleteFile 

func (p *CompatibleProvider) DeleteFile(ctx context.Context, id string) (*core.FileDeleteResponse, error)

func (*CompatibleProvider) DeleteResponse 

func (p *CompatibleProvider) DeleteResponse(ctx context.Context, id string) (*core.ResponseDeleteResponse, error)

func (*CompatibleProvider) Do 

func (p *CompatibleProvider) Do(ctx context.Context, req llmclient.Request, result any) error

func (*CompatibleProvider) Embeddings 

func (p *CompatibleProvider) Embeddings(ctx context.Context, req *core.EmbeddingRequest) (*core.EmbeddingResponse, error)

func (*CompatibleProvider) GetBaseURL 

func (p *CompatibleProvider) GetBaseURL() string

GetBaseURL returns the provider's current base URL. It reads from the client so it always reflects SetBaseURL overrides (used to derive the realtime websocket target without a separately retained, staleable copy).

func (*CompatibleProvider) GetBatch 

func (p *CompatibleProvider) GetBatch(ctx context.Context, id string) (*core.BatchResponse, error)

func (*CompatibleProvider) GetBatchResults 

func (p *CompatibleProvider) GetBatchResults(ctx context.Context, id string) (*core.BatchResultsResponse, error)

func (*CompatibleProvider) GetFile 

func (p *CompatibleProvider) GetFile(ctx context.Context, id string) (*core.FileObject, error)

func (*CompatibleProvider) GetFileContent 

func (p *CompatibleProvider) GetFileContent(ctx context.Context, id string) (*core.FileContentResponse, error)

func (*CompatibleProvider) GetResponse 

func (p *CompatibleProvider) GetResponse(ctx context.Context, id string, params core.ResponseRetrieveParams) (*core.ResponsesResponse, error)

func (*CompatibleProvider) HTTPClient added in v0.2.0 

func (p *CompatibleProvider) HTTPClient() *http.Client

HTTPClient returns the client the provider's requests go through, carrying its outbound proxy and the gateway's TLS trust.

func (*CompatibleProvider) ListBatches 

func (p *CompatibleProvider) ListBatches(ctx context.Context, limit int, after string) (*core.BatchListResponse, error)

func (*CompatibleProvider) ListFiles 

func (p *CompatibleProvider) ListFiles(ctx context.Context, purpose string, limit int, after string) (*core.FileListResponse, error)

func (*CompatibleProvider) ListModels 

func (p *CompatibleProvider) ListModels(ctx context.Context) (*core.ModelsResponse, error)

ListModels lists the models the server serves. A max_model_len an entry reports is kept on the model, where no response serializes it: only models.vllm_discovery_compat decides whether a listing advertises one.

func (*CompatibleProvider) ListModelsWithMaxModelLen added in v0.1.96 

func (p *CompatibleProvider) ListModelsWithMaxModelLen(ctx context.Context) (*core.ModelsResponse, error)

ListModelsWithMaxModelLen lists models like ListModels but also reports the max_model_len field vLLM-style servers (vLLM, SGLang, llm-d) add to each entry as the model's context window: for a self-hosted server the running process is the only source that knows the real limit.

func (*CompatibleProvider) ListResponseInputItems 

func (p *CompatibleProvider) ListResponseInputItems(ctx context.Context, id string, params core.ResponseInputItemsParams) (*core.ResponseInputItemListResponse, error)

func (*CompatibleProvider) Passthrough 

func (p *CompatibleProvider) Passthrough(ctx context.Context, req *core.PassthroughRequest) (*core.PassthroughResponse, error)

func (*CompatibleProvider) ProviderName added in v0.1.108 

func (p *CompatibleProvider) ProviderName() string

ProviderName returns the name this provider reports to clients: the configured instance name when there is one, the provider type otherwise. Providers that translate the Responses API themselves pass it on, so their responses name the same instance the errors do.

func (*CompatibleProvider) Responses 

func (p *CompatibleProvider) Responses(ctx context.Context, req *core.ResponsesRequest) (*core.ResponsesResponse, error)

func (*CompatibleProvider) SetBaseURL 

func (p *CompatibleProvider) SetBaseURL(url string)

func (*CompatibleProvider) SetRequestMutator 

func (p *CompatibleProvider) SetRequestMutator(mutator RequestMutator)

func (*CompatibleProvider) StreamChatCompletion 

func (p *CompatibleProvider) StreamChatCompletion(ctx context.Context, req *core.ChatRequest) (io.ReadCloser, error)

func (*CompatibleProvider) StreamResponses 

func (p *CompatibleProvider) StreamResponses(ctx context.Context, req *core.ResponsesRequest) (io.ReadCloser, error)

type CompatibleProviderConfig 

type CompatibleProviderConfig struct {
	ProviderName string
	BaseURL      string
	// HTTPClient overrides the shared default client while retaining the
	// factory-provided retry, circuit-breaker, and observability settings.
	// Providers use this for transport-level concerns such as request signing.
	HTTPClient     *http.Client
	SetHeaders     func(*http.Request, string)
	RequestMutator RequestMutator
	// AdaptChatRequest rewrites the typed chat request before provider
	// dispatch on ChatCompletion and StreamChatCompletion. Providers use it
	// for parameter quirks (e.g. remapping reasoning effort levels) instead
	// of overriding the chat methods, so Responses-via-chat translation picks
	// the adaptation up automatically. It must not mutate its argument;
	// return a shallow copy when changes are needed.
	AdaptChatRequest func(*core.ChatRequest) (*core.ChatRequest, error)
	// ChatRequestHeaders returns extra per-request HTTP headers for
	// ChatCompletion and StreamChatCompletion, derived from the request
	// context and body (e.g. conversation affinity headers). Nil results are
	// ignored.
	ChatRequestHeaders func(context.Context, *core.ChatRequest) http.Header
	// ForwardAllowedTools sends an allowed_tools tool_choice upstream
	// unchanged. Set it only for upstreams documented to honor the choice.
	// When unset, the declared tools are narrowed to the allowed subset
	// (core.NarrowToAllowedTools), so an upstream that ignores or rejects the
	// unknown choice type cannot call a tool outside it.
	ForwardAllowedTools bool
}

type FileSurface 

type FileSurface struct {
	// contains filtered or unexported fields
}

FileSurface is an embeddable facet exposing CompatibleProvider's OpenAI-compatible files API. Partial-surface providers embed it to advertise exactly the core.NativeFileProvider capability without inheriting the rest of the full OpenAI surface.

func NewFileSurface 

func NewFileSurface(compat *CompatibleProvider) *FileSurface

NewFileSurface wraps compat's file endpoints as an embeddable facet.

func (*FileSurface) CreateFile 

func (s *FileSurface) CreateFile(ctx context.Context, req *core.FileCreateRequest) (*core.FileObject, error)

CreateFile uploads a file through the OpenAI-compatible /files API.

func (*FileSurface) DeleteFile 

func (s *FileSurface) DeleteFile(ctx context.Context, id string) (*core.FileDeleteResponse, error)

DeleteFile deletes a file object through the OpenAI-compatible /files API.

func (*FileSurface) GetFile 

func (s *FileSurface) GetFile(ctx context.Context, id string) (*core.FileObject, error)

GetFile retrieves one file object through the OpenAI-compatible /files API.

func (*FileSurface) GetFileContent 

func (s *FileSurface) GetFileContent(ctx context.Context, id string) (*core.FileContentResponse, error)

GetFileContent fetches raw file bytes through the /files/{id}/content API.

func (*FileSurface) ListFiles 

func (s *FileSurface) ListFiles(ctx context.Context, purpose string, limit int, after string) (*core.FileListResponse, error)

ListFiles lists files through the OpenAI-compatible /files API.

type Provider 

type Provider struct {
	*CompatibleProvider
}

Provider implements the core.Provider interface for OpenAI. Credentials and the realtime base URL are both read live from the embedded CompatibleProvider, so SetBaseURL overrides and key rotation are honored on the realtime websocket dial target too (see realtime.go).

func (*Provider) ChatCompletion added in v0.1.99 

func (p *Provider) ChatCompletion(ctx context.Context, req *core.ChatRequest) (*core.ChatResponse, error)

ChatCompletion serves the request through Chat Completions, or through the Responses API when Chat Completions cannot serve it (see needsResponses). Stop sequences the model rejects are emulated (see takeStopSequences).

func (*Provider) CountChatTokens added in v0.1.100 

func (p *Provider) CountChatTokens(ctx context.Context, req *core.ChatRequest) (int, error)

CountChatTokens counts a chat request's input tokens with OpenAI's /responses/input_tokens endpoint, for the request as the path that will serve it sends it: a request served through the Responses API is counted as translated, one served through Chat Completions with its chat adaptations applied and, when it carries tools on GPT-5 and later, the Chat tools preamble added. Older families have their own, smaller Chat preamble (6 tokens on gpt-4o-mini) that is not added, so a Chat-served tool request there counts slightly below the billed input; OpenAI offers no count for Chat Completions itself. A request the translation cannot carry exactly answers core.ErrMessagesTokenCountUnsupported, so the caller estimates instead.

func (*Provider) CreateSpeech added in v0.1.108 

func (p *Provider) CreateSpeech(ctx context.Context, req *core.AudioSpeechRequest) (*core.AudioResponse, error)

CreateSpeech fills OpenAI's required voice before synthesizing.

func (*Provider) Decisions added in v0.1.106 

func (p *Provider) Decisions(ctx context.Context, req *core.PassthroughRequest) (*core.PassthroughResponse, error)

Decisions forwards an OpenAI Decisions request to /decisions as sent. It is defined on Provider rather than CompatibleProvider: the Decisions API is OpenAI's own, and the OpenAI-compatible upstreams built on the shared adapter do not serve it.

func (*Provider) RealtimeCallTarget 

func (p *Provider) RealtimeCallTarget(ctx context.Context, req *core.RealtimeRequest) (*core.RealtimeHTTPTarget, error)

RealtimeCallTarget implements core.RealtimeCallProvider for OpenAI's WebRTC SDP exchange (POST https://api.openai.com/v1/realtime/calls, or /v1/realtime/translations/calls for translation sessions). The gateway appends the model query parameter or session form field itself, so the target is the bare calls endpoint.

func (*Provider) RealtimeClientSecretTarget 

func (p *Provider) RealtimeClientSecretTarget(ctx context.Context, req *core.RealtimeRequest) (*core.RealtimeHTTPTarget, error)

RealtimeClientSecretTarget implements core.RealtimeCallProvider for minting ephemeral realtime client secrets (POST https://api.openai.com/v1/realtime/client_secrets, or /v1/realtime/translations/client_secrets for translation sessions).

func (*Provider) RealtimeTarget 

func (p *Provider) RealtimeTarget(ctx context.Context, req *core.RealtimeRequest) (*core.RealtimeTarget, error)

RealtimeTarget implements core.RealtimeProvider for OpenAI's realtime websocket (wss://api.openai.com/v1/realtime). The endpoint is derived from the configured base URL so endpoint overrides and OpenAI-compatible realtime backends work without extra config. Bearer auth is injected here and must never be logged. A request carrying a CallID attaches to an existing WebRTC/SIP call as a sideband channel instead of opening a fresh model session; an intent selects the transcription or translation session surface instead of a conversation.

func (*Provider) StreamChatCompletion added in v0.1.99 

func (p *Provider) StreamChatCompletion(ctx context.Context, req *core.ChatRequest) (io.ReadCloser, error)

StreamChatCompletion is ChatCompletion for streaming requests; a request served through the Responses API is streamed back as chat completion chunks.

func (*Provider) SupportsRealtimeIntent added in v0.1.82 

func (p *Provider) SupportsRealtimeIntent(intent string) bool

SupportsRealtimeIntent implements core.RealtimeIntentProvider: OpenAI serves both specialized session surfaces — transcription and translation — on every realtime endpoint the gateway exposes. Any other intent is unknown here and is rejected upstream of the provider rather than served as a conversation.

type RequestMutator 

type RequestMutator func(*llmclient.Request)

type ResponsesUpstream added in v0.1.108 

type ResponsesUpstream interface {
	Responses(ctx context.Context, req *core.ResponsesRequest) (*core.ResponsesResponse, error)
	StreamResponses(ctx context.Context, req *core.ResponsesRequest) (io.ReadCloser, error)
}

ResponsesUpstream is the Responses surface ChatViaResponses serves a chat request on.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.