openai_compatible

package
v0.0.10 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 4, 2026 License: MIT Imports: 11 Imported by: 0

Documentation

Overview

Package openai_compatible provides a reusable implementation for LLM providers that use OpenAI-compatible APIs.

This includes providers like OpenAI, Groq, Together AI, Fireworks AI, and others that implement the OpenAI chat completions API format.

Basic usage:

provider, _ := openai_compatible.New("groq", "gsk_xxx", "https://api.groq.com")
proxy := llmproxy.NewProxy(provider)

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ParseOpenAIRequest

func ParseOpenAIRequest(body io.ReadCloser) (llmproxy.BodyMetadata, []byte, error)

ParseOpenAIRequest is a convenience function that parses an OpenAI-compatible request body and returns the metadata and raw bytes.

func ParseOpenAIRequestBody

func ParseOpenAIRequestBody(data []byte) (llmproxy.BodyMetadata, error)

ParseOpenAIRequestBody parses raw JSON bytes as an OpenAI-compatible request. It returns only the metadata, not the raw bytes.

func ParseResponsesRequest added in v0.0.6

func ParseResponsesRequest(body io.ReadCloser) (llmproxy.BodyMetadata, []byte, error)

func ParseResponsesRequestBody added in v0.0.6

func ParseResponsesRequestBody(data []byte) (llmproxy.BodyMetadata, error)

Types

type CompletionTokensDetails added in v0.0.3

type CompletionTokensDetails struct {
	ReasoningTokens          int `json:"reasoning_tokens,omitempty"`
	AudioTokens              int `json:"audio_tokens,omitempty"`
	AcceptedPredictionTokens int `json:"accepted_prediction_tokens,omitempty"`
	RejectedPredictionTokens int `json:"rejected_prediction_tokens,omitempty"`
}

CompletionTokensDetails contains detailed completion token breakdown.

type Enricher

type Enricher struct {
	// APIKey is the API key for authentication.
	APIKey string
}

Enricher implements llmproxy.RequestEnricher for OpenAI-compatible APIs. It sets the required Authorization header with a Bearer token.

func NewEnricher

func NewEnricher(apiKey string) *Enricher

NewEnricher creates a new enricher with the given API key.

func (*Enricher) Enrich

func (e *Enricher) Enrich(req *http.Request, meta llmproxy.BodyMetadata, rawBody []byte) error

Enrich adds the Authorization and Content-Type headers to the request. It sets:

  • Authorization: Bearer <APIKey>
  • Content-Type: application/json

type Extractor

type Extractor struct{}

Extractor implements llmproxy.ResponseExtractor for OpenAI-compatible responses. It parses the response JSON and extracts token usage, choices, and other metadata.

func NewExtractor

func NewExtractor() *Extractor

NewExtractor creates a new OpenAI-compatible response extractor.

func (*Extractor) Extract

func (e *Extractor) Extract(resp *http.Response) (llmproxy.ResponseMetadata, []byte, error)

Extract reads the response body and parses it as an OpenAI-compatible response. It extracts the ID, model, usage statistics, and completion choices.

Returns:

  • metadata: Parsed response metadata
  • rawBody: The original response body bytes (preserved for forwarding)
  • error: Any parsing error

The raw body is returned so it can be re-attached to the response for the caller, preserving any custom/unsupported fields in the original JSON.

type MultiAPIExtractor added in v0.0.6

type MultiAPIExtractor struct {
	// contains filtered or unexported fields
}

func NewMultiAPIExtractor added in v0.0.6

func NewMultiAPIExtractor() *MultiAPIExtractor

func (*MultiAPIExtractor) Extract added in v0.0.6

type MultiAPIParser added in v0.0.6

type MultiAPIParser struct {
	// contains filtered or unexported fields
}

func NewMultiAPIParser added in v0.0.6

func NewMultiAPIParser() *MultiAPIParser

func (*MultiAPIParser) Parse added in v0.0.6

type OpenAIRequest

type OpenAIRequest struct {
	// Model is the model identifier (e.g., "gpt-4", "llama-2-70b").
	Model string `json:"model"`
	// Messages is the conversation history.
	Messages []llmproxy.Message `json:"messages"`
	// MaxTokens limits the generation length.
	MaxTokens int `json:"max_tokens,omitempty"`
	// Stream enables streaming responses.
	Stream bool `json:"stream"`
	// Custom holds provider-specific parameters not in the standard schema.
	Custom map[string]interface{} `json:"-"`
}

OpenAIRequest represents an OpenAI-compatible chat completion request. It includes standard fields and captures custom fields for provider extensions.

func (*OpenAIRequest) UnmarshalJSON

func (r *OpenAIRequest) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshaling to capture unknown fields.

type OpenAIResponse

type OpenAIResponse struct {
	// ID is the unique response identifier.
	ID string `json:"id"`
	// Object is the object type (e.g., "chat.completion").
	Object string `json:"object"`
	// Created is the Unix timestamp of creation.
	Created int64 `json:"created"`
	// Model is the model used for completion.
	Model string `json:"model"`
	// Usage contains token consumption statistics.
	Usage UsageInfo `json:"usage"`
	// Choices contains the completion choices.
	Choices []ResponseChoice `json:"choices"`
}

OpenAIResponse represents an OpenAI-compatible chat completion response.

type Parser

type Parser struct{}

Parser implements llmproxy.BodyParser for OpenAI-compatible request formats. It extracts model, messages, and other fields into a unified BodyMetadata structure.

func (*Parser) Parse

func (p *Parser) Parse(body io.ReadCloser) (llmproxy.BodyMetadata, []byte, error)

Parse reads an OpenAI-compatible request body and extracts metadata. It returns both the parsed metadata and the raw body bytes for later use.

The parser handles standard OpenAI fields and captures unknown fields in the Custom map for provider-specific extensions.

type PromptTokensDetails added in v0.0.3

type PromptTokensDetails struct {
	CachedTokens int `json:"cached_tokens,omitempty"`
	AudioTokens  int `json:"audio_tokens,omitempty"`
}

PromptTokensDetails contains detailed prompt token breakdown.

type Provider

type Provider struct {
	*llmproxy.BaseProvider
}

Provider is an OpenAI-compatible provider implementation. It embeds llmproxy.BaseProvider and can be further customized.

func New

func New(name, apiKey, baseURL string) (*Provider, error)

New creates a new OpenAI-compatible provider with the given configuration.

Parameters:

  • name: A unique identifier for the provider (e.g., "openai", "groq")
  • apiKey: The API key for authentication
  • baseURL: The provider's API base URL (e.g., "https://api.openai.com")

Example:

provider, _ := openai_compatible.New("groq", "gsk_xxx", "https://api.groq.com")

func NewMultiAPI added in v0.0.6

func NewMultiAPI(name, apiKey, baseURL string) (*Provider, error)

func NewWithProvider

func NewWithProvider(name string, p *llmproxy.BaseProvider) *Provider

NewWithProvider creates a Provider that wraps an existing BaseProvider. Use this when you need to customize individual components before creating the provider.

Example:

base := llmproxy.NewBaseProvider("custom",
    llmproxy.WithBodyParser(&Parser{}),
    llmproxy.WithRequestEnricher(customEnricher),
)
provider := openai_compatible.NewWithProvider("custom", base)

func (*Provider) WebSocketURL added in v0.0.8

func (p *Provider) WebSocketURL(meta llmproxy.BodyMetadata) (*url.URL, error)

type Resolver

type Resolver struct {
	BaseURL *url.URL
	APIType llmproxy.APIType
}

func NewResolver

func NewResolver(baseURL string) (*Resolver, error)

func NewResolverWithAPIType added in v0.0.6

func NewResolverWithAPIType(baseURL string, apiType llmproxy.APIType) (*Resolver, error)

func (*Resolver) Resolve

func (r *Resolver) Resolve(meta llmproxy.BodyMetadata) (*url.URL, error)

func (*Resolver) WebSocketURL added in v0.0.8

func (r *Resolver) WebSocketURL(meta llmproxy.BodyMetadata) (*url.URL, error)

WebSocketURL converts the provider HTTP base URL to a WebSocket URL.

https://api.openai.com -> wss://api.openai.com/v1/responses
http://localhost:8080 -> ws://localhost:8080/v1/responses

type ResponseChoice

type ResponseChoice struct {
	// Index is the choice position.
	Index int `json:"index"`
	// Message contains the completed message (non-streaming).
	Message *ResponseMessage `json:"message,omitempty"`
	// Delta contains the partial message (streaming).
	Delta *ResponseMessage `json:"delta,omitempty"`
	// FinishReason indicates why completion stopped.
	FinishReason string `json:"finish_reason"`
}

ResponseChoice represents a single completion choice.

type ResponseMessage

type ResponseMessage struct {
	Role    string `json:"role"`
	Content string `json:"content"`
}

ResponseMessage represents a message in a completion choice.

type ResponsesError added in v0.0.6

type ResponsesError struct {
	Type    string `json:"type"`
	Code    string `json:"code"`
	Message string `json:"message"`
}

type ResponsesExtractor added in v0.0.6

type ResponsesExtractor struct{}

func NewResponsesExtractor added in v0.0.6

func NewResponsesExtractor() *ResponsesExtractor

func (*ResponsesExtractor) Extract added in v0.0.6

type ResponsesInputDetails added in v0.0.6

type ResponsesInputDetails struct {
	CachedTokens int `json:"cached_tokens,omitempty"`
}

type ResponsesOutputAnnotation added in v0.0.7

type ResponsesOutputAnnotation struct {
	Type       string `json:"type"`
	Title      string `json:"title,omitempty"`
	URL        string `json:"url,omitempty"`
	Index      *int   `json:"index,omitempty"`
	StartIndex *int   `json:"start_index,omitempty"`
	EndIndex   *int   `json:"end_index,omitempty"`
}

type ResponsesOutputContent added in v0.0.6

type ResponsesOutputContent struct {
	Type        string                      `json:"type"`
	Text        string                      `json:"text,omitempty"`
	Annotations []ResponsesOutputAnnotation `json:"annotations,omitempty"`
	Logprobs    interface{}                 `json:"logprobs,omitempty"`
}

type ResponsesOutputDetails added in v0.0.6

type ResponsesOutputDetails struct {
	ReasoningTokens int `json:"reasoning_tokens,omitempty"`
}

type ResponsesOutputItem added in v0.0.6

type ResponsesOutputItem struct {
	ID        string                   `json:"id"`
	Type      string                   `json:"type"`
	Status    string                   `json:"status"`
	Role      string                   `json:"role,omitempty"`
	Content   []ResponsesOutputContent `json:"content,omitempty"`
	Name      string                   `json:"name,omitempty"`
	Arguments string                   `json:"arguments,omitempty"`
	Summary   []ResponsesOutputSummary `json:"summary,omitempty"`
}

type ResponsesOutputSummary added in v0.0.7

type ResponsesOutputSummary struct {
	Type string `json:"type"`
	Text string `json:"text,omitempty"`
}

type ResponsesParser added in v0.0.6

type ResponsesParser struct{}

func (*ResponsesParser) Parse added in v0.0.6

type ResponsesRequest added in v0.0.6

type ResponsesRequest struct {
	Model           string                 `json:"model"`
	Input           interface{}            `json:"input,omitempty"`
	Instructions    string                 `json:"instructions,omitempty"`
	MaxOutputTokens int                    `json:"max_output_tokens,omitempty"`
	Temperature     *float64               `json:"temperature,omitempty"`
	TopP            *float64               `json:"top_p,omitempty"`
	Stream          bool                   `json:"stream"`
	Tools           []interface{}          `json:"tools,omitempty"`
	Truncation      string                 `json:"truncation,omitempty"`
	Custom          map[string]interface{} `json:"-"`
}

func (*ResponsesRequest) UnmarshalJSON added in v0.0.6

func (r *ResponsesRequest) UnmarshalJSON(data []byte) error

type ResponsesResponse added in v0.0.6

type ResponsesResponse struct {
	ID      string                `json:"id"`
	Object  string                `json:"object"`
	Created int64                 `json:"created"`
	Model   string                `json:"model"`
	Status  string                `json:"status"`
	Output  []ResponsesOutputItem `json:"output"`
	Usage   ResponsesUsage        `json:"usage"`
	Error   *ResponsesError       `json:"error,omitempty"`
}

type ResponsesStreamingExtractor added in v0.0.8

type ResponsesStreamingExtractor struct {
	*ResponsesExtractor
}

func NewResponsesStreamingExtractor added in v0.0.8

func NewResponsesStreamingExtractor() *ResponsesStreamingExtractor

func (*ResponsesStreamingExtractor) ExtractStreamingWithController added in v0.0.8

func (*ResponsesStreamingExtractor) IsStreamingResponse added in v0.0.8

func (e *ResponsesStreamingExtractor) IsStreamingResponse(resp *http.Response) bool

type ResponsesUsage added in v0.0.6

type ResponsesUsage struct {
	InputTokens         int                     `json:"input_tokens"`
	OutputTokens        int                     `json:"output_tokens"`
	TotalTokens         int                     `json:"total_tokens"`
	InputTokensDetails  *ResponsesInputDetails  `json:"input_tokens_details,omitempty"`
	OutputTokensDetails *ResponsesOutputDetails `json:"output_tokens_details,omitempty"`
}

type StreamingExtractor added in v0.0.6

type StreamingExtractor struct {
	*Extractor
}

func NewStreamingExtractor added in v0.0.6

func NewStreamingExtractor() *StreamingExtractor

func (*StreamingExtractor) ExtractStreamingWithController added in v0.0.6

func (e *StreamingExtractor) ExtractStreamingWithController(resp *http.Response, w http.ResponseWriter, rc *http.ResponseController) (llmproxy.ResponseMetadata, error)

func (*StreamingExtractor) IsStreamingResponse added in v0.0.6

func (e *StreamingExtractor) IsStreamingResponse(resp *http.Response) bool

type StreamingMultiAPIExtractor added in v0.0.6

type StreamingMultiAPIExtractor struct {
	*MultiAPIExtractor
	// contains filtered or unexported fields
}

func NewStreamingMultiAPIExtractor added in v0.0.6

func NewStreamingMultiAPIExtractor() *StreamingMultiAPIExtractor

func (*StreamingMultiAPIExtractor) ExtractStreamingWithController added in v0.0.6

func (*StreamingMultiAPIExtractor) IsStreamingResponse added in v0.0.6

func (e *StreamingMultiAPIExtractor) IsStreamingResponse(resp *http.Response) bool

type UsageInfo

type UsageInfo struct {
	PromptTokens            int                      `json:"prompt_tokens"`
	CompletionTokens        int                      `json:"completion_tokens"`
	TotalTokens             int                      `json:"total_tokens"`
	PromptTokensDetails     *PromptTokensDetails     `json:"prompt_tokens_details,omitempty"`
	CompletionTokensDetails *CompletionTokensDetails `json:"completion_tokens_details,omitempty"`
}

UsageInfo tracks token usage in an OpenAI-compatible response.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL