handlers

package
v8.0.21 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 8, 2026 License: MIT Imports: 35 Imported by: 1

Documentation

Overview

Package handlers provides core API handler functionality for the CLI Proxy API server. It includes common types, client management, load balancing, and error handling shared across all API endpoint handlers (OpenAI, Claude, Gemini).

Index

Constants

View Source
const ModelDetailIDContextKey = "cliproxy.model_detail_id"

ModelDetailIDContextKey requests selection from the final, plugin-filtered catalog.

Variables

This section is empty.

Functions

func BuildErrorResponseBody

func BuildErrorResponseBody(status int, errText string) []byte

BuildErrorResponseBody builds an OpenAI-compatible JSON error response body. If errText is already valid JSON, it is compacted to ensure single-line SSE compatibility while preserving upstream error payloads.

func BuildErrorResponseBodyWithError

func BuildErrorResponseBodyWithError(status int, errText string, err error) []byte

BuildErrorResponseBodyWithError builds an OpenAI-compatible JSON error response body, preserving structured classifications (such as terminal upstream auth failures and retryable flags) present in err.

func BuildOpenAIResponsesStreamErrorChunk

func BuildOpenAIResponsesStreamErrorChunk(status int, errText string, sequenceNumber int) []byte

BuildOpenAIResponsesStreamErrorChunk builds an OpenAI Responses streaming error chunk. It matches the official responses streaming event shape where the error details are nested.

func BuildOpenAIResponsesStreamFailedChunk

func BuildOpenAIResponsesStreamFailedChunk(status int, errText string, sequenceNumber int) []byte

BuildOpenAIResponsesStreamFailedChunk builds the terminal Responses event used by official Codex clients. It is intentionally separate from BuildOpenAIResponsesStreamErrorChunk so existing clients keep the legacy shape.

func EnrichContextWithSessionHierarchy

func EnrichContextWithSessionHierarchy(ctx context.Context, headers http.Header, payload []byte, metadata map[string]any) context.Context

EnrichContextWithSessionHierarchy extracts canonical session and parent session identities from headers, payload, and metadata and records them in ClientRequestMetadata.

func ExecutionErrorMessage

func ExecutionErrorMessage(err error) *interfaces.ErrorMessage

func FilterUpstreamHeaders

func FilterUpstreamHeaders(src http.Header) http.Header

FilterUpstreamHeaders returns a copy of src with hop-by-hop and security-sensitive headers removed. Returns nil if src is nil or empty after filtering.

func IsCPAReservedResponseHeader

func IsCPAReservedResponseHeader(name string) bool

IsCPAReservedResponseHeader reports whether a downstream response header is managed by CPA.

func NonStreamingKeepAliveInterval

func NonStreamingKeepAliveInterval(cfg *config.SDKConfig) time.Duration

NonStreamingKeepAliveInterval returns the keep-alive interval for non-streaming responses. Returning 0 disables keep-alives (default when unset).

func PassthroughHeadersEnabled

func PassthroughHeadersEnabled(cfg *config.SDKConfig) bool

PassthroughHeadersEnabled returns whether upstream response headers should be forwarded to clients. Default is false.

func PendingStreamError

func PendingStreamError(errs <-chan *interfaces.ErrorMessage) (*interfaces.ErrorMessage, bool)

PendingStreamError returns an immediately available non-nil stream error.

func PreparedStreamPluginExecutor

func PreparedStreamPluginExecutor(ctx context.Context) string

PreparedStreamPluginExecutor returns the executor plugin ID if the prepared route targets a plugin executor.

func PreparedStreamProviderRoute

func PreparedStreamProviderRoute(ctx context.Context) (string, string)

PreparedStreamProviderRoute returns the provider and target model if the prepared route targets a provider route override.

func ReadRequestBody

func ReadRequestBody(c *gin.Context) ([]byte, error)

ReadRequestBody reads the incoming request body and decodes supported Content-Encoding values before handlers inspect JSON fields.

func StreamingBootstrapRetries

func StreamingBootstrapRetries(cfg *config.SDKConfig) int

StreamingBootstrapRetries returns how many times a streaming request may be retried before any bytes are sent.

func StreamingKeepAliveInterval

func StreamingKeepAliveInterval(cfg *config.SDKConfig) time.Duration

StreamingKeepAliveInterval returns the streaming keep-alive interval for this server (SSE heartbeats and WebSocket Ping frames). Returning 0 disables keep-alives (default when unset).

func WithDisallowFreeAuth

func WithDisallowFreeAuth(ctx context.Context) context.Context

WithDisallowFreeAuth returns a child context that requests skipping known free-tier credentials.

func WithExecutionSessionID

func WithExecutionSessionID(ctx context.Context, sessionID string) context.Context

WithExecutionSessionID returns a child context tagged with a long-lived execution session ID.

func WithPinnedAuthID

func WithPinnedAuthID(ctx context.Context, authID string) context.Context

WithPinnedAuthID returns a child context that requests execution on a specific auth ID.

func WithSelectedAuthIDCallback

func WithSelectedAuthIDCallback(ctx context.Context, callback func(string)) context.Context

WithSelectedAuthIDCallback returns a child context that receives the selected auth ID.

func WriteUpstreamHeaders

func WriteUpstreamHeaders(dst http.Header, src http.Header)

WriteUpstreamHeaders writes filtered upstream headers to the gin response writer. Headers already set by CPA (e.g., Content-Type) are NOT overwritten.

Types

type APIHandlerCancelFunc

type APIHandlerCancelFunc func(params ...interface{})

APIHandlerCancelFunc is a function type for canceling an API handler's context. It can optionally accept parameters, which are used for logging the response.

type BaseAPIHandler

type BaseAPIHandler struct {
	// AuthManager manages auth lifecycle and execution in the new architecture.
	AuthManager *coreauth.Manager

	// Cfg holds the current application configuration.
	Cfg *config.SDKConfig

	// PluginHost optionally applies plugin interceptors around upstream execution.
	PluginHost PluginInterceptorHost

	// ModelRouterHost optionally routes matching requests to a plugin executor, the router's own
	// executor, or a built-in provider before model-to-provider resolution and auth selection.
	ModelRouterHost PluginModelRouterHost
}

BaseAPIHandler contains the handlers for API endpoints. It holds a pool of clients to interact with the backend service and manages load balancing, client selection, and configuration.

func NewBaseAPIHandlers

func NewBaseAPIHandlers(cfg *config.SDKConfig, authManager *coreauth.Manager) *BaseAPIHandler

NewBaseAPIHandlers creates a new API handlers instance. It takes a slice of clients and configuration as input.

Parameters:

  • cliClients: A slice of AI service clients
  • cfg: The application configuration

Returns:

  • *BaseAPIHandler: A new API handlers instance

func (*BaseAPIHandler) ExecuteCountWithAuthManager

func (h *BaseAPIHandler) ExecuteCountWithAuthManager(ctx context.Context, handlerType, modelName string, rawJSON []byte, alt string) ([]byte, http.Header, *interfaces.ErrorMessage)

ExecuteCountWithAuthManager executes a non-streaming request via the core auth manager. This path is the only supported execution route.

func (*BaseAPIHandler) ExecuteImageStreamWithAuthManager

func (h *BaseAPIHandler) ExecuteImageStreamWithAuthManager(ctx context.Context, handlerType, modelName string, rawJSON []byte, alt string) (<-chan []byte, http.Header, <-chan *interfaces.ErrorMessage)

ExecuteImageStreamWithAuthManager executes a streaming OpenAI-compatible image endpoint request.

func (*BaseAPIHandler) ExecuteImageWithAuthManager

func (h *BaseAPIHandler) ExecuteImageWithAuthManager(ctx context.Context, handlerType, modelName string, rawJSON []byte, alt string) ([]byte, http.Header, *interfaces.ErrorMessage)

ExecuteImageWithAuthManager executes an OpenAI-compatible image endpoint request.

func (*BaseAPIHandler) ExecuteModel

ExecuteModel executes an internal non-streaming model request. Host model callbacks are non-recursive for their caller: when skip plugin IDs are set, that plugin's interceptors and router are skipped for the nested model execution while other plugins may still run.

func (*BaseAPIHandler) ExecuteModelStream

ExecuteModelStream executes an internal streaming model request. Host model callbacks are non-recursive for their caller: when skip plugin IDs are set, that plugin's interceptors and router are skipped for the nested model execution while other plugins may still run.

func (*BaseAPIHandler) ExecuteProtocolStreamWithAuthManager

func (h *BaseAPIHandler) ExecuteProtocolStreamWithAuthManager(ctx context.Context, req ProtocolExecutionRequest) (ModelExecutionStream, *interfaces.ErrorMessage)

ExecuteProtocolStreamWithAuthManager executes a route-level streaming request with explicit protocols.

func (*BaseAPIHandler) ExecuteProtocolWithAuthManager

ExecuteProtocolWithAuthManager executes a route-level non-streaming request with explicit protocols.

func (*BaseAPIHandler) ExecuteStreamWithAuthManager

func (h *BaseAPIHandler) ExecuteStreamWithAuthManager(ctx context.Context, handlerType, modelName string, rawJSON []byte, alt string) (<-chan []byte, http.Header, <-chan *interfaces.ErrorMessage)

ExecuteStreamWithAuthManager executes a streaming request via the core auth manager. This path is the only supported execution route. The returned http.Header carries upstream response headers captured before streaming begins.

func (*BaseAPIHandler) ExecuteWithAuthManager

func (h *BaseAPIHandler) ExecuteWithAuthManager(ctx context.Context, handlerType, modelName string, rawJSON []byte, alt string) ([]byte, http.Header, *interfaces.ErrorMessage)

ExecuteWithAuthManager executes a non-streaming request via the core auth manager. This path is the only supported execution route.

func (*BaseAPIHandler) ForwardStream

func (h *BaseAPIHandler) ForwardStream(c *gin.Context, flusher http.Flusher, cancel func(error), data <-chan []byte, errs <-chan *interfaces.ErrorMessage, opts StreamForwardOptions)

func (*BaseAPIHandler) GetAlt

func (h *BaseAPIHandler) GetAlt(c *gin.Context) string

GetAlt extracts the 'alt' parameter from the request query string. It checks both 'alt' and '$alt' parameters and returns the appropriate value.

Parameters:

  • c: The Gin context containing the HTTP request

Returns:

  • string: The alt parameter value, or empty string if it's "sse"

func (*BaseAPIHandler) GetContextWithCancel

func (h *BaseAPIHandler) GetContextWithCancel(handler interfaces.APIHandler, c *gin.Context, ctx context.Context) (context.Context, APIHandlerCancelFunc)

GetContextWithCancel creates a new context with cancellation capabilities. It embeds the Gin context and the API handler into the new context for later use. The returned cancel function also handles logging the API response if request logging is enabled.

Parameters:

  • handler: The API handler associated with the request.
  • c: The Gin context of the current request.
  • ctx: The parent context (caller values/deadlines are preserved; request context adds cancellation and request ID).

Returns:

  • context.Context: The new context with cancellation and embedded values.
  • APIHandlerCancelFunc: A function to cancel the context and log the response.

func (*BaseAPIHandler) LoggingAPIResponseError

func (h *BaseAPIHandler) LoggingAPIResponseError(ctx context.Context, err *interfaces.ErrorMessage)

func (*BaseAPIHandler) PrepareStreamModelRoute

func (h *BaseAPIHandler) PrepareStreamModelRoute(ctx context.Context, handlerType string, modelName string, rawJSON []byte) (context.Context, bool)

PrepareStreamModelRoute resolves a stream route once and stores it on the returned context for execution. The boolean reports whether the route overrides normal model-to-provider resolution.

func (*BaseAPIHandler) SetModelRouterHost

func (h *BaseAPIHandler) SetModelRouterHost(host PluginModelRouterHost)

SetModelRouterHost configures the optional plugin model router host.

func (*BaseAPIHandler) SetPluginHost

func (h *BaseAPIHandler) SetPluginHost(host PluginInterceptorHost)

SetPluginHost configures the optional plugin interceptor host.

func (*BaseAPIHandler) StartNonStreamingKeepAlive

func (h *BaseAPIHandler) StartNonStreamingKeepAlive(c *gin.Context, ctx context.Context) func()

StartNonStreamingKeepAlive emits blank lines every 5 seconds while waiting for a non-streaming response. It returns a stop function that must be called before writing the final response.

func (*BaseAPIHandler) SupportsApplyPatchModel added in v8.0.9

func (h *BaseAPIHandler) SupportsApplyPatchModel(model string) bool

SupportsApplyPatchModel follows the same public, non-image routing as execution.

func (*BaseAPIHandler) UpdateClients

func (h *BaseAPIHandler) UpdateClients(cfg *config.SDKConfig)

UpdateClients updates the handlers' client list and configuration. This method is called when the configuration or authentication tokens change.

Parameters:

  • clients: The new slice of AI service clients
  • cfg: The new application configuration

func (*BaseAPIHandler) WriteErrorResponse

func (h *BaseAPIHandler) WriteErrorResponse(c *gin.Context, msg *interfaces.ErrorMessage)

WriteErrorResponse writes an error message to the response writer using the HTTP status embedded in the message.

func (*BaseAPIHandler) WriteModelListResponse

func (h *BaseAPIHandler) WriteModelListResponse(c *gin.Context, sourceFormat string, payload any)

WriteModelListResponse serializes the model-list payload, applies plugin response interceptors if a plugin host is configured, and writes the resulting headers and body to the Gin context.

type ErrorDetail

type ErrorDetail struct {
	// Message is a human-readable message providing more details about the error.
	Message string `json:"message"`

	// Type is the category of error that occurred (e.g., "invalid_request_error").
	Type string `json:"type"`

	// Code is a short code identifying the error, if applicable.
	Code string `json:"code,omitempty"`

	// Retryable optionally indicates whether a retry might fix the issue automatically.
	Retryable *bool `json:"retryable,omitempty"`
}

ErrorDetail provides specific information about an error that occurred. It includes a human-readable message, an error type, and an optional error code.

type ErrorResponse

type ErrorResponse struct {
	// Error contains detailed information about the error that occurred.
	Error ErrorDetail `json:"error"`
}

ErrorResponse represents a standard error response format for the API. It contains a single ErrorDetail field.

type ModelExecutionChunk

type ModelExecutionChunk struct {
	Payload []byte
	Err     *ModelExecutionStreamError
}

ModelExecutionChunk carries either a streaming payload or a terminal stream error.

type ModelExecutionRequest

type ModelExecutionRequest struct {
	EntryProtocol           string
	ExitProtocol            string
	Model                   string
	Stream                  bool
	Body                    []byte
	Headers                 http.Header
	Query                   url.Values
	Alt                     string
	SkipInterceptorPluginID string
	SkipRouterPluginID      string
	ForcedProvider          string
	AuthID                  string
	ProxyURL                string
	Path                    string
}

ModelExecutionRequest describes an internal model execution request.

type ModelExecutionResponse

type ModelExecutionResponse struct {
	StatusCode int
	Headers    http.Header
	Body       []byte
}

ModelExecutionResponse describes a non-streaming internal model execution response.

type ModelExecutionStream

type ModelExecutionStream struct {
	StatusCode int
	Headers    http.Header
	Chunks     <-chan ModelExecutionChunk
}

ModelExecutionStream describes a streaming internal model execution response.

type ModelExecutionStreamError

type ModelExecutionStreamError struct {
	StatusCode int         `json:"status_code"`
	Message    string      `json:"message"`
	Headers    http.Header `json:"headers"`
}

ModelExecutionStreamError carries a JSON-friendly terminal stream error.

func (*ModelExecutionStreamError) Error

func (e *ModelExecutionStreamError) Error() string

Error returns the stream error message or the HTTP status text.

type PluginExecutorHost

PluginExecutorHost executes a routed request with a specific plugin executor.

type PluginInterceptorHost

PluginInterceptorHost applies plugin interceptors around handler execution.

type PluginModelRouterHost

type PluginModelRouterHost interface {
	RouteModel(context.Context, pluginapi.ModelRouteRequest) (pluginapi.ModelRouteResponse, bool)
}

PluginModelRouterHost routes matching requests to a plugin executor, the router's own executor, or a built-in provider before model-to-provider resolution and auth selection.

type ProtocolExecutionRequest

type ProtocolExecutionRequest struct {
	EntryProtocol      string
	ExitProtocol       string
	ForcedProvider     string
	AuthSelectionModel string
	Model              string
	Stream             bool
	Body               []byte
	Headers            http.Header
	Query              url.Values
	Alt                string
	Path               string
}

ProtocolExecutionRequest describes a route-level model execution request with explicit protocols.

type StreamForwardOptions

type StreamForwardOptions struct {
	// KeepAliveInterval overrides the configured streaming keep-alive interval.
	// If nil, the configured default is used. If set to <= 0, keep-alives are disabled.
	KeepAliveInterval *time.Duration

	// WriteChunk writes a single data chunk to the response body. It should not flush.
	WriteChunk func(chunk []byte)

	// ChunkError optionally reports that WriteChunk emitted a terminal failure.
	// The failure is passed to cancel without writing another terminal payload.
	ChunkError func() *interfaces.ErrorMessage

	// ChunkDone reports successful terminal delivery after WriteChunk and Flush.
	ChunkDone func() bool

	// Flush optionally exposes transport flush errors hidden by http.Flusher.
	Flush func() error

	// NormalizeTerminalError optionally replaces an upstream error before it is
	// written or passed to cancel.
	NormalizeTerminalError func(errMsg *interfaces.ErrorMessage) *interfaces.ErrorMessage

	// WriteTerminalError writes an error payload to the response body when streaming fails
	// after headers have already been committed. It should not flush.
	WriteTerminalError func(errMsg *interfaces.ErrorMessage)

	// CloseError optionally validates a clean upstream channel close before WriteDone.
	// Returning an error surfaces it through WriteTerminalError instead of completing the stream.
	CloseError func() *interfaces.ErrorMessage

	// WriteDone optionally writes a terminal marker when the upstream data channel closes
	// without an error (e.g. OpenAI's `[DONE]`). It should not flush.
	WriteDone func()

	// WriteKeepAlive optionally writes a keep-alive heartbeat. It should not flush.
	// When nil, a standard SSE comment heartbeat is used.
	WriteKeepAlive func()
}

Directories

Path Synopsis
Package claude provides HTTP handlers for Claude API code-related functionality.
Package claude provides HTTP handlers for Claude API code-related functionality.
Package gemini provides HTTP handlers for Gemini API endpoints.
Package gemini provides HTTP handlers for Gemini API endpoints.
Package openai provides HTTP handlers for OpenAI API endpoints.
Package openai provides HTTP handlers for OpenAI API endpoints.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL