handlers

package
v7.2.8 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 16, 2026 License: MIT Imports: 25 Imported by: 0

Documentation

Overview

Package handlers provides core API handler functionality for the CLI Proxy API server. It includes common types, client management, load balancing, and error handling shared across all API endpoint handlers (OpenAI, Claude, Gemini).

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func BuildErrorResponseBody 

func BuildErrorResponseBody(status int, errText string) []byte

BuildErrorResponseBody builds an OpenAI-compatible JSON error response body. If errText is already valid JSON, it is returned as-is to preserve upstream error payloads.

func BuildOpenAIResponsesStreamErrorChunk 

func BuildOpenAIResponsesStreamErrorChunk(status int, errText string, sequenceNumber int) []byte

BuildOpenAIResponsesStreamErrorChunk builds an OpenAI Responses streaming error chunk.

Important: OpenAI's HTTP error bodies are shaped like {"error":{...}}; those are valid for non-streaming responses, but streaming clients validate SSE `data:` payloads against a union of chunks that requires a top-level `type` field.

func FilterUpstreamHeaders 

func FilterUpstreamHeaders(src http.Header) http.Header

FilterUpstreamHeaders returns a copy of src with hop-by-hop and security-sensitive headers removed. Returns nil if src is nil or empty after filtering.

func NonStreamingKeepAliveInterval 

func NonStreamingKeepAliveInterval(cfg *config.SDKConfig) time.Duration

NonStreamingKeepAliveInterval returns the keep-alive interval for non-streaming responses. Returning 0 disables keep-alives (default when unset).

func PassthroughHeadersEnabled 

func PassthroughHeadersEnabled(cfg *config.SDKConfig) bool

PassthroughHeadersEnabled returns whether upstream response headers should be forwarded to clients. Default is false.

func ReadRequestBody added in v7.0.8 

func ReadRequestBody(c *gin.Context) ([]byte, error)

ReadRequestBody reads the incoming request body and decodes supported Content-Encoding values before handlers inspect JSON fields.

func StreamingBootstrapRetries 

func StreamingBootstrapRetries(cfg *config.SDKConfig) int

StreamingBootstrapRetries returns how many times a streaming request may be retried before any bytes are sent.

func StreamingKeepAliveInterval 

func StreamingKeepAliveInterval(cfg *config.SDKConfig) time.Duration

StreamingKeepAliveInterval returns the SSE keep-alive interval for this server. Returning 0 disables keep-alives (default when unset).

func WithDisallowFreeAuth 

func WithDisallowFreeAuth(ctx context.Context) context.Context

WithDisallowFreeAuth returns a child context that requests skipping known free-tier credentials.

func WithExecutionSessionID 

func WithExecutionSessionID(ctx context.Context, sessionID string) context.Context

WithExecutionSessionID returns a child context tagged with a long-lived execution session ID.

func WithPinnedAuthID 

func WithPinnedAuthID(ctx context.Context, authID string) context.Context

WithPinnedAuthID returns a child context that requests execution on a specific auth ID.

func WithSelectedAuthIDCallback 

func WithSelectedAuthIDCallback(ctx context.Context, callback func(string)) context.Context

WithSelectedAuthIDCallback returns a child context that receives the selected auth ID.

func WriteUpstreamHeaders 

func WriteUpstreamHeaders(dst http.Header, src http.Header)

WriteUpstreamHeaders writes filtered upstream headers to the gin response writer. Headers already set by CPA (e.g., Content-Type) are NOT overwritten.

Types

type APIHandlerCancelFunc 

type APIHandlerCancelFunc func(params ...interface{})

APIHandlerCancelFunc is a function type for canceling an API handler's context. It can optionally accept parameters, which are used for logging the response.

type BaseAPIHandler 

type BaseAPIHandler struct {
	// AuthManager manages auth lifecycle and execution in the new architecture.
	AuthManager *coreauth.Manager

	// Cfg holds the current application configuration.
	Cfg *config.SDKConfig

	// PluginHost optionally applies plugin interceptors around upstream execution.
	PluginHost PluginInterceptorHost

	// ModelRouterHost optionally routes matching requests to a plugin executor, the router's own
	// executor, or a built-in provider before model-to-provider resolution and auth selection.
	ModelRouterHost PluginModelRouterHost
}

BaseAPIHandler contains the handlers for API endpoints. It holds a pool of clients to interact with the backend service and manages load balancing, client selection, and configuration.

func NewBaseAPIHandlers 

func NewBaseAPIHandlers(cfg *config.SDKConfig, authManager *coreauth.Manager) *BaseAPIHandler

NewBaseAPIHandlers creates a new API handlers instance. It takes a slice of clients and configuration as input.

Parameters:

  • cliClients: A slice of AI service clients
  • cfg: The application configuration

Returns:

  • *BaseAPIHandler: A new API handlers instance

func (*BaseAPIHandler) ExecuteCountWithAuthManager 

func (h *BaseAPIHandler) ExecuteCountWithAuthManager(ctx context.Context, handlerType, modelName string, rawJSON []byte, alt string) ([]byte, http.Header, *interfaces.ErrorMessage)

ExecuteCountWithAuthManager executes a non-streaming request via the core auth manager. This path is the only supported execution route.

func (*BaseAPIHandler) ExecuteImageStreamWithAuthManager added in v7.1.15 

func (h *BaseAPIHandler) ExecuteImageStreamWithAuthManager(ctx context.Context, handlerType, modelName string, rawJSON []byte, alt string) (<-chan []byte, http.Header, <-chan *interfaces.ErrorMessage)

ExecuteImageStreamWithAuthManager executes a streaming OpenAI-compatible image endpoint request.

func (*BaseAPIHandler) ExecuteImageWithAuthManager added in v7.1.15 

func (h *BaseAPIHandler) ExecuteImageWithAuthManager(ctx context.Context, handlerType, modelName string, rawJSON []byte, alt string) ([]byte, http.Header, *interfaces.ErrorMessage)

ExecuteImageWithAuthManager executes an OpenAI-compatible image endpoint request.

func (*BaseAPIHandler) ExecuteModel added in v7.1.67 

func (h *BaseAPIHandler) ExecuteModel(ctx context.Context, req ModelExecutionRequest) (ModelExecutionResponse, *interfaces.ErrorMessage)

ExecuteModel executes an internal non-streaming model request. Host model callbacks are non-recursive for their caller: when skip plugin IDs are set, that plugin's interceptors and router are skipped for the nested model execution while other plugins may still run.

func (*BaseAPIHandler) ExecuteModelStream added in v7.1.67 

func (h *BaseAPIHandler) ExecuteModelStream(ctx context.Context, req ModelExecutionRequest) (ModelExecutionStream, *interfaces.ErrorMessage)

ExecuteModelStream executes an internal streaming model request. Host model callbacks are non-recursive for their caller: when skip plugin IDs are set, that plugin's interceptors and router are skipped for the nested model execution while other plugins may still run.

func (*BaseAPIHandler) ExecuteStreamWithAuthManager 

func (h *BaseAPIHandler) ExecuteStreamWithAuthManager(ctx context.Context, handlerType, modelName string, rawJSON []byte, alt string) (<-chan []byte, http.Header, <-chan *interfaces.ErrorMessage)

ExecuteStreamWithAuthManager executes a streaming request via the core auth manager. This path is the only supported execution route. The returned http.Header carries upstream response headers captured before streaming begins.

func (*BaseAPIHandler) ExecuteWithAuthManager 

func (h *BaseAPIHandler) ExecuteWithAuthManager(ctx context.Context, handlerType, modelName string, rawJSON []byte, alt string) ([]byte, http.Header, *interfaces.ErrorMessage)

ExecuteWithAuthManager executes a non-streaming request via the core auth manager. This path is the only supported execution route.

func (*BaseAPIHandler) ForwardStream 

func (h *BaseAPIHandler) ForwardStream(c *gin.Context, flusher http.Flusher, cancel func(error), data <-chan []byte, errs <-chan *interfaces.ErrorMessage, opts StreamForwardOptions)

func (*BaseAPIHandler) GetAlt 

func (h *BaseAPIHandler) GetAlt(c *gin.Context) string

GetAlt extracts the 'alt' parameter from the request query string. It checks both 'alt' and '$alt' parameters and returns the appropriate value.

Parameters:

  • c: The Gin context containing the HTTP request

Returns:

  • string: The alt parameter value, or empty string if it's "sse"

func (*BaseAPIHandler) GetContextWithCancel 

func (h *BaseAPIHandler) GetContextWithCancel(handler interfaces.APIHandler, c *gin.Context, ctx context.Context) (context.Context, APIHandlerCancelFunc)

GetContextWithCancel creates a new context with cancellation capabilities. It embeds the Gin context and the API handler into the new context for later use. The returned cancel function also handles logging the API response if request logging is enabled.

Parameters:

  • handler: The API handler associated with the request.
  • c: The Gin context of the current request.
  • ctx: The parent context (caller values/deadlines are preserved; request context adds cancellation and request ID).

Returns:

  • context.Context: The new context with cancellation and embedded values.
  • APIHandlerCancelFunc: A function to cancel the context and log the response.

func (*BaseAPIHandler) LoggingAPIResponseError 

func (h *BaseAPIHandler) LoggingAPIResponseError(ctx context.Context, err *interfaces.ErrorMessage)

func (*BaseAPIHandler) SetModelRouterHost added in v7.2.8 

func (h *BaseAPIHandler) SetModelRouterHost(host PluginModelRouterHost)

SetModelRouterHost configures the optional plugin model router host.

func (*BaseAPIHandler) SetPluginHost added in v7.1.55 

func (h *BaseAPIHandler) SetPluginHost(host PluginInterceptorHost)

SetPluginHost configures the optional plugin interceptor host.

func (*BaseAPIHandler) StartNonStreamingKeepAlive 

func (h *BaseAPIHandler) StartNonStreamingKeepAlive(c *gin.Context, ctx context.Context) func()

StartNonStreamingKeepAlive emits blank lines every 5 seconds while waiting for a non-streaming response. It returns a stop function that must be called before writing the final response.

func (*BaseAPIHandler) UpdateClients 

func (h *BaseAPIHandler) UpdateClients(cfg *config.SDKConfig)

UpdateClients updates the handlers' client list and configuration. This method is called when the configuration or authentication tokens change.

Parameters:

  • clients: The new slice of AI service clients
  • cfg: The new application configuration

func (*BaseAPIHandler) WriteErrorResponse 

func (h *BaseAPIHandler) WriteErrorResponse(c *gin.Context, msg *interfaces.ErrorMessage)

WriteErrorResponse writes an error message to the response writer using the HTTP status embedded in the message.

type ErrorDetail 

type ErrorDetail struct {
	// Message is a human-readable message providing more details about the error.
	Message string `json:"message"`

	// Type is the category of error that occurred (e.g., "invalid_request_error").
	Type string `json:"type"`

	// Code is a short code identifying the error, if applicable.
	Code string `json:"code,omitempty"`
}

ErrorDetail provides specific information about an error that occurred. It includes a human-readable message, an error type, and an optional error code.

type ErrorResponse 

type ErrorResponse struct {
	// Error contains detailed information about the error that occurred.
	Error ErrorDetail `json:"error"`
}

ErrorResponse represents a standard error response format for the API. It contains a single ErrorDetail field.

type ModelExecutionChunk added in v7.1.67 

type ModelExecutionChunk struct {
	Payload []byte
	Err     *ModelExecutionStreamError
}

ModelExecutionChunk carries either a streaming payload or a terminal stream error.

type ModelExecutionRequest added in v7.1.67 

type ModelExecutionRequest struct {
	EntryProtocol           string
	ExitProtocol            string
	Model                   string
	Stream                  bool
	Body                    []byte
	Headers                 http.Header
	Query                   url.Values
	Alt                     string
	SkipInterceptorPluginID string
	SkipRouterPluginID      string
}

ModelExecutionRequest describes an internal model execution request.

type ModelExecutionResponse added in v7.1.67 

type ModelExecutionResponse struct {
	StatusCode int
	Headers    http.Header
	Body       []byte
}

ModelExecutionResponse describes a non-streaming internal model execution response.

type ModelExecutionStream added in v7.1.67 

type ModelExecutionStream struct {
	StatusCode int
	Headers    http.Header
	Chunks     <-chan ModelExecutionChunk
}

ModelExecutionStream describes a streaming internal model execution response.

type ModelExecutionStreamError added in v7.1.67 

type ModelExecutionStreamError struct {
	StatusCode int         `json:"status_code"`
	Message    string      `json:"message"`
	Headers    http.Header `json:"headers"`
}

ModelExecutionStreamError carries a JSON-friendly terminal stream error.

func (*ModelExecutionStreamError) Error added in v7.1.67 

func (e *ModelExecutionStreamError) Error() string

Error returns the stream error message or the HTTP status text.

type PluginExecutorHost added in v7.2.8 

type PluginExecutorHost interface {
	ExecutePluginExecutor(context.Context, string, coreexecutor.Request, coreexecutor.Options) (coreexecutor.Response, error)
	ExecutePluginExecutorStream(context.Context, string, coreexecutor.Request, coreexecutor.Options) (*coreexecutor.StreamResult, error)
	CountPluginExecutor(context.Context, string, coreexecutor.Request, coreexecutor.Options) (coreexecutor.Response, error)
}

PluginExecutorHost executes a routed request with a specific plugin executor.

type PluginInterceptorHost added in v7.1.55 

type PluginInterceptorHost interface {
	InterceptRequestBeforeAuth(context.Context, pluginapi.RequestInterceptRequest) pluginapi.RequestInterceptResponse
	InterceptRequestAfterAuth(context.Context, pluginapi.RequestInterceptRequest) pluginapi.RequestInterceptResponse
	InterceptResponse(context.Context, pluginapi.ResponseInterceptRequest) pluginapi.ResponseInterceptResponse
	InterceptStreamChunk(context.Context, pluginapi.StreamChunkInterceptRequest) pluginapi.StreamChunkInterceptResponse
}

PluginInterceptorHost applies plugin interceptors around handler execution.

type PluginModelRouterHost added in v7.2.8 

type PluginModelRouterHost interface {
	RouteModel(context.Context, pluginapi.ModelRouteRequest) (pluginapi.ModelRouteResponse, bool)
}

PluginModelRouterHost routes matching requests to a plugin executor, the router's own executor, or a built-in provider before model-to-provider resolution and auth selection.

type StreamForwardOptions 

type StreamForwardOptions struct {
	// KeepAliveInterval overrides the configured streaming keep-alive interval.
	// If nil, the configured default is used. If set to <= 0, keep-alives are disabled.
	KeepAliveInterval *time.Duration

	// WriteChunk writes a single data chunk to the response body. It should not flush.
	WriteChunk func(chunk []byte)

	// WriteTerminalError writes an error payload to the response body when streaming fails
	// after headers have already been committed. It should not flush.
	WriteTerminalError func(errMsg *interfaces.ErrorMessage)

	// WriteDone optionally writes a terminal marker when the upstream data channel closes
	// without an error (e.g. OpenAI's `[DONE]`). It should not flush.
	WriteDone func()

	// WriteKeepAlive optionally writes a keep-alive heartbeat. It should not flush.
	// When nil, a standard SSE comment heartbeat is used.
	WriteKeepAlive func()
}

Source Files

View all Source files

Directories

Path Synopsis
Package claude provides HTTP handlers for Claude API code-related functionality.
 Package claude provides HTTP handlers for Claude API code-related functionality.
Package gemini provides HTTP handlers for Gemini CLI API functionality.
 Package gemini provides HTTP handlers for Gemini CLI API functionality.
Package openai provides HTTP handlers for OpenAI API endpoints.
 Package openai provides HTTP handlers for OpenAI API endpoints.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.