Documentation
¶
Overview ¶
Package handlers provides core API handler functionality for the switchAILocal server. It includes common types, client management, load balancing, and error handling shared across all API endpoint handlers (OpenAI, Claude, Gemini).
Index ¶
- func BuildErrorResponseBody(status int, errText string) []byte
- func StreamingBootstrapRetries(cfg *config.SDKConfig) int
- func StreamingKeepAliveInterval(cfg *config.SDKConfig) time.Duration
- type APIHandlerCancelFunc
- type BaseAPIHandler
- func (h *BaseAPIHandler) Classify(ctx context.Context, prompt string) (string, error)
- func (h *BaseAPIHandler) ExecuteCountWithAuthManager(ctx context.Context, handlerType, modelName string, rawJSON []byte, alt string) ([]byte, *interfaces.ErrorMessage)
- func (h *BaseAPIHandler) ExecuteMultimodalWithAuthManager(ctx context.Context, handlerType, modelName string, rawJSON []byte, alt string, ...) ([]byte, *interfaces.ErrorMessage)
- func (h *BaseAPIHandler) ExecuteStreamMultimodalWithAuthManager(ctx context.Context, handlerType, modelName string, rawJSON []byte, alt string, ...) (<-chan []byte, <-chan *interfaces.ErrorMessage)
- func (h *BaseAPIHandler) ExecuteStreamWithAuthManager(ctx context.Context, handlerType, modelName string, rawJSON []byte, alt string) (<-chan []byte, <-chan *interfaces.ErrorMessage)
- func (h *BaseAPIHandler) ExecuteWithAuthManager(ctx context.Context, handlerType, modelName string, rawJSON []byte, alt string) ([]byte, *interfaces.ErrorMessage)
- func (h *BaseAPIHandler) ForwardStream(c *gin.Context, flusher http.Flusher, cancel func(error), data <-chan []byte, ...)
- func (h *BaseAPIHandler) GetAlt(c *gin.Context) string
- func (h *BaseAPIHandler) GetContextWithCancel(handler interfaces.APIHandler, c *gin.Context, ctx context.Context) (context.Context, APIHandlerCancelFunc)
- func (h *BaseAPIHandler) LoggingAPIResponseError(ctx context.Context, err *interfaces.ErrorMessage)
- func (h *BaseAPIHandler) UpdateClients(cfg *config.SDKConfig)
- func (h *BaseAPIHandler) WriteErrorResponse(c *gin.Context, msg *interfaces.ErrorMessage)
- type ErrorDetail
- type ErrorResponse
- type PipelineIntegrator
- type StreamForwardOptions
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func BuildErrorResponseBody ¶
BuildErrorResponseBody builds an OpenAI-compatible JSON error response body. If errText is already valid JSON, it is returned as-is to preserve upstream error payloads.
func StreamingBootstrapRetries ¶
StreamingBootstrapRetries returns how many times a streaming request may be retried before any bytes are sent.
Types ¶
type APIHandlerCancelFunc ¶
type APIHandlerCancelFunc func(params ...interface{})
APIHandlerCancelFunc is a function type for canceling an API handler's context. It can optionally accept parameters, which are used for logging the response.
type BaseAPIHandler ¶
type BaseAPIHandler struct {
// AuthManager manages auth lifecycle and execution in the new architecture.
AuthManager *coreauth.Manager
// LuaEngine manages LUA plugins for request/response modification.
LuaEngine *plugin.LuaEngine
// Cfg holds the current application configuration.
Cfg *config.SDKConfig
// PipelineIntegrator integrates steering and memory into request processing.
// This is optional - if nil, intelligent systems are disabled.
PipelineIntegrator PipelineIntegrator
// AutoResolver provides intelligent auto-routing when model="auto".
// This is optional - if nil, legacy auto-model-priority fallback is used.
AutoResolver *autoroute.AutoResolver
// CircuitBreaker manages provider-level overload protection.
CircuitBreaker *circuit.Registry
// VirtualRouter routes public virtual models (ail-compound) to concrete
// provider/model members with capability filtering.
VirtualRouter *virtualmodels.Router
}
BaseAPIHandler contains the handlers for API endpoints. It holds a pool of clients to interact with the backend service and manages load balancing, client selection, and configuration.
func NewBaseAPIHandlers ¶
func NewBaseAPIHandlers(cfg *config.SDKConfig, authManager *coreauth.Manager, luaEngine *plugin.LuaEngine, pipelineIntegrator PipelineIntegrator, autoResolver *autoroute.AutoResolver, circuitBreaker *circuit.Registry) *BaseAPIHandler
NewBaseAPIHandlers creates a new API handlers instance. It takes a slice of clients and configuration as input.
Parameters:
- cfg: The application configuration
- authManager: The authentication manager
- luaEngine: The LUA plugin engine
- pipelineIntegrator: Optional pipeline integrator for intelligent systems (can be nil)
- autoResolver: Optional intelligent routing resolver (can be nil)
Returns:
- *BaseAPIHandler: A new API handlers instance
func (*BaseAPIHandler) Classify ¶
Classify performs an LLM-based classification using the RouterModel. It is used by the Lua engine to support intelligent routing decisions.
func (*BaseAPIHandler) ExecuteCountWithAuthManager ¶
func (h *BaseAPIHandler) ExecuteCountWithAuthManager(ctx context.Context, handlerType, modelName string, rawJSON []byte, alt string) ([]byte, *interfaces.ErrorMessage)
ExecuteCountWithAuthManager executes a non-streaming request via the core auth manager. This path is the only supported execution route.
func (*BaseAPIHandler) ExecuteMultimodalWithAuthManager ¶
func (h *BaseAPIHandler) ExecuteMultimodalWithAuthManager(ctx context.Context, handlerType, modelName string, rawJSON []byte, alt string, operation string, contentType string) ([]byte, *interfaces.ErrorMessage)
ExecuteMultimodalWithAuthManager executes a multimodal request (audio/image) via the core auth manager. It injects operation and content-type metadata for the executor.
func (*BaseAPIHandler) ExecuteStreamMultimodalWithAuthManager ¶ added in v0.3.0
func (h *BaseAPIHandler) ExecuteStreamMultimodalWithAuthManager(ctx context.Context, handlerType, modelName string, rawJSON []byte, alt string, operation, contentType string) (<-chan []byte, <-chan *interfaces.ErrorMessage)
ExecuteStreamMultimodalWithAuthManager is the streaming counterpart to ExecuteMultimodalWithAuthManager: it injects operation + content-type metadata so multimodal executors (currently MiniMax music) can dispatch to a non-chat streaming path. Binary chunks (e.g. raw MP3 bytes) flow through the same []byte channel; the caller is responsible for setting the correct Content-Type on the HTTP response.
func (*BaseAPIHandler) ExecuteStreamWithAuthManager ¶
func (h *BaseAPIHandler) ExecuteStreamWithAuthManager(ctx context.Context, handlerType, modelName string, rawJSON []byte, alt string) (<-chan []byte, <-chan *interfaces.ErrorMessage)
ExecuteStreamWithAuthManager executes a streaming request via the core auth manager. This path is the only supported execution route.
func (*BaseAPIHandler) ExecuteWithAuthManager ¶
func (h *BaseAPIHandler) ExecuteWithAuthManager(ctx context.Context, handlerType, modelName string, rawJSON []byte, alt string) ([]byte, *interfaces.ErrorMessage)
ExecuteWithAuthManager executes a non-streaming request via the core auth manager. This path is the only supported execution route.
func (*BaseAPIHandler) ForwardStream ¶
func (h *BaseAPIHandler) ForwardStream(c *gin.Context, flusher http.Flusher, cancel func(error), data <-chan []byte, errs <-chan *interfaces.ErrorMessage, opts StreamForwardOptions)
func (*BaseAPIHandler) GetAlt ¶
func (h *BaseAPIHandler) GetAlt(c *gin.Context) string
GetAlt extracts the 'alt' parameter from the request query string. It checks both 'alt' and '$alt' parameters and returns the appropriate value.
Parameters:
- c: The Gin context containing the HTTP request
Returns:
- string: The alt parameter value, or empty string if it's "sse"
func (*BaseAPIHandler) GetContextWithCancel ¶
func (h *BaseAPIHandler) GetContextWithCancel(handler interfaces.APIHandler, c *gin.Context, ctx context.Context) (context.Context, APIHandlerCancelFunc)
GetContextWithCancel creates a new context with cancellation capabilities. It embeds the Gin context and the API handler into the new context for later use. The returned cancel function also handles logging the API response if request logging is enabled.
Parameters:
- handler: The API handler associated with the request.
- c: The Gin context of the current request.
- ctx: The parent context (caller values/deadlines are preserved; request context adds cancellation and request ID).
Returns:
- context.Context: The new context with cancellation and embedded values.
- APIHandlerCancelFunc: A function to cancel the context and log the response.
func (*BaseAPIHandler) LoggingAPIResponseError ¶
func (h *BaseAPIHandler) LoggingAPIResponseError(ctx context.Context, err *interfaces.ErrorMessage)
func (*BaseAPIHandler) UpdateClients ¶
func (h *BaseAPIHandler) UpdateClients(cfg *config.SDKConfig)
UpdateClients updates the handlers' client list and configuration. This method is called when the configuration or authentication tokens change.
Parameters:
- clients: The new slice of AI service clients
- cfg: The new application configuration
func (*BaseAPIHandler) WriteErrorResponse ¶
func (h *BaseAPIHandler) WriteErrorResponse(c *gin.Context, msg *interfaces.ErrorMessage)
WriteErrorResponse writes an error message to the response writer using the HTTP status embedded in the message.
type ErrorDetail ¶
type ErrorDetail struct {
// Message is a human-readable message providing more details about the error.
Message string `json:"message"`
// Type is the category of error that occurred (e.g., "invalid_request_error").
Type string `json:"type"`
// Code is a short code identifying the error, if applicable.
Code string `json:"code,omitempty"`
}
ErrorDetail provides specific information about an error that occurred. It includes a human-readable message, an error type, and an optional error code.
type ErrorResponse ¶
type ErrorResponse struct {
// Error contains detailed information about the error that occurred.
Error ErrorDetail `json:"error"`
}
ErrorResponse represents a standard error response format for the API. It contains a single ErrorDetail field.
type PipelineIntegrator ¶
type PipelineIntegrator interface {
// ApplySteering evaluates steering rules and modifies the request if rules match.
ApplySteering(ctx interface{}, messages []map[string]string) (string, []map[string]string, error)
// RecordRouting records a routing decision to the memory system.
RecordRouting(decision interface{}) error
// UpdateOutcome updates a routing decision with its outcome.
UpdateOutcome(decision interface{}) error
// EmitRoutingEvent emits a routing decision event to the event bus.
EmitRoutingEvent(decision interface{}) error
}
PipelineIntegrator defines the interface for request pipeline integration. This allows the handler to integrate with intelligent systems (steering, memory, events) without creating a direct dependency on the integration package.
type StreamForwardOptions ¶
type StreamForwardOptions struct {
// KeepAliveInterval overrides the configured streaming keep-alive interval.
// If nil, the configured default is used. If set to <= 0, keep-alives are disabled.
KeepAliveInterval *time.Duration
// WriteChunk writes a single data chunk to the response body. It should not flush.
WriteChunk func(chunk []byte)
// WriteTerminalError writes an error payload to the response body when streaming fails
// after headers have already been committed. It should not flush.
WriteTerminalError func(errMsg *interfaces.ErrorMessage)
// WriteDone optionally writes a terminal marker when the upstream data channel closes
// without an error (e.g. OpenAI's `[DONE]`). It should not flush.
WriteDone func()
// WriteKeepAlive optionally writes a keep-alive heartbeat. It should not flush.
// When nil, a standard SSE comment heartbeat is used.
WriteKeepAlive func()
}
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package claude provides HTTP handlers for Claude API code-related functionality.
|
Package claude provides HTTP handlers for Claude API code-related functionality. |
|
Package gemini provides HTTP handlers for Gemini CLI API functionality.
|
Package gemini provides HTTP handlers for Gemini CLI API functionality. |
|
Package notifications implements outbound notification relays so sandboxed clients (Tytus pods, restricted agents, etc.) can emit messages to external services like Telegram without ever holding the upstream credentials.
|
Package notifications implements outbound notification relays so sandboxed clients (Tytus pods, restricted agents, etc.) can emit messages to external services like Telegram without ever holding the upstream credentials. |
|
Package openai provides HTTP handlers for OpenAI API endpoints.
|
Package openai provides HTTP handlers for OpenAI API endpoints. |