Documentation
¶
Index ¶
- Constants
- func IsClientCanceled(err error) bool
- func ListLLMApiHandlerTypes() []string
- func RegisterLLMApiHandlerType(handlerType string)
- func RequestBodyErrorStatus(err error, fallback int) int
- func RewriteLLMRoutePath(r *http.Request, prefix string) *http.Request
- func WriteDispatchError(logger *zap.Logger, protocol, routeID, model string, status int, ...) error
- func WriteHttpErrorLog(logger *zap.Logger, w http.ResponseWriter, r *http.Request, status int, ...)
- func WriteProviderErrorLog(logger *zap.Logger, w http.ResponseWriter, r *http.Request, protocol string, ...) (int, string)
- type Handler
- type HandlerOptions
- type LLMApiHandler
- type NextHandler
- type PreparedLLMApiRequest
- type ResponseFlusher
Constants ¶
const ( MaxLLMRequestBodyBytes = 16 << 20 MaxMCPRequestBodyBytes = 4 << 20 MaxACPRequestBodyBytes = 4 << 20 )
const StatusClientClosedRequest = 499
Variables ¶
This section is empty.
Functions ¶
func IsClientCanceled ¶
func ListLLMApiHandlerTypes ¶
func ListLLMApiHandlerTypes() []string
ListLLMApiHandlerTypes returns the types of all registered LLM API handlers.
func RegisterLLMApiHandlerType ¶
func RegisterLLMApiHandlerType(handlerType string)
RegisterLLMApiHandlerType registers an available LLM API handler type.
func RequestBodyErrorStatus ¶ added in v0.3.0
func RewriteLLMRoutePath ¶ added in v0.1.0
RewriteLLMRoutePath returns a cloned request with the matched LLM route prefix stripped.
func WriteDispatchError ¶ added in v0.1.0
func WriteHttpErrorLog ¶ added in v0.1.0
Types ¶
type Handler ¶
type Handler struct {
// contains filtered or unexported fields
}
Handler dispatches gateway requests to the route-selected LLM API handler.
func NewHandler ¶
func NewHandler(agentGateway *gateway.AgentGateway, apiHandlers map[string]LLMApiHandler, logger *zap.Logger, opts HandlerOptions) *Handler
NewHandler constructs a runtime dispatcher handler.
func (*Handler) Dispatch ¶
func (h *Handler) Dispatch(w http.ResponseWriter, r *http.Request, next NextHandler) error
Dispatch handles a request and optionally delegates unmatched requests to next.
type HandlerOptions ¶ added in v0.1.0
type LLMApiHandler ¶
type LLMApiHandler interface {
Name() string
MatchLLMApi(*http.Request) bool
PrepareLLMApiRequest(*http.Request) (*PreparedLLMApiRequest, llmroutepkg.RequestRequirements, error)
ServeLLMApi(http.ResponseWriter, *http.Request, provider.Provider, *PreparedLLMApiRequest) error
}
type NextHandler ¶
type NextHandler interface {
ServeHTTP(http.ResponseWriter, *http.Request) error
}
NextHandler is the small subset of a middleware next handler needed by the dispatcher.
type PreparedLLMApiRequest ¶
type PreparedLLMApiRequest struct {
Type provider.LLMApiRequestType
ChatRequest *provider.ChatRequest
EmbeddingRequest *provider.EmbeddingRequest
ResponsesRequest *provider.ResponsesRequest
StreamRequested bool
RawRequest any
}
func (*PreparedLLMApiRequest) IsValid ¶
func (r *PreparedLLMApiRequest) IsValid() bool
func (*PreparedLLMApiRequest) Model ¶
func (r *PreparedLLMApiRequest) Model() string
func (*PreparedLLMApiRequest) SetModel ¶
func (r *PreparedLLMApiRequest) SetModel(model string)
func (*PreparedLLMApiRequest) Stream ¶
func (r *PreparedLLMApiRequest) Stream() bool
type ResponseFlusher ¶ added in v0.3.0
type ResponseFlusher struct {
// contains filtered or unexported fields
}
ResponseFlusher flushes through ResponseController so wrapped middleware writers that expose capabilities via Unwrap still stream incrementally.
func NewResponseFlusher ¶ added in v0.3.0
func NewResponseFlusher(w http.ResponseWriter) ResponseFlusher
NewResponseFlusher creates a best-effort flusher for streaming responses.
func (ResponseFlusher) Flush ¶ added in v0.3.0
func (f ResponseFlusher) Flush() bool
Flush attempts to flush pending response bytes and reports whether it worked.
Source Files
Directories