Documentation
¶
Overview ¶
Package x is an experimental package that provides the customizability of the AI Gateway filter.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrNoMatchingRule = errors.New("no matching rule found")
ErrNoMatchingRule is the error the router function must return if there is no matching rule.
Functions ¶
This section is empty.
Types ¶
type NewCustomRouterFn ¶
NewCustomRouterFn is the function signature for NewCustomRouter.
It accepts the exptproc config passed to the AI Gateway filter and returns a Router. This is called when the new configuration is loaded.
The defaultRouter can be used to delegate the calculation to the default router implementation.
var NewCustomRouter NewCustomRouterFn
NewCustomRouter is the function to create a custom router over the default router. This is nil by default and can be set by the custom build of external processor.
type Router ¶
type Router interface {
// Calculate determines the backend to route to based on the request headers.
//
// The request headers include the populated [filterapi.Config.ModelNameHeaderKey]
// with the parsed model name based on the [filterapi.Config] given to the NewCustomRouterFn.
//
// Returns the backend.
Calculate(requestHeaders map[string]string) (backend *filterapi.Backend, err error)
}
Router is the interface for the router.
Router must be goroutine-safe as it is shared across multiple requests.
Source Files