plugins/

Details

Path	Synopsis
gateway
algorithms
algorithms/pd Package pd contains the scoring types used by the PD (prefill-decode) disaggregated-inference router.	Package pd contains the scoring types used by the PD (prefill-decode) disaggregated-inference router.
algorithms/pd/engine
algorithms/pd/prefill
algorithms/pd/selector Package selector defines the PodSelector contract for PD-disaggregated routing.	Package selector defines the PodSelector contract for PD-disaggregated routing.
algorithms/pd/transfer
algorithms/vtc Package vtc implements the Virtual Token Counter routing algorithms focused on fairness and utilization	Package vtc implements the Virtual Token Counter routing algorithms focused on fairness and utilization
configprofiles Package configprofiles parses the model.aibrix.ai/config annotation (or ConfigMap) and supports multiple named profiles selectable at runtime via config-profile header.	Package configprofiles parses the model.aibrix.ai/config annotation (or ConfigMap) and supports multiple named profiles selectable at runtime via config-profile header.
queue
ratelimiter
statesync Package statesync provides a generic Redis-backed state sync layer for use across multiple gateway (or other) replicas.	Package statesync provides a generic Redis-backed state sync layer for use across multiple gateway (or other) replicas.