README
¶
Envoy AI Gateway
Envoy AI Gateway is an open source project for using Envoy Gateway to handle request traffic from application clients to Generative AI services.
Contact
- Slack: Join the Envoy Slack workspace if you're not already a member. Otherwise, use the Envoy AI Gateway channel to start collaborating with the community.
Get Involved
We adhere to the CNCF Code of conduct
The Envoy AI Gateway team and community members meet every Thursday. Please register for the meeting, add agenda points, and get involved. The meeting details are available in the public document.
To contribute to the project via pull requests, please read the CONTRIBUTING.md file which includes information on how to build and test the project.
Background
The proposal of using Envoy Gateway as a Cloud Native LLM Gateway inspired the initiation of this project.
Directories
¶
| Path | Synopsis |
|---|---|
|
api
|
|
|
v1alpha1
Package v1alpha1 contains API schema definitions for the aigateway.envoyproxy.io API group.
|
Package v1alpha1 contains API schema definitions for the aigateway.envoyproxy.io API group. |
|
cmd
|
|
|
controller
command
|
|
|
extproc
command
|
|
|
examples
|
|
|
extproc_custom_router
command
|
|
|
Package filterapi provides the configuration for the AI Gateway-implemented filter which is currently an external processor (See https://github.com/envoyproxy/ai-gateway/issues/90).
|
Package filterapi provides the configuration for the AI Gateway-implemented filter which is currently an external processor (See https://github.com/envoyproxy/ai-gateway/issues/90). |
|
x
Package x is an experimental package that provides the customizability of the AI Gateway filter.
|
Package x is an experimental package that provides the customizability of the AI Gateway filter. |
|
internal
|
|
|
apischema/openai
Package openai contains the following is the OpenAI API schema definitions.
|
Package openai contains the following is the OpenAI API schema definitions. |
|
controller/rotators
Package rotators provides credential rotation implementations.
|
Package rotators provides credential rotation implementations. |
|
llmcostcel
Package llmcostcel provides functions to create and evaluate CEL programs to calculate costs.
|
Package llmcostcel provides functions to create and evaluate CEL programs to calculate costs. |
|
tests
|
|
|
tools
module
|
Click to show internal directories.
Click to hide internal directories.