ai-gateway

module

v1.2.0 Latest Latest Go to latest Published: Oct 6, 2026 License: Apache-2.0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/envoyproxy/ai-gateway

Links

Open Source Insights

README ¶

The open source control plane for AI and agent traffic, powered by Envoy.
An Agentic AI Foundation project. Formerly Envoy AI Gateway.

Documentation · Quickstart · Blog · Release Notes · Talks

Agent Router

Agent Router gives application teams one consistent, OpenAI-compatible API for every model and tool, from hosted providers to self-hosted inference and MCP servers. Platform teams keep credentials, routing, quotas, failover, and usage attribution in one place, enforced by Envoy and Envoy Gateway.

Agent Router controls. Envoy carries.

Formerly Envoy AI Gateway

Envoy AI Gateway is now Agent Router, an Agentic AI Foundation project. Same code, same maintainers, same release cadence and Apache 2.0 license. Nothing you deploy is renamed:

CRDs and API group are unchanged: AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy, and the aigateway.envoyproxy.io API group stay as they are.
The CLI is still aigw, and the envoy-ai-gateway-system namespace is unchanged.
Container images, Helm charts, and the Go module path are unchanged (docker.io/envoyproxy/ai-gateway-*, github.com/envoyproxy/ai-gateway).
The repository moved to theagentrouter/agent-router; old envoyproxy/ai-gateway links redirect.
The website moved to theagentrouter.ai; every aigateway.envoyproxy.io link redirects to the same page.

Your manifests from yesterday apply tomorrow.

Quick start

Run Agent Router as a standalone, OpenAI-compatible router on your laptop with one command:

OPENAI_API_KEY=sk-your-key aigw run

Then point any OpenAI-compatible client at http://localhost:1975/v1. See the CLI guide for installation and provider auto-configuration, or the Getting Started guide to deploy on Kubernetes with Envoy Gateway.

Usage

When using Agent Router, we refer to a two-tier gateway pattern. The Tier One Gateway functions as a centralized entry point, and the Tier Two Gateway handles ingress traffic to a self-hosted model serving cluster.

The Tier One Gateway handles authentication, top-level routing, and global rate limiting
The Tier Two Gateway provides fine-grained control over self-hosted model access, with endpoint picker support for LLM inference optimization.

Supported AI Providers

Agent Router supports a wide range of AI providers, making it easy to integrate with your preferred LLM services:

_OpenAI	_{Azure OpenAI}	_{Google Gemini}	_{Vertex AI}	_{AWS Bedrock}
_Mistral	_Cohere	_Groq	_{Together AI}	_DeepInfra
_DeepSeek	_Hunyuan	_SambaNova	_Grok	_{Tetrate Agent Router Service}
_Anthropic

Documentation

Documentation for Agent Router.
Quickstart to use Agent Router in a few simple steps.
CLI guide to run Agent Router standalone with aigw run.
Concepts to understand the architecture and resources of Agent Router.
Blog for announcements, features, and reference architectures.
Talks and Presentations about Agent Router.

Contact

Chat: join the Agent Router Discord to ask questions and collaborate with the community.
Weekly meeting: the maintainers and community meet every Monday. Register, add agenda points, and get involved via the public meeting document.

Get Involved

We follow the LF Projects Code of Conduct, as an Agentic AI Foundation project.

To contribute to the project via pull requests, please read the CONTRIBUTING.md file which includes information on how to build and test the project.

Background

The proposal of using Envoy Gateway as a Cloud Native LLM Gateway inspired the initiation of this project.

Agent Router is an Agentic AI Foundation project, powered by Envoy. Copyright © Agent Router a Series of LF Projects, LLC. For web site terms of use, trademark policy and other project policies please see https://lfprojects.org.

Directories ¶

Path	Synopsis
api
v1alpha1 Package v1alpha1 contains API schema definitions for the aigateway.envoyproxy.io API group.	Package v1alpha1 contains API schema definitions for the aigateway.envoyproxy.io API group.
v1alpha1/client/clientset/versioned
v1alpha1/client/clientset/versioned/fake This package has the automatically generated fake clientset.	This package has the automatically generated fake clientset.
v1alpha1/client/clientset/versioned/scheme This package contains the scheme of the automatically generated clientset.	This package contains the scheme of the automatically generated clientset.
v1alpha1/client/clientset/versioned/typed/api/v1alpha1 This package has the automatically generated typed clients.	This package has the automatically generated typed clients.
v1alpha1/client/clientset/versioned/typed/api/v1alpha1/fake Package fake has the automatically generated clients.	Package fake has the automatically generated clients.
v1alpha1/client/informers/externalversions
v1alpha1/client/informers/externalversions/api
v1alpha1/client/informers/externalversions/api/v1alpha1
v1alpha1/client/informers/externalversions/internalinterfaces
v1alpha1/client/listers/api/v1alpha1
v1beta1 Package v1beta1 contains API schema definitions for the aigateway.envoyproxy.io API group.	Package v1beta1 contains API schema definitions for the aigateway.envoyproxy.io API group.
v1beta1/client/clientset/versioned
v1beta1/client/clientset/versioned/fake This package has the automatically generated fake clientset.	This package has the automatically generated fake clientset.
v1beta1/client/clientset/versioned/scheme This package contains the scheme of the automatically generated clientset.	This package contains the scheme of the automatically generated clientset.
v1beta1/client/clientset/versioned/typed/api/v1beta1 This package has the automatically generated typed clients.	This package has the automatically generated typed clients.
v1beta1/client/clientset/versioned/typed/api/v1beta1/fake Package fake has the automatically generated clients.	Package fake has the automatically generated clients.
v1beta1/client/informers/externalversions
v1beta1/client/informers/externalversions/api
v1beta1/client/informers/externalversions/api/v1beta1
v1beta1/client/informers/externalversions/internalinterfaces
v1beta1/client/listers/api/v1beta1
cmd
aigw command
controller command
extproc command
extproc/mainlib
internal
apischema/anthropic
apischema/awsbedrock
apischema/cohere Package cohere contains Cohere API schema definitions.	Package cohere contains Cohere API schema definitions.
apischema/gcp
apischema/openai Package openai contains the following is the OpenAI API schema definitions.	Package openai contains the following is the OpenAI API schema definitions.
apischema/openai/tokenize Package tokenize contains Tokenize API schema definitions.	Package tokenize contains Tokenize API schema definitions.
apischema/typesafe Package typesafe contains TypeSafe AI API schema definitions.	Package typesafe contains TypeSafe AI API schema definitions.
autoconfig
backendauth
bodymutator
controller
controller/rotators Package rotators provides credential rotation implementations.	Package rotators provides credential rotation implementations.
controller/tokenprovider
endpointspec Package endpointspec defines the EndpointSpec which is to bundle the translator, tracing and most importantly request and response types for different API endpoints.	Package endpointspec defines the EndpointSpec which is to bundle the translator, tracing and most importantly request and response types for different API endpoints.
extensionserver
extproc
filterapi Package filterapi provides the configuration for the AI Gateway-implemented filter which is currently an external processor (See https://github.com/envoyproxy/ai-gateway/issues/90).	Package filterapi provides the configuration for the AI Gateway-implemented filter which is currently an external processor (See https://github.com/envoyproxy/ai-gateway/issues/90).
gcpauth Package gcpauth provides shared GCP authentication utilities.	Package gcpauth provides shared GCP authentication utilities.
headermutator
internalapi Package internalapi provides constants and functions used across the boundary among controller, extension server and extproc.	Package internalapi provides constants and functions used across the boundary among controller, extension server and extproc.
json
lang
llmcostcel Package llmcostcel provides functions to create and evaluate CEL programs to calculate costs.	Package llmcostcel provides functions to create and evaluate CEL programs to calculate costs.
mcpproxy
metrics
pprof
ratelimit/runner
ratelimit/translator
redaction Package redaction provides utilities for redacting sensitive information from requests and responses for safe debug logging.	Package redaction provides utilities for redacting sensitive information from requests and responses for safe debug logging.
requestheaderattrs
testing
testing/testotel Package testotel provides test utilities for OpenTelemetry tracing, metrics, and logs tests.	Package testotel provides test utilities for OpenTelemetry tracing, metrics, and logs tests.
tracing
tracing/httperr Package httperr maps upstream HTTP status codes to the error identifiers each tracing semantic convention reports.	Package httperr maps upstream HTTP status codes to the error identifiers each tracing semantic convention reports.
tracing/openinference Package openinference provides OpenInference semantic conventions for OpenTelemetry tracing.	Package openinference provides OpenInference semantic conventions for OpenTelemetry tracing.
tracing/openinference/anthropic
tracing/openinference/cohere Package cohere provides OpenInference semantic conventions hooks for Cohere instrumentation used by the ExtProc router filter.	Package cohere provides OpenInference semantic conventions hooks for Cohere instrumentation used by the ExtProc router filter.
tracing/openinference/openai Package openai provides OpenInference semantic conventions hooks for OpenAI instrumentation used by the ExtProc router filter.	Package openai provides OpenInference semantic conventions hooks for OpenAI instrumentation used by the ExtProc router filter.
tracing/openinference/typesafe Package typesafe provides OpenInference semantic conventions hooks for TypeSafe System One instrumentation used by the ExtProc router filter.	Package typesafe provides OpenInference semantic conventions hooks for TypeSafe System One instrumentation used by the ExtProc router filter.
tracing/otelgenai Package otelgenai implements the OpenTelemetry GenAI semantic conventions for span recording.	Package otelgenai implements the OpenTelemetry GenAI semantic conventions for span recording.
tracing/tracingapi Package tracingapi provides types for OpenTelemetry tracing support, notably to reduce chance of cyclic imports.	Package tracingapi provides types for OpenTelemetry tracing support, notably to reduce chance of cyclic imports.
tracing/tracingenv Package tracingenv provides environment variable parsing helpers shared by the tracing semantic convention implementations.	Package tracingenv provides environment variable parsing helpers shared by the tracing semantic convention implementations.
translator
version
xdg
tests
data-plane-mcp
internal
internal/dataplaneenv
internal/e2elib
internal/testextauth
internal/testextauth/testextauthserver command
internal/testmcp
internal/testmcp/testmcpserver command
internal/testopenai Package testopenai provides Azure OpenAI support for VCR cassette recording and playback.	Package testopenai provides Azure OpenAI support for VCR cassette recording and playback.
internal/testopeninference Package testopeninference provides OpenInference span recording and caching for testing AI Gateway's OpenTelemetry tracing implementation.	Package testopeninference provides OpenInference span recording and caching for testing AI Gateway's OpenTelemetry tracing implementation.
internal/testupstreamlib
internal/testupstreamlib/testupstream command
tools module

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL