agent-gateway

agent-gateway

agent-gateway is the open-source core (Apache 2.0) of AgentGuide — the control plane for enterprise agent systems: observe, govern, and orchestrate AI agents through one self-hosted gateway. AgentGuide is the product name; agent-gateway is this repository and the gateway engine it builds.

The gateway serves LLM, MCP, and native ACP workloads. It provides OpenAI-compatible and Anthropic-compatible ingress, route-based provider dispatch, VirtualKey authentication, dynamic config backed by SQLite, and Admin APIs for gateway operations.

This repository builds three binaries:

• agw: Caddy-based gateway runtime
• agwd: standalone gateway daemon
• agwctl: management CLI for the gateway and Caddy Admin APIs

[!IMPORTANT] When upgrading from the integrated CLI-auth implementation, install the separate agw-auth executable on the gateway's PATH, or configure its full command with credential_refresh_command (Caddy) or --credential-refresh-command plus repeatable --credential-refresh-arg flags (agwd). OAuth credential refresh is now request-driven, so the first request after an idle period can add up to 30 seconds of refresh latency. A refresh failure rejects that credential for the current attempt; the request fails if no alternate credential or configured provider authentication is available. Set AGW_AUTH_PROXY_URL or AGW_AUTH_TRANSPORT_PROFILE on the Gateway process when the refresh subprocess needs those network settings.

The store keeps the historical cliauth_credentials table name, but this breaking cutover does not translate legacy credential or route type values. Before starting the new binary against an existing SQLite store, back up the database and run:

UPDATE cliauth_credentials
SET data = json_set(data, '$.type', 'oauth_token')
WHERE json_extract(data, '$.type') = 'cliauth_token';

UPDATE routes
SET config = replace(config, '"cliauth_token"', '"oauth_token"')
WHERE config LIKE '%"cliauth_token"%';

What It Does

• expose OpenAI-compatible and Anthropic-compatible HTTP APIs
• route requests to direct providers or logical model targets
• manage providers, routes, VirtualKeys, and credentials through an Admin API
• consume oauth_token credentials created by an external tool and invoke the configured refresh command for expiring tokens on the request path
• support MCP gateway routing, discovery, execution, and runtime inspection
• expose the first native ACP control surface for codex/opencode agent routing
• host builtin agents in-process on eino ADK and serve them through the same POST /<agent-route>/turn SSE contract as ACP-backed Agents; interactive tool permissions can suspend a turn on an ADK checkpoint until a human allows or denies each call
• execute ACP and builtin Agents through one runtime-neutral backend registry and unified AgentRoutes, with shared run ids, ordered event sequencing, normalized errors, and cross-runtime usage correlation
• run with either a Caddyfile-based runtime or a standalone daemon with a config store

Architecture

agent-gateway is a multi-protocol gateway that connects, manages, and observes AI agents on a single binary. It exposes two distinct API surfaces, one for each direction of traffic:

• Agent Control API (ACP / HTTP) — how consumers reach agents. Users and business apps call the gateway to launch and drive an agent.
• Resource Access API (LLM / MCP) — how agents reach the outside world. An agent calls back through the gateway to use LLM providers and MCP tools.

A typical request flows in four hops: ① a consumer calls the gateway over HTTP / ACP → ② the gateway launches and drives an agent → ③ the agent calls back through the gateway for LLM / MCP → ④ the gateway proxies that call to upstream resources (LLM providers, MCP servers). For builtin agents, hops ② and ③ happen in-process inside the gateway host — still routed, still observed. Because both directions pass through the gateway, it adds VirtualKey auth, routing, and config across the board, and observability is first-class: every hop is captured as audit logs, token usage, and call-chain traces for multi-agent governance.

The gateway supports three integration paths for the agents themselves:

• No-code (builtin): define an agent as pure configuration — model through a gateway LLM route, tools through gateway-managed MCP services, topology (single, sequential, parallel, loop, supervisor, planexecute, deep) — and the gateway's in-process eino ADK host materializes and runs it. No process to ship, no code to write.
• Low-code: drive off-the-shelf CLI coding agents (Codex, OpenCode) directly over ACP — no custom code required. (Claude Code plugs in as a gateway consumer through the Claude Code-compatible cc API profile, not over ACP.)
• Full control: bring your own agent in any stack and let the gateway manage and observe it.

Quick Start

Build the binaries:

make build

Keep the Caddyfile minimal and declare providers, routes, and VirtualKeys in bundle YAML files applied at runtime via agwctl apply.

Create a minimal Caddyfile (no providers or routes). LLM providers and routes can also be declared statically in the Caddyfile; see docs/getting-started/quickstart-llm.md for that flow:

{
	admin localhost:2019

	agent_gateway {
		config_store sqlite {
			path ./data/configstore.db
		}
		credential_refresh_command agw-auth refresh
	}
}

http://localhost:8019 {
	route /admin/* {
		basic_auth {
			admin <hashed-password>
		}
		agent_gateway_admin
	}
}

http://127.0.0.1:8080 {
	agent_route_dispatcher {
		llm_api openai
		llm_api anthropic
		llm_api cc
		mcp
		acp
	}
}

Generate the hash with ./agw hash-password --plaintext 'your-password' and run the gateway:

./agw hash-password --plaintext 'your-password'
OPENAI_API_KEY=sk-... ./agw run --config ./Caddyfile

LLM Quick Start

Declare an LLM provider, route, and VirtualKey in a bundle YAML applied at runtime via agwctl apply.

Create a bundle file gateway.bundle.yaml:

apiVersion: gateway.agw/v1alpha1
kind: GatewayBundle
providers:
  - id: openai-main
    provider_type: openai
    api_key: ${OPENAI_API_KEY}
    default_model: gpt-4.1
    options:
      compact: none
llmRoutes:
  - id: openai-chat
    protocol: openai
    match_policy:
      path_prefix: /
    auth_policy:
      require_virtual_key: true
    target_policy:
      provider_target:
        provider_id: openai-main
virtualKeys:
  - id: test-key
    allowed_route_ids:
      - openai-chat
    rate_limits:
      llm:
        requests_per_minute: 60
        burst: 10

rate_limits is optional. It independently limits LLM, MCP, and agent ingress for a VirtualKey with in-memory token buckets; agent configures separate ACP and builtin buckets. Exceeded requests return 429 with Retry-After.

Set the admin Basic Auth credentials for agwctl as an environment variable, then apply the bundle:

export AGW_ADMIN_BASIC_AUTH=admin:your-password

OPENAI_API_KEY=sk-... ./agwctl apply -f gateway.bundle.yaml

apply is idempotent — it creates objects that do not exist, updates those that have changed, and skips unchanged ones:

apply: gateway.bundle.yaml
  create provider openai-main
  create llm_route openai-chat
  create virtual_key test-key
summary: create=3 update=0 skip=0 error=0

Retrieve the generated VirtualKey value and verify the OpenAI-compatible data plane directly:

AGW_API_KEY=$(./agwctl virtualkey get test-key | jq -r '.key')
curl -sS http://127.0.0.1:8080/v1/chat/completions \
  -H "Authorization: Bearer $AGW_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"hello"}]}'

curl -sS http://127.0.0.1:8080/v1/responses \
  -H "Authorization: Bearer $AGW_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"input":"hello"}'

agwctl is a control-plane management CLI; protocol data-plane checks use the corresponding HTTP, MCP, or Agent client directly.

The AGW_ADMIN_ADDR environment variable sets the admin API address (default: http://localhost:8019). Run agwctl export to dump the current gateway state as a bundle YAML.

Provider options.compact selects compatibility request shaping. In Caddyfile provider blocks, configure it as option compact <cc|codex|none>. In bundle YAML, configure it as options.compact. Providers ignore modes they do not implement.

Built-in provider_type values: openai, anthropic, claudecode, codex, gemini, ollama, openrouter, deepseek, zhipu, qwen. The zhipu provider supports GLM Coding Plan Chat Completions, including request-level thinking, reasoning_content replay, and streamed tool arguments; set its base_url to https://open.bigmodel.cn/api/coding/paas/v4, or set options.api_profile to coding_plan when using a custom proxy URL. For Claude Code through an Anthropic-compatible endpoint such as GLM Coding Plan, use provider_type claudecode; it preserves signed and redacted thinking blocks across tool turns and accepts generic capability overrides such as context_window, max_output_tokens, default_max_tokens, and vision. The qwen provider targets DashScope's OpenAI-compatible mode and supports an optional options.enable_thinking (bool) to control Qwen thinking mode; per-request reasoning fields override it.

MCP Quick Start

The MCP gateway uses the same minimal Caddyfile as the Quick Start. Enable mcp in the dispatcher, then apply an MCP bundle.

Create or reuse the minimal Caddyfile from the Quick Start above (the mcp directive is already present in agent_route_dispatcher). Run the gateway, then create a bundle file gateway.bundle.mcp.yaml.

Streamable HTTP upstream (remote MCP server over HTTP):

apiVersion: gateway.agw/v1alpha1
kind: GatewayBundle
mcpServices:
  - id: mcp-main
    name: Main MCP Service
    transport: streamable_http
    url: ${MCP_SERVICE_URL}
mcpRoutes:
  - id: mcp-main-route
    service_id: mcp-main
    match_policy:
      path_prefix: /mcp
    auth_policy:
      require_virtual_key: true
virtualKeys:
  - id: mcp-key
    allowed_route_ids:
      - mcp-main-route

Route ids must be slash-free. If id is omitted the gateway auto-generates a deterministic id mcp:<service_id>:<path-slug> (the path prefix lowercased with non-alphanumeric runs collapsed to -, / → root), so /mcp on mcp-main becomes mcp:mcp-main:mcp. The id above is set explicitly only for readability; because the auto-generated id is predictable you may also reference it directly in allowed_route_ids.

stdio upstream (local subprocess, e.g. @modelcontextprotocol/server-filesystem):

apiVersion: gateway.agw/v1alpha1
kind: GatewayBundle
mcpServices:
  - id: mcp-fs
    name: Filesystem MCP Service
    transport: stdio
    command: npx
    args: ["-y", "@modelcontextprotocol/server-filesystem", "/tmp"]
mcpRoutes:
  - id: mcp-fs-route
    service_id: mcp-fs
    match_policy:
      path_prefix: /mcp
    auth_policy:
      require_virtual_key: true
virtualKeys:
  - id: mcp-key
    allowed_route_ids:
      - mcp-fs-route

Apply and verify:

export AGW_ADMIN_BASIC_AUTH=admin:your-password

MCP_SERVICE_URL=https://your-mcp-server/mcp ./agwctl apply -f gateway.bundle.mcp.yaml

# Discover tools via admin API
./agwctl mcp-service tools mcp-main

# Retrieve the VirtualKey and send an MCP request
MCP_API_KEY=$(./agwctl virtualkey get mcp-key | jq -r '.key')

curl -s http://127.0.0.1:8080/mcp \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer $MCP_API_KEY" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

MCP route IDs are auto-generated as the deterministic, slash-free mcp:<service_id>:<path-slug> when id is omitted (the path prefix lowercased with non-alphanumeric runs collapsed to -, / → root). See docs/getting-started/quickstart-mcp.md for the full walkthrough.

ACP Agent Quick Start

Enable agent in agent_route_dispatcher, then apply an ACP-backed Agent and a unified AgentRoute. ACP runtime configuration is owned directly by the Agent; there is no separate ACP service object or ACP-specific ingress route.

For Codex, install the adapter and create the working directory:

npm install -g @zed-industries/codex-acp
mkdir -p /tmp/acp-codex-test

Create a bundle file gateway.bundle.acp.yaml:

apiVersion: gateway.agw/v1alpha1
kind: GatewayBundle

agents:
  - id: codex-main
    name: Codex
    runtime:
      type: acp
      acp:
        agent_type: codex
        cwd: /tmp/acp-codex-test
        allowed_roots: [/tmp/acp-codex-test]
        max_instances: 4
        permission_mode: auto_approve
    routes: {}
    resources: {}
    policy: {}
    disabled: false

agentRoutes:
  - id: acp-codex
    agent_id: codex-main
    match_policy:
      path_prefix: /acp/codex
    auth_policy:
      require_virtual_key: true

virtualKeys:
  - id: acp-key
    allowed_route_ids:
      - acp-codex

Apply and verify:

export AGW_ADMIN_BASIC_AUTH=admin:your-password

./agwctl apply -f gateway.bundle.acp.yaml

./agwctl agent get codex-main
./agwctl agent-route list
./agwctl acp-runtime get

Send a streamed turn through the dispatcher:

ACP_API_KEY=$(./agwctl virtualkey get acp-key | jq -r '.key')

curl -N -s http://127.0.0.1:8080/acp/codex/turn \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer $ACP_API_KEY" \
  -d '{"input":"Reply with exactly one word: pong","options":{"version":"v1","runtime":{"thread_id":"t-demo-1"}}}'

List sessions or replay a transcript through the same AgentRoute and VirtualKey:

curl -s "http://127.0.0.1:8080/acp/codex/sessions?cwd=/tmp/acp-codex-test" \
  -H "Authorization: Bearer $ACP_API_KEY"

curl -s "http://127.0.0.1:8080/acp/codex/sessions/<session-id>/transcript" \
  -H "Authorization: Bearer $ACP_API_KEY"

Inspect sessions, replay transcripts, or operate on runtime state through Agent-level capability APIs:

./agwctl agent sessions codex-main --cwd /tmp/acp-codex-test
./agwctl agent transcript codex-main <session-id>

./agwctl acp-runtime inflight
./agwctl acp-runtime close-thread codex-main t-demo-1

Agent route IDs are auto-generated as agent:<agent_id>:<path-slug> when omitted. Permission modes are deny, auto_approve, and interactive; clients must follow the runtime capability's advertised resume mode.

See docs/getting-started/quickstart-acp.md, docs/architecture/acp-architecture.md, docs/reference/acp-technical-spec.md, and docs/reference/acp-api.md for the full ACP documentation.

Metrics Admin API

Usage metrics are backed by the SQLite usage event tables (llm_usage_events, mcp_usage_events, acp_usage_events, builtin_usage_events) when the sqlite config store backend is active. Unified Agent ingress keeps route_kind=agent/route_protocol=agent and selects the ACP or builtin typed table through runtime_type; new ACP Agent events use agent_id directly and do not populate the historical service_id column. Prometheus counters use only the bounded route_kind and runtime_type labels.

GET /admin/metrics                       # per-kind summaries + pipeline health counters
GET /admin/metrics/prometheus            # O(1) in-process counter snapshot (text exposition)

GET /admin/metrics/llm/events            # recent LLM events
GET /admin/metrics/llm/timeseries        # LLM aggregates bucketed over time
GET /admin/metrics/llm/breakdown         # LLM aggregates grouped by a dimension

GET /admin/metrics/mcp/events            # recent MCP events
GET /admin/metrics/mcp/timeseries        # MCP aggregates bucketed over time
GET /admin/metrics/mcp/breakdown         # MCP aggregates grouped by a dimension
GET /admin/metrics/mcp/tools/summary     # MCP breakdown fixed to tool_name

GET /admin/metrics/acp/events            # recent ACP events
GET /admin/metrics/acp/timeseries        # ACP aggregates bucketed over time
GET /admin/metrics/acp/breakdown         # ACP aggregates grouped by a dimension
GET /admin/metrics/acp/summary           # ACP breakdown (operation/agent_type)

GET /admin/metrics/interactions          # cross-protocol interaction events
GET /admin/metrics/interactions/summary  # cross-protocol breakdown

timeseries and breakdown share one shape per protocol: from, to, limit, group_by, and the protocol's filter keys (LLM: route_id/provider_id/virtual_key_id/upstream_model/llm_api; MCP: route_id/service_id/virtual_key_id/method/tool_name/result_status; ACP: route_id/route_protocol/virtual_key_id/agent_type/operation). timeseries also takes bucket (minute|hour|day, short forms like m/h/d, or Grafana-style durations like 3h/5m/1d; empty defaults to hour). See docs/reference/admin-api-reference.md for the full Admin API surface.

Runtimes

• agw uses a Caddyfile plus the shared config store
• agwd runs as a standalone daemon with --config-store and optional --static-config
• agwctl talks to the gateway Admin API or the Caddy admin API; interactive CLI login is provided by the separate agw-auth project

See docs/README.md for runtime-specific guides and references.

Documentation

• docs/README.md: documentation index and split plan
• docs/architecture/architecture-overview.md: current architecture overview
• docs/architecture/mcp-architecture.md: MCP gateway architecture
• docs/architecture/acp-architecture.md: ACP gateway architecture
• docs/architecture/configstore-architecture.md: config store architecture
• docs/design/agents-control-plane.md: cross-runtime agent control-plane design
• docs/design/request-pipeline.md: synchronous gateway request pipelines and the upper-layer Temporal/business-workflow boundary
• docs/design/builtin-agent-runtime.md: builtin ADK runtime design and implementation status
• docs/design/gateway-bundle-yaml.md: bundle YAML design

Current Limits

• OpenAI-compatible chat and Anthropic-compatible messages are the primary mature LLM paths
• OpenAI embeddings and Anthropic token counting are not fully implemented
• MCP is active in the dispatcher and Admin API surface, but some adjacent subsystems are still evolving
• ACP is a functional native route/admin/dispatcher surface with a reusable stdio runtime driver and thin codex/opencode agent adapters; crash retry and the in-repo Codex app-server bridge remain deferred
• metrics Admin APIs expose durable SQLite-backed summaries (with pipeline drop/failure counters), recent LLM/MCP/ACP interaction events, and aggregate breakdowns; a Prometheus exposition endpoint (GET /admin/metrics/prometheus) serves O(1) in-process counters, and usage events can be exported as OpenTelemetry spans to an OTLP collector (metrics { otlp { endpoint ... } } in the Caddyfile or --metrics-otlp-* agwd flags; see the commented OTLP profile in examples/Caddyfile.example) — events carry W3C trace/span/parent ids, so the collector sees the full interaction span tree including builtin-agent internal calls; an optional components toggle nests one span per eino chat-model component call under the interaction span
• the agents control plane is active: pkg/agent, the agents config store, gateway-bundle parity, and /admin/agents CRUD plus workspace/activity/usage/interactions/resources/health; the M3 runtime control plane adds capabilities, exact-run list/cancel, one-shot permissions, and capability-gated sessions/transcripts; usage events carry optional agent_id, run_id, and runtime_type correlation
• memory is not shipped in v0.4.x; /admin/memory/... is a reserved Admin API family whose endpoints return 501 Not Implemented

Development

Useful commands:

go test ./...
go test ./pkg/admin ./pkg/gateway ./pkg/dispatcher/...
go test ./pkg/llm/provider/...

./agw adapt --config ./Caddyfile
./agw run --config ./Caddyfile