TrustGate

TrustGate

The high-performance data-plane gateway for LLM and agent traffic — built from scratch in Go

Documentation  |  Quick Start  |  Architecture  |  Community

✨ Features

• 🚀 High Performance — Built in Go on top of Fiber, tuned for low latency and high concurrency.
• 🌍 Multi-Provider — First-class adapters for OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Google Gemini, Vertex AI, Groq, Mistral and DeepSeek.
• 🧭 Smart Routing & Load Balancing — Round-robin, weighted round-robin and IP-hash strategies with health checks and fallback targets.
• 🔌 Plugin System — Policy stages with built-in plugins: rate limiting, token rate limiting, request size guard, semantic cache and CORS.
• 🧠 Semantic Cache — Embedding-based response caching to cut cost and latency on repeated prompts.
• 🔒 Security & Multi-Tenancy — Per-gateway consumers, API-key auth, and policies scoped globally or per consumer.
• 📊 Observability — Built-in metrics, request telemetry streamed to Kafka by default, with an opt-in per-gateway OpenTelemetry (OTLP) exporter.
• ⚙️ Two Independent Planes — Admin and Proxy run as separate processes so you can scale them independently.
• ☁️ Cloud Agnostic — Single static binary, Docker image and Kubernetes manifests. Deploy anywhere.

🚀 Quick Start

One-line install

Clones the repo, seeds .env, brings up the full stack in Docker and — when Go is installed — compiles the trustgate binary and installs it on your PATH:

curl -fsSL https://raw.githubusercontent.com/NeuralTrust/TrustGate/main/scripts/install.sh | bash

Requires git, docker and Docker Compose (plus Go to build the CLI). Re-running updates the checkout and never overwrites your .env. Useful overrides:

• AG_REF=develop — install a different branch/tag/commit
• AG_DIR=/path/to/dir — where to clone
• AG_BIN_DIR=~/.local/bin — where to install the trustgate CLI
• AG_INSTALL_CLI=0 — skip building the CLI · AG_NO_START=1 — skip docker compose up

Using Docker Compose

# Clone the repository
git clone https://github.com/NeuralTrust/TrustGate.git
cd TrustGate

# Copy the env template and adjust as needed
cp .env.example .env

# One command to bring up everything (Postgres, Redis, Kafka, Zookeeper) + admin, proxy & mcp
make up

# Tail the logs / tear everything down
make logs
make down

Then hit the health probes:

curl localhost:8080/healthz       # admin
curl localhost:8081/healthz       # proxy
curl localhost:8082/healthz       # mcp
curl localhost:8080/__/version    # build info (version, commit, build date)

The image is pinned to linux/amd64 because confluent-kafka-go only bundles an amd64 librdkafka; on Apple Silicon the build runs under emulation out of the box.

Local Development

Run the infra in Docker and the binary on your machine so you can attach a debugger:

# 1. Boot the local dev infra (Postgres, Redis, Kafka, Zookeeper)
make compose-up

# 2a. Run admin + proxy together in a single process (simplest, single-node)
make run-all        # applies migrations, starts admin on :8080 and proxy on :8081

# 2b. ...or run each plane in its own terminal (closer to production)
make run-admin      # terminal 1 — applies migrations, starts admin on :8080
make run-proxy      # terminal 2 — applies migrations, starts proxy on :8081
make run-mcp        # terminal 3 — (optional) starts the MCP server on :8082

# 3. Stop the infra (add -v to wipe volumes)
make compose-down

Using Kubernetes

Manifests live under k8s/.

kubectl apply -k k8s/

Run Tests

make test            # unit tests
make test-race       # unit tests with the race detector
make test-cover      # unit tests with coverage profile
make test-functional # functional tests against a real admin server

🧪 Your first request

The Admin plane (:8080) configures gateways, providers and consumers; the Proxy plane (:8081) serves OpenAI-compatible traffic. The proxy resolves the gateway from the X-AG-Gateway-Slug header and the consumer from its X-AG-API-Key. End-to-end, from zero to a forwarded completion:

make up   # admin :8080, proxy :8081 + Postgres/Redis/Kafka

ADMIN="http://localhost:8080"
PROXY="http://localhost:8081"
TOKEN="$ADMIN_TOKEN"   # admin JWT, see "Admin token" below

# 1. Create a gateway (slug becomes its host/subdomain)
GW=$(curl -s -X POST "$ADMIN/v1/gateways" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"name":"My Gateway","slug":"demo"}')
GW_ID=$(echo "$GW" | jq -r .id); GW_SLUG=$(echo "$GW" | jq -r .slug)

# 2. Register an upstream LLM provider (OpenAI here)
REG=$(curl -s -X POST "$ADMIN/v1/gateways/$GW_ID/registries" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"name":"openai-primary","provider":"openai",
       "auth":{"type":"api_key","api_key":{"api_key":"'"$OPENAI_API_KEY"'"}}}')
REG_ID=$(echo "$REG" | jq -r .id)

# 3. Create a consumer bound to that registry
CON=$(curl -s -X POST "$ADMIN/v1/gateways/$GW_ID/consumers" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"name":"my-app","registries":[{"id":"'"$REG_ID"'"}]}')
CON_ID=$(echo "$CON" | jq -r .id); CON_SLUG=$(echo "$CON" | jq -r .slug)

# 4. Mint a consumer API key (returned in cleartext once)
AUTH=$(curl -s -X POST "$ADMIN/v1/gateways/$GW_ID/auths" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"name":"my-app-key","type":"api_key"}')
AUTH_ID=$(echo "$AUTH" | jq -r .id); API_KEY=$(echo "$AUTH" | jq -r .api_key)

# 5. Attach the key to the consumer
curl -s -X POST "$ADMIN/v1/gateways/$GW_ID/consumers/$CON_ID/auths/$AUTH_ID" \
  -H "Authorization: Bearer $TOKEN"

# 6. Call the proxy (OpenAI-compatible)
curl -s -X POST "$PROXY/$CON_SLUG/v1/chat/completions" \
  -H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Hello!"}]}'

From an application, point any OpenAI SDK at the proxy — no client changes beyond the base URL and two headers:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8081/my-app",  # /{consumer_slug}
    api_key="unused",                          # the provider key lives in the gateway
    default_headers={
        "X-AG-Gateway-Slug": "demo",
        "X-AG-API-Key": "<consumer api key>",
    },
)

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)

Other entrypoints follow the same /{consumer_slug}/... shape: /v1/messages (Anthropic format) and /v1/responses (OpenAI Responses format).

Admin token

The Admin API expects a JWT (HS256) signed with SERVER_SECRET_KEY from your .env. Mint a short-lived one for local use:

export SERVER_SECRET_KEY="$(grep ^SERVER_SECRET_KEY .env | cut -d= -f2-)"
export ADMIN_TOKEN=$(python3 - <<'PY'
import jwt, os, time
secret = os.environ["SERVER_SECRET_KEY"]
print(jwt.encode({"sub": "admin", "iat": int(time.time()), "exp": int(time.time()) + 3600}, secret, algorithm="HS256"))
PY
)

🏗️ Architecture

TrustGate ships a single binary that boots one HTTP server, selected by argv[1] (default: proxy). In production each pod runs one container with the appropriate argument, so the Admin, Proxy and MCP planes scale independently.

./trustgate              # → proxy (default)
./trustgate proxy        # → proxy
./trustgate admin        # → admin
./trustgate mcp          # → mcp (Model Context Protocol server)
./trustgate run          # → admin + proxy together in one process (single-node)

flowchart LR
    subgraph Clients["Clients & Agents"]
        APP["Apps / SDKs / Agents"]
    end

    subgraph AG["TrustGate"]
        direction TB
        ADMIN["Admin Plane :8080\nGateways · Registries · Consumers\nAuth · Policies · Catalog"]
        PROXY["Proxy Plane :8081\nRouting · Load Balancing\nPolicy Stages · Plugins"]
        MCP["MCP Plane :8082\nMCP targets & tools for agents"]
    end

    subgraph Plugins["Policy Plugins"]
        RL["Rate Limit"]
        TRL["Token Rate Limit"]
        RS["Request Size"]
        SC["Semantic Cache"]
        CORS["CORS"]
    end

    subgraph Providers["LLM Providers"]
        P1["OpenAI · Anthropic\nAzure · Bedrock"]
        P2["Gemini · Vertex\nGroq · Mistral"]
    end

    subgraph Infra["Infrastructure"]
        PG[("Postgres")]
        RD[("Redis")]
        KFK[["Kafka"]]
    end

    APP -->|API key| PROXY
    APP -->|MCP| MCP
    PROXY --> Plugins
    PROXY -->|load balance| Providers
    ADMIN -. config .-> PROXY
    ADMIN -. config .-> MCP
    ADMIN --- PG
    PROXY --- PG
    PROXY --- RD
    MCP --- PG
    PROXY -->|telemetry| KFK

Request lifecycle

1. A client calls the Proxy with a consumer API key.
2. The gateway resolves the consumer, gateway config and applicable policies.
3. Policy stages run their plugins (rate limit, token rate limit, request size, semantic cache, CORS).
4. The load balancer picks a healthy upstream target (round-robin / weighted / IP-hash) with fallback.
5. The request is forwarded to the selected provider adapter (OpenAI, Anthropic, Bedrock, …), streaming when supported.
6. The response is returned, the semantic cache is populated, and telemetry is emitted to Kafka.

Planes

🔌 Plugins

Plugins run inside ordered policy stages and can execute sequentially or in parallel.

🌍 Providers

⚙️ Configuration

All configuration is read from environment variables. In development, copy .env.example to .env and godotenv loads it automatically. Production deployments inject env vars directly (Helm values, ECS task definitions, k8s ConfigMap + Secret).

# Server (HTTP listeners)
SERVER_ADMIN_PORT=8080
SERVER_PROXY_PORT=8081
SERVER_MCP_PORT=8082

# Host suffixes returned in the gateway response ({slug}.<base-domain>)
GATEWAY_BASE_DOMAIN=llm.neuraltrust.ai  # proxy plane host
MCP_BASE_DOMAIN=mcp.neuraltrust.ai      # mcp plane host

# Database (Postgres via pgx/pgxpool)
DB_HOST=localhost
DB_PORT=5432
DB_NAME=trustgate

# Redis & Kafka
REDIS_HOST=localhost
KAFKA_BROKERS=localhost:9092

# Telemetry & Metrics
TELEMETRY_ENABLED=true
TELEMETRY_KAFKA_TOPIC=trustgate.requests
METRICS_ENABLED=true

# OTLP exporter defaults (opt-in per gateway; per-gateway settings override these)
OTEL_EXPORTER_OTLP_ENDPOINT=collector:4317
OTEL_EXPORTER_OTLP_PROTOCOL=grpc

See .env.example for the full set with safe defaults.

Telemetry exporters (Kafka default + OTLP opt-in)

Every completed request is turned into a sanitized business event and fanned out to one or more exporters. Kafka is the config-driven default (feeds kafka-connect → ClickHouse → data-plane-api). A gateway can additionally opt into the otlp exporter, which ships the same event to an external OpenTelemetry Collector as a single OTLP log record per request (event.name="gateway.request"); the Collector fans out to any vendor backend. Enabling otlp does not replace Kafka — both fire (explicit exporters merge with the global defaults).

Add it to a gateway's telemetry.exporters:

{
  "telemetry": {
    "exporters": [
      {
        "name": "otlp",
        "settings": {
          "endpoint": "collector:4317",
          "protocol": "grpc",
          "headers": { "authorization": "Bearer <token>" },
          "compression": "gzip",
          "timeout": "10s",
          "insecure": false
        }
      }
    ]
  }
}

With Kafka still configured, the event is delivered to the Collector and Kafka (the ClickHouse path keeps populating with schema_version=2 intact).

otlp settings

Any key absent from a gateway's settings falls back to the process-level OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_PROTOCOL, OTEL_EXPORTER_OTLP_HEADERS, OTEL_EXPORTER_OTLP_TIMEOUT, OTEL_EXPORTER_OTLP_INSECURE, and OTEL_EXPORTER_OTLP_COMPRESSION env vars (per-gateway settings win). OTEL_EXPORTER_OTLP_TIMEOUT accepts an integer of milliseconds per the OpenTelemetry spec (e.g. 10000); a Go duration string (e.g. 10s) is also accepted. Settings are validated structurally on gateway create/update with no network I/O; export is non-blocking (bounded queue, drop-on-full) so a slow or unreachable Collector never affects request latency or the Kafka path.

Migrations

Migrations are in-code Go files under pkg/infra/database/migrations/. Each file is named <unix_timestamp>_<snake_name>.go and registers itself via database.RegisterMigration in its init(). The pgx-backed runner commits each migration's DDL plus its migration_version row in a single transaction, applying any pending migrations automatically on boot.

📚 API Docs

The Admin API is fully annotated and ships Swagger 2.0 and OpenAPI 3 specs:

make swagger   # generate docs/swagger.{json,yaml} + docs.go
make openapi   # convert to docs/openapi.json (OpenAPI 3)
make docs      # regenerate everything

Specs live under docs/ (swagger.json, swagger.yaml, openapi.json).

🗂️ Repository layout

cmd/trustgate/         # entry point (single binary: proxy | admin | mcp | run)
pkg/version/           # ldflag-fed build info
pkg/config/            # env-only config loader (.env via godotenv in dev)
pkg/domain/            # domain entities, value objects and port interfaces
pkg/app/               # application services (use cases)
pkg/infra/providers/   # provider adapters (openai, anthropic, bedrock, …)
pkg/infra/plugins/     # policy plugins (ratelimit, semanticcache, …)
pkg/infra/loadbalancer/# routing strategies + health checks
pkg/infra/database/    # pgxpool + in-code Go migrations registry
pkg/infra/telemetry/   # Kafka (default) + OTLP exporters
pkg/api/handler/http/  # per-route HTTP handlers
pkg/server/            # Server interface + admin / proxy routers
pkg/container/         # dig DI container + one module per context

🤝 Contributing

We love contributions! To get started:

1. Fork the repository
2. Create your feature branch (git checkout -b feat/my-feature)
3. Run make lint && make test before committing
4. Push to your branch and open a Pull Request

📜 License

TrustGate is licensed under the Apache License 2.0 — see the LICENSE file for details.

📫 Community & Support

• Documentation
• Slack Community
• GitHub Issues
• Twitter
• Blog