ai

LLM control plane, RAG, and model hub for the Hanzo platform. Native Go model routing with no Python middlemen.

Quick start

docker pull ghcr.io/hanzoai/ai:latest

curl -H "Authorization: Bearer hk-YOUR-API-KEY" \
  https://api.hanzo.ai/v1/chat/completions \
  -d '{"model":"zen4-pro","messages":[{"role":"user","content":"Hello"}]}'

What this is

ai is the canonical LLM control plane for the Hanzo platform — model routing, RAG, model hub, MCP management. OpenAI-compatible JSON over HTTP (/v1/chat/completions, /v1/models). Three auth modes (IAM API key hk-*, JWT, provider key sk-*). Static model routing — 66+ models mapped to upstream providers in pure Go. Per-request usage tracking fire-and-forget to IAM. KMS-resolved provider secrets, org-scoped. Renamed from hanzoai/cloud on 2026-05-19 when the unified binary took the cloud name per HIP-0106; ai mounts as the ai subsystem inside hanzoai/cloud.

Specs

Implements:

• HIP-0037 AI Cloud Platform
• HIP-0106 Unified Cloud Binary (ai subsystem)

Architecture

   user / api key  ->  hanzoai/gateway  ->  hanzoai/ai (zip.App, Go)
                                                  |
                              auth: hk-* | JWT | sk-* (provider passthrough)
                                                  |
                              model routing: 66+ models -> upstream providers
                                                  |
                          +--------+--------+--------+--------+
                          |        |        |        |        |
                       DO-AI  Fireworks  OpenAI   Anthropic   ...
                          |        |        |        |        |
                              KMS-resolved provider secrets
                                                  |
                              IAM usage tracking (async)

Hanzo Cloud

AI Cloud OS — Native Go model routing, IAM-integrated auth, usage tracking, and KMS secrets management. Zero middlemen, pure performance.

Architecture

Hanzo Cloud implements the ZAP (Zero-overhead API Protocol) — a native Go model routing layer that connects users directly to upstream AI providers with no intermediaries.

User → cloud-api (Go, ZAP gateway) → upstream providers (DO-AI, Fireworks, OpenAI)
                ↕                              ↕
           Hanzo IAM                      Hanzo KMS
      (auth, billing, usage)          (multi-tenant secrets)

ZAP Protocol

ZAP defines the fast native path from API gateway to AI inference:

• OpenAI-compatible JSON over HTTP (/v1/chat/completions, /v1/models)
• Three auth modes: IAM API key (hk-*), JWT (hanzo.id OAuth), Provider key (sk-*)
• Static model routing — 66+ models mapped to 3 upstream providers in pure Go
• Per-request usage tracking — async fire-and-forget to IAM
• KMS-resolved secrets — provider API keys from Infisical with org-scoped projects
• Zero Python — no legacy proxy middleware, no extra hops

Supported Models

Free Tier (DO-AI) — 28 models

gpt-4o, gpt-5, gpt-5-mini, claude-opus-4-6, claude-sonnet-4-5, claude-haiku-4-5, o3, o3-mini, qwen3-32b, deepseek-r1-distill-70b, llama-3.3-70b, and more.

Premium (Fireworks) — 17 models

fireworks/deepseek-r1, fireworks/deepseek-v3, fireworks/kimi-k2, fireworks/qwen3-235b-a22b, fireworks/qwen3-coder-480b, fireworks/cogito-671b, and more.

Premium (OpenAI Direct) — 5 models

openai-direct/gpt-4o, openai-direct/gpt-5, openai-direct/o3, openai-direct/o3-mini, openai-direct/gpt-4o-mini

Zen Models (Premium, 3X) — 8 models

zen4-mini, zen4-pro, zen4-max, zen4-ultra, zen4-coder-flash, zen4-coder-pro, zen-vl, zen3-omni

Full model list: GET /api/models

Quick Start

# Build
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-w -s" -o cloud-api-server .

# Run
./cloud-api-server

# Test
curl -H "Authorization: Bearer hk-YOUR-API-KEY" \
  https://api.hanzo.ai/v1/chat/completions \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"Hello"}]}'

Configuration

Set via conf/app.conf or environment variables:

Deploy

# Docker
docker pull ghcr.io/hanzoai/cloud:latest

# Kubernetes (production)
kubectl apply -f k8s/kms-secrets.yaml
kubectl apply -f k8s/configmap.yaml
kubectl apply -f k8s/deployment.yaml
kubectl apply -f k8s/service.yaml

GitHub Actions production deploy (.github/workflows/deploy-production.yml) resolves deployment credentials from Hanzo KMS. Preferred setup is a single GitHub secret: HANZO_API_KEY. Universal Auth (KMS_CLIENT_ID + KMS_CLIENT_SECRET) remains as a fallback.

License

Apache-2.0