proxy

Overview ¶

Example: OpenAI-compatible proxy server.

This example wires the Nexus proxy with the full streaming surface:

• SSE (default, OpenAI-compatible) at POST /v1/chat/completions
• NDJSON via Accept: application/x-ndjson or ?stream_format=ndjson
• Native nexus SSE via Accept: application/vnd.nexus.events+sse
• WebSocket bidirectional streaming at /v1/realtime
• Stream record-and-replay caching (in-memory)
• Plugin lifecycle metrics for streamed traffic

Run:

OPENAI_API_KEY=sk-... go run ./_examples/proxy

The proxy requires a gateway key (nxs_...). The example creates a tenant and a key at startup and prints the key once; the in-memory store forgets it on restart. Then from Python:

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1", api_key="nxs_...")
resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
)
for chunk in resp:
    print(chunk.choices[0].delta.content or "", end="", flush=True)