model-router

Overview ¶

Command model-router is the sidecar that runs alongside every agent pod. It exposes an OpenAI-compatible API on :8080 and a Gemini-compatible API on :8082. runtime decision logging Prometheus metrics (trace-event client counters) are on :9091/metrics. All LLM calls from the agent are proxied through here, enabling:

• Multi-model routing (rule-based + LLM meta-router)
• Kubernetes JWT authentication (TokenReview)
• Conversation state checkpointing
• Unified tool dispatch (Tool CRDs, MCP servers, agent-as-tool; in-process executor, no :8081)
• Cost tracking and budget enforcement