Documentation
¶
Index ¶
- Constants
- func DefaultProxyTransport() *http.Transport
- func NewAnthropicProxyHandler(upstream *url.URL, transport http.RoundTripper) http.Handler
- func NewAuthSwapTransport(next http.RoundTripper, token string) http.RoundTripper
- func NewHealthzHandler() http.Handler
- func NewModelRouter(routes []ModelRoute, defaultProviderName string, defaultHandler http.Handler, ...) http.Handler
- func NewNotFoundHandler() http.Handler
- func NewSetLoglevelHandler() http.Handler
- func NewSetLoglevelHandlerWithRevert(autoRevert time.Duration) http.Handler
- type ModelRoute
Constants ¶
const ( SetLoglevelDefault = 1 SetLoglevelAutoRevert = 5 * time.Minute )
SetLoglevelDefault is the verbosity we revert to after autoRevertAfter. V(1) keeps the structured per-request line; the operator bumps to 2/3 for alias/route detail or to higher tiers for deep debug.
Variables ¶
This section is empty.
Functions ¶
func DefaultProxyTransport ¶ added in v0.3.0
DefaultProxyTransport returns an http.Transport with explicit timeouts suitable for upstream LLM API calls. Long ResponseHeaderTimeout because LLM completions can take 30s+ for the first byte (SSE); short Dial because connections are local-network or quick HTTPS to api.anthropic.com.
func NewAnthropicProxyHandler ¶ added in v0.3.0
NewAnthropicProxyHandler returns an HTTP handler that reverse-proxies every incoming request to upstream (typically https://api.anthropic.com).
The Authorization header passes through unchanged — this is what lets Claude Code's subscription OAuth bearer travel through the router to Anthropic without the router ever holding it. No body parsing, no model-based routing: that's task 3. v1 of task 2 = single upstream, verbatim forward.
Upstream errors (connection refused, 5xx before body, etc.) are logged server-side with the full error string for debugging, but the client sees only a generic "502 Bad Gateway / upstream unavailable" — the internal error details (IPs, TLS handshake failures, connection strings) are not leaked.
If transport is nil, DefaultProxyTransport is used.
func NewAuthSwapTransport ¶ added in v0.4.0
func NewAuthSwapTransport(next http.RoundTripper, token string) http.RoundTripper
NewAuthSwapTransport wraps next so each outbound request has its Authorization header replaced with `Bearer <token>`. Used by the model router to swap the client's subscription OAuth bearer for a per-provider API token (MiniMax, Ollama, vLLM) before forwarding.
If token is empty, the wrapper is a no-op and returns next.
func NewHealthzHandler ¶
NewHealthzHandler returns a handler that responds 200 with body "OK".
func NewModelRouter ¶ added in v0.4.0
func NewModelRouter( routes []ModelRoute, defaultProviderName string, defaultHandler http.Handler, aliases map[string]string, sampler liblog.Sampler, ) http.Handler
NewModelRouter returns an HTTP handler that body-parses each request's JSON `model` field, resolves it through the aliases map (single-hop, case-sensitive exact match), then dispatches to the first matching ModelRoute. Unmatched models (and non-JSON / no-model requests) fall through to defaultHandler (logged as provider=defaultProviderName). The body is fully read and replayed for the downstream handler — fine for /v1/messages JSON payloads (typically <100 KB); not suitable for unbounded upload bodies.
aliases may be nil or empty — both mean "no alias rewriting". On a hit, the body's top-level .model field is re-marshaled to the resolved value before route dispatch, so the upstream sees the full model name.
One structured `[req]` log line per request at V(1):
[req] POST /v1/messages model=m3 alias=MiniMax-M3-highspeed provider=minimax status=200 latency=842ms
Non-200 responses are ALWAYS logged; 200 responses are gated by the sampler. `log.DefaultSamplerFactory` gives the canonical OR-combo: at most once per 10s, OR unconditionally when glog `-v` ≥ 4. This keeps the steady-state log readable while preserving every error event and giving full visibility once the operator bumps verbosity via `/setloglevel/4`.
At V(2), alias resolution and route match get their own `[alias]` / `[route]` detail lines (independent of the sampler — V(2) detail is already operator-opt-in, additional gating buys nothing).
func NewNotFoundHandler ¶ added in v0.7.0
NewNotFoundHandler returns a 404 handler that logs the unknown path before responding. Registered at `/` in the factory's mux so it catches everything not matched by a more specific route (`/v1/`, `/healthz`, `/readiness`, `/metrics`, `/setloglevel/`, `/gc`).
Logged at glog V(1) — same level as `[req]` — so unknown-path probes surface in the operator's default log alongside real traffic. Useful for catching misconfigured clients (wrong base URL, typo in `/v1/messages`) and any probing of the listener.
func NewSetLoglevelHandler ¶
NewSetLoglevelHandler returns a handler that flips glog's -v verbosity at runtime. Convenience wrapper for the production default (SetLoglevelAutoRevert = 5 min); tests use NewSetLoglevelHandlerWithRevert with a short window.
func NewSetLoglevelHandlerWithRevert ¶ added in v0.6.0
NewSetLoglevelHandlerWithRevert returns a handler that flips glog's -v verbosity at runtime. URL shape is `/setloglevel/<level>`; the integer suffix is parsed and passed to `log.LogLevelSetter.Set`, which auto- reverts to SetLoglevelDefault after autoRevert so a forgotten bump can't leave the router in verbose mode indefinitely.
The LogLevelSetter is created once at handler construction (single instance, shared across requests); `Set()` itself spawns the auto- revert goroutine per call — that's upstream bborbe/log behavior, and `resetLogLevel` is idempotent so overlapping timers are harmless.
Level validation: negative values are rejected with 400; glog itself accepts any int32, but a negative verbosity is meaningless and almost always a typo (`-1` instead of `1`).
Example:
$ curl http://127.0.0.1:8788/setloglevel/3 set loglevel to 3 completed
Stdlib-mux compatible: parses the level from URL.Path directly rather than relying on gorilla/mux path vars.
Types ¶
type ModelRoute ¶ added in v0.4.0
ModelRoute pairs a glob pattern (filepath.Match syntax) with the provider name + handler to invoke when an incoming request's `model` field matches. ProviderName is what appears in the structured log (`provider=minimax`) and is the same key as in the YAML config's `providers:` map.
Source Files