Documentation
¶
Overview ¶
Package proxy is the second inference flow: transparent forwarding of caller-supplied upstream credentials. Dispatched from the inference handler when X-WR-Proxy-Mode: Proxy is set.
Differences from app/pipeline:
- No Policy / Model / HostBinding resolution. The caller picks the upstream Host by slug via X-WR-Upstream-Host; the inference handler hands us the resolved BaseURL.
- No keypool, no per-key circuit breaker, no retry/failover. The caller's Authorization is the upstream credential; one attempt.
- System-owned rate limits only (inference-api-proxy for authed, inference-api-proxy-anonymous for anonymous), keyed by relay-key hash or client IP respectively.
- Same tee + detached post-flight pattern as app/pipeline: the post-flight goroutine commits the reservation and extracts tokens for usage logging. Never blocks the response.
The package is HTTP-aware only at the upstream-call boundary (one outbound http.Request). The inbound *http.Request is consumed by the handler; this package takes pre-resolved bytes + headers + the upstream Authorization value.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrNoUpstreamAuth = errors.New("proxy: upstream authorization required")
ErrNoUpstreamAuth signals the handler forgot to populate UpstreamAuth.
Functions ¶
This section is empty.
Types ¶
type Pipeline ¶
type Pipeline struct {
Limiter *pkgratelimit.Limiter
Lifecycle *lifecycle.Registry
Client *http.Client
Logger *slog.Logger
}
Pipeline is the orchestrator. Constructed once at boot; Run() is goroutine-safe.
func New ¶
func New(limiter *pkgratelimit.Limiter, registry *lifecycle.Registry, logger *slog.Logger) *Pipeline
New constructs a Pipeline with a sensible http.Client default. The client has no overall timeout — streaming responses can take minutes. Use net/http transport timeouts for hop-level safety instead.
lifecycle is optional; pass nil if post-flight observers aren't wired in this deployment (tests, minimal smoke).
type Request ¶
type Request struct {
// Method + Path are the inbound HTTP method and path appended to
// HostBaseURL when building the upstream URL. Path includes the
// leading slash (e.g. "/v1/messages").
Method string
Path string
// Body is the inbound request body. Streamed through to upstream
// verbatim; the proxy does not buffer it. Transport-layer code that
// wants to inspect the body (for usage stamping, deep validation,
// etc.) should ReadAll first and pass a *bytes.Reader here — proxy
// stays shape-blind either way.
Body io.Reader
// Headers is the inbound header set with relay-internal headers
// already stripped (X-WR-*, Cookie, hop-by-hop).
// Authorization is set separately via UpstreamAuth — do not include
// it here.
Headers http.Header
// HostBaseURL is the resolved upstream base URL from the Host row.
HostBaseURL string
// UpstreamAuth is the verbatim "Authorization" value the caller
// supplied (typically "Bearer …"). Forwarded as-is.
UpstreamAuth string
// RateScope is the limiter bucket subject — the relay-key hash for
// authed, the client IP for anonymous. Empty disables limiting.
RateScope string
// Rules is the resolved rule set (system-owned for proxy mode).
Rules []pkgratelimit.Rule
// Extractor parses the upstream response body for usage logging.
// nil disables extraction (post-flight still commits the reservation).
Extractor TokenExtractor
// ModelName is for logging only — proxy mode does not consult the
// catalog Model row.
ModelName string
// Lifecycle is the per-request shared context, constructed by the
// handler before Run. Post-flight observers see it via the registered
// lifecycle observers (Finalize). Optional — nil skips observer dispatch.
Lifecycle *lifecycle.Context
}
Request is the pre-resolved input to Run.
type Result ¶
type Result struct {
Status int
Headers http.Header
// Body MUST be Closed by the caller. Closing triggers the post-
// flight goroutine (if it hasn't run already).
Body io.ReadCloser
}
Result is what the handler streams back to the caller.
type TokenExtractor ¶
TokenExtractor extracts upstream usage from a response body. The inference handler picks the right extractor by inbound endpoint (OpenAI shape for /v1/chat/completions, Anthropic shape for /v1/messages) and passes it in.