proxy

package
v0.9.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 7, 2026 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Overview

Package proxy is the second inference flow: transparent forwarding of caller-supplied upstream credentials. Dispatched from the inference handler when X-WR-Proxy-Mode: Proxy is set.

Differences from app/pipeline:

  • No Policy / Model / HostBinding resolution. The caller picks the upstream Host by slug via X-WR-Upstream-Host; the inference handler hands us the resolved BaseURL.
  • No keypool, no per-key circuit breaker, no retry/failover. The caller's Authorization is the upstream credential; one attempt.
  • System-owned rate limits only (inference-api-proxy for authed, inference-api-proxy-anonymous for anonymous), keyed by relay-key hash or client IP respectively.
  • Same tee + detached post-flight pattern as app/pipeline: the post-flight goroutine commits the reservation and extracts tokens for usage logging. Never blocks the response.

The package is HTTP-aware only at the upstream-call boundary (one outbound http.Request). The inbound *http.Request is consumed by the handler; this package takes pre-resolved bytes + headers + the upstream Authorization value.

Index

Constants

This section is empty.

Variables

View Source
var ErrNoUpstreamAuth = errors.New("proxy: upstream authorization required")

ErrNoUpstreamAuth signals the handler forgot to populate UpstreamAuth.

Functions

This section is empty.

Types

type Pipeline

type Pipeline struct {
	Limiter   *pkgratelimit.Limiter
	Lifecycle *lifecycle.Registry
	Client    *http.Client
	Logger    *slog.Logger
}

Pipeline is the orchestrator. Constructed once at boot; Run() is goroutine-safe.

func New

func New(limiter *pkgratelimit.Limiter, registry *lifecycle.Registry, logger *slog.Logger) *Pipeline

New constructs a Pipeline with a sensible http.Client default. The client has no overall timeout — streaming responses can take minutes. Use net/http transport timeouts for hop-level safety instead.

lifecycle is optional; pass nil if post-flight observers aren't wired in this deployment (tests, minimal smoke).

func (*Pipeline) Run

func (p *Pipeline) Run(ctx context.Context, req *Request) (res *Result, err error)

Run forwards one proxy-mode request. The returned Body MUST be Closed.

type Request

type Request struct {
	// Method + Path are the inbound HTTP method and path appended to
	// HostBaseURL when building the upstream URL. Path includes the
	// leading slash (e.g. "/v1/messages").
	Method string
	Path   string

	// Body is the inbound request body. Streamed through to upstream
	// verbatim; the proxy does not buffer it. Transport-layer code that
	// wants to inspect the body (for usage stamping, deep validation,
	// etc.) should ReadAll first and pass a *bytes.Reader here — proxy
	// stays shape-blind either way.
	Body io.Reader

	// ContentLength is the inbound request's Content-Length, applied to
	// the upstream request explicitly because a non-seekable streamed Body
	// gives http.NewRequest no length to infer — without it the forward
	// degrades to chunked transfer. <= 0 means unknown.
	ContentLength int64

	// Headers is the inbound header set with relay-internal headers
	// already stripped (X-WR-*, Cookie, hop-by-hop).
	// Authorization is set separately via UpstreamAuth — do not include
	// it here.
	Headers http.Header

	// HostBaseURL is the resolved upstream base URL from the Host row.
	HostBaseURL string

	// UpstreamAuth is the verbatim "Authorization" value the caller
	// supplied (typically "Bearer …"). Forwarded as-is.
	UpstreamAuth string

	// RateScope is the limiter bucket subject — the relay-key hash for
	// authed, the client IP for anonymous. Empty disables limiting.
	RateScope string
	// Rules is the resolved rule set (system-owned for proxy mode).
	Rules []pkgratelimit.Rule

	// Extractor parses the upstream response body for usage logging.
	// nil disables extraction (post-flight still commits the reservation).
	Extractor TokenExtractor

	// ModelName is for logging only — proxy mode does not consult the
	// catalog Model row.
	ModelName string

	// Lifecycle is the per-request shared context, constructed by the
	// handler before Run. Post-flight observers see it via the registered
	// lifecycle observers (Finalize). Optional — nil skips observer dispatch.
	Lifecycle *lifecycle.Context
}

Request is the pre-resolved input to Run.

type Result

type Result struct {
	Status  int
	Headers http.Header
	// Body MUST be Closed by the caller. Closing triggers the post-
	// flight goroutine (if it hasn't run already).
	Body io.ReadCloser
}

Result is what the handler streams back to the caller.

type TokenExtractor

type TokenExtractor interface {
	ExtractTokens(body []byte) sdkusage.Tokens
}

TokenExtractor extracts upstream usage from a response body. The inference handler picks the right extractor by inbound endpoint (OpenAI shape for /v1/chat/completions, Anthropic shape for /v1/messages) and passes it in.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL