extproc-replay-server

Overview ¶

replay-server reads paperd wire-trace bundles and serves the captured upstream responses back over HTTP, so extproc can be exercised against real upstream streaming bytes inside a cluster instead of against synthetic curls.

Synthetic adversarial probes (reducer_adversarial_test.go) cover the shapes that can drive the reducer into the empty-Content surface — empty body, compressed payload, JSON error envelope, post-message_start truncation, out-of-shape chunks. Synthetic probes confirm the reducer would fail on any of them in isolation; only real captures answer which shape an upstream actually emitted on a given turn. This binary reads turn-* capture bundles produced by a proxy/extproc build with explicit wire-capture support and serves the captured response.sse files back as if it were the upstream LLM provider.

Layout expected ¶

<captures-dir>/turn-<ns>-<seq>/
  request.json     { method, url, headers, body_b64, ... }
  response.sse     raw upstream bytes
  meta.json        { status, content_type, content_encoding, ... }

Wire behaviour ¶

On every request, the server picks one bundle and replays:

• The captured HTTP status from meta.json
• The captured Content-Type (and Content-Encoding, if any)
• The raw response.sse bytes verbatim

Selection order, first match wins:

1. X-Replay-Turn: <turn-dir-name> header → that specific bundle
2. ?turn=<turn-dir-name> query param → same as above
3. Round-robin over all loaded bundles

Endpoints ¶

• GET /_replay/healthz → 200 with bundle count
• GET /_replay/list → JSON list of every bundle
• any other path → replay one bundle as described

Usage:

replay-server --captures-dir /path/to/wire-captures

Listening ¶

The default bind is loopback. What this server hands out is captured upstream traffic — real model responses, and whatever was in the prompts that produced them — so binding every interface by default would publish that to anything able to reach the host, which for the ad-hoc captures this is usually pointed at is a disclosure rather than an inconvenience.

Exercising extproc through Envoy inside a cluster needs a reachable address, and that stays supported — it just has to be asked for:

replay-server --captures-dir /captures --listen 0.0.0.0:8080