Documentation
¶
Overview ¶
Package shadow implements the mirror pipeline: halo-kong posts a Job for sampled first-turn requests, halo-shadow replays it against control and candidate (non-streaming) and stores the pair for later LLM-judge grading.
Index ¶
Constants ¶
const DefaultBudgetUSD = 50
DefaultBudgetUSD caps estimated spend when Config.BudgetUSD is unset.
const TokenHeader = "X-Halo-Shadow-Token" //nolint:gosec // header name, not a credential
TokenHeader authenticates halo-kong to halo-shadow.
Variables ¶
var ReplayHeaders = map[string]bool{"anthropic-version": true, "anthropic-beta": true, "openai-beta": true}
ReplayHeaders is the only client-originated header set halo-shadow forwards.
Functions ¶
func NewHTTPClient ¶
NewHTTPClient never follows redirects (a 3xx is recorded as the response: credentials must not travel to wherever an upstream points) and bounds every phase of the call.
Types ¶
type Config ¶
type Config struct {
Queue int // bounded queue length (default 64)
Workers int // default 4
Token string
// Policy returns the current org (gateway.Snapshot.Get). Both sides of a
// pair are resolved from it; jobs never name upstreams. Required.
Policy func() (*policy.Org, error)
// UpstreamHeaders are halo-shadow's own credentials, keyed by policy upstream
// name; they are only ever sent to that upstream's URL.
UpstreamHeaders map[string]map[string]string
// BudgetUSD caps estimated spend (default DefaultBudgetUSD). Each job
// reserves a worst-case estimate before anything is sent, so concurrency
// can't overshoot; the reservation is settled to actual usage afterwards.
BudgetUSD float64
PriceInPerMTok float64
PriceOutPerMTok float64
HTTP *http.Client // default: NewHTTPClient()
Log *slog.Logger
}
Config for the halo-shadow server.
type FileStore ¶
type FileStore struct {
// contains filtered or unexported fields
}
FileStore appends JSONL to a file (mode 0600).
func NewFileStore ¶
func NewFileStore(path string, o StoreOptions) (*FileStore, error)
type Job ¶
type Job struct {
Experiment string `json:"experiment"`
Variant string `json:"variant"`
Ring string `json:"ring,omitempty"`
SessionID string `json:"session_id,omitempty"`
Protocol string `json:"protocol"`
Method string `json:"method"`
Path string `json:"path"`
Headers map[string]string `json:"headers,omitempty"`
Body json.RawMessage `json:"body"`
}
Job is what halo-kong / halo-proxy post. It deliberately carries NO upstream URLs or models: halo-shadow resolves both sides from its own policy snapshot (experiment + variant + the alias in Body/Path), so a caller holding the token can't point halo-shadow (and its credentials) anywhere else. Body is the ORIGINAL client body (alias model). Only allowlisted Headers are replayed. Ring/SessionID are recorded metadata only; they never affect routing.
func Jobs ¶
Jobs builds the mirror jobs for a prepared gateway request: one per shadow target, only the ReplayHeaders allowlist (never client credentials) and never an upstream (halo-shadow resolves both sides from its own policy). It returns nil when nothing should be mirrored or body exceeds maxBytes.
type Mirrorer ¶
type Mirrorer struct {
// contains filtered or unexported fields
}
Mirrorer is the halo-kong side: Submit never blocks the request path. Jobs go through a small bounded queue to a single sender; when full they're dropped and counted.
func NewMirrorer ¶
NewMirrorer validates url and starts the single sender goroutine.
func SharedMirrorer ¶
SharedMirrorer returns one Mirrorer per halo-shadow URL (Kong instantiates a plugin config per route/service; they share the queue).
type Pair ¶
type Pair struct {
ID string `json:"id"`
Time time.Time `json:"time"`
Experiment string `json:"experiment"`
Ring string `json:"ring,omitempty"`
SessionID string `json:"session_id,omitempty"`
Protocol string `json:"protocol"`
Request json.RawMessage `json:"request"`
Control Side `json:"control"`
Candidate Side `json:"candidate"`
CostUSD float64 `json:"cost_usd_estimate"`
}
Pair is one JSONL row. Request + both responses are everything a judge needs.
type Server ¶
type Server struct {
// contains filtered or unexported fields
}
Server is halo-shadow.
func (*Server) MetricsHandler ¶
MetricsHandler serves /metrics without the token, for a separate scrape-only listener (Prometheus can't send X-Halo-Shadow-Token). It exposes counters and spend gauges only; keep it off untrusted networks.
type Side ¶
type Side struct {
Target Target `json:"target"`
Status int `json:"status"`
LatencyMS int64 `json:"latency_ms"`
InputTokens int64 `json:"input_tokens"`
OutputTokens int64 `json:"output_tokens"`
Response json.RawMessage `json:"response,omitempty"`
Error string `json:"error,omitempty"`
}
Side is one response, flattened for the judge.
type StoreOptions ¶
type StoreOptions struct {
// Key (32 bytes) enables AES-256-GCM at rest. Rows become
// {"id","time","kid","enc"}; id/time stay clear for retention pruning.
Key []byte
// Retention: Prune drops rows older than this (0 = keep forever).
Retention time.Duration
}
StoreOptions tunes FileStore. Zero value: plaintext, kept forever.
Source Files