Documentation
¶
Overview ¶
Package gateway holds the traffic-plane logic shared by the halo-kong plugin and halo-shadow: request inspection, cohort decision and model rewrite. It is pure (no Kong, no network) so the plugin stays a thin adapter.
Index ¶
- Constants
- func EventStreamToSSE(rc io.ReadCloser) io.ReadCloser
- func ForceNonStreaming(protocol, path string, body []byte) (string, []byte, error)
- func HarnessFromUA(ua string) string
- func IdentityHeaderFor(org *policy.Org, override string) string
- func Inspect(protocol, path string, body []byte) (model string, firstTurn bool)
- func IsCountTokens(p string) bool
- func ParseUA(ua string) (harness, version string)
- func ProtocolForPath(p string) string
- func RewriteRequest(protocol, path string, body []byte, model string) (string, []byte, error)
- func RingGate(org *policy.Org, ring, gate string) string
- func RingPins(org *policy.Org, ring string) map[string][]string
- func RouteOrder(alias string, r policy.ModelRoute, key string) []policy.RouteTarget
- func SessionID(h http.Header) string
- func StickyKey(userID, sessionID string) string
- func Target(raw string) (scheme, host string, port int, pathPrefix string, err error)
- type Breaker
- type Decision
- type Finding
- type KillEnvelope
- type KillList
- type KillSwitch
- type Outbound
- type PostureClient
- type PostureVerdict
- type Rejection
- type RequestInfo
- type Result
- type RouteRow
- type ShadowTarget
- type Snapshot
- type TargetRow
Constants ¶
const ( HaloPrefix = "x-halo-" HeaderRing = "x-halo-ring" HeaderRelease = "x-halo-release" HeaderExperiment = "x-halo-experiment" HeaderVariant = "x-halo-variant" RingUnknown = "unknown" )
Headers the gateway stamps on upstream requests. Everything under HaloPrefix is owned by the gateway; client-supplied values are always discarded.
const ( GatePosture = "posture" GateVersion = "version" )
Gates the traffic plane applies per ring (policy Ring.Posture / Ring.VersionGate).
const ( DefaultPostureTTL = time.Minute // a verdict is reused this long before halo-server is asked again DefaultPostureGrace = 15 * time.Minute // a cached verdict outlives an unreachable halo-server this long )
Posture defaults.
const ( ProtoAnthropic = "anthropic-messages" ProtoBedrock = "bedrock-invoke" ProtoResponses = "openai-responses" ProtoGemini = "gemini" )
Wire protocols (match policy.Gateway.Protocols values).
const ( PathMessages = "/v1/messages" PathCountTokens = "/v1/messages/count_tokens" //nolint:gosec // URL path, not a credential PathResponses = "/v1/responses" )
Exact model-call paths. Anything else under a model-bearing prefix (/v1/messages/*, /v1/responses/*, /model/*) is rejected by Prepare.
const ( DefaultKillInterval = 10 * time.Second DefaultKillMaxAge = 10 * time.Minute )
Kill-switch poller defaults.
Variables ¶
This section is empty.
Functions ¶
func EventStreamToSSE ¶
func EventStreamToSSE(rc io.ReadCloser) io.ReadCloser
EventStreamToSSE converts an AWS event stream (Bedrock invoke-with-response-stream) carrying Anthropic events into the SSE the Messages API streams: each chunk's {"bytes": base64(event json)} becomes "event: <type>\ndata: <json>\n\n". Exception frames become an SSE error event.
func ForceNonStreaming ¶
ForceNonStreaming makes a request non-streaming (used for shadow copies).
func HarnessFromUA ¶
HarnessFromUA maps a User-Agent to a harness adapter name.
func IdentityHeaderFor ¶
IdentityHeaderFor picks the configured header, falling back to the org's.
func Inspect ¶
Inspect extracts the requested model alias and whether the request is a first turn. Bedrock carries the alias in the path, the others in the body. Unparseable bodies yield ("", false): never shadowed, routed by default.
func IsCountTokens ¶
IsCountTokens reports whether p is a token-counting call (never mirrored, excluded from request metrics).
func ParseUA ¶
ParseUA returns the harness and CLI version a User-Agent names. The formats (sources in docs/adr/0011-gateway-enforced-posture-and-version.md):
claude-cli/2.1.287 (external, sdk-cli) Claude Code codex_cli_rs/0.99.0 (Mac OS 15.1.0; arm64) iTerm.app Codex (originator/version (os; arch) terminal) GeminiCLI/0.26.0/gemini-2.5-pro (darwin; x64) Gemini CLI (GeminiCLI[-client]/version/model (platform; arch[; surface]))
The version is the token after the first '/', up to a space, '/', '(' or ';'. version is "" when the harness is known but no semver follows; harness is "" for an unrecognised UA (Copilot CLI's is not documented and has none).
func ProtocolForPath ¶
ProtocolForPath classifies a request path by exact match; "" means "not a model call". count_tokens is anthropic-messages (it carries a model alias).
func RewriteRequest ¶
RewriteRequest points a request at upstream model `model`: the JSON "model" field for anthropic-messages / openai-responses, the /model/{id}/ path segment for bedrock-invoke. Only the request is touched; responses stream through untouched. Empty model or unknown protocol is a no-op.
func RingGate ¶
RingGate is ring's mode for gate (GatePosture or GateVersion); off when the ring is unknown.
func RingPins ¶
RingPins returns the CLI versions ring r's devices may run per harness: the ring profile's pin plus the variant profiles of its running client-axis experiment (the gateway cannot tell which variant a device applied).
func RouteOrder ¶
func RouteOrder(alias string, r policy.ModelRoute, key string) []policy.RouteTarget
RouteOrder returns the route's targets in attempt order for key. Targets are grouped into priority tiers (ascending). Within a tier the weighted pick (internal/assign) goes first and the rest follow in policy order; a tier without weights keeps policy order. The first element is the target a request lands on when everything is healthy.
Types ¶
type Breaker ¶
type Breaker struct {
Threshold int // consecutive failures that open the circuit (default 5)
Cooldown time.Duration // open time before a half-open probe (default 30s)
// contains filtered or unexported fields
}
Breaker is a per-key circuit breaker. After Threshold consecutive failures a key is open for Cooldown; then exactly one probe request is let through (half-open): success closes it, failure re-opens it for another Cooldown. Safe for concurrent use. Allow must be followed by Done or Cancel.
func (*Breaker) Cancel ¶
Cancel releases an allowed request that never produced a verdict (the client went away), so a half-open probe slot is not leaked.
type Decision ¶
type Decision struct {
Ring string
Release string
Experiment string
Variant string
UpstreamName string
Upstream string // URL; "" = leave Kong's default service
UpstreamModel string // "" = leave the model untouched
// Route is the ordered attempt list (failover order) for the alias;
// UpstreamName/UpstreamModel/Upstream are its first element. halo-kong
// uses only those; halo-proxy walks Route.
Route []policy.RouteTarget
Headers map[string]string
Shadow []ShadowTarget
}
Decision is the routing outcome for one request.
type Finding ¶
type Finding struct {
Gate string // GatePosture | GateVersion
Mode string // policy.GateWarn | policy.GateEnforce
Reason string
}
Finding is a failed gate check under a ring whose mode is warn or enforce.
func CheckPosture ¶
func CheckPosture(ctx context.Context, c *PostureClient, org *policy.Org, ring string, sub *policy.Subject) (f *Finding, unknown bool)
CheckPosture applies ring's posture gate to a verified subject. unknown reports that the gate is on but no verdict was available (no client, an anonymous caller, or halo-server unreachable past the grace window): the request passes and callers count it.
func CheckVersion ¶
CheckVersion applies ring's versionGate to a request's User-Agent. nil = pass (or gate off). A ring that pins no version for the detected harness passes: there is nothing to compare against.
type KillEnvelope ¶
type KillEnvelope struct {
Payload []byte `json:"payload"` // base64 in JSON
Signature []byte `json:"signature"` // base64 in JSON
}
KillEnvelope is GET /api/v1/gateway/killswitch: Payload is the JSON KillList, Signature is ed25519 over killDomain+Payload.
func SignKillList ¶
func SignKillList(key ed25519.PrivateKey, l KillList) (KillEnvelope, error)
SignKillList seals l for gateways.
type KillList ¶
type KillList struct {
Version uint64 `json:"version"` // grows with every kill/unkill; informational
Experiments []string `json:"experiments"`
// Toggles are killed feature-toggle names: treated as off everywhere, whatever
// their rules say. Part of the signed payload, so covered by the same
// signature, freshness and monotonicity checks as Experiments.
Toggles []string `json:"toggles,omitempty"`
IssuedAt time.Time `json:"issuedAt"`
}
KillList is the set of experiments halo-server has killed: gateways treat them as not running (control routing, no shadow) without waiting for a policy PR to merge and a snapshot to ship.
type KillSwitch ¶
type KillSwitch struct {
URL string
Token string
PubKey ed25519.PublicKey
Interval time.Duration // default DefaultKillInterval
MaxAge time.Duration // default DefaultKillMaxAge
Client *http.Client // default: 10s timeout, no redirects
Log *slog.Logger
Now func() time.Time
// contains filtered or unexported fields
}
KillSwitch polls halo-server for the signed kill list.
Failure policy: a fetch or verification failure keeps the last accepted list (a kill never silently lapses because the control plane is down); if no list was ever accepted the set is empty and the policy snapshot's statuses alone decide. A list is accepted only if it is strictly newer (IssuedAt) than the current one, so a replayed older envelope cannot undo a kill. A nil *KillSwitch is valid and kills nothing.
func NewKillSwitch ¶
func NewKillSwitch(url, token string, pub ed25519.PublicKey, interval time.Duration, log *slog.Logger, allowInsecureInCluster bool) (*KillSwitch, error)
NewKillSwitch validates the config (see bundle.ParseEd25519Verifier for PEM keys). allowInsecureInCluster permits plain http to a non-loopback host.
func (*KillSwitch) Apply ¶
func (k *KillSwitch) Apply(org *policy.Org) *policy.Org
Apply returns org with every killed experiment's status forced to "paused", so Decide routes its users to default (control) and mirrors nothing. org itself is never mutated; it is returned as-is when nothing applies. ponytail: copies per request while a kill is active; cache by (org, version) if it shows in profiles.
func (*KillSwitch) Killed ¶
func (k *KillSwitch) Killed() KillList
Killed returns the current list (a copy).
func (*KillSwitch) Refresh ¶
func (k *KillSwitch) Refresh(ctx context.Context) error
Refresh fetches and verifies the list once; on error the previous list stays.
func (*KillSwitch) Restore ¶
func (k *KillSwitch) Restore(l KillList)
Restore seeds a last-known list persisted by the caller (halod keeps it in its state file) so kills survive a restart and envelopes issued before it are refused as replays. It is a no-op unless l is newer than the current list. The list is trusted as given: pass only one that was verified.
func (*KillSwitch) Run ¶
func (k *KillSwitch) Run(ctx context.Context)
Run refreshes immediately and then every Interval until ctx ends.
type Outbound ¶
type Outbound struct {
Path string
RawQuery string
SetQuery bool // replace the client's query string with RawQuery (else keep it)
Body []byte
// SSE: the response is a Bedrock event stream the anthropic-messages client
// cannot read; convert it with EventStreamToSSE.
SSE bool
}
Outbound is the request an upstream expects, derived from the client's request. Path is escaped and relative to the upstream's base URL.
func BuildOutbound ¶
func BuildOutbound(proto string, up policy.Upstream, model, path string, body []byte) (Outbound, error)
BuildOutbound rewrites the client's request (protocol proto, escaped path, body) for upstream up serving model. Same-wire upstreams only get the model swapped (RewriteRequest). Translations:
anthropic-messages -> bedrock POST /model/{id}/invoke[-with-response-stream], body without model/stream + anthropic_version
anthropic-messages -> vertex POST .../publishers/anthropic/models/{id}:[stream]rawPredict, body without model + anthropic_version
openai-responses -> azure POST /openai/v1/responses (or /openai/responses?api-version=), model = deployment
Responses stay in the provider's format except Bedrock streams, which are converted to SSE (see Outbound.SSE).
type PostureClient ¶
type PostureClient struct {
URL, Token string
TTL, Grace time.Duration
Client *http.Client
Now func() time.Time
Log *slog.Logger
// contains filtered or unexported fields
}
PostureClient asks halo-server for a subject's device posture and caches the verdict. It fails closed only on a verdict halo-server returned: when the server cannot be reached, the last verdict serves for Grace, and after that the posture is unknown (callers let the request through and count it).
func NewPostureClient ¶
func NewPostureClient(url, token string, ttl, grace time.Duration, log *slog.Logger, allowInsecureInCluster bool) (*PostureClient, error)
NewPostureClient validates the config (https unless loopback or allowInsecureInCluster).
func (*PostureClient) Check ¶
func (c *PostureClient) Check(ctx context.Context, subject string) (v PostureVerdict, known bool)
Check returns subject's verdict; known=false when halo-server could not be asked and no verdict younger than Grace is cached. ponytail: concurrent misses for one subject each call halo-server; add singleflight if a burst after TTL expiry shows up in the server's load.
type PostureVerdict ¶
type PostureVerdict struct {
Compliant bool `json:"compliant"`
Reason string `json:"reason,omitempty"`
}
PostureVerdict is halo-server's answer for one subject (GET /api/v1/gateway/posture).
type Rejection ¶
type Rejection struct {
Status int
Protocol string // selects the error body shape
Message string
Code string // OpenAI Responses error code; "" = derived from Status
}
Rejection is a request the gateway answers itself. Fail closed: unknown model endpoints (404), unreadable/oversize bodies (413), models that are not a configured alias (400: a permanent client error, which the CLIs do not retry, unlike 403/5xx/429).
func RejectTooLarge ¶
RejectTooLarge is the 413 for a model call whose body could not be read in full (Kong spooled it to disk, or it exceeds the proxy limit).
type RequestInfo ¶
type RequestInfo struct {
UserID string // from the auth gateway's identity header only
Groups []string
ModelAlias string
SessionID string
Harness string
Protocol string
FirstTurn bool
}
RequestInfo is everything Decide may look at.
func NewRequestInfo ¶
func NewRequestInfo(h http.Header, path string, body []byte) RequestInfo
NewRequestInfo builds RequestInfo from the request alone. UserID/Groups are left empty: cohort comes only from a verified identity (PrepareVerified).
type Result ¶
type Result struct {
Decision Decision
Protocol string
// ClearHeaders are client-supplied x-halo-* names to drop (all of them; the
// ones Decision sets are then re-set with gateway-owned values).
ClearHeaders []string
// Path/Body are the rewritten forms; Rewritten is false when unchanged.
Path string
Body []byte
Rewritten bool
// RewriteErr is non-nil when a rewrite was needed but failed (Reject is
// then set too: an un-rewritten alias must never reach the upstream).
RewriteErr error
// Reject, when non-nil, must be answered by the adapter itself; the
// request must NOT be forwarded.
Reject *Rejection
}
Result is everything the Kong plugin must apply to the upstream request.
func PrepareVerified ¶
func PrepareVerified(org *policy.Org, sub *policy.Subject, h http.Header, path string, body []byte) Result
PrepareVerified inspects a request and computes the full set of mutations, or a Rejection the adapter must answer instead of forwarding. Cohort comes only from sub, verified by internal/identity (JWT, or trusted-proxy headers); sub == nil means anonymous: default routing, no experiments. Client-supplied headers (x-halo-* or identity) never influence the outcome.
type RouteRow ¶
type RouteRow struct {
Alias string `json:"alias"`
Ring string `json:"ring"`
Experiment string `json:"experiment,omitempty"` // traffic-axis experiment that supplied the route
Variant string `json:"variant,omitempty"`
Targets []TargetRow `json:"targets"`
}
RouteRow is the effective route of one alias for a caller.
func RouteTable ¶
func RouteTable(org *policy.Org, req RequestInfo) []RouteRow
RouteTable returns every alias's effective routes for req (UserID, Groups and SessionID decide the ring, experiment variant and weighted pick; ModelAlias is ignored). Anonymous callers (no user, no session) see policy order.
type ShadowTarget ¶
ShadowTarget is one candidate a request should be mirrored to.
type Snapshot ¶
type Snapshot struct {
// contains filtered or unexported fields
}
Snapshot serves a compiled org policy (policy.Compile output, or a bare json.Marshal of *policy.Org) from a file and reloads it when the mtime/size changes. A failed reload keeps serving the last good policy.
func NewSnapshot ¶
type TargetRow ¶
type TargetRow struct {
Order int `json:"order"`
Selected bool `json:"selected"`
Upstream string `json:"upstream"`
Kind string `json:"kind"`
Model string `json:"model"`
Weight float64 `json:"weight,omitempty"`
Priority int `json:"priority"`
TimeoutSeconds int `json:"timeoutSeconds,omitempty"`
SkippedBy []string `json:"skippedByHarnesses,omitempty"` // harnesses whose wire protocol this target cannot serve
}
TargetRow is one attempt-order entry; Selected marks where a request lands when every target is healthy.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package adapters renders the "put halo-proxy in front of / behind stack X" config snippets shipped under deploy/integrations.
|
Package adapters renders the "put halo-proxy in front of / behind stack X" config snippets shipped under deploy/integrations. |
|
Package kong generates the decK declarative config for Kong OSS (db-less) that fronts an org's model traffic and attaches the halo-kong plugin.
|
Package kong generates the decK declarative config for Kong OSS (db-less) that fronts an org's model traffic and attaches the halo-kong plugin. |
|
Package upstreamauth signs requests to provider upstreams that need a gateway-held credential instead of the caller's, so halo-proxy can talk to Amazon Bedrock directly (no orchestrator, no SigV4 sidecar).
|
Package upstreamauth signs requests to provider upstreams that need a gateway-held credential instead of the caller's, so halo-proxy can talk to Amazon Bedrock directly (no orchestrator, no SigV4 sidecar). |