gateway

package
v0.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 12, 2026 License: Apache-2.0 Imports: 28 Imported by: 0

Documentation

Overview

Package gateway holds the traffic-plane logic shared by the halo-kong plugin and halo-shadow: request inspection, cohort decision and model rewrite. It is pure (no Kong, no network) so the plugin stays a thin adapter.

Index

Constants

View Source
const (
	HaloPrefix       = "x-halo-"
	HeaderRing       = "x-halo-ring"
	HeaderRelease    = "x-halo-release"
	HeaderExperiment = "x-halo-experiment"
	HeaderVariant    = "x-halo-variant"

	RingUnknown = "unknown"
)

Headers the gateway stamps on upstream requests. Everything under HaloPrefix is owned by the gateway; client-supplied values are always discarded.

View Source
const (
	GatePosture = "posture"
	GateVersion = "version"
)

Gates the traffic plane applies per ring (policy Ring.Posture / Ring.VersionGate).

View Source
const (
	DefaultPostureTTL   = time.Minute      // a verdict is reused this long before halo-server is asked again
	DefaultPostureGrace = 15 * time.Minute // a cached verdict outlives an unreachable halo-server this long

)

Posture defaults.

View Source
const (
	ProtoAnthropic = "anthropic-messages"
	ProtoBedrock   = "bedrock-invoke"
	ProtoResponses = "openai-responses"
	ProtoGemini    = "gemini"
)

Wire protocols (match policy.Gateway.Protocols values).

View Source
const (
	PathMessages    = "/v1/messages"
	PathCountTokens = "/v1/messages/count_tokens" //nolint:gosec // URL path, not a credential
	PathResponses   = "/v1/responses"
)

Exact model-call paths. Anything else under a model-bearing prefix (/v1/messages/*, /v1/responses/*, /model/*) is rejected by Prepare.

View Source
const (
	DefaultKillInterval = 10 * time.Second
	DefaultKillMaxAge   = 10 * time.Minute
)

Kill-switch poller defaults.

Variables

This section is empty.

Functions

func EventStreamToSSE

func EventStreamToSSE(rc io.ReadCloser) io.ReadCloser

EventStreamToSSE converts an AWS event stream (Bedrock invoke-with-response-stream) carrying Anthropic events into the SSE the Messages API streams: each chunk's {"bytes": base64(event json)} becomes "event: <type>\ndata: <json>\n\n". Exception frames become an SSE error event.

func ForceNonStreaming

func ForceNonStreaming(protocol, path string, body []byte) (string, []byte, error)

ForceNonStreaming makes a request non-streaming (used for shadow copies).

func HarnessFromUA

func HarnessFromUA(ua string) string

HarnessFromUA maps a User-Agent to a harness adapter name.

func IdentityHeaderFor

func IdentityHeaderFor(org *policy.Org, override string) string

IdentityHeaderFor picks the configured header, falling back to the org's.

func Inspect

func Inspect(protocol, path string, body []byte) (model string, firstTurn bool)

Inspect extracts the requested model alias and whether the request is a first turn. Bedrock carries the alias in the path, the others in the body. Unparseable bodies yield ("", false): never shadowed, routed by default.

func IsCountTokens

func IsCountTokens(p string) bool

IsCountTokens reports whether p is a token-counting call (never mirrored, excluded from request metrics).

func ParseUA

func ParseUA(ua string) (harness, version string)

ParseUA returns the harness and CLI version a User-Agent names. The formats (sources in docs/adr/0011-gateway-enforced-posture-and-version.md):

claude-cli/2.1.287 (external, sdk-cli)               Claude Code
codex_cli_rs/0.99.0 (Mac OS 15.1.0; arm64) iTerm.app  Codex (originator/version (os; arch) terminal)
GeminiCLI/0.26.0/gemini-2.5-pro (darwin; x64)        Gemini CLI (GeminiCLI[-client]/version/model (platform; arch[; surface]))

The version is the token after the first '/', up to a space, '/', '(' or ';'. version is "" when the harness is known but no semver follows; harness is "" for an unrecognised UA (Copilot CLI's is not documented and has none).

func ProtocolForPath

func ProtocolForPath(p string) string

ProtocolForPath classifies a request path by exact match; "" means "not a model call". count_tokens is anthropic-messages (it carries a model alias).

func RewriteRequest

func RewriteRequest(protocol, path string, body []byte, model string) (string, []byte, error)

RewriteRequest points a request at upstream model `model`: the JSON "model" field for anthropic-messages / openai-responses, the /model/{id}/ path segment for bedrock-invoke. Only the request is touched; responses stream through untouched. Empty model or unknown protocol is a no-op.

func RingGate

func RingGate(org *policy.Org, ring, gate string) string

RingGate is ring's mode for gate (GatePosture or GateVersion); off when the ring is unknown.

func RingPins

func RingPins(org *policy.Org, ring string) map[string][]string

RingPins returns the CLI versions ring r's devices may run per harness: the ring profile's pin plus the variant profiles of its running client-axis experiment (the gateway cannot tell which variant a device applied).

func RouteOrder

func RouteOrder(alias string, r policy.ModelRoute, key string) []policy.RouteTarget

RouteOrder returns the route's targets in attempt order for key. Targets are grouped into priority tiers (ascending). Within a tier the weighted pick (internal/assign) goes first and the rest follow in policy order; a tier without weights keeps policy order. The first element is the target a request lands on when everything is healthy.

func SessionID

func SessionID(h http.Header) string

SessionID is the harness session id: Claude Code's header, then Codex's.

func StickyKey

func StickyKey(userID, sessionID string) string

StickyKey is what weighted route choice hashes: the verified identity plus the harness session, so a session stays on one target. "" (anonymous, no session) means no stickiness is possible: the first-priority target is used.

func Target

func Target(raw string) (scheme, host string, port int, pathPrefix string, err error)

Target splits an upstream URL for Kong's balancer.

Types

type Breaker

type Breaker struct {
	Threshold int           // consecutive failures that open the circuit (default 5)
	Cooldown  time.Duration // open time before a half-open probe (default 30s)
	// contains filtered or unexported fields
}

Breaker is a per-key circuit breaker. After Threshold consecutive failures a key is open for Cooldown; then exactly one probe request is let through (half-open): success closes it, failure re-opens it for another Cooldown. Safe for concurrent use. Allow must be followed by Done or Cancel.

func (*Breaker) Allow

func (b *Breaker) Allow(key string) bool

Allow reports whether a request may be sent to key now.

func (*Breaker) Cancel

func (b *Breaker) Cancel(key string)

Cancel releases an allowed request that never produced a verdict (the client went away), so a half-open probe slot is not leaked.

func (*Breaker) Done

func (b *Breaker) Done(key string, ok bool)

Done records the outcome of an allowed request.

func (*Breaker) State

func (b *Breaker) State(key string) string

State is "closed", "open" or "half-open" (an expired open circuit).

type Decision

type Decision struct {
	Ring          string
	Release       string
	Experiment    string
	Variant       string
	UpstreamName  string
	Upstream      string // URL; "" = leave Kong's default service
	UpstreamModel string // "" = leave the model untouched
	// Route is the ordered attempt list (failover order) for the alias;
	// UpstreamName/UpstreamModel/Upstream are its first element. halo-kong
	// uses only those; halo-proxy walks Route.
	Route   []policy.RouteTarget
	Headers map[string]string
	Shadow  []ShadowTarget
}

Decision is the routing outcome for one request.

func Decide

func Decide(org *policy.Org, req RequestInfo) Decision

Decide is pure and deterministic for a given (org, req).

type Finding

type Finding struct {
	Gate   string // GatePosture | GateVersion
	Mode   string // policy.GateWarn | policy.GateEnforce
	Reason string
}

Finding is a failed gate check under a ring whose mode is warn or enforce.

func CheckPosture

func CheckPosture(ctx context.Context, c *PostureClient, org *policy.Org, ring string, sub *policy.Subject) (f *Finding, unknown bool)

CheckPosture applies ring's posture gate to a verified subject. unknown reports that the gate is on but no verdict was available (no client, an anonymous caller, or halo-server unreachable past the grace window): the request passes and callers count it.

func CheckVersion

func CheckVersion(org *policy.Org, ring, ua string) *Finding

CheckVersion applies ring's versionGate to a request's User-Agent. nil = pass (or gate off). A ring that pins no version for the detected harness passes: there is nothing to compare against.

func (*Finding) Message

func (f *Finding) Message() string

Message is what the developer's CLI shows on a refusal: what failed and what to run.

func (*Finding) Reject

func (f *Finding) Reject(proto string) *Rejection

Reject is the 403 to answer in enforce mode, in the caller's wire format; nil in warn mode.

type KillEnvelope

type KillEnvelope struct {
	Payload   []byte `json:"payload"`   // base64 in JSON
	Signature []byte `json:"signature"` // base64 in JSON
}

KillEnvelope is GET /api/v1/gateway/killswitch: Payload is the JSON KillList, Signature is ed25519 over killDomain+Payload.

func SignKillList

func SignKillList(key ed25519.PrivateKey, l KillList) (KillEnvelope, error)

SignKillList seals l for gateways.

type KillList

type KillList struct {
	Version     uint64   `json:"version"` // grows with every kill/unkill; informational
	Experiments []string `json:"experiments"`
	// Toggles are killed feature-toggle names: treated as off everywhere, whatever
	// their rules say. Part of the signed payload, so covered by the same
	// signature, freshness and monotonicity checks as Experiments.
	Toggles  []string  `json:"toggles,omitempty"`
	IssuedAt time.Time `json:"issuedAt"`
}

KillList is the set of experiments halo-server has killed: gateways treat them as not running (control routing, no shadow) without waiting for a policy PR to merge and a snapshot to ship.

func VerifyKillList

func VerifyKillList(pub ed25519.PublicKey, env KillEnvelope, now time.Time, maxAge time.Duration) (KillList, error)

VerifyKillList checks the signature and freshness (IssuedAt within maxAge of now, at most a minute in the future) and returns the list.

type KillSwitch

type KillSwitch struct {
	URL      string
	Token    string
	PubKey   ed25519.PublicKey
	Interval time.Duration // default DefaultKillInterval
	MaxAge   time.Duration // default DefaultKillMaxAge
	Client   *http.Client  // default: 10s timeout, no redirects
	Log      *slog.Logger
	Now      func() time.Time
	// contains filtered or unexported fields
}

KillSwitch polls halo-server for the signed kill list.

Failure policy: a fetch or verification failure keeps the last accepted list (a kill never silently lapses because the control plane is down); if no list was ever accepted the set is empty and the policy snapshot's statuses alone decide. A list is accepted only if it is strictly newer (IssuedAt) than the current one, so a replayed older envelope cannot undo a kill. A nil *KillSwitch is valid and kills nothing.

func NewKillSwitch

func NewKillSwitch(url, token string, pub ed25519.PublicKey, interval time.Duration, log *slog.Logger, allowInsecureInCluster bool) (*KillSwitch, error)

NewKillSwitch validates the config (see bundle.ParseEd25519Verifier for PEM keys). allowInsecureInCluster permits plain http to a non-loopback host.

func (*KillSwitch) Apply

func (k *KillSwitch) Apply(org *policy.Org) *policy.Org

Apply returns org with every killed experiment's status forced to "paused", so Decide routes its users to default (control) and mirrors nothing. org itself is never mutated; it is returned as-is when nothing applies. ponytail: copies per request while a kill is active; cache by (org, version) if it shows in profiles.

func (*KillSwitch) Killed

func (k *KillSwitch) Killed() KillList

Killed returns the current list (a copy).

func (*KillSwitch) Refresh

func (k *KillSwitch) Refresh(ctx context.Context) error

Refresh fetches and verifies the list once; on error the previous list stays.

func (*KillSwitch) Restore

func (k *KillSwitch) Restore(l KillList)

Restore seeds a last-known list persisted by the caller (halod keeps it in its state file) so kills survive a restart and envelopes issued before it are refused as replays. It is a no-op unless l is newer than the current list. The list is trusted as given: pass only one that was verified.

func (*KillSwitch) Run

func (k *KillSwitch) Run(ctx context.Context)

Run refreshes immediately and then every Interval until ctx ends.

type Outbound

type Outbound struct {
	Path     string
	RawQuery string
	SetQuery bool // replace the client's query string with RawQuery (else keep it)
	Body     []byte
	// SSE: the response is a Bedrock event stream the anthropic-messages client
	// cannot read; convert it with EventStreamToSSE.
	SSE bool
}

Outbound is the request an upstream expects, derived from the client's request. Path is escaped and relative to the upstream's base URL.

func BuildOutbound

func BuildOutbound(proto string, up policy.Upstream, model, path string, body []byte) (Outbound, error)

BuildOutbound rewrites the client's request (protocol proto, escaped path, body) for upstream up serving model. Same-wire upstreams only get the model swapped (RewriteRequest). Translations:

anthropic-messages -> bedrock   POST /model/{id}/invoke[-with-response-stream], body without model/stream + anthropic_version
anthropic-messages -> vertex    POST .../publishers/anthropic/models/{id}:[stream]rawPredict, body without model + anthropic_version
openai-responses   -> azure     POST /openai/v1/responses (or /openai/responses?api-version=), model = deployment

Responses stay in the provider's format except Bedrock streams, which are converted to SSE (see Outbound.SSE).

type PostureClient

type PostureClient struct {
	URL, Token string
	TTL, Grace time.Duration
	Client     *http.Client
	Now        func() time.Time
	Log        *slog.Logger
	// contains filtered or unexported fields
}

PostureClient asks halo-server for a subject's device posture and caches the verdict. It fails closed only on a verdict halo-server returned: when the server cannot be reached, the last verdict serves for Grace, and after that the posture is unknown (callers let the request through and count it).

func NewPostureClient

func NewPostureClient(url, token string, ttl, grace time.Duration, log *slog.Logger, allowInsecureInCluster bool) (*PostureClient, error)

NewPostureClient validates the config (https unless loopback or allowInsecureInCluster).

func (*PostureClient) Check

func (c *PostureClient) Check(ctx context.Context, subject string) (v PostureVerdict, known bool)

Check returns subject's verdict; known=false when halo-server could not be asked and no verdict younger than Grace is cached. ponytail: concurrent misses for one subject each call halo-server; add singleflight if a burst after TTL expiry shows up in the server's load.

type PostureVerdict

type PostureVerdict struct {
	Compliant bool   `json:"compliant"`
	Reason    string `json:"reason,omitempty"`
}

PostureVerdict is halo-server's answer for one subject (GET /api/v1/gateway/posture).

type Rejection

type Rejection struct {
	Status   int
	Protocol string // selects the error body shape
	Message  string
	Code     string // OpenAI Responses error code; "" = derived from Status
}

Rejection is a request the gateway answers itself. Fail closed: unknown model endpoints (404), unreadable/oversize bodies (413), models that are not a configured alias (400: a permanent client error, which the CLIs do not retry, unlike 403/5xx/429).

func RejectTooLarge

func RejectTooLarge(path string) *Rejection

RejectTooLarge is the 413 for a model call whose body could not be read in full (Kong spooled it to disk, or it exceeds the proxy limit).

func (*Rejection) Error

func (r *Rejection) Error() string

func (*Rejection) JSON

func (r *Rejection) JSON() []byte

JSON renders the error in the caller's wire format so the CLI shows Message.

type RequestInfo

type RequestInfo struct {
	UserID     string // from the auth gateway's identity header only
	Groups     []string
	ModelAlias string
	SessionID  string
	Harness    string
	Protocol   string
	FirstTurn  bool
}

RequestInfo is everything Decide may look at.

func NewRequestInfo

func NewRequestInfo(h http.Header, path string, body []byte) RequestInfo

NewRequestInfo builds RequestInfo from the request alone. UserID/Groups are left empty: cohort comes only from a verified identity (PrepareVerified).

type Result

type Result struct {
	Decision Decision
	Protocol string
	// ClearHeaders are client-supplied x-halo-* names to drop (all of them; the
	// ones Decision sets are then re-set with gateway-owned values).
	ClearHeaders []string
	// Path/Body are the rewritten forms; Rewritten is false when unchanged.
	Path      string
	Body      []byte
	Rewritten bool
	// RewriteErr is non-nil when a rewrite was needed but failed (Reject is
	// then set too: an un-rewritten alias must never reach the upstream).
	RewriteErr error
	// Reject, when non-nil, must be answered by the adapter itself; the
	// request must NOT be forwarded.
	Reject *Rejection
}

Result is everything the Kong plugin must apply to the upstream request.

func PrepareVerified

func PrepareVerified(org *policy.Org, sub *policy.Subject, h http.Header, path string, body []byte) Result

PrepareVerified inspects a request and computes the full set of mutations, or a Rejection the adapter must answer instead of forwarding. Cohort comes only from sub, verified by internal/identity (JWT, or trusted-proxy headers); sub == nil means anonymous: default routing, no experiments. Client-supplied headers (x-halo-* or identity) never influence the outcome.

type RouteRow

type RouteRow struct {
	Alias      string      `json:"alias"`
	Ring       string      `json:"ring"`
	Experiment string      `json:"experiment,omitempty"` // traffic-axis experiment that supplied the route
	Variant    string      `json:"variant,omitempty"`
	Targets    []TargetRow `json:"targets"`
}

RouteRow is the effective route of one alias for a caller.

func RouteTable

func RouteTable(org *policy.Org, req RequestInfo) []RouteRow

RouteTable returns every alias's effective routes for req (UserID, Groups and SessionID decide the ring, experiment variant and weighted pick; ModelAlias is ignored). Anonymous callers (no user, no session) see policy order.

type ShadowTarget

type ShadowTarget struct {
	Experiment string
	Variant    string
	Upstream   string // URL
	Model      string
}

ShadowTarget is one candidate a request should be mirrored to.

type Snapshot

type Snapshot struct {
	// contains filtered or unexported fields
}

Snapshot serves a compiled org policy (policy.Compile output, or a bare json.Marshal of *policy.Org) from a file and reloads it when the mtime/size changes. A failed reload keeps serving the last good policy.

func NewSnapshot

func NewSnapshot(path string) *Snapshot

func (*Snapshot) Get

func (s *Snapshot) Get() (*policy.Org, error)

Get returns the current org. err is non-nil if the latest load/reload failed; org may still be the previous good snapshot (nil if none ever loaded).

type TargetRow

type TargetRow struct {
	Order          int      `json:"order"`
	Selected       bool     `json:"selected"`
	Upstream       string   `json:"upstream"`
	Kind           string   `json:"kind"`
	Model          string   `json:"model"`
	Weight         float64  `json:"weight,omitempty"`
	Priority       int      `json:"priority"`
	TimeoutSeconds int      `json:"timeoutSeconds,omitempty"`
	SkippedBy      []string `json:"skippedByHarnesses,omitempty"` // harnesses whose wire protocol this target cannot serve
}

TargetRow is one attempt-order entry; Selected marks where a request lands when every target is healthy.

Directories

Path Synopsis
Package adapters renders the "put halo-proxy in front of / behind stack X" config snippets shipped under deploy/integrations.
Package adapters renders the "put halo-proxy in front of / behind stack X" config snippets shipped under deploy/integrations.
Package kong generates the decK declarative config for Kong OSS (db-less) that fronts an org's model traffic and attaches the halo-kong plugin.
Package kong generates the decK declarative config for Kong OSS (db-less) that fronts an org's model traffic and attaches the halo-kong plugin.
Package upstreamauth signs requests to provider upstreams that need a gateway-held credential instead of the caller's, so halo-proxy can talk to Amazon Bedrock directly (no orchestrator, no SigV4 sidecar).
Package upstreamauth signs requests to provider upstreams that need a gateway-held credential instead of the caller's, so halo-proxy can talk to Amazon Bedrock directly (no orchestrator, no SigV4 sidecar).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL