Documentation
¶
Overview ¶
Package gateway holds the traffic-plane logic shared by the halo-kong plugin and halo-shadow: request inspection, cohort decision and model rewrite. It is pure (no Kong, no network) so the plugin stays a thin adapter.
Index ¶
- Constants
- Variables
- func Background(proto string, body []byte) bool
- func CheckDirectoryFailMode(m string) error
- func CheckDirectoryMode(m string) error
- func DetectorIDs() []string
- func EstimateUsage(r UsageResult, e Estimate) budget.Usage
- func EventStreamToSSE(rc io.ReadCloser) io.ReadCloser
- func FailOpenSubject(sub *policy.Subject) *policy.Subject
- func ForceNonStreaming(protocol, path string, body []byte) (string, []byte, error)
- func HarnessFromUA(ua string) string
- func IdentityHeaderFor(org *policy.Org, override string) string
- func Inspect(protocol, path string, body []byte) (model string, firstTurn bool)
- func IsCountTokens(p string) bool
- func MaxOutputTokens(body []byte) int64
- func ParseUA(ua string) (harness, version string)
- func ProtocolForPath(p string) string
- func ReasoningBudget(proto string, body []byte) (bool, int64)
- func RedactionMarker(detector, hash string) string
- func RevocationSubjects(sub *policy.Subject) []string
- func RewriteRequest(protocol, path string, body []byte, model string) (string, []byte, error)
- func RingGate(org *policy.Org, ring, gate string) string
- func RingPins(org *policy.Org, ring string) map[string][]string
- func RouteOrder(alias string, r policy.ModelRoute, key string) []policy.RouteTarget
- func SecretHash(secret string) string
- func ServerStateInput(proto string, body []byte) bool
- func SessionID(h http.Header) string
- func StickyKey(userID, sessionID string) string
- func SubjectUnits(org *policy.Org, sub *policy.Subject) []string
- func Target(raw string) (scheme, host string, port int, pathPrefix string, err error)
- func UsageTap(proto, contentType, contentEncoding string, rc io.ReadCloser, ...) io.ReadCloser
- func UsageTapText(proto, contentType, contentEncoding string, rc io.ReadCloser, ...) io.ReadCloser
- type Breaker
- type BudgetCheck
- type BudgetClient
- func (c *BudgetClient) Check(ctx context.Context, p budget.Principal) (st []budget.Status, known bool)
- func (c *BudgetClient) Counters() (reported, failed, dropped int64)
- func (c *BudgetClient) FlushNow(ctx context.Context) error
- func (c *BudgetClient) Record(p budget.Principal, s budget.Spend, at time.Time)
- func (c *BudgetClient) Run(ctx context.Context)
- func (c *BudgetClient) Time() time.Time
- type BudgetUsage
- type BudgetVerdict
- type DLPBatch
- type DLPEvent
- type DLPFinding
- type DLPReport
- type DLPReporter
- type DLPResponseScanner
- type Decision
- type DirectoryClient
- type DirectoryEntry
- type Estimate
- type Finding
- type KillEnvelope
- type KillList
- type KillSwitch
- type Outbound
- type PostureClient
- type PostureVerdict
- type Rejection
- type RequestInfo
- type Result
- type RevocationList
- type RouteRow
- type ShadowTarget
- type Snapshot
- type TargetRow
- type Tool
- type UsageResult
Constants ¶
const ( // HeaderActor carries the verified actor chain ("planner>reviewer") and // HeaderActorDepth its length upstream. Gateway-owned like every x-halo-*: // a client copy is always discarded. HeaderActor = "x-halo-actor" HeaderActorDepth = "x-halo-actor-depth" // CodeAgentNotPermitted is the Rejection code of every admitAgent refusal. CodeAgentNotPermitted = "agent_not_permitted" )
const ( DefaultBudgetTTL = 30 * time.Second // a verdict is reused this long per principal DefaultBudgetGrace = 15 * time.Minute // a cached verdict outlives an unreachable halo-server this long DefaultBudgetFlush = 5 * time.Second // usage report interval )
Budget client defaults (ADR-0017).
const ( HaloPrefix = "x-halo-" HeaderRing = "x-halo-ring" HeaderRelease = "x-halo-release" HeaderExperiment = "x-halo-experiment" HeaderVariant = "x-halo-variant" // HeaderGate carries warn-mode and unknown gate outcomes (posture=unknown, // version=warn) and HeaderKillswitch a misconfigured kill switch, for the // upstream's log pipeline. Like every owned name they are always cleared, // even when the host's header view was truncated. HeaderGate = "x-halo-gate" HeaderKillswitch = "x-halo-killswitch" RingUnknown = "unknown" )
Headers the gateway stamps on upstream requests. Everything under HaloPrefix is owned by the gateway; client-supplied values are always discarded.
const ( DefaultDirectoryTTL = time.Minute // an entry is reused this long before halo-server is asked again DefaultDirectoryGrace = 5 * time.Minute // a cached entry outlives an unreachable halo-server this long // DefaultRevocationTTL is how long the revocation list (ADR-0026) is // used before halo-server is asked again: with halo-server reachable, a // revocation reaches the gateway within this (plus one fetch). DefaultRevocationTTL = 30 * time.Second )
Directory defaults.
const ( // DirectoryFailOpen (default) keeps the verified subject with no groups: // default routing, no experiments, and still the posture gate and per-user // budgets. A deactivated user is never demoted to an anonymous caller. DirectoryFailOpen = "open" // DirectoryFailClosed refuses the request (ErrDirectoryUnavailable, a 503). DirectoryFailClosed = "closed" )
Directory fail modes (directory.failMode): what Resolve does when the directory cannot be asked and no entry younger than Grace is cached.
const ( // DirectoryModeFull (default) resolves every verified subject in SCIM and // checks halo-server credentials against the revocation list. DirectoryModeFull = "full" // DirectoryModeRevocations only checks halo-server credentials (workload, // delegated agent) against the revocation list (ADR-0026): no SCIM lookup, // so it works on a halo-server without SCIM. DirectoryModeRevocations = "revocations" )
Directory modes (directory.mode): what the directory client asks halo-server.
const ( DLPSecret = "secret" DLPInjection = "injection" // DLPTruncated is DLPReport.Blocked (never a finding kind) when the secret // action is block and the body could not be scanned in full. DLPTruncated = "truncated" )
DLP finding kinds.
const ( CodeSecretDetected = "secret_detected" CodeInjectionDetected = "prompt_injection_detected" CodeScanIncomplete = "dlp_scan_incomplete" )
Codes on a DLP block (OpenAI Responses error code; Rejection.Code).
const ( DefaultDLPFlush = 5 * time.Second MaxDLPEvents = 500 // per batch; halo-server refuses more )
DLP reporter bounds.
const ( DLPRequest = "request" DLPResponse = "response" )
Directions of a DLP event.
const ( GatePosture = "posture" GateVersion = "version" )
Gates the traffic plane applies per ring (policy Ring.Posture / Ring.VersionGate).
const ( DefaultPostureTTL = time.Minute // a verdict is reused this long before halo-server is asked again DefaultPostureGrace = 15 * time.Minute // a cached verdict outlives an unreachable halo-server this long )
Posture defaults.
const ( ProtoAnthropic = "anthropic-messages" ProtoBedrock = "bedrock-invoke" ProtoResponses = "openai-responses" ProtoGemini = "gemini" )
Wire protocols (match policy.Gateway.Protocols values).
const ( PathMessages = "/v1/messages" PathCountTokens = "/v1/messages/count_tokens" //nolint:gosec // URL path, not a credential PathResponses = "/v1/responses" )
Exact model-call paths. Anything else under a model-bearing prefix (/v1/messages/*, /v1/responses/*, /model/*) is rejected by Prepare.
const ( DefaultKillInterval = 10 * time.Second DefaultKillMaxAge = 10 * time.Minute )
Kill-switch poller defaults.
const ( // DefaultOutputCeiling caps the output estimate of a request that sets no // maximum output tokens: no supported model generates more in one call. DefaultOutputCeiling = 128_000 // DefaultServerStateInputTokens is the input charged, when final usage is // missing, for a request that names server-side state (ServerStateInput): // a full long-context window, since its real size is invisible here. DefaultServerStateInputTokens = 200_000 )
Estimate defaults.
const NoRevocationWarning = "" /* 200-byte string literal not displayed */
NoRevocationWarning is the startup warning a gateway logs when it accepts halo-server credentials (workload keys set) without a directory URL: it cannot see revocations (ADR-0026).
Variables ¶
var ErrCredentialRevoked = errors.New("credential revoked")
ErrCredentialRevoked is the answer for a credential the revocation list kills.
ErrDirectoryUnavailable is Resolve's answer under FailClosed when halo-server cannot be asked and no entry younger than Grace is cached. Callers answer RejectDirectoryUnavailable.
var ErrNotProvisioned = errors.New("gateway: subject is deactivated or not provisioned in the directory")
ErrNotProvisioned is Resolve's answer for a verified subject the directory does not list as active (deactivated, deleted or never provisioned). Callers refuse the request (RejectNotProvisioned): demoting it to an anonymous caller would skip the posture gate and per-user budgets.
Functions ¶
func Background ¶ added in v0.3.0
Background reports an OpenAI Responses background request.
func CheckDirectoryFailMode ¶ added in v0.3.0
CheckDirectoryFailMode validates a directory.failMode value ("" = open).
func CheckDirectoryMode ¶ added in v0.4.0
CheckDirectoryMode validates a directory.mode value ("" = full).
func DetectorIDs ¶ added in v0.4.0
func DetectorIDs() []string
DetectorIDs lists the built-in detector ids, sorted.
func EstimateUsage ¶ added in v0.3.0
func EstimateUsage(r UsageResult, e Estimate) budget.Usage
EstimateUsage is the pessimistic charge for a metered 2xx response whose final usage was not read (r.Complete false), so hanging up before the last event never makes a call free:
- input as reported so far (Anthropic's message_start), else request bytes / 3; at least e.ServerStateInput when the request names server-side state, whose size the request bytes do not show;
- output the larger of the output reported so far and a measure of what was generated: for a decoded stream the delta payload text / 4 (at least one token per delta event), so SSE framing and JSON escaping are not charged; otherwise response bytes / 4. With reasoning on, at least the thinking budget (else the limit); in background mode the limit. Capped at the request's maximum output tokens, else e.OutputCeiling;
- never zero.
func EventStreamToSSE ¶
func EventStreamToSSE(rc io.ReadCloser) io.ReadCloser
EventStreamToSSE converts an AWS event stream (Bedrock invoke-with-response-stream) carrying Anthropic events into the SSE the Messages API streams: each chunk's {"bytes": base64(event json)} becomes "event: <type>\ndata: <json>\n\n". Exception frames become an SSE error event.
func FailOpenSubject ¶ added in v0.4.0
FailOpenSubject is sub kept with no groups, as Resolve answers when the directory cannot be asked under the open fail mode. Kind, Delegation and Credential stay: a delegated agent keeps its chain, so the agent policy still applies, and the credential stays checkable against revocations.
func ForceNonStreaming ¶
ForceNonStreaming makes a request non-streaming (used for shadow copies).
func HarnessFromUA ¶
HarnessFromUA maps a User-Agent to a harness adapter name.
func IdentityHeaderFor ¶
IdentityHeaderFor picks the configured header, falling back to the org's.
func Inspect ¶
Inspect extracts the requested model alias and whether the request is a first turn. Bedrock carries the alias in the path, the others in the body. Unparseable bodies yield ("", false): never shadowed, routed by default.
func IsCountTokens ¶
IsCountTokens reports whether p is a token-counting call (never mirrored, excluded from request metrics).
func MaxOutputTokens ¶ added in v0.3.0
MaxOutputTokens is a model request's output token limit on any supported wire; 0 when it sets none or does not parse.
func ParseUA ¶
ParseUA returns the harness and CLI version a User-Agent names. The formats (sources in docs/adr/0011-gateway-enforced-posture-and-version.md):
claude-cli/2.1.287 (external, sdk-cli) Claude Code codex_cli_rs/0.99.0 (Mac OS 15.1.0; arm64) iTerm.app Codex (originator/version (os; arch) terminal) GeminiCLI/0.26.0/gemini-2.5-pro (darwin; x64) Gemini CLI (GeminiCLI[-client]/version/model (platform; arch[; surface])) opencode/1.19.3 ai-sdk/provider-utils/4.0.46 runtime/bun/1.3.14 OpenCode (opencode/version, then the AI SDK and runtime)
The version is the token after the first '/', up to a space, '/', '(' or ';'. version is "" when the harness is known but no semver follows; harness is "" for an unrecognised UA (Copilot CLI's is not documented and has none).
func ProtocolForPath ¶
ProtocolForPath classifies a request path by exact match; "" means "not a model call". count_tokens is anthropic-messages (it carries a model alias).
func ReasoningBudget ¶ added in v0.3.0
ReasoningBudget reports whether a request enables hidden reasoning, and the token budget it sets (0 = none): OpenAI Responses reasoning (unless effort "none"); Anthropic thinking enabled or adaptive (budget_tokens) or an output_config effort, also under Bedrock Converse additionalModelRequestFields; Gemini generationConfig.thinkingConfig.
func RedactionMarker ¶ added in v0.4.0
RedactionMarker replaces a secret in a redacted body.
func RevocationSubjects ¶ added in v0.4.0
RevocationSubjects are the canonical subjects whose epochs apply to sub's credential (its root and every agent of its chain); nil without a credential.
func RewriteRequest ¶
RewriteRequest points a request at upstream model `model`: the JSON "model" field for anthropic-messages / openai-responses, the /model/{id}/ path segment for bedrock-invoke. Only the request is touched; responses stream through untouched. Empty model or unknown protocol is a no-op.
func RingGate ¶
RingGate is ring's mode for gate (GatePosture or GateVersion); off when the ring is unknown.
func RingPins ¶
RingPins returns the CLI versions ring r's devices may run per harness: the ring profile's pin plus the variant profiles of its running client-axis experiment (the gateway cannot tell which variant a device applied).
func RouteOrder ¶
func RouteOrder(alias string, r policy.ModelRoute, key string) []policy.RouteTarget
RouteOrder returns the route's targets in attempt order for key. Targets are grouped into priority tiers (ascending). Within a tier the weighted pick (internal/assign) goes first and the rest follow in policy order; a tier without weights keeps policy order. The first element is the target a request lands on when everything is healthy.
func SecretHash ¶ added in v0.4.0
SecretHash is the 8-hex SHA-256 prefix a finding and a redaction marker carry: enough to correlate one secret across events, never to recover it.
func ServerStateInput ¶ added in v0.3.0
ServerStateInput reports whether a request makes the provider bill input tokens the request body does not carry, so request bytes underestimate it:
- OpenAI Responses: previous_response_id, conversation, a stored prompt id, background mode, input_file by file_id or file_url, input_image by file_id or an http(s) URL, item_reference, and the file_search, web_search*, mcp and code_interpreter tools;
- Anthropic Messages (and Bedrock invoke): server tools (web_search, web_fetch, code_execution, tool_search), MCP servers, container uploads, and document or image blocks whose source is a file or a URL.
func StickyKey ¶
StickyKey is what weighted route choice hashes: the verified identity plus the harness session, so a session stays on one target. "" (anonymous, no session) means no stickiness is possible: the first-priority target is used.
func SubjectUnits ¶ added in v0.4.0
SubjectUnits is the caller's org unit chain, its own unit first; nil when anonymous or in no unit.
func UsageTap ¶ added in v0.3.0
func UsageTap(proto, contentType, contentEncoding string, rc io.ReadCloser, done func(UsageResult)) io.ReadCloser
UsageTap wraps a model response body in the caller's wire protocol and reports its token usage once the body ends (EOF, read error or Close), exactly once. It never changes the bytes the client receives. A body with Content-Encoding gzip or deflate is decompressed for the scan only, by a goroutine that exits when the body ends (the caller must Close it); any other encoding is not scanned and reports Complete false.
The response is scanned as it streams: SSE line by line (Anthropic message_start/message_delta, OpenAI response.completed, Gemini chunks), an AWS event stream frame by frame (Bedrock invoke/converse streams), or, for a plain JSON body, its last 64 KiB. Counts merge by maximum, since every wire reports cumulative totals.
func UsageTapText ¶ added in v0.4.0
func UsageTapText(proto, contentType, contentEncoding string, rc io.ReadCloser, done func(UsageResult), text func(string)) io.ReadCloser
UsageTapText is UsageTap that also hands each piece of generated output text (what OutputText counts: text, thinking and tool-argument strings of streamed delta events) to text, on the goroutine that scans, before done runs. text may be nil. DLP response scanning reads the output this way.
Types ¶
type Breaker ¶
type Breaker struct {
Threshold int // consecutive failures that open the circuit (default 5)
Cooldown time.Duration // open time before a half-open probe (default 30s)
// contains filtered or unexported fields
}
Breaker is a per-key circuit breaker. After Threshold consecutive failures a key is open for Cooldown; then exactly one probe request is let through (half-open): success closes it, failure re-opens it for another Cooldown. Safe for concurrent use. Allow must be followed by Done or Cancel.
func (*Breaker) Cancel ¶
Cancel releases an allowed request that never produced a verdict (the client went away), so a half-open probe slot is not leaked.
type BudgetCheck ¶ added in v0.3.0
BudgetCheck is halo-server's POST <url>/check body.
type BudgetClient ¶ added in v0.3.0
type BudgetClient struct {
URL, Token string // URL: .../api/v1/gateway/budget
TTL, Grace, Flush time.Duration
Client *http.Client
Now func() time.Time
Log *slog.Logger
// contains filtered or unexported fields
}
BudgetClient is one halo-proxy replica's side of the budget design: halo-server holds the authoritative counters; the replica caches each principal's verdict for TTL and reports its own usage every Flush. Between refreshes the replica adds the usage it has seen to the cached verdict, so one busy caller cannot outrun the cache on a single replica.
func NewBudgetClient ¶ added in v0.3.0
func NewBudgetClient(url, token string, ttl, grace, flush time.Duration, log *slog.Logger, allowInsecureInCluster bool) (*BudgetClient, error)
NewBudgetClient validates the config (https unless loopback or allowInsecureInCluster).
func (*BudgetClient) Check ¶ added in v0.3.0
func (c *BudgetClient) Check(ctx context.Context, p budget.Principal) (st []budget.Status, known bool)
Check returns the counters p is charged to, with this replica's unreported usage added. known=false when halo-server could not be asked and no verdict younger than Grace is cached. ponytail: concurrent misses for one principal each call halo-server, as in PostureClient; add singleflight if a burst after TTL expiry shows up.
func (*BudgetClient) Counters ¶ added in v0.3.0
func (c *BudgetClient) Counters() (reported, failed, dropped int64)
Counters for /metrics: reports sent, reports failed, records dropped.
func (*BudgetClient) FlushNow ¶ added in v0.3.0
func (c *BudgetClient) FlushNow(ctx context.Context) error
FlushNow sends pending usage in batches until none is left or a send fails. A failed batch is kept and resent with the same id.
func (*BudgetClient) Record ¶ added in v0.3.0
Record queues usage for the next report and charges it to p's cached verdict.
func (*BudgetClient) Run ¶ added in v0.3.0
func (c *BudgetClient) Run(ctx context.Context)
Run reports usage every Flush until ctx ends. Call FlushNow after it returns to send what the last interval left.
func (*BudgetClient) Time ¶ added in v0.3.0
func (c *BudgetClient) Time() time.Time
Time is the client's clock (Now, or the wall clock).
type BudgetUsage ¶ added in v0.3.0
BudgetUsage is halo-server's POST <url>/usage body. Batch is stable across retries so the server applies a batch once.
type BudgetVerdict ¶ added in v0.3.0
BudgetVerdict is halo-server's answer: every counter the principal is charged to.
type DLPBatch ¶ added in v0.4.0
DLPBatch is halo-server's POST /api/v1/gateway/dlp/events body. Batch is stable across retries of the same events.
type DLPEvent ¶ added in v0.4.0
type DLPEvent struct {
At time.Time `json:"at"`
Gateway string `json:"gateway"` // halo-proxy | halo-kong
Subject string `json:"subject,omitempty"` // verified caller; "" = anonymous
Chain string `json:"chain,omitempty"` // verified delegation chain (ADR-0024)
Ring string `json:"ring,omitempty"`
Unit string `json:"unit,omitempty"`
Protocol string `json:"protocol"`
Harness string `json:"harness,omitempty"`
Direction string `json:"direction"` // DLPRequest | DLPResponse
Action string `json:"action"` // monitor | redact | block
DLPFinding
}
DLPEvent is one finding as audited: who, where, what kind, never the secret.
type DLPFinding ¶ added in v0.4.0
type DLPFinding struct {
Kind string `json:"kind"` // DLPSecret | DLPInjection
Detector string `json:"detector"` // detector id; "injection" for injection
Hash string `json:"hash,omitempty"` // first 8 hex of SHA-256(secret)
Location string `json:"location"` // JSON path in the request, or "response"
Score int `json:"score,omitempty"` // injection score
Signals []string `json:"signals,omitempty"` // injection signals that scored
ToolData bool `json:"toolData,omitempty"` // found in a tool result or document
}
DLPFinding is one secret or injection signal. It never holds the secret.
type DLPReport ¶ added in v0.4.0
type DLPReport struct {
SecretAction, InjectionAction string // effective actions (policy.DLP*)
Findings []DLPFinding
// Redacted is the body with every secret replaced, nil when unchanged
// (no redaction). It is the pre-model-rewrite body: what the shadow mirror sends.
Redacted []byte
Truncated bool // the body was larger than the scan cap or the scan ran out of time
OverCap bool // Truncated because the body was larger than the scan cap
Blocked string // DLPSecret | DLPInjection | DLPTruncated when the request is refused
Took time.Duration
}
DLPReport is a request scan's outcome.
func ScanRequest ¶ added in v0.4.0
ScanRequest scans a model-call body under d for a caller in ring and unit chain units. It returns nil when DLP is off for the caller. The scan stops at d.ScanLimit() bytes or dlpScanBudget, whichever first.
type DLPReporter ¶ added in v0.4.0
type DLPReporter struct {
URL, Token string
Flush time.Duration
Client *http.Client
Log *slog.Logger
Now func() time.Time
// contains filtered or unexported fields
}
DLPReporter queues events and posts them to halo-server. The zero of *DLPReporter (nil) drops everything, so callers need no "configured" branch. Report never blocks the request path: past maxDLPQueue events are dropped and counted. The same finding (subject, detector, hash, location, direction) is reported once per hour; a conversation carries its history on every turn.
func NewDLPReporter ¶ added in v0.4.0
func NewDLPReporter(url, token string, flush time.Duration, log *slog.Logger, allowInsecureInCluster bool) (*DLPReporter, error)
NewDLPReporter validates the config (https unless loopback or allowInsecureInCluster).
func (*DLPReporter) Counters ¶ added in v0.4.0
func (r *DLPReporter) Counters() (sent, failed, dropped, deduped int64)
Counters for /metrics: events sent, failed posts, events dropped (queue full) and deduplicated.
func (*DLPReporter) FlushNow ¶ added in v0.4.0
func (r *DLPReporter) FlushNow(ctx context.Context) error
FlushNow sends queued events in batches until none is left or a send fails; a failed batch is resent with the same id.
func (*DLPReporter) Report ¶ added in v0.4.0
func (r *DLPReporter) Report(evs ...DLPEvent)
Report queues events. Safe for concurrent use; nil-safe.
func (*DLPReporter) Run ¶ added in v0.4.0
func (r *DLPReporter) Run(ctx context.Context)
Run posts every Flush until ctx ends. Call FlushNow after it returns.
type DLPResponseScanner ¶ added in v0.4.0
type DLPResponseScanner struct {
Truncated bool
// contains filtered or unexported fields
}
DLPResponseScanner looks for secrets in a response's generated text as it streams. It never changes the response. Not safe for concurrent use: one UsageTap feeds it from one goroutine, and Findings is read once the body ended.
func NewDLPResponseScanner ¶ added in v0.4.0
func NewDLPResponseScanner(d *policy.DLP, ring string, units []string) *DLPResponseScanner
NewDLPResponseScanner returns a scanner, or nil when DLP or response scanning is off for the caller.
func (*DLPResponseScanner) Findings ¶ added in v0.4.0
func (s *DLPResponseScanner) Findings() []DLPFinding
Findings scans what is left and returns every distinct secret seen.
func (*DLPResponseScanner) Text ¶ added in v0.4.0
func (s *DLPResponseScanner) Text(t string)
Text adds generated text.
type Decision ¶
type Decision struct {
Ring string
Release string
Experiment string
Variant string
UpstreamName string
Upstream string // URL; "" = leave Kong's default service
UpstreamModel string // "" = leave the model untouched
// Route is the ordered attempt list (failover order) for the alias;
// UpstreamName/UpstreamModel/Upstream are its first element. halo-kong
// uses only those; halo-proxy walks Route.
Route []policy.RouteTarget
Headers map[string]string
Shadow []ShadowTarget
}
Decision is the routing outcome for one request.
type DirectoryClient ¶ added in v0.3.0
type DirectoryClient struct {
URL, Token string
TTL, Grace time.Duration
Client *http.Client
Now func() time.Time
Log *slog.Logger
// FailClosed: refuse (ErrDirectoryUnavailable) instead of failing open
// when the directory cannot be asked (DirectoryFailClosed).
FailClosed bool
// Observe, when set, is told every Resolve outcome of a looked-up subject:
// ok, stale (cached entry served through an outage), failopen,
// unavailable (FailClosed) or not_provisioned. It is also told
// revocations_unavailable each time the revocation list cannot be fetched
// (whatever the fail mode then does). For metrics.
Observe func(outcome string)
// RevocationTTL: the revocation list is refetched once it is older
// (0 = DefaultRevocationTTL). It is fetched only for callers holding a
// halo-server credential (workload or delegated agent).
RevocationTTL time.Duration
// RevocationsOnly (DirectoryModeRevocations) checks halo-server credentials
// against the revocation list and never looks a subject up in SCIM: groups
// stay the verified credential's. For deployments without SCIM.
RevocationsOnly bool
// contains filtered or unexported fields
}
DirectoryClient makes halo-server's SCIM directory authoritative for a verified subject (ADR-0022): the subject's groups are the directory's, and a subject the directory does not list as active is refused (ErrNotProvisioned). Groups never come from the request.
func NewDirectoryClient ¶ added in v0.3.0
func NewDirectoryClient(url, token string, ttl, grace time.Duration, log *slog.Logger, allowInsecureInCluster bool) (*DirectoryClient, error)
NewDirectoryClient validates the config (https unless loopback or allowInsecureInCluster).
func (*DirectoryClient) Resolve ¶ added in v0.3.0
func (c *DirectoryClient) Resolve(ctx context.Context, sub *policy.Subject) (*policy.Subject, error)
Resolve returns sub with the directory's groups. It returns nil, nil (unknown caller) only when sub is nil, and ErrNotProvisioned when the directory does not list sub as active, when halo-server answers the lookup with a non-200 status below 500, or when the last cached answer for sub was inactive, however old. When halo-server cannot be asked (transport error or 5xx) and no entry younger than Grace is cached it returns sub with no groups (fail open), or ErrDirectoryUnavailable under FailClosed: never nil, which would let an outage turn a deactivated user into an anonymous caller that skips the posture gate and per-user budgets. Workload subjects (ADR-0023) are not in SCIM: they are returned unchanged without a lookup. Kind and Delegation always come from the verified credential, never from the directory: a delegated agent (ADR-0024) is looked up as its root principal, so an agent acting for a deprovisioned user is refused like the user, and keeps its chain (and the agent policy) on every path, fail open included. A caller holding a halo-server credential (workload or delegated agent) is first checked against the revocation list (ADR-0026, see revocations): a revoked credential is ErrCredentialRevoked, workload-rooted chains included. ponytail: concurrent misses for one subject each call halo-server, as PostureClient.Check does.
type DirectoryEntry ¶ added in v0.3.0
type DirectoryEntry struct {
Active bool `json:"active"`
Groups []string `json:"groups,omitempty"`
}
DirectoryEntry is halo-server's SCIM directory answer for one subject (GET /api/v1/gateway/directory). Inactive covers deactivated, deleted and never provisioned users alike.
type Estimate ¶ added in v0.3.0
type Estimate struct {
RequestBytes int
// MaxOutput is the request's maximum output tokens (MaxOutputTokens); 0 = none.
MaxOutput int64
// OutputCeiling caps the output estimate when MaxOutput is 0 (0 = DefaultOutputCeiling).
OutputCeiling int64
// ServerStateInput is the input to charge when the request makes the
// provider bill input its bytes do not show (ServerStateInput true); 0 = it does not.
ServerStateInput int64
// Reasoning: the request enables hidden reasoning (ReasoningBudget), so
// the output estimate is at least ThinkingBudget, or the output limit when
// ThinkingBudget is 0: reasoning tokens are billed as output but never
// streamed, so a stream cut before the final usage event shows none of them.
Reasoning bool
ThinkingBudget int64
// Background: an OpenAI Responses background request; its response is
// queued (usage null), so the output is charged at the limit.
Background bool
}
Estimate is what EstimateUsage knows about the request (RequestEstimate).
func RequestEstimate ¶ added in v0.3.0
RequestEstimate is the Estimate for a metered request body: maximum output, server-side state (charged serverStateInput, else DefaultServerStateInputTokens), reasoning and background mode.
type Finding ¶
type Finding struct {
Gate string // GatePosture | GateVersion
Mode string // policy.GateWarn | policy.GateEnforce
Reason string
}
Finding is a failed gate check under a ring whose mode is warn or enforce.
func CheckPosture ¶
func CheckPosture(ctx context.Context, c *PostureClient, org *policy.Org, ring string, sub *policy.Subject) (f *Finding, unknown bool)
CheckPosture applies ring's posture gate to a verified subject. unknown reports that the gate is on but no verdict was available (an anonymous caller, or halo-server unreachable past the grace window): the request passes and callers count it. A nil client means the posture gate is not configured: a gateway whose posture config is invalid must not pass nil (halo-proxy refuses to start; halo-kong fails closed on enforce rings).
func CheckVersion ¶
CheckVersion applies ring's versionGate to a request's User-Agent. nil = pass (or gate off). A ring that pins no version for the detected harness passes: there is nothing to compare against.
type KillEnvelope ¶
type KillEnvelope struct {
Payload []byte `json:"payload"` // base64 in JSON
Signature []byte `json:"signature"` // base64 in JSON
// Alg is the signature algorithm (bundle.AlgEd25519 or bundle.AlgECDSAP256,
// ADR-0038); absent means ed25519, as every envelope before it. It only
// selects how to verify: a verifier accepts it from its own keys only.
Alg string `json:"alg,omitempty"`
// KeyID is bundle.KeyID of the signing key: informational (rotation).
KeyID string `json:"keyId,omitempty"`
}
KillEnvelope is GET /api/v1/gateway/killswitch: Payload is the JSON KillList, Signature is over killDomain+Payload with algorithm Alg.
func SignKillList ¶
SignKillList seals l for gateways with s (a file, KMS or external signer).
type KillList ¶
type KillList struct {
Version uint64 `json:"version"` // grows with every kill/unkill; informational
Experiments []string `json:"experiments"`
// Toggles are killed feature-toggle names: treated as off everywhere, whatever
// their rules say. Part of the signed payload, so covered by the same
// signature, freshness and monotonicity checks as Experiments.
Toggles []string `json:"toggles,omitempty"`
IssuedAt time.Time `json:"issuedAt"`
}
KillList is the set of experiments halo-server has killed: gateways treat them as not running (control routing, no shadow) without waiting for a policy PR to merge and a snapshot to ship.
type KillSwitch ¶
type KillSwitch struct {
URL string
Token string
// PubKey verifies the envelopes: a bundle.Ed25519Verifier, an
// ECDSAVerifier, or a bundle.Keyring while rotating the kill key.
PubKey bundle.Verifier
Interval time.Duration // default DefaultKillInterval
MaxAge time.Duration // default DefaultKillMaxAge
Client *http.Client // default: 10s timeout, no redirects
Log *slog.Logger
Now func() time.Time
// contains filtered or unexported fields
}
KillSwitch polls halo-server for the signed kill list.
Failure policy: a fetch or verification failure keeps the last accepted list (a kill never silently lapses because the control plane is down); if no list was ever accepted the set is empty and the policy snapshot's statuses alone decide. A list is accepted only if it is strictly newer (IssuedAt) than the current one, so a replayed older envelope cannot undo a kill. A nil *KillSwitch is valid and kills nothing.
func NewKillSwitch ¶
func NewKillSwitch(url, token string, pub bundle.Verifier, interval time.Duration, log *slog.Logger, allowInsecureInCluster bool) (*KillSwitch, error)
NewKillSwitch validates the config (see bundle.ParseVerifier for PEM keys). allowInsecureInCluster permits plain http to a non-loopback host.
func (*KillSwitch) Apply ¶
func (k *KillSwitch) Apply(org *policy.Org) *policy.Org
Apply returns org with every killed experiment's status forced to "paused", so Decide routes its users to default (control) and mirrors nothing. org itself is never mutated; it is returned as-is when nothing applies. ponytail: copies per request while a kill is active; cache by (org, version) if it shows in profiles.
func (*KillSwitch) Killed ¶
func (k *KillSwitch) Killed() KillList
Killed returns the current list (a copy).
func (*KillSwitch) Refresh ¶
func (k *KillSwitch) Refresh(ctx context.Context) error
Refresh fetches and verifies the list once; on error the previous list stays.
func (*KillSwitch) Restore ¶
func (k *KillSwitch) Restore(l KillList)
Restore seeds a last-known list persisted by the caller (halod keeps it in its state file) so kills survive a restart and envelopes issued before it are refused as replays. It is a no-op unless l is newer than the current list. The list is trusted as given: pass only one that was verified.
func (*KillSwitch) Run ¶
func (k *KillSwitch) Run(ctx context.Context)
Run refreshes immediately and then every Interval until ctx ends.
type Outbound ¶
type Outbound struct {
Path string
RawQuery string
SetQuery bool // replace the client's query string with RawQuery (else keep it)
Body []byte
// SSE: the response is a Bedrock event stream the anthropic-messages client
// cannot read; convert it with EventStreamToSSE.
SSE bool
}
Outbound is the request an upstream expects, derived from the client's request. Path is escaped and relative to the upstream's base URL.
func BuildOutbound ¶
func BuildOutbound(proto string, up policy.Upstream, model, path string, body []byte) (Outbound, error)
BuildOutbound rewrites the client's request (protocol proto, escaped path, body) for upstream up serving model. Same-wire upstreams only get the model swapped (RewriteRequest). Translations:
anthropic-messages -> bedrock POST /model/{id}/invoke[-with-response-stream], body without model/stream + anthropic_version
anthropic-messages -> vertex POST .../publishers/anthropic/models/{id}:[stream]rawPredict, body without model + anthropic_version
openai-responses -> azure POST /openai/v1/responses (or /openai/responses?api-version=), model = deployment
Responses stay in the provider's format except Bedrock streams, which are converted to SSE (see Outbound.SSE).
type PostureClient ¶
type PostureClient struct {
URL, Token string
TTL, Grace time.Duration
Client *http.Client
Now func() time.Time
Log *slog.Logger
// contains filtered or unexported fields
}
PostureClient asks halo-server for a subject's device posture and caches the verdict. It fails closed on a verdict halo-server returned and on a lookup it refused (a non-200 status below 500: a rejected token, posture not configured on the server, a redirect): that is a misconfiguration, never an outage. When the server cannot be reached (transport error or 5xx), the last verdict serves for Grace, and after that the posture is unknown (callers let the request through and count it).
func NewPostureClient ¶
func NewPostureClient(url, token string, ttl, grace time.Duration, log *slog.Logger, allowInsecureInCluster bool) (*PostureClient, error)
NewPostureClient validates the config (https unless loopback or allowInsecureInCluster).
func (*PostureClient) Check ¶
func (c *PostureClient) Check(ctx context.Context, subject string) (v PostureVerdict, known bool)
Check returns subject's verdict; known=false when halo-server could not be asked and no verdict younger than Grace is cached. A refused lookup is a non-compliant verdict, not cached (a fixed token heals at once). ponytail: concurrent misses for one subject each call halo-server; add singleflight if a burst after TTL expiry shows up in the server's load.
type PostureVerdict ¶
type PostureVerdict struct {
Compliant bool `json:"compliant"`
Reason string `json:"reason,omitempty"`
}
PostureVerdict is halo-server's answer for one subject (GET /api/v1/gateway/posture).
type Rejection ¶
type Rejection struct {
Status int
Protocol string // selects the error body shape
Message string
Code string // OpenAI Responses error code; "" = derived from Status
// RetryAfter, when > 0, is sent as Retry-After (whole seconds, rounded up).
RetryAfter time.Duration
}
Rejection is a request the gateway answers itself. Fail closed: unknown model endpoints (404), unreadable/oversize bodies (413), models that are not a configured alias (400: a permanent client error, which the CLIs do not retry, unlike 403/5xx/429).
func DirectoryReject ¶ added in v0.4.0
DirectoryReject maps a DirectoryClient.Resolve error to the rejection a gateway answers, in the caller's protocol.
func RejectCredentialRevoked ¶ added in v0.4.0
RejectCredentialRevoked is the 401, in the caller's protocol, for ErrCredentialRevoked.
func RejectDirectoryUnavailable ¶ added in v0.3.0
RejectDirectoryUnavailable is the 503, in the caller's protocol, for ErrDirectoryUnavailable.
func RejectNotProvisioned ¶ added in v0.3.0
RejectNotProvisioned is the 403, in the caller's protocol, for ErrNotProvisioned.
func RejectTooLarge ¶
RejectTooLarge is the 413 for a model call whose body could not be read in full (Kong spooled it to disk, or it exceeds the proxy limit).
func (*Rejection) JSON ¶
JSON renders the error in the caller's wire format so the CLI shows Message.
func (*Rejection) SetHeaders ¶ added in v0.3.0
SetHeaders adds the response headers that go with the body: Retry-After, and for Bedrock the x-amzn-ErrorType its SDKs classify errors by.
type RequestInfo ¶
type RequestInfo struct {
UserID string // from the auth gateway's identity header only
Groups []string
ModelAlias string
SessionID string
Harness string
Protocol string
FirstTurn bool
// AnyTurn: a parsed, replayable model call at any step of a session
// (eligible for shadow experiments with turns: all).
AnyTurn bool
// Workload: the verified caller is a CI / cloud workload (ADR-0023). It
// gets no experiment or shadow unless policy workloads.experiments is true.
Workload bool
// Agent: the verified caller is a delegated agent acting for UserID
// (ADR-0024). It gets no experiment or shadow unless policy
// agents.experiments is true (and, for a workload root, workloads.experiments).
Agent bool
}
RequestInfo is everything Decide may look at.
func NewRequestInfo ¶
func NewRequestInfo(h http.Header, path string, body []byte) RequestInfo
NewRequestInfo builds RequestInfo from the request alone. UserID/Groups are left empty: cohort comes only from a verified identity (PrepareVerified).
type Result ¶
type Result struct {
Decision Decision
Protocol string
// ClearHeaders are client-supplied x-halo-* names to drop (all of them; the
// ones Decision sets are then re-set with gateway-owned values).
ClearHeaders []string
// Path/Body are the rewritten forms; Rewritten is false when unchanged.
Path string
Body []byte
Rewritten bool
// RewriteErr is non-nil when a rewrite was needed but failed (Reject is
// then set too: an un-rewritten alias must never reach the upstream).
RewriteErr error
// Reject, when non-nil, must be answered by the adapter itself; the
// request must NOT be forwarded.
Reject *Rejection
// DLP is the request's secret and injection scan (ADR-0033); nil when the
// policy has no gateway.dlp or it is off for the caller. When it redacted,
// Body (and Rewritten) carry the redacted body.
DLP *DLPReport
}
Result is everything the Kong plugin must apply to the upstream request.
func PrepareVerified ¶
func PrepareVerified(org *policy.Org, sub *policy.Subject, h http.Header, path string, body []byte) Result
PrepareVerified inspects a request and computes the full set of mutations, or a Rejection the adapter must answer instead of forwarding. Cohort comes only from sub, verified by internal/identity (JWT, or trusted-proxy headers); sub == nil means anonymous: default routing, no experiments. Client-supplied headers (x-halo-* or identity) never influence the outcome.
type RevocationList ¶ added in v0.4.0
type RevocationList struct {
// JTIs maps a revoked credential id to when the entry lapses (unix seconds).
JTIs map[string]int64 `json:"jtis"`
// Subjects maps a canonical subject (policy.CanonicalUser of a human root
// id, a "workload:" subject or "agent:<name>") to its revoked-before epoch
// (unix seconds): see Check.
Subjects map[string]int64 `json:"subjects"`
}
RevocationList is halo-server's active credential revocations (ADR-0026), GET /api/v1/gateway/directory?revocations=1. It holds only revocations that can still matter: a jti until the longest a credential can live (policy.MaxWorkloadTTL) has passed since it was revoked, a subject epoch until that long after the epoch. So its size is bounded by what admins revoked in the last hour, and halo-server caps it (maxActiveRevocations).
func (*RevocationList) Check ¶ added in v0.4.0
func (l *RevocationList) Check(sub *policy.Subject) error
Check returns ErrCredentialRevoked (wrapped with the reason) when the credential sub was verified from is dead:
- its jti, or the jti of any credential it was exchanged from, is revoked;
- its root principal has an epoch at or after the root's authentication (a delegated credential's auth_time, a workload credential's iat);
- an agent of its delegation chain has an epoch at or after its iat.
A subject with no halo-server credential (an IdP user) is never revoked here.
type RouteRow ¶
type RouteRow struct {
Alias string `json:"alias"`
Ring string `json:"ring"`
Experiment string `json:"experiment,omitempty"` // traffic-axis experiment that supplied the route
Variant string `json:"variant,omitempty"`
Targets []TargetRow `json:"targets"`
}
RouteRow is the effective route of one alias for a caller.
func RouteTable ¶
func RouteTable(org *policy.Org, req RequestInfo) []RouteRow
RouteTable returns every alias's effective routes for req (UserID, Groups and SessionID decide the ring, experiment variant and weighted pick; ModelAlias is ignored). Anonymous callers (no user, no session) see policy order.
type ShadowTarget ¶
ShadowTarget is one candidate a request should be mirrored to.
type Snapshot ¶
type Snapshot struct {
// contains filtered or unexported fields
}
Snapshot serves a compiled org policy (policy.Compile output, or a bare json.Marshal of *policy.Org) from a file and reloads it when the mtime/size changes. A failed reload keeps serving the last good policy.
func NewSnapshot ¶
type TargetRow ¶
type TargetRow struct {
Order int `json:"order"`
Selected bool `json:"selected"`
Upstream string `json:"upstream"`
Kind string `json:"kind"`
Model string `json:"model"`
Weight float64 `json:"weight,omitempty"`
Priority int `json:"priority"`
TimeoutSeconds int `json:"timeoutSeconds,omitempty"`
SkippedBy []string `json:"skippedByHarnesses,omitempty"` // harnesses whose wire protocol this target cannot serve
}
TargetRow is one attempt-order entry; Selected marks where a request lands when every target is healthy.
type Tool ¶ added in v0.3.0
Tool is one tool a model call declares. Hosted marks a tool the provider runs or builds in (web search, code execution, a remote MCP server, a versioned Anthropic tool): Name is then its type, its Gemini or Bedrock tool key, or mcp:<server>, and policy matches it as a prefix (see policy.Agents.DeniesTool).
func ToolNames ¶ added in v0.3.0
ToolNames returns every tool a model call declares, or an error when the body or its tool declarations cannot be read: a body nested past maxJSONDepth, trailing data, a tools member of the wrong shape, a tool without a name, or a (protocol, path) without an extractor. The inspectable shapes are an explicit list:
- Anthropic Messages and count_tokens: tools[].name and tools[].type (hosted unless "custom"), mcp_servers[].name as mcp:<name>.
- OpenAI Responses: tools[].name of function and custom tools; any other tools[].type is hosted, mcp as mcp:<server_label>.
- Bedrock invoke and converse: the Anthropic shape, toolConfig.tools[].toolSpec.name, toolConfig.tools[].systemTool.name (hosted), any other toolConfig.tools[] member (hosted) and additionalModelRequestFields.tools.
- Gemini generateContent, streamGenerateContent and countTokens: tools[].functionDeclarations[].name, every other tools[] member key (codeExecution, googleSearch, urlContext, ...) as hosted, and the same under generateContentRequest.
Keys match ignoring case and underscores (function_declarations is functionDeclarations) and duplicate keys all count, so a body cannot show the gateway one tool list and the upstream parser another.
type UsageResult ¶ added in v0.3.0
type UsageResult struct {
Usage budget.Usage // field-wise maximum over every usage object read
Found bool // at least one usage object was read
// Complete: the upstream body reached its clean end (EOF) and a usage
// object was read, so Usage is the provider's final count. A client that
// hangs up early, an upstream cut, a Content-Encoding the tap cannot decode
// or a body without usage leave it false: charge EstimateUsage instead.
Complete bool
Bytes int64 // response bytes seen (decoded bytes when the tap decoded them)
// Streamed: the body was an SSE or AWS event stream the tap decoded, so
// OutputText and OutputEvents measure the generated output.
Streamed bool
// OutputText is the UTF-8 bytes of generated output (text, thinking, tool
// arguments) in the stream's delta events, without framing or JSON
// escaping; OutputEvents counts those events.
OutputText, OutputEvents int64
}
UsageResult is what a UsageTap saw by the time its body ended.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package adapters renders the "put halo-proxy in front of / behind stack X" config snippets shipped under deploy/integrations.
|
Package adapters renders the "put halo-proxy in front of / behind stack X" config snippets shipped under deploy/integrations. |
|
Package kong generates the decK declarative config for Kong OSS (db-less) that fronts an org's model traffic and attaches the halo-kong plugin.
|
Package kong generates the decK declarative config for Kong OSS (db-less) that fronts an org's model traffic and attaches the halo-kong plugin. |
|
Package upstreamauth signs requests to provider upstreams that need a gateway-held credential instead of the caller's, so halo-proxy can talk to Amazon Bedrock directly (no orchestrator, no SigV4 sidecar).
|
Package upstreamauth signs requests to provider upstreams that need a gateway-held credential instead of the caller's, so halo-proxy can talk to Amazon Bedrock directly (no orchestrator, no SigV4 sidecar). |