gateway

package
v0.6.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 3, 2026 License: Apache-2.0 Imports: 40 Imported by: 0

Documentation

Overview

Package gateway holds the traffic-plane logic shared by the halo-kong plugin and halo-shadow: request inspection, cohort decision and model rewrite. It is pure (no Kong, no network) so the plugin stays a thin adapter.

Index

Constants

View Source
const (
	// HeaderActor carries the verified actor chain ("planner>reviewer") and
	// HeaderActorDepth its length upstream. Gateway-owned like every x-halo-*:
	// a client copy is always discarded.
	HeaderActor      = "x-halo-actor"
	HeaderActorDepth = "x-halo-actor-depth"

	// CodeAgentNotPermitted is the Rejection code of every admitAgent refusal.
	CodeAgentNotPermitted = "agent_not_permitted"
)
View Source
const (
	DefaultBudgetTTL   = 30 * time.Second // a verdict is reused this long per principal
	DefaultBudgetGrace = 15 * time.Minute // a cached verdict outlives an unreachable halo-server this long
	DefaultBudgetFlush = 5 * time.Second  // usage report interval

)

Budget client defaults (ADR-0017).

View Source
const (
	HaloPrefix       = "x-halo-"
	HeaderRing       = "x-halo-ring"
	HeaderRelease    = "x-halo-release"
	HeaderExperiment = "x-halo-experiment"
	HeaderVariant    = "x-halo-variant"
	// HeaderGate carries warn-mode and unknown gate outcomes (posture=unknown,
	// version=warn) and HeaderKillswitch a misconfigured kill switch, for the
	// upstream's log pipeline. Like every owned name they are always cleared,
	// even when the host's header view was truncated.
	HeaderGate       = "x-halo-gate"
	HeaderKillswitch = "x-halo-killswitch"

	RingUnknown = "unknown"
)

Headers the gateway stamps on upstream requests. Everything under HaloPrefix is owned by the gateway; client-supplied values are always discarded.

View Source
const (
	DefaultDirectoryTTL   = time.Minute     // an entry is reused this long before halo-server is asked again
	DefaultDirectoryGrace = 5 * time.Minute // a cached entry outlives an unreachable halo-server this long
	// DefaultRevocationTTL is how long the revocation list (ADR-0026) is
	// used before halo-server is asked again: with halo-server reachable, a
	// revocation reaches the gateway within this (plus one fetch).
	DefaultRevocationTTL = 30 * time.Second
)

Directory defaults.

View Source
const (
	// DirectoryFailOpen (default) keeps the verified subject with no groups:
	// default routing, no experiments, and still the posture gate and per-user
	// budgets. A deactivated user is never demoted to an anonymous caller.
	DirectoryFailOpen = "open"
	// DirectoryFailClosed refuses the request (ErrDirectoryUnavailable, a 503).
	DirectoryFailClosed = "closed"
)

Directory fail modes (directory.failMode): what Resolve does when the directory cannot be asked and no entry younger than Grace is cached.

View Source
const (
	// DirectoryModeFull (default) resolves every verified subject in SCIM and
	// checks halo-server credentials against the revocation list.
	DirectoryModeFull = "full"
	// DirectoryModeRevocations only checks halo-server credentials (workload,
	// delegated agent) against the revocation list (ADR-0026): no SCIM lookup,
	// so it works on a halo-server without SCIM.
	DirectoryModeRevocations = "revocations"
)

Directory modes (directory.mode): what the directory client asks halo-server.

View Source
const (
	DLPSecret    = "secret"
	DLPInjection = "injection"
	// DLPTruncated is DLPReport.Blocked (never a finding kind) when the secret
	// action is block and the body could not be scanned in full.
	DLPTruncated = "truncated"
)

DLP finding kinds.

View Source
const (
	CodeSecretDetected    = "secret_detected"
	CodeInjectionDetected = "prompt_injection_detected"
	CodeScanIncomplete    = "dlp_scan_incomplete"
)

Codes on a DLP block (OpenAI Responses error code; Rejection.Code).

View Source
const (
	DefaultDLPFlush = 5 * time.Second
	MaxDLPEvents    = 500 // per batch; halo-server refuses more

)

DLP reporter bounds.

View Source
const (
	DLPRequest  = "request"
	DLPResponse = "response"
)

Directions of a DLP event.

View Source
const (
	GatePosture = "posture"
	GateVersion = "version"
)

Gates the traffic plane applies per ring (policy Ring.Posture / Ring.VersionGate).

View Source
const (
	DefaultPostureTTL   = time.Minute      // a verdict is reused this long before halo-server is asked again
	DefaultPostureGrace = 15 * time.Minute // a cached verdict outlives an unreachable halo-server this long

)

Posture defaults.

View Source
const (
	ProtoAnthropic = "anthropic-messages"
	ProtoBedrock   = "bedrock-invoke"
	ProtoResponses = "openai-responses"
	ProtoGemini    = "gemini"
)

Wire protocols (match policy.Gateway.Protocols values).

View Source
const (
	PathMessages    = "/v1/messages"
	PathCountTokens = "/v1/messages/count_tokens" //nolint:gosec // URL path, not a credential
	PathResponses   = "/v1/responses"
)

Exact model-call paths. Anything else under a model-bearing prefix (/v1/messages/*, /v1/responses/*, /model/*) is rejected by Prepare.

View Source
const (
	DefaultKillInterval = 10 * time.Second
	DefaultKillMaxAge   = 10 * time.Minute
)

Kill-switch poller defaults.

View Source
const (
	// DefaultOutputCeiling caps the output estimate of a request that sets no
	// maximum output tokens: no supported model generates more in one call.
	DefaultOutputCeiling = 128_000
	// DefaultServerStateInputTokens is the input charged, when final usage is
	// missing, for a request that names server-side state (ServerStateInput):
	// a full long-context window, since its real size is invisible here.
	DefaultServerStateInputTokens = 200_000
)

Estimate defaults.

View Source
const NoRevocationWarning = "" /* 200-byte string literal not displayed */

NoRevocationWarning is the startup warning a gateway logs when it accepts halo-server credentials (workload keys set) without a directory URL: it cannot see revocations (ADR-0026).

Variables

View Source
var ErrCredentialRevoked = errors.New("credential revoked")

ErrCredentialRevoked is the answer for a credential the revocation list kills.

View Source
var ErrDirectoryUnavailable = errors.New("gateway: directory unavailable")

ErrDirectoryUnavailable is Resolve's answer under FailClosed when halo-server cannot be asked and no entry younger than Grace is cached. Callers answer RejectDirectoryUnavailable.

View Source
var ErrNotProvisioned = errors.New("gateway: subject is deactivated or not provisioned in the directory")

ErrNotProvisioned is Resolve's answer for a verified subject the directory does not list as active (deactivated, deleted or never provisioned). Callers refuse the request (RejectNotProvisioned): demoting it to an anonymous caller would skip the posture gate and per-user budgets.

Functions

func Background added in v0.3.0

func Background(proto string, body []byte) bool

Background reports an OpenAI Responses background request.

func CheckDirectoryFailMode added in v0.3.0

func CheckDirectoryFailMode(m string) error

CheckDirectoryFailMode validates a directory.failMode value ("" = open).

func CheckDirectoryMode added in v0.4.0

func CheckDirectoryMode(m string) error

CheckDirectoryMode validates a directory.mode value ("" = full).

func DetectorIDs added in v0.4.0

func DetectorIDs() []string

DetectorIDs lists the built-in detector ids, sorted.

func EstimateUsage added in v0.3.0

func EstimateUsage(r UsageResult, e Estimate) budget.Usage

EstimateUsage is the pessimistic charge for a metered 2xx response whose final usage was not read (r.Complete false), so hanging up before the last event never makes a call free:

  • input as reported so far (Anthropic's message_start), else request bytes / 3; at least e.ServerStateInput when the request names server-side state, whose size the request bytes do not show;
  • output the larger of the output reported so far and a measure of what was generated: for a decoded stream the delta payload text / 4 (at least one token per delta event), so SSE framing and JSON escaping are not charged; otherwise response bytes / 4. With reasoning on, at least the thinking budget (else the limit); in background mode the limit. Capped at the request's maximum output tokens, else e.OutputCeiling;
  • never zero.

func EventStreamToSSE

func EventStreamToSSE(rc io.ReadCloser) io.ReadCloser

EventStreamToSSE converts an AWS event stream (Bedrock invoke-with-response-stream) carrying Anthropic events into the SSE the Messages API streams: each chunk's {"bytes": base64(event json)} becomes "event: <type>\ndata: <json>\n\n". Exception frames become an SSE error event.

func FailOpenSubject added in v0.4.0

func FailOpenSubject(sub *policy.Subject) *policy.Subject

FailOpenSubject is sub kept with no groups, as Resolve answers when the directory cannot be asked under the open fail mode. Kind, Delegation and Credential stay: a delegated agent keeps its chain, so the agent policy still applies, and the credential stays checkable against revocations.

func ForceNonStreaming

func ForceNonStreaming(protocol, path string, body []byte) (string, []byte, error)

ForceNonStreaming makes a request non-streaming (used for shadow copies).

func HarnessFromUA

func HarnessFromUA(ua string) string

HarnessFromUA maps a User-Agent to a harness adapter name.

func IdentityHeaderFor

func IdentityHeaderFor(org *policy.Org, override string) string

IdentityHeaderFor picks the configured header, falling back to the org's.

func Inspect

func Inspect(protocol, path string, body []byte) (model string, firstTurn bool)

Inspect extracts the requested model alias and whether the request is a first turn. Bedrock carries the alias in the path, the others in the body. Unparseable bodies yield ("", false): never shadowed, routed by default.

func IsCountTokens

func IsCountTokens(p string) bool

IsCountTokens reports whether p is a token-counting call (never mirrored, excluded from request metrics).

func MaxOutputTokens added in v0.3.0

func MaxOutputTokens(body []byte) int64

MaxOutputTokens is a model request's output token limit on any supported wire; 0 when it sets none or does not parse.

func ParseUA

func ParseUA(ua string) (harness, version string)

ParseUA returns the harness and CLI version a User-Agent names. The formats (sources in docs/adr/0011-gateway-enforced-posture-and-version.md):

claude-cli/2.1.287 (external, sdk-cli)               Claude Code
codex_cli_rs/0.99.0 (Mac OS 15.1.0; arm64) iTerm.app  Codex (originator/version (os; arch) terminal)
GeminiCLI/0.26.0/gemini-2.5-pro (darwin; x64)        Gemini CLI (GeminiCLI[-client]/version/model (platform; arch[; surface]))
opencode/1.19.3 ai-sdk/provider-utils/4.0.46 runtime/bun/1.3.14   OpenCode (opencode/version, then the AI SDK and runtime)

The version is the token after the first '/', up to a space, '/', '(' or ';'. version is "" when the harness is known but no semver follows; harness is "" for an unrecognised UA (Copilot CLI's is not documented and has none).

func ProtocolForPath

func ProtocolForPath(p string) string

ProtocolForPath classifies a request path by exact match; "" means "not a model call". count_tokens is anthropic-messages (it carries a model alias).

func ReasoningBudget added in v0.3.0

func ReasoningBudget(proto string, body []byte) (bool, int64)

ReasoningBudget reports whether a request enables hidden reasoning, and the token budget it sets (0 = none): OpenAI Responses reasoning (unless effort "none"); Anthropic thinking enabled or adaptive (budget_tokens) or an output_config effort, also under Bedrock Converse additionalModelRequestFields; Gemini generationConfig.thinkingConfig.

func RedactionMarker added in v0.4.0

func RedactionMarker(detector, hash string) string

RedactionMarker replaces a secret in a redacted body.

func RevocationSubjects added in v0.4.0

func RevocationSubjects(sub *policy.Subject) []string

RevocationSubjects are the canonical subjects whose epochs apply to sub's credential (its root and every agent of its chain); nil without a credential.

func RewriteRequest

func RewriteRequest(protocol, path string, body []byte, model string) (string, []byte, error)

RewriteRequest points a request at upstream model `model`: the JSON "model" field for anthropic-messages / openai-responses, the /model/{id}/ path segment for bedrock-invoke. Only the request is touched; responses stream through untouched. Empty model or unknown protocol is a no-op.

func RingGate

func RingGate(org *policy.Org, ring, gate string) string

RingGate is ring's mode for gate (GatePosture or GateVersion); off when the ring is unknown.

func RingPins

func RingPins(org *policy.Org, ring string) map[string][]string

RingPins returns the CLI versions ring r's devices may run per harness: the ring profile's pin plus the variant profiles of its running client-axis experiment (the gateway cannot tell which variant a device applied).

func RouteOrder

func RouteOrder(alias string, r policy.ModelRoute, key string) []policy.RouteTarget

RouteOrder returns the route's targets in attempt order for key. Targets are grouped into priority tiers (ascending). Within a tier the weighted pick (internal/assign) goes first and the rest follow in policy order; a tier without weights keeps policy order. The first element is the target a request lands on when everything is healthy.

func SecretHash added in v0.4.0

func SecretHash(secret string) string

SecretHash is the 8-hex SHA-256 prefix a finding and a redaction marker carry: enough to correlate one secret across events, never to recover it.

func ServerStateInput added in v0.3.0

func ServerStateInput(proto string, body []byte) bool

ServerStateInput reports whether a request makes the provider bill input tokens the request body does not carry, so request bytes underestimate it:

  • OpenAI Responses: previous_response_id, conversation, a stored prompt id, background mode, input_file by file_id or file_url, input_image by file_id or an http(s) URL, item_reference, and the file_search, web_search*, mcp and code_interpreter tools;
  • Anthropic Messages (and Bedrock invoke): server tools (web_search, web_fetch, code_execution, tool_search), MCP servers, container uploads, and document or image blocks whose source is a file or a URL.

func SessionID

func SessionID(h http.Header) string

SessionID is the harness session id: Claude Code's header, then Codex's.

func StickyKey

func StickyKey(userID, sessionID string) string

StickyKey is what weighted route choice hashes: the verified identity plus the harness session, so a session stays on one target. "" (anonymous, no session) means no stickiness is possible: the first-priority target is used.

func SubjectUnits added in v0.4.0

func SubjectUnits(org *policy.Org, sub *policy.Subject) []string

SubjectUnits is the caller's org unit chain, its own unit first; nil when anonymous or in no unit.

func Target

func Target(raw string) (scheme, host string, port int, pathPrefix string, err error)

Target splits an upstream URL for Kong's balancer.

func UsageTap added in v0.3.0

func UsageTap(proto, contentType, contentEncoding string, rc io.ReadCloser, done func(UsageResult)) io.ReadCloser

UsageTap wraps a model response body in the caller's wire protocol and reports its token usage once the body ends (EOF, read error or Close), exactly once. It never changes the bytes the client receives. A body with Content-Encoding gzip or deflate is decompressed for the scan only, by a goroutine that exits when the body ends (the caller must Close it); any other encoding is not scanned and reports Complete false.

The response is scanned as it streams: SSE line by line (Anthropic message_start/message_delta, OpenAI response.completed, Gemini chunks), an AWS event stream frame by frame (Bedrock invoke/converse streams), or, for a plain JSON body, its last 64 KiB. Counts merge by maximum, since every wire reports cumulative totals.

func UsageTapText added in v0.4.0

func UsageTapText(proto, contentType, contentEncoding string, rc io.ReadCloser, done func(UsageResult), text func(string)) io.ReadCloser

UsageTapText is UsageTap that also hands each piece of generated output text (what OutputText counts: text, thinking and tool-argument strings of streamed delta events) to text, on the goroutine that scans, before done runs. text may be nil. DLP response scanning reads the output this way.

Types

type Breaker

type Breaker struct {
	Threshold int           // consecutive failures that open the circuit (default 5)
	Cooldown  time.Duration // open time before a half-open probe (default 30s)
	// contains filtered or unexported fields
}

Breaker is a per-key circuit breaker. After Threshold consecutive failures a key is open for Cooldown; then exactly one probe request is let through (half-open): success closes it, failure re-opens it for another Cooldown. Safe for concurrent use. Allow must be followed by Done or Cancel.

func (*Breaker) Allow

func (b *Breaker) Allow(key string) bool

Allow reports whether a request may be sent to key now.

func (*Breaker) Cancel

func (b *Breaker) Cancel(key string)

Cancel releases an allowed request that never produced a verdict (the client went away), so a half-open probe slot is not leaked.

func (*Breaker) Done

func (b *Breaker) Done(key string, ok bool)

Done records the outcome of an allowed request.

func (*Breaker) State

func (b *Breaker) State(key string) string

State is "closed", "open" or "half-open" (an expired open circuit).

type BudgetCheck added in v0.3.0

type BudgetCheck struct {
	budget.Principal
}

BudgetCheck is halo-server's POST <url>/check body.

type BudgetClient added in v0.3.0

type BudgetClient struct {
	URL, Token        string // URL: .../api/v1/gateway/budget
	TTL, Grace, Flush time.Duration
	Client            *http.Client
	Now               func() time.Time
	Log               *slog.Logger
	// contains filtered or unexported fields
}

BudgetClient is one halo-proxy replica's side of the budget design: halo-server holds the authoritative counters; the replica caches each principal's verdict for TTL and reports its own usage every Flush. Between refreshes the replica adds the usage it has seen to the cached verdict, so one busy caller cannot outrun the cache on a single replica.

func NewBudgetClient added in v0.3.0

func NewBudgetClient(url, token string, ttl, grace, flush time.Duration, log *slog.Logger, allowInsecureInCluster bool) (*BudgetClient, error)

NewBudgetClient validates the config (https unless loopback or allowInsecureInCluster).

func (*BudgetClient) Check added in v0.3.0

func (c *BudgetClient) Check(ctx context.Context, p budget.Principal) (st []budget.Status, known bool)

Check returns the counters p is charged to, with this replica's unreported usage added. known=false when halo-server could not be asked and no verdict younger than Grace is cached. ponytail: concurrent misses for one principal each call halo-server, as in PostureClient; add singleflight if a burst after TTL expiry shows up.

func (*BudgetClient) Counters added in v0.3.0

func (c *BudgetClient) Counters() (reported, failed, dropped int64)

Counters for /metrics: reports sent, reports failed, records dropped.

func (*BudgetClient) FlushNow added in v0.3.0

func (c *BudgetClient) FlushNow(ctx context.Context) error

FlushNow sends pending usage in batches until none is left or a send fails. A failed batch is kept and resent with the same id.

func (*BudgetClient) Record added in v0.3.0

func (c *BudgetClient) Record(p budget.Principal, s budget.Spend, at time.Time)

Record queues usage for the next report and charges it to p's cached verdict.

func (*BudgetClient) Run added in v0.3.0

func (c *BudgetClient) Run(ctx context.Context)

Run reports usage every Flush until ctx ends. Call FlushNow after it returns to send what the last interval left.

func (*BudgetClient) Time added in v0.3.0

func (c *BudgetClient) Time() time.Time

Time is the client's clock (Now, or the wall clock).

type BudgetUsage added in v0.3.0

type BudgetUsage struct {
	Batch   string          `json:"batch"`
	Records []budget.Record `json:"records"`
}

BudgetUsage is halo-server's POST <url>/usage body. Batch is stable across retries so the server applies a batch once.

type BudgetVerdict added in v0.3.0

type BudgetVerdict struct {
	Budgets []budget.Status `json:"budgets"`
}

BudgetVerdict is halo-server's answer: every counter the principal is charged to.

type DLPBatch added in v0.4.0

type DLPBatch struct {
	Batch  string     `json:"batch"`
	Events []DLPEvent `json:"events"`
}

DLPBatch is halo-server's POST /api/v1/gateway/dlp/events body. Batch is stable across retries of the same events.

type DLPEvent added in v0.4.0

type DLPEvent struct {
	At        time.Time `json:"at"`
	Gateway   string    `json:"gateway"`           // halo-proxy | halo-kong
	Subject   string    `json:"subject,omitempty"` // verified caller; "" = anonymous
	Chain     string    `json:"chain,omitempty"`   // verified delegation chain (ADR-0024)
	Ring      string    `json:"ring,omitempty"`
	Unit      string    `json:"unit,omitempty"`
	Protocol  string    `json:"protocol"`
	Harness   string    `json:"harness,omitempty"`
	Direction string    `json:"direction"` // DLPRequest | DLPResponse
	Action    string    `json:"action"`    // monitor | redact | block
	DLPFinding
}

DLPEvent is one finding as audited: who, where, what kind, never the secret.

func DLPEvents added in v0.4.0

func DLPEvents(rep *DLPReport, base DLPEvent) []DLPEvent

DLPEvents builds the audit events for a request scan.

type DLPFinding added in v0.4.0

type DLPFinding struct {
	Kind     string   `json:"kind"`               // DLPSecret | DLPInjection
	Detector string   `json:"detector"`           // detector id; "injection" for injection
	Hash     string   `json:"hash,omitempty"`     // first 8 hex of SHA-256(secret)
	Location string   `json:"location"`           // JSON path in the request, or "response"
	Score    int      `json:"score,omitempty"`    // injection score
	Signals  []string `json:"signals,omitempty"`  // injection signals that scored
	ToolData bool     `json:"toolData,omitempty"` // found in a tool result or document
}

DLPFinding is one secret or injection signal. It never holds the secret.

type DLPReport added in v0.4.0

type DLPReport struct {
	SecretAction, InjectionAction string // effective actions (policy.DLP*)
	Findings                      []DLPFinding
	// Redacted is the body with every secret replaced, nil when unchanged
	// (no redaction). It is the pre-model-rewrite body: what the shadow mirror sends.
	Redacted  []byte
	Truncated bool   // the body was larger than the scan cap or the scan ran out of time
	OverCap   bool   // Truncated because the body was larger than the scan cap
	Blocked   string // DLPSecret | DLPInjection | DLPTruncated when the request is refused
	Took      time.Duration
}

DLPReport is a request scan's outcome.

func ScanRequest added in v0.4.0

func ScanRequest(d *policy.DLP, proto, ring string, units []string, body []byte) *DLPReport

ScanRequest scans a model-call body under d for a caller in ring and unit chain units. It returns nil when DLP is off for the caller. The scan stops at d.ScanLimit() bytes or dlpScanBudget, whichever first.

func (*DLPReport) Count added in v0.4.0

func (r *DLPReport) Count(kind string) int

Count returns the findings of kind.

func (*DLPReport) Reject added in v0.4.0

func (r *DLPReport) Reject(proto string) *Rejection

Reject is the refusal for a blocked request, in the caller's wire format: 400 (a permanent client error, which the CLIs show and do not retry), or 413 for a body over the scan cap under a block secret action.

type DLPReporter added in v0.4.0

type DLPReporter struct {
	URL, Token string
	Flush      time.Duration
	Client     *http.Client
	Log        *slog.Logger
	Now        func() time.Time
	// contains filtered or unexported fields
}

DLPReporter queues events and posts them to halo-server. The zero of *DLPReporter (nil) drops everything, so callers need no "configured" branch. Report never blocks the request path: past maxDLPQueue events are dropped and counted. The same finding (subject, detector, hash, location, direction) is reported once per hour; a conversation carries its history on every turn.

func NewDLPReporter added in v0.4.0

func NewDLPReporter(url, token string, flush time.Duration, log *slog.Logger, allowInsecureInCluster bool) (*DLPReporter, error)

NewDLPReporter validates the config (https unless loopback or allowInsecureInCluster).

func (*DLPReporter) Counters added in v0.4.0

func (r *DLPReporter) Counters() (sent, failed, dropped, deduped int64)

Counters for /metrics: events sent, failed posts, events dropped (queue full) and deduplicated.

func (*DLPReporter) FlushNow added in v0.4.0

func (r *DLPReporter) FlushNow(ctx context.Context) error

FlushNow sends queued events in batches until none is left or a send fails; a failed batch is resent with the same id.

func (*DLPReporter) Report added in v0.4.0

func (r *DLPReporter) Report(evs ...DLPEvent)

Report queues events. Safe for concurrent use; nil-safe.

func (*DLPReporter) Run added in v0.4.0

func (r *DLPReporter) Run(ctx context.Context)

Run posts every Flush until ctx ends. Call FlushNow after it returns.

type DLPResponseScanner added in v0.4.0

type DLPResponseScanner struct {
	Truncated bool
	// contains filtered or unexported fields
}

DLPResponseScanner looks for secrets in a response's generated text as it streams. It never changes the response. Not safe for concurrent use: one UsageTap feeds it from one goroutine, and Findings is read once the body ended.

func NewDLPResponseScanner added in v0.4.0

func NewDLPResponseScanner(d *policy.DLP, ring string, units []string) *DLPResponseScanner

NewDLPResponseScanner returns a scanner, or nil when DLP or response scanning is off for the caller.

func (*DLPResponseScanner) Findings added in v0.4.0

func (s *DLPResponseScanner) Findings() []DLPFinding

Findings scans what is left and returns every distinct secret seen.

func (*DLPResponseScanner) Text added in v0.4.0

func (s *DLPResponseScanner) Text(t string)

Text adds generated text.

type Decision

type Decision struct {
	Ring          string
	Release       string
	Experiment    string
	Variant       string
	UpstreamName  string
	Upstream      string // URL; "" = leave Kong's default service
	UpstreamModel string // "" = leave the model untouched
	// Route is the ordered attempt list (failover order) for the alias;
	// UpstreamName/UpstreamModel/Upstream are its first element. halo-kong
	// uses only those; halo-proxy walks Route.
	Route   []policy.RouteTarget
	Headers map[string]string
	Shadow  []ShadowTarget
}

Decision is the routing outcome for one request.

func Decide

func Decide(org *policy.Org, req RequestInfo) Decision

Decide is pure and deterministic for a given (org, req).

type DirectoryClient added in v0.3.0

type DirectoryClient struct {
	URL, Token string
	TTL, Grace time.Duration
	Client     *http.Client
	Now        func() time.Time
	Log        *slog.Logger
	// FailClosed: refuse (ErrDirectoryUnavailable) instead of failing open
	// when the directory cannot be asked (DirectoryFailClosed).
	FailClosed bool
	// Observe, when set, is told every Resolve outcome of a looked-up subject:
	// ok, stale (cached entry served through an outage), failopen,
	// unavailable (FailClosed) or not_provisioned. It is also told
	// revocations_unavailable each time the revocation list cannot be fetched
	// (whatever the fail mode then does). For metrics.
	Observe func(outcome string)
	// RevocationTTL: the revocation list is refetched once it is older
	// (0 = DefaultRevocationTTL). It is fetched only for callers holding a
	// halo-server credential (workload or delegated agent).
	RevocationTTL time.Duration
	// RevocationsOnly (DirectoryModeRevocations) checks halo-server credentials
	// against the revocation list and never looks a subject up in SCIM: groups
	// stay the verified credential's. For deployments without SCIM.
	RevocationsOnly bool
	// contains filtered or unexported fields
}

DirectoryClient makes halo-server's SCIM directory authoritative for a verified subject (ADR-0022): the subject's groups are the directory's, and a subject the directory does not list as active is refused (ErrNotProvisioned). Groups never come from the request.

func NewDirectoryClient added in v0.3.0

func NewDirectoryClient(url, token string, ttl, grace time.Duration, log *slog.Logger, allowInsecureInCluster bool) (*DirectoryClient, error)

NewDirectoryClient validates the config (https unless loopback or allowInsecureInCluster).

func (*DirectoryClient) Resolve added in v0.3.0

func (c *DirectoryClient) Resolve(ctx context.Context, sub *policy.Subject) (*policy.Subject, error)

Resolve returns sub with the directory's groups. It returns nil, nil (unknown caller) only when sub is nil, and ErrNotProvisioned when the directory does not list sub as active, when halo-server answers the lookup with a non-200 status below 500, or when the last cached answer for sub was inactive, however old. When halo-server cannot be asked (transport error or 5xx) and no entry younger than Grace is cached it returns sub with no groups (fail open), or ErrDirectoryUnavailable under FailClosed: never nil, which would let an outage turn a deactivated user into an anonymous caller that skips the posture gate and per-user budgets. Workload subjects (ADR-0023) are not in SCIM: they are returned unchanged without a lookup. Kind and Delegation always come from the verified credential, never from the directory: a delegated agent (ADR-0024) is looked up as its root principal, so an agent acting for a deprovisioned user is refused like the user, and keeps its chain (and the agent policy) on every path, fail open included. A caller holding a halo-server credential (workload or delegated agent) is first checked against the revocation list (ADR-0026, see revocations): a revoked credential is ErrCredentialRevoked, workload-rooted chains included. ponytail: concurrent misses for one subject each call halo-server, as PostureClient.Check does.

type DirectoryEntry added in v0.3.0

type DirectoryEntry struct {
	Active bool     `json:"active"`
	Groups []string `json:"groups,omitempty"`
}

DirectoryEntry is halo-server's SCIM directory answer for one subject (GET /api/v1/gateway/directory). Inactive covers deactivated, deleted and never provisioned users alike.

type Estimate added in v0.3.0

type Estimate struct {
	RequestBytes int
	// MaxOutput is the request's maximum output tokens (MaxOutputTokens); 0 = none.
	MaxOutput int64
	// OutputCeiling caps the output estimate when MaxOutput is 0 (0 = DefaultOutputCeiling).
	OutputCeiling int64
	// ServerStateInput is the input to charge when the request makes the
	// provider bill input its bytes do not show (ServerStateInput true); 0 = it does not.
	ServerStateInput int64
	// Reasoning: the request enables hidden reasoning (ReasoningBudget), so
	// the output estimate is at least ThinkingBudget, or the output limit when
	// ThinkingBudget is 0: reasoning tokens are billed as output but never
	// streamed, so a stream cut before the final usage event shows none of them.
	Reasoning      bool
	ThinkingBudget int64
	// Background: an OpenAI Responses background request; its response is
	// queued (usage null), so the output is charged at the limit.
	Background bool
}

Estimate is what EstimateUsage knows about the request (RequestEstimate).

func RequestEstimate added in v0.3.0

func RequestEstimate(proto string, body []byte, serverStateInput, outputCeiling int64) Estimate

RequestEstimate is the Estimate for a metered request body: maximum output, server-side state (charged serverStateInput, else DefaultServerStateInputTokens), reasoning and background mode.

type Finding

type Finding struct {
	Gate   string // GatePosture | GateVersion
	Mode   string // policy.GateWarn | policy.GateEnforce
	Reason string
}

Finding is a failed gate check under a ring whose mode is warn or enforce.

func CheckPosture

func CheckPosture(ctx context.Context, c *PostureClient, org *policy.Org, ring string, sub *policy.Subject) (f *Finding, unknown bool)

CheckPosture applies ring's posture gate to a verified subject. unknown reports that the gate is on but no verdict was available (an anonymous caller, or halo-server unreachable past the grace window): the request passes and callers count it. A nil client means the posture gate is not configured: a gateway whose posture config is invalid must not pass nil (halo-proxy refuses to start; halo-kong fails closed on enforce rings).

func CheckVersion

func CheckVersion(org *policy.Org, ring, ua string) *Finding

CheckVersion applies ring's versionGate to a request's User-Agent. nil = pass (or gate off). A ring that pins no version for the detected harness passes: there is nothing to compare against.

func (*Finding) Message

func (f *Finding) Message() string

Message is what the developer's CLI shows on a refusal: what failed and what to run.

func (*Finding) Reject

func (f *Finding) Reject(proto string) *Rejection

Reject is the 403 to answer in enforce mode, in the caller's wire format; nil in warn mode.

type KillEnvelope

type KillEnvelope struct {
	Payload   []byte `json:"payload"`   // base64 in JSON
	Signature []byte `json:"signature"` // base64 in JSON
	// Alg is the signature algorithm (bundle.AlgEd25519 or bundle.AlgECDSAP256,
	// ADR-0038); absent means ed25519, as every envelope before it. It only
	// selects how to verify: a verifier accepts it from its own keys only.
	Alg string `json:"alg,omitempty"`
	// KeyID is bundle.KeyID of the signing key: informational (rotation).
	KeyID string `json:"keyId,omitempty"`
}

KillEnvelope is GET /api/v1/gateway/killswitch: Payload is the JSON KillList, Signature is over killDomain+Payload with algorithm Alg.

func SignKillList

func SignKillList(ctx context.Context, s bundle.Signer, l KillList) (KillEnvelope, error)

SignKillList seals l for gateways with s (a file, KMS or external signer).

type KillList

type KillList struct {
	Version     uint64   `json:"version"` // grows with every kill/unkill; informational
	Experiments []string `json:"experiments"`
	// Toggles are killed feature-toggle names: treated as off everywhere, whatever
	// their rules say. Part of the signed payload, so covered by the same
	// signature, freshness and monotonicity checks as Experiments.
	Toggles  []string  `json:"toggles,omitempty"`
	IssuedAt time.Time `json:"issuedAt"`
}

KillList is the set of experiments halo-server has killed: gateways treat them as not running (control routing, no shadow) without waiting for a policy PR to merge and a snapshot to ship.

func VerifyKillList

func VerifyKillList(ctx context.Context, v bundle.Verifier, env KillEnvelope, now time.Time, maxAge time.Duration) (KillList, error)

VerifyKillList checks the signature and freshness (IssuedAt within maxAge of now, at most a minute in the future) and returns the list.

type KillSwitch

type KillSwitch struct {
	URL   string
	Token string
	// PubKey verifies the envelopes: a bundle.Ed25519Verifier, an
	// ECDSAVerifier, or a bundle.Keyring while rotating the kill key.
	PubKey   bundle.Verifier
	Interval time.Duration // default DefaultKillInterval
	MaxAge   time.Duration // default DefaultKillMaxAge
	Client   *http.Client  // default: 10s timeout, no redirects
	Log      *slog.Logger
	Now      func() time.Time
	// contains filtered or unexported fields
}

KillSwitch polls halo-server for the signed kill list.

Failure policy: a fetch or verification failure keeps the last accepted list (a kill never silently lapses because the control plane is down); if no list was ever accepted the set is empty and the policy snapshot's statuses alone decide. A list is accepted only if it is strictly newer (IssuedAt) than the current one, so a replayed older envelope cannot undo a kill. A nil *KillSwitch is valid and kills nothing.

func NewKillSwitch

func NewKillSwitch(url, token string, pub bundle.Verifier, interval time.Duration, log *slog.Logger, allowInsecureInCluster bool) (*KillSwitch, error)

NewKillSwitch validates the config (see bundle.ParseVerifier for PEM keys). allowInsecureInCluster permits plain http to a non-loopback host.

func (*KillSwitch) Apply

func (k *KillSwitch) Apply(org *policy.Org) *policy.Org

Apply returns org with every killed experiment's status forced to "paused", so Decide routes its users to default (control) and mirrors nothing. org itself is never mutated; it is returned as-is when nothing applies. ponytail: copies per request while a kill is active; cache by (org, version) if it shows in profiles.

func (*KillSwitch) Killed

func (k *KillSwitch) Killed() KillList

Killed returns the current list (a copy).

func (*KillSwitch) Refresh

func (k *KillSwitch) Refresh(ctx context.Context) error

Refresh fetches and verifies the list once; on error the previous list stays.

func (*KillSwitch) Restore

func (k *KillSwitch) Restore(l KillList)

Restore seeds a last-known list persisted by the caller (halod keeps it in its state file) so kills survive a restart and envelopes issued before it are refused as replays. It is a no-op unless l is newer than the current list. The list is trusted as given: pass only one that was verified.

func (*KillSwitch) Run

func (k *KillSwitch) Run(ctx context.Context)

Run refreshes immediately and then every Interval until ctx ends.

type Outbound

type Outbound struct {
	Path     string
	RawQuery string
	SetQuery bool // replace the client's query string with RawQuery (else keep it)
	Body     []byte
	// SSE: the response is a Bedrock event stream the anthropic-messages client
	// cannot read; convert it with EventStreamToSSE.
	SSE bool
}

Outbound is the request an upstream expects, derived from the client's request. Path is escaped and relative to the upstream's base URL.

func BuildOutbound

func BuildOutbound(proto string, up policy.Upstream, model, path string, body []byte) (Outbound, error)

BuildOutbound rewrites the client's request (protocol proto, escaped path, body) for upstream up serving model. Same-wire upstreams only get the model swapped (RewriteRequest). Translations:

anthropic-messages -> bedrock   POST /model/{id}/invoke[-with-response-stream], body without model/stream + anthropic_version
anthropic-messages -> vertex    POST .../publishers/anthropic/models/{id}:[stream]rawPredict, body without model + anthropic_version
openai-responses   -> azure     POST /openai/v1/responses (or /openai/responses?api-version=), model = deployment

Responses stay in the provider's format except Bedrock streams, which are converted to SSE (see Outbound.SSE).

type PostureClient

type PostureClient struct {
	URL, Token string
	TTL, Grace time.Duration
	Client     *http.Client
	Now        func() time.Time
	Log        *slog.Logger
	// contains filtered or unexported fields
}

PostureClient asks halo-server for a subject's device posture and caches the verdict. It fails closed on a verdict halo-server returned and on a lookup it refused (a non-200 status below 500: a rejected token, posture not configured on the server, a redirect): that is a misconfiguration, never an outage. When the server cannot be reached (transport error or 5xx), the last verdict serves for Grace, and after that the posture is unknown (callers let the request through and count it).

func NewPostureClient

func NewPostureClient(url, token string, ttl, grace time.Duration, log *slog.Logger, allowInsecureInCluster bool) (*PostureClient, error)

NewPostureClient validates the config (https unless loopback or allowInsecureInCluster).

func (*PostureClient) Check

func (c *PostureClient) Check(ctx context.Context, subject string) (v PostureVerdict, known bool)

Check returns subject's verdict; known=false when halo-server could not be asked and no verdict younger than Grace is cached. A refused lookup is a non-compliant verdict, not cached (a fixed token heals at once). ponytail: concurrent misses for one subject each call halo-server; add singleflight if a burst after TTL expiry shows up in the server's load.

type PostureVerdict

type PostureVerdict struct {
	Compliant bool   `json:"compliant"`
	Reason    string `json:"reason,omitempty"`
}

PostureVerdict is halo-server's answer for one subject (GET /api/v1/gateway/posture).

type Rejection

type Rejection struct {
	Status   int
	Protocol string // selects the error body shape
	Message  string
	Code     string // OpenAI Responses error code; "" = derived from Status
	// RetryAfter, when > 0, is sent as Retry-After (whole seconds, rounded up).
	RetryAfter time.Duration
}

Rejection is a request the gateway answers itself. Fail closed: unknown model endpoints (404), unreadable/oversize bodies (413), models that are not a configured alias (400: a permanent client error, which the CLIs do not retry, unlike 403/5xx/429).

func DirectoryReject added in v0.4.0

func DirectoryReject(path string, err error) *Rejection

DirectoryReject maps a DirectoryClient.Resolve error to the rejection a gateway answers, in the caller's protocol.

func RejectCredentialRevoked added in v0.4.0

func RejectCredentialRevoked(path string) *Rejection

RejectCredentialRevoked is the 401, in the caller's protocol, for ErrCredentialRevoked.

func RejectDirectoryUnavailable added in v0.3.0

func RejectDirectoryUnavailable(path string) *Rejection

RejectDirectoryUnavailable is the 503, in the caller's protocol, for ErrDirectoryUnavailable.

func RejectNotProvisioned added in v0.3.0

func RejectNotProvisioned(path string) *Rejection

RejectNotProvisioned is the 403, in the caller's protocol, for ErrNotProvisioned.

func RejectTooLarge

func RejectTooLarge(path string) *Rejection

RejectTooLarge is the 413 for a model call whose body could not be read in full (Kong spooled it to disk, or it exceeds the proxy limit).

func (*Rejection) Error

func (r *Rejection) Error() string

func (*Rejection) JSON

func (r *Rejection) JSON() []byte

JSON renders the error in the caller's wire format so the CLI shows Message.

func (*Rejection) SetHeaders added in v0.3.0

func (r *Rejection) SetHeaders(h http.Header)

SetHeaders adds the response headers that go with the body: Retry-After, and for Bedrock the x-amzn-ErrorType its SDKs classify errors by.

type RequestInfo

type RequestInfo struct {
	UserID     string // from the auth gateway's identity header only
	Groups     []string
	ModelAlias string
	SessionID  string
	Harness    string
	Protocol   string
	FirstTurn  bool
	// AnyTurn: a parsed, replayable model call at any step of a session
	// (eligible for shadow experiments with turns: all).
	AnyTurn bool
	// Workload: the verified caller is a CI / cloud workload (ADR-0023). It
	// gets no experiment or shadow unless policy workloads.experiments is true.
	Workload bool
	// Agent: the verified caller is a delegated agent acting for UserID
	// (ADR-0024). It gets no experiment or shadow unless policy
	// agents.experiments is true (and, for a workload root, workloads.experiments).
	Agent bool
}

RequestInfo is everything Decide may look at.

func NewRequestInfo

func NewRequestInfo(h http.Header, path string, body []byte) RequestInfo

NewRequestInfo builds RequestInfo from the request alone. UserID/Groups are left empty: cohort comes only from a verified identity (PrepareVerified).

type Result

type Result struct {
	Decision Decision
	Protocol string
	// ClearHeaders are client-supplied x-halo-* names to drop (all of them; the
	// ones Decision sets are then re-set with gateway-owned values).
	ClearHeaders []string
	// Path/Body are the rewritten forms; Rewritten is false when unchanged.
	Path      string
	Body      []byte
	Rewritten bool
	// RewriteErr is non-nil when a rewrite was needed but failed (Reject is
	// then set too: an un-rewritten alias must never reach the upstream).
	RewriteErr error
	// Reject, when non-nil, must be answered by the adapter itself; the
	// request must NOT be forwarded.
	Reject *Rejection
	// DLP is the request's secret and injection scan (ADR-0033); nil when the
	// policy has no gateway.dlp or it is off for the caller. When it redacted,
	// Body (and Rewritten) carry the redacted body.
	DLP *DLPReport
}

Result is everything the Kong plugin must apply to the upstream request.

func PrepareVerified

func PrepareVerified(org *policy.Org, sub *policy.Subject, h http.Header, path string, body []byte) Result

PrepareVerified inspects a request and computes the full set of mutations, or a Rejection the adapter must answer instead of forwarding. Cohort comes only from sub, verified by internal/identity (JWT, or trusted-proxy headers); sub == nil means anonymous: default routing, no experiments. Client-supplied headers (x-halo-* or identity) never influence the outcome.

type RevocationList added in v0.4.0

type RevocationList struct {
	// JTIs maps a revoked credential id to when the entry lapses (unix seconds).
	JTIs map[string]int64 `json:"jtis"`
	// Subjects maps a canonical subject (policy.CanonicalUser of a human root
	// id, a "workload:" subject or "agent:<name>") to its revoked-before epoch
	// (unix seconds): see Check.
	Subjects map[string]int64 `json:"subjects"`
}

RevocationList is halo-server's active credential revocations (ADR-0026), GET /api/v1/gateway/directory?revocations=1. It holds only revocations that can still matter: a jti until the longest a credential can live (policy.MaxWorkloadTTL) has passed since it was revoked, a subject epoch until that long after the epoch. So its size is bounded by what admins revoked in the last hour, and halo-server caps it (maxActiveRevocations).

func (*RevocationList) Check added in v0.4.0

func (l *RevocationList) Check(sub *policy.Subject) error

Check returns ErrCredentialRevoked (wrapped with the reason) when the credential sub was verified from is dead:

  • its jti, or the jti of any credential it was exchanged from, is revoked;
  • its root principal has an epoch at or after the root's authentication (a delegated credential's auth_time, a workload credential's iat);
  • an agent of its delegation chain has an epoch at or after its iat.

A subject with no halo-server credential (an IdP user) is never revoked here.

type RouteRow

type RouteRow struct {
	Alias      string      `json:"alias"`
	Ring       string      `json:"ring"`
	Experiment string      `json:"experiment,omitempty"` // traffic-axis experiment that supplied the route
	Variant    string      `json:"variant,omitempty"`
	Targets    []TargetRow `json:"targets"`
}

RouteRow is the effective route of one alias for a caller.

func RouteTable

func RouteTable(org *policy.Org, req RequestInfo) []RouteRow

RouteTable returns every alias's effective routes for req (UserID, Groups and SessionID decide the ring, experiment variant and weighted pick; ModelAlias is ignored). Anonymous callers (no user, no session) see policy order.

type ShadowTarget

type ShadowTarget struct {
	Experiment string
	Variant    string
	Upstream   string // URL
	Model      string
}

ShadowTarget is one candidate a request should be mirrored to.

type Snapshot

type Snapshot struct {
	// contains filtered or unexported fields
}

Snapshot serves a compiled org policy (policy.Compile output, or a bare json.Marshal of *policy.Org) from a file and reloads it when the mtime/size changes. A failed reload keeps serving the last good policy.

func NewSnapshot

func NewSnapshot(path string) *Snapshot

func (*Snapshot) Get

func (s *Snapshot) Get() (*policy.Org, error)

Get returns the current org. err is non-nil if the latest load/reload failed; org may still be the previous good snapshot (nil if none ever loaded).

func (*Snapshot) Refused added in v0.3.0

func (s *Snapshot) Refused() uint64

Refused counts snapshots refused for security validation errors (policy.SecurityIssues): a compiled snapshot that skipped `halo validate` never activates.

type TargetRow

type TargetRow struct {
	Order          int      `json:"order"`
	Selected       bool     `json:"selected"`
	Upstream       string   `json:"upstream"`
	Kind           string   `json:"kind"`
	Model          string   `json:"model"`
	Weight         float64  `json:"weight,omitempty"`
	Priority       int      `json:"priority"`
	TimeoutSeconds int      `json:"timeoutSeconds,omitempty"`
	SkippedBy      []string `json:"skippedByHarnesses,omitempty"` // harnesses whose wire protocol this target cannot serve
}

TargetRow is one attempt-order entry; Selected marks where a request lands when every target is healthy.

type Tool added in v0.3.0

type Tool struct {
	Name   string
	Hosted bool
}

Tool is one tool a model call declares. Hosted marks a tool the provider runs or builds in (web search, code execution, a remote MCP server, a versioned Anthropic tool): Name is then its type, its Gemini or Bedrock tool key, or mcp:<server>, and policy matches it as a prefix (see policy.Agents.DeniesTool).

func ToolNames added in v0.3.0

func ToolNames(protocol, path string, body []byte) ([]Tool, error)

ToolNames returns every tool a model call declares, or an error when the body or its tool declarations cannot be read: a body nested past maxJSONDepth, trailing data, a tools member of the wrong shape, a tool without a name, or a (protocol, path) without an extractor. The inspectable shapes are an explicit list:

  • Anthropic Messages and count_tokens: tools[].name and tools[].type (hosted unless "custom"), mcp_servers[].name as mcp:<name>.
  • OpenAI Responses: tools[].name of function and custom tools; any other tools[].type is hosted, mcp as mcp:<server_label>.
  • Bedrock invoke and converse: the Anthropic shape, toolConfig.tools[].toolSpec.name, toolConfig.tools[].systemTool.name (hosted), any other toolConfig.tools[] member (hosted) and additionalModelRequestFields.tools.
  • Gemini generateContent, streamGenerateContent and countTokens: tools[].functionDeclarations[].name, every other tools[] member key (codeExecution, googleSearch, urlContext, ...) as hosted, and the same under generateContentRequest.

Keys match ignoring case and underscores (function_declarations is functionDeclarations) and duplicate keys all count, so a body cannot show the gateway one tool list and the upstream parser another.

type UsageResult added in v0.3.0

type UsageResult struct {
	Usage budget.Usage // field-wise maximum over every usage object read
	Found bool         // at least one usage object was read
	// Complete: the upstream body reached its clean end (EOF) and a usage
	// object was read, so Usage is the provider's final count. A client that
	// hangs up early, an upstream cut, a Content-Encoding the tap cannot decode
	// or a body without usage leave it false: charge EstimateUsage instead.
	Complete bool
	Bytes    int64 // response bytes seen (decoded bytes when the tap decoded them)
	// Streamed: the body was an SSE or AWS event stream the tap decoded, so
	// OutputText and OutputEvents measure the generated output.
	Streamed bool
	// OutputText is the UTF-8 bytes of generated output (text, thinking, tool
	// arguments) in the stream's delta events, without framing or JSON
	// escaping; OutputEvents counts those events.
	OutputText, OutputEvents int64
}

UsageResult is what a UsageTap saw by the time its body ended.

Directories

Path Synopsis
Package adapters renders the "put halo-proxy in front of / behind stack X" config snippets shipped under deploy/integrations.
Package adapters renders the "put halo-proxy in front of / behind stack X" config snippets shipped under deploy/integrations.
Package kong generates the decK declarative config for Kong OSS (db-less) that fronts an org's model traffic and attaches the halo-kong plugin.
Package kong generates the decK declarative config for Kong OSS (db-less) that fronts an org's model traffic and attaches the halo-kong plugin.
Package upstreamauth signs requests to provider upstreams that need a gateway-held credential instead of the caller's, so halo-proxy can talk to Amazon Bedrock directly (no orchestrator, no SigV4 sidecar).
Package upstreamauth signs requests to provider upstreams that need a gateway-held credential instead of the caller's, so halo-proxy can talk to Amazon Bedrock directly (no orchestrator, no SigV4 sidecar).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL