middleware

package
v0.74.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 3, 2026 License: BSD-3-Clause, AGPL-3.0 Imports: 20 Imported by: 0

Documentation

Overview

Package middleware defines the per-target middleware chain that runs inside the reverse proxy hot path. It is the only chain wired into the request path.

Concepts:

  • Slot: the position a middleware occupies in the chain. A middleware lives in exactly one slot — separate concerns become separate middlewares.
  • Decision: the on_request slot can DENY; on_response and terminal slots can only PASSTHROUGH. The dispatcher clamps decisions that violate this contract.
  • Metadata: the only side-channel between middlewares. Each middleware declares an allowlist of keys it may emit; the merger enforces caps and namespace rules.

Index

Constants

View Source 
const (
	ErrorKindPanic       = "panic"
	ErrorKindTimeout     = "timeout"
	ErrorKindInvokeError = "invoke_error"
)

Dispatcher reliability kinds reported via proxy.middleware.errors_total{kind=...}.

View Source 
const (
	// LLM request-side metadata (emitted by llm_request_parser).
	KeyLLMProvider         = "llm.provider"
	KeyLLMModel            = "llm.model"
	KeyLLMStream           = "llm.stream"
	KeyLLMRequestPromptRaw = "llm.request_prompt_raw"
	KeyLLMCaptureTruncated = "llm.capture_truncated"
	// KeyLLMSessionID groups requests of the same conversation / coding
	// session, read from the per-provider session marker in the request
	// body. Empty for clients that don't send one.
	KeyLLMSessionID = "llm.session_id"

	// LLM response-side metadata (emitted by llm_response_parser).
	//nolint:gosec // metadata key name, not a credential
	KeyLLMInputTokens = "llm.input_tokens"
	//nolint:gosec // metadata key name, not a credential
	KeyLLMOutputTokens = "llm.output_tokens"
	//nolint:gosec // metadata key name, not a credential
	KeyLLMTotalTokens = "llm.total_tokens"
	// LLM cached-input bucket. For OpenAI it's the SUBSET of input
	// tokens that hit the prompt cache (prompt_tokens_details.
	// cached_tokens) — billed at the cached_input_per_1k rate when
	// configured. For Anthropic it's cache_read_input_tokens, which
	// is ADDITIVE to llm.input_tokens — billed at cache_read_per_1k.
	// cost_meter switches formula on llm.provider.
	//nolint:gosec // metadata key name, not a credential
	KeyLLMCachedInputTokens = "llm.cached_input_tokens"
	// LLM cache-creation bucket (Anthropic only). ADDITIVE to
	// llm.input_tokens; billed at cache_creation_per_1k.
	//nolint:gosec // metadata key name, not a credential
	KeyLLMCacheCreationTokens = "llm.cache_creation_tokens"
	KeyLLMResponseCompletion  = "llm.response_completion"

	// Guardrail outcomes (emitted by llm_guardrail). The guardrail
	// also re-emits llm.request_prompt as a redacted variant of the
	// raw prompt and drops llm.request_prompt_raw from the bag.
	KeyLLMRequestPrompt  = "llm.request_prompt"
	KeyLLMPolicyDecision = "llm_policy.decision"
	KeyLLMPolicyReason   = "llm_policy.reason"

	// LLM router routing decision (emitted by llm_router). The router
	// stamps the resolved provider id so downstream middlewares and
	// the access-log emitter can attribute the request without
	// re-parsing the body.
	KeyLLMResolvedProviderID = "llm.resolved_provider_id"

	// LLM authorising groups for this request (emitted by llm_router
	// on the allow path). Carries the comma-separated intersection of
	// the caller's UserGroups with the resolved route's
	// AllowedGroupIDs — i.e. the groups that actually authorise this
	// specific request, NOT every group the peer happens to be in.
	// Identity-stamping middlewares use this for per-request tag
	// attribution so unrelated group memberships don't leak into
	// downstream gateways' spend logs.
	KeyLLMAuthorisingGroups = "llm.authorising_groups"

	// LLM policy attribution (emitted by llm_limit_check on the allow
	// path). Names the policy that paid for this request and the
	// dimension counters the post-flight llm_limit_record middleware
	// must tick. Empty when no applicable policy has any caps
	// configured (catch-all-allow attribution).
	KeyLLMSelectedPolicyID   = "llm.selected_policy_id"
	KeyLLMAttributionGroupID = "llm.attribution_group_id"
	KeyLLMAttributionWindowS = "llm.attribution_window_seconds"

	// Cost metering (emitted by cost_meter).
	KeyCostUSDTotal = "cost.usd_total"
	KeyCostSkipped  = "cost.skipped"

	// Framework-emitted error markers. Use the mw.<id>.* prefix to
	// distinguish framework-injected entries from middleware-emitted
	// metadata.
	KeyFrameworkErrorKindFmt = "mw.%s.error_kind"
)

Metadata key namespace constants shared across the built-in middlewares. Each domain owns a prefix; middlewares declare their per-key allowlist drawn from these constants. Agents implementing the G2 middlewares import this file so the dashboard's expanded-row viewer and the access-log writer see a stable key surface.

Key shape rules (enforced by the metadata accumulator):

  • Lowercase ASCII letters, digits, dot, underscore, hyphen.
  • At least one dot separating namespace from leaf.
  • Max length: MaxMetadataKeyBytes.
View Source 
const (
	MetadataReasonBadKey         = "bad_key"
	MetadataReasonNotAllowlisted = "not_allowlisted"
	MetadataReasonKeyTooLong     = "key_too_long"
	MetadataReasonValueTooLong   = "value_too_long"
	MetadataReasonMiddlewareCap  = "middleware_cap"
	MetadataReasonRequestCap     = "request_cap"
)

Rejection reasons reported by Accumulator.Emit.

View Source 
const (
	// MaxBodyCapBytes is the proxy-wide upper bound for per-direction
	// body capture. Sized to hold a full LLM streaming response (token
	// usage rides the trailing SSE event, so the captured prefix must
	// reach the end of the stream); a single response is bounded by the
	// model's max output tokens, so this is a real ceiling, not a
	// treadmill. Request capture stays well under this — oversized
	// requests use the tolerant routing scan instead of buffering.
	MaxBodyCapBytes int64 = 8 << 20
	// MinTimeout is the proxy-wide lower bound for per-middleware
	// Invoke timeouts.
	MinTimeout = 10 * time.Millisecond
	// MaxTimeout is the proxy-wide upper bound for per-middleware
	// Invoke timeouts.
	MaxTimeout = 5 * time.Second
	// DefaultTimeout is used when the per-target timeout is zero or
	// unset.
	DefaultTimeout = 500 * time.Millisecond

	// MaxMiddlewareMetadataBytes is the per-middleware metadata total
	// cap.
	MaxMiddlewareMetadataBytes = 16 << 10
	// MaxRequestMetadataBytes is the per-request metadata total cap
	// across all middlewares in the chain. Earlier middlewares win
	// when the budget is exhausted.
	MaxRequestMetadataBytes = 32 << 10
	// MaxMetadataKeyBytes is the maximum length of a metadata key.
	MaxMetadataKeyBytes = 96
	// MaxMetadataValueBytes is the maximum length of a metadata value.
	MaxMetadataValueBytes = 4 << 10
	// MaxMiddlewaresPerChain caps the number of middleware entries
	// accepted per chain at the proxy translator and the management
	// REST API. Mirrors the chain invocation cap so a misconfigured
	// mapping cannot push the chain clone cost beyond a known bound.
	MaxMiddlewaresPerChain = 16
)

Resource limits enforced by the proxy at config apply time and by the dispatcher at runtime. Per-target values supplied by management are clamped to these bounds.

Variables

View Source 
var ErrContentLengthMismatch = errors.New("body replace rejected: content-length mismatch (short read)")

ErrContentLengthMismatch is returned when the client-advertised Content-Length disagrees with the number of bytes actually read from the body (short-read).

View Source 
var ErrExpectContinue = errors.New("body replace rejected: request has Expect: 100-continue")

ErrExpectContinue is returned when a middleware attempts to replace the body of a request that advertised Expect: 100-continue.

View Source 
var ErrOriginalNotDrained = errors.New("body replace rejected: original body not drained")

ErrOriginalNotDrained is returned when the original body was not fully consumed before replacement. This prevents the backend from seeing a mix of original bytes and the replacement.

Functions

func ApplyBodyReplace 

func ApplyBodyReplace(r *http.Request, newBody []byte)

ApplyBodyReplace swaps r.Body for a reader over newBody, recomputes Content-Length, and strips Transfer-Encoding and Trailer so no stale framing reaches the backend.

func IsHeaderMutable 

func IsHeaderMutable(name string) bool

IsHeaderMutable reports whether a middleware is allowed to mutate the named header. The check is case-insensitive and honours both exact matches and the compiled-in prefix denylist.

func RenderDenyResponse 

func RenderDenyResponse(w http.ResponseWriter, middlewareID string, reason *DenyReason, defaultStatus int)

RenderDenyResponse writes a structured JSON deny body. Status is clamped to [400, 499] excluding 401 (to avoid conflicts with the proxy's auth flow). All middleware-supplied strings are redacted and truncated. On any validation failure the function writes a generic 403.

func Scan 

func Scan(value string) string

Scan redacts high-signal secret patterns from value. Matches are replaced with `[REDACTED:<kind>]`. Non-matching input is returned unchanged.

func ValidateBodyReplace 

func ValidateBodyReplace(r *http.Request, newBody []byte, originalDrained bool) error

ValidateBodyReplace runs the smuggling-prevention rules before a body replacement is applied. Callers must pass originalDrained=true once they have read r.Body to EOF.

Types

type Accumulator 

type Accumulator struct {
	// contains filtered or unexported fields
}

Accumulator enforces per-middleware and per-request metadata caps. Not safe for concurrent use; callers hold one inside a single chain execution.

func NewAccumulator 

func NewAccumulator(maxPerRequest int) *Accumulator

NewAccumulator returns an accumulator configured for the per-request total cap. A maxPerRequest of zero means use MaxRequestMetadataBytes.

func (*Accumulator) Emit 

func (a *Accumulator) Emit(middlewareID string, allow []string, out []KV) ([]KV, []MetadataRejection)

Emit validates the candidate metadata against the middleware's allowlist and the global caps, redacts each accepted value, and returns the accepted entries plus any rejections for metric emission.

type AuthHeader 

type AuthHeader struct {
	Name  string
	Value string
}

AuthHeader is a single name/value pair the proxy injects on the upstream request after stripping the client's auth headers.

type Chain 

type Chain struct {
	// contains filtered or unexported fields
}

Chain is the ordered set of middlewares that run for a specific target. Chains are immutable once built; Manager produces a new Chain on every Rebuild.

Ordering: middlewares are kept in registration order. RunRequest iterates the SlotOnRequest middlewares in order; RunResponse iterates the SlotOnResponse middlewares in reverse order (middleware-style LIFO so the last to see the request is the first to see the response); RunTerminal iterates the SlotTerminal middlewares in registration order, after every on_response slot has emitted, so the metadata bag they observe is complete.

Close drains in-flight invocations and tears down each middleware. Callers swapping a chain via Manager invoke Close on the old chain after the swap so live requests finish on the previous instance.

func NewChain 

func NewChain(targetID string, bound []boundMiddleware, d *Dispatcher) *Chain

NewChain assembles a Chain from the bound middlewares. The slice order is the registration order; the chain captures index slices per slot so iteration does not re-scan the slot field per call.

func (*Chain) Close 

func (c *Chain) Close(ctx context.Context) error

Close waits for outstanding invocations against this chain to finish (bounded by ctx) and releases the middleware instances bound to it. Safe to call once the chain has been removed from the routing snapshot. Subsequent Run* calls are still safe (return without invoking) but Close itself is one-shot.

func (*Chain) Empty 

func (c *Chain) Empty() bool

Empty reports whether the chain has no middlewares.

func (*Chain) IDs 

func (c *Chain) IDs() []string

IDs returns the ordered list of middleware IDs bound to this chain.

func (*Chain) RunRequest 

func (c *Chain) RunRequest(ctx context.Context, r *http.Request, in *Input, acc *Accumulator) (denied *Output, merged []KV, rewrite *UpstreamRewrite, err error)

RunRequest iterates the on_request slot in registration order. Deny short-circuits the remaining middlewares and returns the deny output. The caller owns applying mutations to the real request and merging the metadata returned in `merged` into the captured-data bag passed to subsequent slots.

Each middleware sees the metadata emitted by earlier middlewares in the same slot — this is how llm_guardrail reads llm.request_prompt_raw from llm_request_parser without a side channel, and how cost_meter reads tokens emitted by llm_response_parser on the response leg.

If any middleware emits a non-nil Mutations.RewriteUpstream while satisfying the mutation gates (CanMutate && MutationsSupported), the latest such value is returned to the caller. Last-write-wins so the last middleware in the slot can override an earlier rewrite.

func (*Chain) RunResponse 

func (c *Chain) RunResponse(ctx context.Context, in *Input, acc *Accumulator) (merged []KV)

RunResponse iterates the on_response slot in reverse registration order, matching the middleware "last in, first out" convention so the last middleware to see the request is the first to see the response. Middlewares cannot deny; they emit metadata.

As with RunRequest, each middleware sees the metadata emitted by earlier middlewares in this slot — accumulated in the order the middlewares run (LIFO of registration). cost_meter relies on this to read llm.input_tokens / llm.output_tokens that llm_response_parser emitted just before it.

func (*Chain) RunTerminal 

func (c *Chain) RunTerminal(ctx context.Context, in *Input, acc *Accumulator) (merged []KV)

RunTerminal iterates the terminal slot in registration order, after every on_response middleware has emitted. Terminal middlewares observe the full metadata bag carried in `in.Metadata` plus any emissions from terminal middlewares that ran before them; they cannot deny and cannot mutate.

func (*Chain) TargetID 

func (c *Chain) TargetID() string

TargetID returns the key used to find this chain.

type Decision 

type Decision int

Decision captures the outcome of a middleware invocation as observed by the dispatcher. Response-phase middlewares always return DecisionPassthrough; the dispatcher clamps any other value. 

const (
	// DecisionAllow lets the request proceed.
	DecisionAllow Decision = 0
	// DecisionDeny stops the chain and returns a rendered deny
	// response. Only honoured in SlotOnRequest.
	DecisionDeny Decision = 1
	// DecisionPassthrough is the response-phase neutral outcome.
	DecisionPassthrough Decision = 2
)

type DenyReason 

type DenyReason struct {
	Code    string
	Message string
	Details map[string]string
}

DenyReason is the structured payload a middleware returns alongside a DecisionDeny. The proxy renders it through a fixed JSON template so middlewares cannot emit arbitrary bytes to the wire.

type Dispatcher 

type Dispatcher struct {
	// contains filtered or unexported fields
}

Dispatcher drives a single middleware invocation with panic recovery, deadline, and output filtering. Safe for concurrent use.

func NewDispatcher 

func NewDispatcher(metrics *Metrics, logger *log.Logger) *Dispatcher

NewDispatcher returns a dispatcher that emits on the provided metrics bundle and logger. A nil metrics bundle falls back to a noop instrument set; a nil logger falls back to the standard logger.

func (*Dispatcher) Invoke 

func (d *Dispatcher) Invoke(ctx context.Context, spec Spec, mw Middleware, in *Input) (*Output, error)

Invoke runs a single middleware under the reliability wrappers: deadline, panic recovery (type + truncated stack only), fail-mode, metric emission, and output filtering. The returned output is always safe to apply.

type Factory 

type Factory interface {
	ID() string
	New(rawConfig []byte) (Middleware, error)
}

Factory builds a configured Middleware instance from raw config bytes shipped on the wire. Each registered middleware ID has a single factory in the registry. Factory.New returns an error when the config is malformed or violates a per-middleware invariant; the chain build path logs the error, increments the resolve_error metric, and skips the middleware.

type FailMode 

type FailMode int

FailMode controls how the dispatcher reacts when a middleware returns an error, times out, or panics. Observer middlewares default to FailOpen; policy middlewares should default to FailClosed. 

const (
	// FailOpen allows the request to proceed when a middleware fails.
	FailOpen FailMode = 0
	// FailClosed denies the request when a middleware fails. Only
	// meaningful for SlotOnRequest middlewares.
	FailClosed FailMode = 1
)

type Input 

type Input struct {
	Slot             Slot
	RequestID        string
	TargetID         string
	Method           string
	URL              string
	Headers          []KV
	Body             []byte
	BodyTruncated    bool
	OriginalBodySize int64

	Status            int
	RespHeaders       []KV
	RespBody          []byte
	RespBodyTruncated bool
	OriginalRespSize  int64

	ServiceID string
	AccountID string
	UserID    string
	// UserEmail is the calling user's email address when the auth path
	// resolves a user record. Empty for non-OIDC schemes (PIN/Password/
	// Header) and for legacy session JWTs minted before the email claim
	// was introduced. Identity-stamping middlewares (e.g.
	// llm_identity_inject) prefer this over UserID for upstream gateways
	// that key budgets / attribution on a human-readable identifier.
	UserEmail  string
	AuthMethod string
	SourceIP   string
	// UserGroups captures the calling peer's group memberships at
	// request time, surfaced from the proxy's auth flow so policy-aware
	// middlewares can authorise without an extra management round-trip.
	UserGroups []string
	// UserGroupNames carries the human-readable display names paired
	// positionally with UserGroups (UserGroupNames[i] is the name of
	// UserGroups[i]). Identity-stamping middlewares prefer names for
	// upstream tags so attribution dashboards stay readable. Slice may
	// be shorter than UserGroups for tokens minted before names were
	// resolvable; consumers should fall back to ids for missing
	// positions.
	UserGroupNames []string
	Metadata       []KV

	// AgentNetwork is true when the target is a synthesised
	// agent-network service. Carried on the input so the access-log
	// terminal middleware can stamp the proto field without re-deriving
	// from the service ID.
	AgentNetwork bool
}

Input is the immutable envelope handed to each middleware. The dispatcher deep-copies Headers, Body, Metadata, RespHeaders, and RespBody before each invocation so middlewares cannot mutate the shared in-flight copies; mutations must flow through Output.Mutations.

type KV 

type KV struct {
	Key   string
	Value string
}

KV is the canonical header/metadata representation used across the middleware boundary. We use a slice of KV instead of http.Header because it preserves key order, is cheap to deep-copy per invocation, and is directly representable in a future protobuf envelope.

func FilterHeaderMutations 

func FilterHeaderMutations(m *Mutations) (filteredAdd []KV, filteredRemove []string, blocked []string)

FilterHeaderMutations returns the subsets of HeadersAdd and HeadersRemove that are safe to apply, plus the list of blocked header names so the dispatcher can increment the blocked-header metric.

type LiveServiceCheck 

type LiveServiceCheck func(serviceID string) bool

LiveServiceCheck reports whether the given service ID is still present in the proxy's live mapping cache. The Manager calls it during InvalidateMiddleware so a chain whose service has been removed since the last Rebuild is not resurrected from the binding cache, closing the auth-revocation race.

type Manager 

type Manager struct {
	// contains filtered or unexported fields
}

Manager owns the per-target middleware chains, the global capture budget, and the shared dispatcher. Readers (ChainFor) are lock-free; writers (Rebuild, Invalidate*) serialise on writeMu so two concurrent mapping updates do not lose writes.

func NewManager 

func NewManager(budgetBytes int64, metrics *Metrics, logger *log.Logger) *Manager

NewManager constructs a Manager with the given capture budget size. A zero or negative budget falls back to bodytap.DefaultCaptureBudgetBytes.

func (*Manager) Budget 

func (m *Manager) Budget() bodytap.Budget

Budget returns the shared capture budget.

func (*Manager) ChainFor 

func (m *Manager) ChainFor(serviceID, pathID string) *Chain

ChainFor returns the chain for serviceID/pathID or nil if none is registered. Lock-free.

func (*Manager) Dispatcher 

func (m *Manager) Dispatcher() *Dispatcher

Dispatcher returns the shared dispatcher (primarily for testing).

func (*Manager) Invalidate 

func (m *Manager) Invalidate(serviceID string)

Invalidate drops every chain for the given service ID.

func (*Manager) InvalidateAll 

func (m *Manager) InvalidateAll()

InvalidateAll drops every chain.

func (*Manager) InvalidateMiddleware 

func (m *Manager) InvalidateMiddleware(id string)

InvalidateMiddleware rebuilds only the chains that reference id.

func (*Manager) Metrics 

func (m *Manager) Metrics() *Metrics

Metrics returns the shared metrics bundle.

func (*Manager) Rebuild 

func (m *Manager) Rebuild(serviceID string, bindings []PathTargetBinding) error

Rebuild replaces every chain keyed by serviceID with the provided bindings. Entries for other services are preserved. Replaced chains are closed asynchronously after the atomic swap so in-flight requests against the previous chain finish before middleware resources are released.

func (*Manager) SetLiveServiceCheck 

func (m *Manager) SetLiveServiceCheck(fn LiveServiceCheck)

SetLiveServiceCheck installs a callback the Manager uses to confirm a service ID still maps to a live mapping before resurrecting its chain from the binding cache during InvalidateMiddleware. A nil fn disables the check.

func (*Manager) SetResolver 

func (m *Manager) SetResolver(r *Resolver)

SetResolver installs the resolver used by Rebuild. Safe to call once at boot before any Rebuild; not safe to swap concurrently.

type MetadataRejection 

type MetadataRejection struct {
	Key    string
	Reason string
}

MetadataRejection describes a single rejected key/value so the dispatcher can emit per-reason counter increments.

type Metrics 

type Metrics struct {
	// contains filtered or unexported fields
}

Metrics is the bundle of OTel instruments emitted by the middleware dispatcher. The constructor falls back to a noop meter when given nil so tests can skip metrics wiring entirely.

func NewMetrics 

func NewMetrics(meter metric.Meter) (*Metrics, error)

NewMetrics registers the proxy.middleware.* instruments on the given meter. A nil meter is treated as the global no-op provider.

func (*Metrics) IncCaptureBypass 

func (m *Metrics) IncCaptureBypass(ctx context.Context, targetID, reason string)

IncCaptureBypass increments the capture-bypass counter for a reason.

func (*Metrics) IncError 

func (m *Metrics) IncError(ctx context.Context, middlewareID, kind string)

IncError increments the error counter with the given failure kind label.

func (*Metrics) IncHeaderMutationBlocked 

func (m *Metrics) IncHeaderMutationBlocked(ctx context.Context, middlewareID, header string)

IncHeaderMutationBlocked increments the blocked-header counter.

func (*Metrics) IncInvocation 

func (m *Metrics) IncInvocation(ctx context.Context, middlewareID string)

IncInvocation increments the heartbeat counter regardless of outcome.

func (*Metrics) IncMetadataRejected 

func (m *Metrics) IncMetadataRejected(ctx context.Context, middlewareID, reason string)

IncMetadataRejected increments the rejected-metadata counter for a reason.

func (*Metrics) IncRequest 

func (m *Metrics) IncRequest(ctx context.Context, middlewareID, targetID, outcome string)

IncRequest increments proxy.middleware.requests_total with the middleware, target, and outcome labels.

func (*Metrics) ObserveDuration 

func (m *Metrics) ObserveDuration(ctx context.Context, middlewareID string, ms int64)

ObserveDuration records the middleware Invoke latency in milliseconds.

type Middleware 

type Middleware interface {
	ID() string
	Version() string
	Slot() Slot

	// AcceptedContentTypes lists the request/response content types
	// the middleware needs the body for. Empty slice means the
	// middleware does not inspect the body.
	AcceptedContentTypes() []string

	// MetadataKeys is the closed set of metadata keys this middleware
	// may emit. The accumulator drops anything outside this allowlist.
	MetadataKeys() []string

	// MutationsSupported reports whether the middleware may emit
	// header / body mutations. A spec with CanMutate=true is honoured
	// only when the implementation also supports mutations.
	MutationsSupported() bool

	Invoke(ctx context.Context, in *Input) (*Output, error)

	Close() error
}

Middleware is the surface exposed by each concrete implementation. The Manager invokes it through the Dispatcher, passing a cloned Input. Each middleware lives in exactly one Slot.

Close releases any resources owned by the middleware instance (background goroutines, file handles). It is invoked when the chain holding the middleware is replaced or torn down. Implementations must be idempotent and safe to call after construction even when Invoke was never called.

type Mutations 

type Mutations struct {
	HeadersAdd      []KV
	HeadersRemove   []string
	BodyReplace     []byte
	RewriteUpstream *UpstreamRewrite
}

Mutations describes the deltas a middleware wants applied to the in-flight request. The dispatcher filters HeadersAdd/HeadersRemove through the compiled-in denylist and runs BodyReplace through the body policy before anything is applied. RewriteUpstream redirects the outbound target (scheme + host) for the request; the chain returns the latest non-nil rewrite to the reverse proxy.

type Output 

type Output struct {
	Decision   Decision
	DenyStatus int
	DenyReason *DenyReason
	Metadata   []KV
	Mutations  *Mutations
}

Output is the value each middleware returns to the dispatcher. The dispatcher applies the output filter (clamp, mutations gate) before any side effect reaches the shared request.

type PathTargetBinding 

type PathTargetBinding struct {
	ServiceID string
	PathID    string
	Specs     []Spec
}

PathTargetBinding is the minimal per-path binding the server passes to Rebuild. It carries the stable keys Manager uses for snapshot lookups plus the validated middleware spec list for that path.

type Registry 

type Registry struct {
	// contains filtered or unexported fields
}

Registry maps middleware IDs to their factories. The proxy installs a single Registry at boot; concrete middlewares register themselves from init() functions inside their own packages so the boot wiring only needs an anonymous import.

Registry is safe for concurrent reads after boot. Register / Unregister take the write lock; Get and IDs take the read lock.

func NewRegistry 

func NewRegistry() *Registry

NewRegistry returns an empty registry.

func (*Registry) Get 

func (r *Registry) Get(id string) Factory

Get returns the factory for id, or nil when no factory is registered.

func (*Registry) IDs 

func (r *Registry) IDs() []string

IDs returns the registered IDs in unspecified order. Used by the management translator to reject specs that reference unknown IDs at apply time.

func (*Registry) IsKnown 

func (r *Registry) IsKnown(id string) bool

IsKnown reports whether id has a registered factory.

func (*Registry) MustRegister 

func (r *Registry) MustRegister(f Factory)

MustRegister panics on error. Intended for init() registration so duplicate IDs surface at startup.

func (*Registry) Register 

func (r *Registry) Register(f Factory) error

Register installs the factory under its ID. Returns an error when an ID is already registered — collisions are programmer errors and must be visible at boot rather than silently last-write-wins.

type Resolver 

type Resolver struct {
	// contains filtered or unexported fields
}

Resolver wraps a Registry and produces a configured Middleware instance from a Spec. The Manager uses this during chain build.

func NewResolver 

func NewResolver(registry *Registry) *Resolver

NewResolver returns a resolver backed by the registry.

func (*Resolver) Resolve 

func (r *Resolver) Resolve(spec Spec) (Middleware, Spec, error)

Resolve builds a Middleware instance and merges runtime-only fields (version, accepted content types, metadata key allowlist, mutation support) onto the spec.

Return semantics:

  • (mw, mergedSpec, nil): instance built, include in chain.
  • (nil, spec, nil): id not registered; silently skip.
  • (nil, spec, err): factory rejected the config (logged + counted by Manager, other middlewares still bind).

type Slot 

type Slot int

Slot identifies where in the request lifecycle a middleware runs. A middleware declares a single slot. Splitting per-purpose work (request parsing vs response parsing vs cost metering) into separate slot-keyed middlewares is the explicit architectural choice for the agent-network use case; no middleware participates in more than one slot. 

const (
	// SlotOnRequest runs before the upstream call. Middlewares in this
	// slot may DENY the request, mutate headers/body (when permitted),
	// and emit metadata derived from the request envelope.
	SlotOnRequest Slot = 1
	// SlotOnResponse runs after the upstream returns. Middlewares in
	// this slot observe the response, emit metadata, and may mutate
	// response headers when permitted. They cannot DENY.
	SlotOnResponse Slot = 2
	// SlotTerminal runs after every SlotOnResponse middleware has
	// emitted. Terminal middlewares observe the full metadata bag and
	// ship it to external sinks (access log, metrics export). They
	// cannot DENY and cannot mutate the response.
	SlotTerminal Slot = 3
)

type Spec 

type Spec struct {
	ID        string
	Slot      Slot
	Version   string
	Enabled   bool
	FailMode  FailMode
	Timeout   time.Duration
	RawConfig []byte
	CanMutate bool

	// Runtime-only fields populated from the registered middleware at
	// chain build time; not sourced from proto.
	MetadataKeys         []string
	AcceptedContentTypes []string
	MutationsSupported   bool
}

Spec is the apply-time, validated representation of a per-target middleware configuration merged with the runtime-only fields compiled into the middleware implementation.

The wire shape is RawConfig (JSON bytes) instead of the older params map[string]string. Each middleware unmarshals RawConfig into its own typed config struct, surfacing structural validation errors at construction rather than per-invocation lookups.

func (Spec) Clone 

func (s Spec) Clone() Spec

Clone returns a deep copy of the spec safe to cache across mapping updates.

type UpstreamRewrite 

type UpstreamRewrite struct {
	Scheme string
	Host   string
	// Path, when non-empty, replaces the path component of the
	// proxy's effective upstream URL. The rewrite path is then joined
	// with the agent's request path by httputil.ProxyRequest.SetURL —
	// e.g. rewrite Path="/v1/{account}/{gateway}/compat" + agent
	// request "/chat/completions" → outbound
	// "/v1/{account}/{gateway}/compat/chat/completions". Used by
	// llm_router to honor the operator-configured upstream path on
	// gateways like Cloudflare AI Gateway whose URL contains
	// account / gateway segments that the agent's app doesn't know
	// about. Empty Path leaves the original target's path
	// untouched (the historical behavior).
	Path string
	// StripPathPrefix, when non-empty, is removed from the front of the agent's
	// request path before it is joined onto the upstream URL. Used for
	// gateway-namespace prefixes (e.g. a client addressing Bedrock as
	// "/bedrock/model/{id}/invoke") that must not reach the real upstream, whose
	// native path is "/model/{id}/invoke". Empty leaves the request path intact.
	StripPathPrefix string
	AuthHeader      *AuthHeader
	StripHeaders    []string
	// SkipTLSVerify, when true, makes the proxy dial the rewritten upstream
	// without verifying its TLS certificate. Set by llm_router from the
	// provider's skip_tls_verification for self-hosted / internal gateways.
	SkipTLSVerify bool
}

UpstreamRewrite redirects the request's outbound target. Only scheme+host are honoured; path, query, and body are untouched. The reverse proxy reads the rewrite (when non-nil) instead of the PathTarget URL configured by the synth, so a single shared synth service can fan out to many upstreams selected per request.

AuthHeader and StripHeaders carry the upstream auth substitution the router needs. They bypass the framework's HeadersAdd / HeadersRemove denylist (which blocks Authorization, Cookie, etc. from middleware mutation) on the grounds that the proxy itself is the entity rewriting auth here, not an arbitrary middleware. The reverse proxy applies them directly to the upstream request after the chain's regular mutation phase, so a malicious or misconfigured middleware can still emit RewriteUpstream but only the proxy's trusted upstream-build path actually unpacks AuthHeader.

Source Files

View all Source files

Directories

Path Synopsis
Package bodytap owns the framework-side body capture used by the middleware chain.
 Package bodytap owns the framework-side body capture used by the middleware chain.
Package builtin holds the package-level middleware registry that concrete middleware packages register themselves into via init().
 Package builtin holds the package-level middleware registry that concrete middleware packages register themselves into via init().
cost_meter
Package cost_meter implements the SlotOnResponse middleware that converts token-usage metadata emitted by llm_response_parser into a per-request USD cost estimate.
 Package cost_meter implements the SlotOnResponse middleware that converts token-usage metadata emitted by llm_response_parser into a per-request USD cost estimate.
llm_guardrail
Package llm_guardrail implements the SlotOnRequest middleware that enforces the per-target LLM guardrail policy: a model allowlist check and an opt-in prompt-capture step that may run a PII redactor before emitting the prompt into the metadata bag.
 Package llm_guardrail implements the SlotOnRequest middleware that enforces the per-target LLM guardrail policy: a model allowlist check and an opt-in prompt-capture step that may run a PII redactor before emitting the prompt into the metadata bag.
llm_identity_inject
Package llm_identity_inject implements the SlotOnRequest middleware that stamps the caller's NetBird identity onto upstream LLM-gateway requests.
 Package llm_identity_inject implements the SlotOnRequest middleware that stamps the caller's NetBird identity onto upstream LLM-gateway requests.
llm_limit_check
Package llm_limit_check is the SlotOnRequest middleware that asks management which agent-network policy "pays" for the current LLM request.
 Package llm_limit_check is the SlotOnRequest middleware that asks management which agent-network policy "pays" for the current LLM request.
llm_limit_record
Package llm_limit_record is the SlotOnResponse middleware that posts the served request's token + cost deltas back to management so the per-(user, group, window) consumption counters tick.
 Package llm_limit_record is the SlotOnResponse middleware that posts the served request's token + cost deltas back to management so the per-(user, group, window) consumption counters tick.
llm_request_parser
Package llm_request_parser implements the SlotOnRequest middleware that detects the LLM provider from the request URL, parses the JSON request body for model and streaming flags, and extracts the user prompt text.
 Package llm_request_parser implements the SlotOnRequest middleware that detects the LLM provider from the request URL, parses the JSON request body for model and streaming flags, and extracts the user prompt text.
llm_response_parser
Package llm_response_parser implements the SlotOnResponse middleware that decodes OpenAI- and Anthropic-shaped LLM responses (buffered or streaming) and emits token usage and completion metadata.
 Package llm_response_parser implements the SlotOnResponse middleware that decodes OpenAI- and Anthropic-shaped LLM responses (buffered or streaming) and emits token usage and completion metadata.
llm_router
Package llm_router implements the SlotOnRequest middleware that routes a request to an upstream LLM provider based on the model name emitted upstream by llm_request_parser.
 Package llm_router implements the SlotOnRequest middleware that routes a request to an upstream LLM provider based on the model name emitted upstream by llm_request_parser.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.