filterapi

package
v0.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 23, 2026 License: Apache-2.0 Imports: 11 Imported by: 0

Documentation

Overview

Package filterapi provides the configuration for the AI Gateway-implemented filter which is currently an external processor (See https://github.com/envoyproxy/ai-gateway/issues/90).

This configuration must be decoupled from the Envoy Gateway types as well as its implementation details. Also, the configuration must not be tied with k8s so it can be tested and iterated without the need for the k8s cluster.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func StartConfigWatcher added in v0.5.0

func StartConfigWatcher(ctx context.Context, path string, rcv ConfigReceiver, l *slog.Logger, tick time.Duration) error

StartConfigWatcher starts a watcher for the given path and Receiver. Periodically checks the file for changes and calls the Receiver's UpdateConfig method.

Types

type APIKeyAuth

type APIKeyAuth struct {
	// Key is the API key as a literal string.
	Key string `json:"key"`
}

APIKeyAuth defines the file that will be mounted to the external proc.

type APISchemaName

type APISchemaName string

APISchemaName corresponds to APISchemaName in api/v1alpha1/api.go.

const (
	// APISchemaOpenAI represents the standard OpenAI API schema.
	APISchemaOpenAI APISchemaName = "OpenAI"
	// APISchemaCohere represents the Cohere API schema.
	APISchemaCohere APISchemaName = "Cohere"
	// APISchemaAWSBedrock represents the AWS Bedrock API schema.
	APISchemaAWSBedrock APISchemaName = "AWSBedrock"
	// APISchemaAzureOpenAI represents the Azure OpenAI API schema.
	APISchemaAzureOpenAI APISchemaName = "AzureOpenAI"
	// APISchemaGCPVertexAI represents the Google Cloud Gemini API schema.
	// Used for Gemini models hosted on Google Cloud Vertex AI.
	APISchemaGCPVertexAI APISchemaName = "GCPVertexAI"
	// APISchemaGCPAnthropic represents the Google Cloud Anthropic API schema.
	// Used for Claude models hosted on Google Cloud Vertex AI.
	APISchemaGCPAnthropic APISchemaName = "GCPAnthropic"
	// APISchemaAnthropic represents the standard Anthropic API schema.
	APISchemaAnthropic APISchemaName = "Anthropic"
	// APISchemaAWSAnthropic represents the AWS Bedrock Anthropic API schema.
	// Used for Claude models hosted on AWS Bedrock using the native Anthropic Messages API.
	APISchemaAWSAnthropic APISchemaName = "AWSAnthropic"
)

type AWSAuth

type AWSAuth struct {
	// CredentialFileLiteral is the literal string of the AWS credential file. E.g.
	// [default]\naws_access_key_id = <access-key-id>\naws_secret_access_key = <secret-access-key>\naws_session_token = <session-token>.
	CredentialFileLiteral string `json:"credentialFileLiteral,omitempty"`
	Region                string `json:"region"`
}

AWSAuth defines the credentials needed to access AWS.

type AnthropicAPIKeyAuth

type AnthropicAPIKeyAuth struct {
	// Key is the Anthropic API key as a literal string.
	Key string `json:"key"`
}

AnthropicAPIKeyAuth defines the Anthropic API key.

type AuthorizationAction added in v0.5.0

type AuthorizationAction string
const (
	// AuthorizationActionAllow is the action to allow the request.
	AuthorizationActionAllow AuthorizationAction = "Allow"
	// AuthorizationActionDeny is the action to deny the request.
	AuthorizationActionDeny AuthorizationAction = "Deny"
)

type AzureAPIKeyAuth

type AzureAPIKeyAuth struct {
	// Key is the Azure API key as a literal string.
	Key string `json:"key"`
}

AzureAPIKeyAuth defines the Azure OpenAI API key.

type AzureAuth

type AzureAuth struct {
	// AccessToken is the access token as a literal string.
	AccessToken string `json:"accessToken"`
}

AzureAuth defines the file containing azure access token that will be mounted to the external proc.

type Backend

type Backend struct {
	// Name of the backend including the route name as well as the route rule index.
	Name              string                        `json:"name"`
	ModelNameOverride internalapi.ModelNameOverride `json:"modelNameOverride"`
	// Schema specifies the API schema of the output format of requests from.
	Schema VersionedAPISchema `json:"schema"`
	// Auth is the authn/z configuration for the backend. Optional.
	Auth *BackendAuth `json:"auth,omitempty"`
	// Sensitive Headers to be removed from the request before sending to the backend. Optional.
	HeaderMutation *HTTPHeaderMutation `json:"httpHeaderMutation,omitempty"`
	// Body mutations to be applied to the request before sending to the backend. Optional.
	BodyMutation *HTTPBodyMutation `json:"httpBodyMutation,omitempty"`
}

Backend corresponds to AIGatewayRouteRuleBackendRef in api/v1alpha1/api.go besides that this abstracts the concept of a backend at Envoy Gateway level to a simple name.

type BackendAuth

type BackendAuth struct {
	// APIKey is a location of the api key secret file.
	APIKey *APIKeyAuth `json:"apiKey,omitempty"`
	// AWSAuth specifies the location of the AWS credential file and region.
	AWSAuth *AWSAuth `json:"aws,omitempty"`
	// AzureAPIKey is the Azure OpenAI API key.
	AzureAPIKey *AzureAPIKeyAuth `json:"azureAPIKey,omitempty"`
	// AnthropicAPIKey is the Anthropic API key.
	AnthropicAPIKey *AnthropicAPIKeyAuth `json:"anthropicAPIKey,omitempty"`
	// AzureAuth specifies the location of Azure access token file.
	AzureAuth *AzureAuth `json:"azure,omitempty"`
	// GCPAuth specifies the location of GCP credential file.
	GCPAuth *GCPAuth `json:"gcp,omitempty"`
}

BackendAuth corresponds partially to BackendSecurityPolicy in api/v1alpha1/api.go.

type BackendAuthHandler added in v0.5.0

type BackendAuthHandler interface {
	// Do performs the backend auth, and make changes to the request headers passed in as `requestHeaders`.
	// It also returns a list of headers that were added or modified as a slice of key-value pairs.
	Do(ctx context.Context, requestHeaders map[string]string, mutatedBody []byte) ([]internalapi.Header, error)
}

BackendAuthHandler is the interface that deals with the backend auth for a specific backend.

type Config

type Config struct {
	// Version is the version of the AI Gateway, e.g., "v0.4.0" derived from internal/version package.
	// This is to ensure compatibility between the filter and the AI Gateway management plane.
	//
	// When there's discrepancy between the version set here (by the controller) and the version of the extproc
	// filter binary, which can happen during rolling upgrade, the filter will not load the configuration,
	// and keep working with the previous configuration.
	Version string `json:"version,omitempty"`
	// UUID is the unique identifier of the filter configuration assigned by the AI Gateway when the configuration is updated.
	UUID string `json:"uuid,omitempty"`
	// LLMRequestCost configures the cost of each LLM-related request. Optional. If this is provided, the filter will populate
	// the "calculated" cost in the filter metadata at the end of the response body processing.
	LLMRequestCosts []LLMRequestCost `json:"llmRequestCosts,omitempty"`
	// Backends is the list of backends that this listener can route to.
	Backends []Backend `json:"backends,omitempty"`
	// Models is the list of models that this route is aware of. Used to populate the "/models" endpoint in OpenAI-compatible APIs.
	Models []Model `json:"models,omitempty"`
	// MCPConfig is the configuration for the MCPRoute implementations.
	MCPConfig *MCPConfig `json:"mcpConfig,omitempty"`
}

Config is the configuration for the Envoy AI Gateway filter.

func UnmarshalConfigYaml

func UnmarshalConfigYaml(path string) (*Config, error)

UnmarshalConfigYaml reads the file at the given path and unmarshals it into a Config struct.

type ConfigReceiver added in v0.5.0

type ConfigReceiver interface {
	// LoadConfig updates the configuration.
	LoadConfig(ctx context.Context, config *Config) error
}

ConfigReceiver is an interface that can receive *Config updates. This is mostly for decoupling and testing purposes.

type GCPAuth

type GCPAuth struct {
	// AccessToken is the access token as a literal string.
	// This token is obtained through GCP Workload Identity Federation and service account impersonation.
	// The token is automatically rotated by the BackendSecurityPolicy controller before expiration.
	AccessToken string `json:"accessToken"`
	// Region is the GCP region to use for the request.
	// This is used in URL path templates when making requests to GCP Vertex AI endpoints.
	// Examples: "us-central1", "europe-west4".
	Region string `json:"region"`
	// ProjectName is the GCP project name to use for the request.
	// This is used in URL path templates when making requests to GCP Vertex AI endpoints.
	// This should be the project where Vertex AI APIs are enabled.
	ProjectName string `json:"projectName"`
}

GCPAuth defines the GCP authentication configuration used to access Google Cloud AI services.

type HTTPBodyField added in v0.5.0

type HTTPBodyField struct {
	// Path is the top-level field name to set in the request body.
	// Examples: "service_tier", "max_tokens", "temperature"
	Path string `json:"path"`
	// Value is the JSON value to set at the specified field. This can be any valid JSON value:
	// string, number, boolean, object, array, or null.
	// The value will be parsed as JSON and inserted at the specified field.
	Value string `json:"value"`
}

HTTPBodyField represents a JSON field name and value for body mutation

type HTTPBodyMutation added in v0.5.0

type HTTPBodyMutation struct {
	// Set overwrites/adds the request body with the given JSON field (name, value)
	// before sending to the backend. Only top-level fields are currently supported.
	Set []HTTPBodyField `json:"set,omitempty"`
	// Remove the given JSON field(s) from the HTTP request body before sending to the backend.
	// The value of Remove is a list of top-level field names to remove.
	Remove []string `json:"remove,omitempty"`
}

HTTPBodyMutation defines the mutation of HTTP request body JSON fields that will be applied to the request

type HTTPHeader

type HTTPHeader struct {
	// Name is the name of the HTTP Header to be matched.
	// This is always ensured to be lower-cased like Envoy does internally.
	Name string `json:"name"`
	// Value is the value of HTTP Header to be matched.
	Value string `json:"value"`
}

HTTPHeader represents an HTTP Header name and value as defined by RFC 7230.

type HTTPHeaderMutation

type HTTPHeaderMutation struct {
	// Set overwrites the request with the given header (name, value)
	// before the action.
	Set []HTTPHeader `json:"set,omitempty"`
	// Remove the given header(s) from the HTTP request before the action. The
	// value of Remove is a list of HTTP header names.
	// This is always ensured to be lower-cased like Envoy does internally.
	Remove []string `json:"remove,omitempty"`
}

HTTPHeaderMutation defines the mutation of HTTP headers that will be applied to the request

type JWTClaim added in v0.5.0

type JWTClaim struct {
	// Name is the name of the claim, supports dotted paths for nested claims.
	Name string `json:"name"`

	// ValueType indicates whether the claim value is expected to be a string or string array.
	ValueType JWTClaimValueType `json:"valueType"`

	// Values are the values that the claim must match.
	// If the claim is a string type, the specified value must match exactly.
	// If the claim is a string array type, the specified value must match one of the values in the array.
	// If multiple values are specified, one of the values must match for the rule to match.
	Values []string `json:"values"`
}

type JWTClaimValueType added in v0.5.0

type JWTClaimValueType string
const (
	JWTClaimValueTypeString      JWTClaimValueType = "String"
	JWTClaimValueTypeStringArray JWTClaimValueType = "StringArray"
)

type JWTSource added in v0.5.0

type JWTSource struct {
	// Scopes defines the list of JWT scopes required for the rule.
	// If multiple scopes are specified, all scopes must be present in the JWT for the rule to match.
	Scopes []string `json:"scopes,omitempty"`

	// Claims defines the list of JWT claims required for the rule. Each claim must exist on the token
	// and have at least one of the expected values.
	Claims []JWTClaim `json:"claims,omitempty"`
}

type LLMRequestCost

type LLMRequestCost struct {
	// MetadataKey is the key of the metadata storing the request cost.
	MetadataKey string `json:"metadataKey"`
	// Type is the kind of the request cost calculation.
	Type LLMRequestCostType `json:"type"`
	// CEL is the CEL expression to calculate the cost of the request.
	// This is not empty when the Type is LLMRequestCostTypeCEL.
	CEL string `json:"cel,omitempty"`
}

LLMRequestCost specifies "where" the request cost is stored in the filter metadata as well as "how" the cost is calculated. By default, the cost is retrieved from "output token" in the response body.

This can be used to subtract the usage token from the usage quota in the rate limit filter when the request completes combined with `apply_on_stream_done` and `hits_addend` fields of the rate limit configuration https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/route/v3/route_components.proto#config-route-v3-ratelimit introduced in Envoy 1.33.

type LLMRequestCostType

type LLMRequestCostType string

LLMRequestCostType specifies the kind of the request cost calculation.

const (
	// LLMRequestCostTypeOutputToken specifies that the request cost is calculated from the output token.
	LLMRequestCostTypeOutputToken LLMRequestCostType = "OutputToken"
	// LLMRequestCostTypeInputToken specifies that the request cost is calculated from the input token.
	LLMRequestCostTypeInputToken LLMRequestCostType = "InputToken"
	// LLMRequestCostTypeCachedInputToken specifies that the request cost is calculated from the cached read input token.
	LLMRequestCostTypeCachedInputToken LLMRequestCostType = "CachedInputToken"
	// LLMRequestCostTypeCacheCreationInputToken specifies that the request cost is calculated from the cache creation input token.
	LLMRequestCostTypeCacheCreationInputToken LLMRequestCostType = "CacheCreationInputToken"
	// LLMRequestCostTypeTotalToken specifies that the request cost is calculated from the total token.
	LLMRequestCostTypeTotalToken LLMRequestCostType = "TotalToken"
	// LLMRequestCostTypeCEL specifies that the request cost is calculated from the CEL expression.
	LLMRequestCostTypeCEL LLMRequestCostType = "CEL"
)

type MCPAuthorizationSource added in v0.5.0

type MCPAuthorizationSource struct {
	// JWT defines the JWT scopes required for this rule to match.
	JWT JWTSource `json:"jwt"`
}

type MCPAuthorizationTarget added in v0.5.0

type MCPAuthorizationTarget struct {
	// Tools defines the list of tools this rule applies to.
	Tools []ToolCall `json:"tools"`
}

type MCPBackend

type MCPBackend struct {
	// Name is the fully qualified identifier of a MCP backend.
	// This name is set in [internalapi.MCPBackendHeader] header to route the request to the specific backend.
	Name MCPBackendName `json:"name"`

	// Path is the HTTP endpoint path of the backend MCP server.
	Path string `json:"path"`

	// ToolSelector filters the tools exposed by this backend. If not set, all tools are exposed.
	ToolSelector *MCPToolSelector `json:"toolSelector,omitempty"`
}

MCPBackend is the MCP backend configuration.

type MCPBackendName

type MCPBackendName = string

MCPBackendName is the name of the MCP backend.

type MCPConfig

type MCPConfig struct {
	// BackendListenerAddr is the address that speaks plain HTTP and can be used to
	// route to each backend directly without interruption.
	//
	// The listener should only listen on the local interface, and equipped with
	// the HCM filter with the plain header-based routing for each backend based
	// on the [internalapi.MCPBackendHeader] header.
	BackendListenerAddr string `json:"backendListenerAddr"`

	// Routes is the list of routes that this listener can route to.
	Routes []MCPRoute `json:"routes,omitempty"`
}

MCPConfig is the configuration for the MCP listener and routing.

type MCPRoute

type MCPRoute struct {
	// Name is the fully qualified identifier of a MCPRoute.
	// This name is set in [internalapi.MCPRouteHeader] header to identify the route.
	Name MCPRouteName `json:"name"`

	// Backends is the list of backends that this route can route to.
	Backends []MCPBackend `json:"backends"`

	// Authorization is the authorization configuration for this route.
	Authorization *MCPRouteAuthorization `json:"authorization,omitempty"`
}

MCPRoute is the route configuration for routing to each MCP backend based on the tool name.

type MCPRouteAuthorization added in v0.5.0

type MCPRouteAuthorization struct {
	// DefaultAction is the action to take when no rules match.
	DefaultAction AuthorizationAction `json:"defaultAction"`

	// Rules defines a list of authorization rules.
	// Requests that match any rule and satisfy the rule's conditions will be allowed.
	// Requests that do not match any rule or fail to satisfy the matched rule's conditions will be denied.
	// If no rules are defined, all requests will be denied.
	Rules []MCPRouteAuthorizationRule `json:"rules,omitempty"`

	// ResourceMetadataURL is the URI of the OAuth Protected Resource Metadata document for this route.
	// This is used to populate the WWW-Authenticate header when scope-based authorization fails.
	ResourceMetadataURL string `json:"resourceMetadataURL,omitempty"`
}

MCPRouteAuthorization defines the authorization configuration for a MCPRoute.

type MCPRouteAuthorizationRule added in v0.5.0

type MCPRouteAuthorizationRule struct {
	// Action is the action to take when the rule matches.
	Action AuthorizationAction `json:"action"`

	// Source defines the authorization source for this rule.
	// If not specified, the rule will match all sources.
	Source *MCPAuthorizationSource `json:"source,omitempty"`

	// Target defines the authorization target for this rule.
	// If not specified, the rule will match all targets.
	Target *MCPAuthorizationTarget `json:"target,omitempty"`

	// CEL specifies a Common Expression Language (CEL) expression for this rule.
	// The expression must evaluate to true for the rule to match.
	// If empty, no CEL condition is applied.
	CEL *string `json:"cel,omitempty"`
}

MCPRouteAuthorizationRule defines an authorization rule for MCPRoute based on the MCP authorization spec. Reference: https://modelcontextprotocol.io/specification/draft/basic/authorization#scope-challenge-handling

type MCPRouteName

type MCPRouteName = string

MCPRouteName is the name of the MCP route.

type MCPToolSelector

type MCPToolSelector struct {
	// Include is a list of tool names to include. Only the specified tools will be available.
	Include []string `json:"include,omitempty"`

	// IncludeRegex is a list of RE2-compatible regular expressions that, when matched, include the tool.
	// Only tools matching these patterns will be available.
	// TODO: regex is almost completely absent in the MCP ecosystem, consider removing this for simplicity.
	IncludeRegex []string `json:"includeRegex,omitempty"`
}

MCPToolSelector filters tools using include patterns with exact matches or regular expressions.

type Model

type Model struct {
	// Name will be exported as the field of "ID" in OpenAI-compatible APIs.
	Name string
	// ownedBy will be exported as the field of "OwnedBy" in OpenAI-compatible API "/models".
	OwnedBy string
	// createdAt will be exported as the field of "Created" in OpenAI-compatible API "/models".
	CreatedAt time.Time
}

Model corresponds to the OpenAI model object in the OpenAI-compatible APIs and is used to populate the "/models" endpoint in OpenAI-compatible APIs.

type NewBackendAuthHandlerFunc added in v0.5.0

type NewBackendAuthHandlerFunc func(ctx context.Context, auth *BackendAuth) (BackendAuthHandler, error)

NewBackendAuthHandlerFunc is a function type that creates a new BackendAuthHandler for a given BackendAuth configuration.

type RouteRuleName

type RouteRuleName string

RouteRuleName is the name of the route rule.

type RuntimeBackend added in v0.5.0

type RuntimeBackend struct {
	// Backend is the filter backend configuration.
	Backend *Backend
	// Handler is the backend auth handler.
	Handler BackendAuthHandler
}

RuntimeBackend is a filter backend with its auth handler that is derived from the filterapi.Backend configuration.

type RuntimeConfig added in v0.5.0

type RuntimeConfig struct {
	// UUID is the unique identifier of the filter configuration, inherited from filterapi.Config.
	UUID string
	// RequestCosts is the list of request costs.
	RequestCosts []RuntimeRequestCost
	// DeclaredModels is the list of declared models.
	DeclaredModels []Model
	// Backends is the map of backends by name.
	Backends map[string]*RuntimeBackend
}

RuntimeConfig is the runtime filter configuration that is derived from the filterapi.Config.

func NewRuntimeConfig added in v0.5.0

func NewRuntimeConfig(ctx context.Context, config *Config, fn NewBackendAuthHandlerFunc) (*RuntimeConfig, error)

NewRuntimeConfig creates a new runtime filter configuration from the given filterapi.Config and a function to create backend auth handlers.

type RuntimeRequestCost added in v0.5.0

type RuntimeRequestCost struct {
	*LLMRequestCost
	CELProg cel.Program
}

RuntimeRequestCost is the configuration for the request cost, optionally with a CEL program. This is derived from the filterapi.LLMRequestCost configuration, and includes the compiled CEL program if provided.

type ToolCall added in v0.5.0

type ToolCall struct {
	// Backend is the name of the backend this tool belongs to.
	Backend string `json:"backend"`

	// Tool is the name of the tool.
	Tool string `json:"tool"`
}

type VersionedAPISchema

type VersionedAPISchema struct {
	// Name is the name of the API schema.
	Name APISchemaName `json:"name"`
	// Version is the version of the API schema. Optional.
	Version string `json:"version,omitempty"`
	// Prefix is the prefix of the API schema. Optional. Currently, only used for OpenAI.
	Prefix string `json:"prefix,omitempty"`
}

VersionedAPISchema corresponds to VersionedAPISchema in api/v1alpha1/api.go.

func (VersionedAPISchema) OpenAIPrefix added in v0.5.0

func (v VersionedAPISchema) OpenAIPrefix() string

OpenAIPrefix returns the OpenAI API prefix for the VersionedAPISchema. This is for backwards compatibility with existing users. This won't be necessary after v0.5 release when we can use Prefix directly.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL