Documentation
¶
Overview ¶
Package bouncer provides token validation, secret pattern matching, and streaming redaction for MCP proxy request/response filtering and security enforcement.
Index ¶
- Constants
- Variables
- func CompileCustomPatterns(configs []PatternConfig) ([]*regexp.Regexp, error)
- func CompileCustomPatternsWithTimeout(configs []PatternConfig, timeout time.Duration) ([]*regexp.Regexp, error)
- func CompilePatterns(configs []PatternConfig) ([]*regexp.Regexp, error)
- func DetectLanguage(filename string) string
- func EmbedFailureCount() uint64
- func EmbedSuccessCount() uint64
- func EmbedToolCall(ctx context.Context, req EmbedRequest)
- func EstimateToolsFromDescription(description string) int
- func FindDangerousPatterns(patterns []string) (unsafe []string, err error)
- func FormatPatternList() string
- func FormatSnapshot(s Snapshot) string
- func GetBuffer() []byte
- func GetPatternNames() []string
- func GlobalEmbedPool() *embedder.Pool
- func IsPatternSafe(pattern string) bool
- func LoadPatternsWithLogging(customConfigs []PatternConfig) ([]*regexp.Regexp, []string)
- func MatchSecret(input string) []string
- func PatternsToRegexps(patterns []SecretPattern) []*regexp.Regexp
- func RedactJSONWithSidecar(ctx context.Context, data []byte, r *Redactor, sidecar SidecarClient, ...) ([]byte, error)
- func RedactSecrets(input string) string
- func RedactWithPatterns(input string, patterns []*regexp.Regexp) string
- func ReturnBuffer(buf []byte)
- func SafeCompile(pattern string) (*regexp.Regexp, error)
- func SafeCompileMust(pattern string) *regexp.Regexp
- func SetGlobalEmbedPool(p *embedder.Pool)
- func SetupEmbedder(cfg embedder.Config, poolCfg embedder.PoolConfig) error
- func StripComments(pattern string) string
- func ValidatePattern(pattern string) error
- func ValidatePatterns() error
- func ValidateReDoS(pattern string, timeout time.Duration) error
- type AlertManager
- type BoilerplateProcessor
- type BoilerplatePruner
- type BoilerplateReport
- type Bouncer
- type Claims
- type Config
- type EmbedOutcome
- type EmbedRequest
- type LoadedPatterns
- type PatternConfig
- type PatternDef
- type RedactionEvent
- type RedactionMeta
- type Redactor
- type SecretPattern
- type SidecarClient
- type Snapshot
- type StreamingRedactor
- type TokenValidator
Constants ¶
const ( // DefaultEstimatedTools is the default tool-count estimate when the // registry entry does not declare one. It matches the average used by // pkg/utils.TokenEstimator.CompareMCPConfigurations. DefaultEstimatedTools = 35 )
const SecretRedacted = "[SECRET_REDACTED]"
const SidecarFallback = "[VALUE_REDACTED]"
SidecarFallback is the sentinel string the sidecar LLM emits when its primary path fails (Ollama down, decode error, empty response). The redactor treats it as "I could not redact anything; do whatever you did without me" rather than as a hard error, so a single Ollama hiccup does not reject every request. See pkg/sidecar.Client.aggressiveRedact.
Variables ¶
var ( // ErrInvalidToken indicates the provided token is invalid. ErrInvalidToken = errors.New("invalid token") // ErrExpiredToken indicates the provided token has expired. ErrExpiredToken = errors.New("expired token") // ErrMissingToken indicates no token was provided. ErrMissingToken = errors.New("missing token") )
var BuiltInPatterns = []SecretPattern{ { Name: "aws-access-key", Pattern: regexp.MustCompile(`AKIA[0-9A-Z]{16}`), Example: "AKIAIOSFODNN7EXAMPLE", Description: "AWS Access Key ID (20 characters, starts with AKIA)", }, { Name: "github-classic-pat", Pattern: regexp.MustCompile(`ghp_[A-Za-z0-9]{36,}`), Example: "ghp_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", Description: "GitHub Classic Personal Access Token (starts with ghp_, 36+ chars after prefix)", }, { Name: "github-fine-grained-pat", Pattern: regexp.MustCompile(`github_pat_[A-Za-z0-9_]{22,}`), Example: "github_pat_11XXXXXXXXXXXXXXXX_XXXXXXXXXXXXXXXXXXXX", Description: "GitHub Fine-grained PAT (starts with github_pat_)", }, { Name: "stripe-secret-key", Pattern: regexp.MustCompile(`sk_live_[A-Za-z0-9]{24}`), Example: "[Stripe Live Secret Key - 24 chars after sk_live_]", Description: "Stripe Live Secret Key (starts with sk_live_)", }, { Name: "stripe-publishable-key", Pattern: regexp.MustCompile(`pk_live_[A-Za-z0-9]{24}`), Example: "[Stripe Publishable Key - 24 chars after pk_live_]", Description: "Stripe Live Publishable Key (starts with pk_live_)", }, { Name: "pem-private-key", Pattern: regexp.MustCompile(`-----BEGIN (?:RSA |EC |DSA |OPENSSH |ENCRYPTED )?PRIVATE KEY-----[\s\S]+?-----END (?:RSA |EC |DSA |OPENSSH |ENCRYPTED )?PRIVATE KEY-----`), Example: "-----BEGIN RSA PRIVATE KEY-----\nMIIEowIBAAKCAQ...\n-----END RSA PRIVATE KEY-----", Description: "Multi-line PEM-encoded private keys (RSA, EC, DSA, OpenSSH, PKCS8, encrypted)", }, { Name: "pem-certificate", Pattern: regexp.MustCompile(`-----BEGIN CERTIFICATE-----[\s\S]+?-----END CERTIFICATE-----`), Example: "-----BEGIN CERTIFICATE-----\nMIIDdzCCAl+gAwIBAgI...\n-----END CERTIFICATE-----", Description: "Multi-line PEM-encoded X.509 certificates", }, { Name: "gcp-service-account", Pattern: regexp.MustCompile(`"type"\s*:\s*"service_account"`), Example: `{"type": "service_account", "project_id": "...", "private_key": "..."}`, Description: "GCP service account JSON key marker (\"type\": \"service_account\")", }, { Name: "gcp-oauth-token", Pattern: regexp.MustCompile(`ya29\.[A-Za-z0-9_-]{20,}`), Example: "ya29.XXXXXXXXXXXXXXXXXXXXXXXXXXXX", Description: "GCP OAuth2 access / refresh token (starts with ya29., 20+ alphanumeric/_/- chars after)", }, { Name: "slack-token", Pattern: regexp.MustCompile(`xox[bpars]-[A-Za-z0-9-]{20,}`), Example: "xoxb-XXXXXXXXXXXXXXXX-XXXXXXXXXXXXXXXX-XXXXXXXXXXXXXXXXXXXXXXXX", Description: "Slack bot/user/app/refresh token (xoxb-, xoxp-, xoxa-, xoxr-, xoxs-, 20+ chars after prefix)", }, { Name: "gitlab-pat", Pattern: regexp.MustCompile(`glpat-[A-Za-z0-9_-]{20,}`), Example: "glpat-XXXXXXXXXXXXXXXXXXXX", Description: "GitLab Personal Access Token (starts with glpat-, 20+ chars after prefix)", }, { Name: "openai-api-key", Pattern: regexp.MustCompile(`sk-[A-Za-z0-9]{40,}`), Example: "sk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", Description: "OpenAI API key (sk- followed by 40+ alphanumeric chars)", }, { Name: "anthropic-api-key", Pattern: regexp.MustCompile(`sk-ant-[A-Za-z0-9_-]{32,}`), Example: "sk-ant-api03-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", Description: "Anthropic API key (starts with sk-ant-, 32+ chars after prefix)", }, { Name: "generic-api-key", Pattern: regexp.MustCompile(`(?i)(api[_-]?key)[_-]?[=]?[A-Za-z0-9]{16,}`), Example: "api_key=abcdefghijklmnopqrstuvwx", Description: "Generic API key pattern (case-insensitive)", }, { Name: "bearer-token", Pattern: regexp.MustCompile(`(?i)bearer\s+[A-Za-z0-9\-_]+\.[A-Za-z0-9\-_]+\.[A-Za-z0-9\-_]+`), Example: "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c", Description: "JWT Bearer token (three base64url segments)", }, { Name: "env-var-value", Pattern: regexp.MustCompile(`\$[A-Z_][A-Z0-9_]{0,30}=([^\s,}]+)`), Example: "$API_KEY=secret123", Description: "Environment variable assignment", }, }
var EmbedResultHandler func(EmbedOutcome)
var (
ErrDangerousPattern = errors.New("dangerous regex pattern detected")
)
var SensitiveJSONFieldNames = []string{
"api_key",
"apikey",
"api-key",
"token",
"password",
"passwd",
"private_key",
"privatekey",
"private-key",
"client_secret",
"clientsecret",
"client-secret",
"secret",
}
SensitiveJSONFieldNames is the set of JSON keys whose value must be redacted regardless of pattern match, when walking a parsed JSON document in RedactJSON. It is keyed lowercase; case-insensitive lookup is performed by sensitiveJSONFieldLookup. The field-name pass catches tokens whose value no built-in regex recognizes (an internal platform token sitting behind a key called "api_key", for example).
The field-name pass is intentionally kept out of the byte-level streaming regex set: a regex matching `"<key>": "<value>"` would either swallow the surrounding quotes (emitting invalid JSON for the common case), or terminate at the first unescaped quote inside the value (silently leaking the tail of a value containing `\"`). The JSON-aware walker already visits every string value in isolation, which is the only context where field-name-based redaction is sound.
Functions ¶
func CompileCustomPatterns ¶ added in v0.2.0
func CompileCustomPatterns(configs []PatternConfig) ([]*regexp.Regexp, error)
func CompileCustomPatternsWithTimeout ¶ added in v0.2.0
func CompilePatterns ¶ added in v0.2.0
func CompilePatterns(configs []PatternConfig) ([]*regexp.Regexp, error)
func DetectLanguage ¶ added in v0.2.0
func EmbedFailureCount ¶ added in v0.8.0
func EmbedFailureCount() uint64
func EmbedSuccessCount ¶ added in v0.8.0
func EmbedSuccessCount() uint64
func EmbedToolCall ¶ added in v0.8.0
func EmbedToolCall(ctx context.Context, req EmbedRequest)
func EstimateToolsFromDescription ¶ added in v0.8.0
EstimateToolsFromDescription derives a coarse tool-count estimate from the size of the registry entry's description. It is intentionally simple so it can be used as a fallback when no other signal is present; callers should prefer ComputeSnapshot with an explicit tool count whenever possible.
func FindDangerousPatterns ¶ added in v0.6.0
func FormatPatternList ¶ added in v0.2.0
func FormatPatternList() string
func FormatSnapshot ¶ added in v0.8.0
FormatSnapshot renders the snapshot as a short human-readable block ready to print to stdout. Lines are kept short for terminal readability.
func GetPatternNames ¶ added in v0.2.0
func GetPatternNames() []string
func GlobalEmbedPool ¶ added in v0.8.0
func IsPatternSafe ¶ added in v0.6.0
func LoadPatternsWithLogging ¶ added in v0.2.0
func LoadPatternsWithLogging(customConfigs []PatternConfig) ([]*regexp.Regexp, []string)
func MatchSecret ¶ added in v0.2.0
func PatternsToRegexps ¶ added in v0.2.0
func PatternsToRegexps(patterns []SecretPattern) []*regexp.Regexp
func RedactJSONWithSidecar ¶ added in v0.8.0
func RedactJSONWithSidecar(ctx context.Context, data []byte, r *Redactor, sidecar SidecarClient, alwaysCallSidecar bool) ([]byte, error)
RedactJSONWithSidecar applies the regex redactor first and then, if a sidecar is configured and either the regex matched nothing or alwaysCallSidecar is true, hands the already-redacted content to the sidecar for a second pass. The sidecar therefore never sees a secret the regex layer could catch (the regex pass runs first regardless), and its output is accepted only if it is valid JSON or matches the sidecar's documented fallback sentinel.
func RedactSecrets ¶ added in v0.2.0
func RedactWithPatterns ¶ added in v0.2.0
func ReturnBuffer ¶ added in v0.2.0
func ReturnBuffer(buf []byte)
func SafeCompileMust ¶ added in v0.6.0
func SetGlobalEmbedPool ¶ added in v0.8.0
func SetupEmbedder ¶ added in v0.8.0
func SetupEmbedder(cfg embedder.Config, poolCfg embedder.PoolConfig) error
func StripComments ¶ added in v0.6.0
func ValidatePattern ¶ added in v0.6.0
func ValidatePatterns ¶ added in v0.2.0
func ValidatePatterns() error
Types ¶
type AlertManager ¶ added in v0.2.0
type AlertManager struct {
// contains filtered or unexported fields
}
func NewAlertManager ¶ added in v0.2.0
func NewAlertManager(verbose bool) *AlertManager
func (*AlertManager) EmitSummary ¶ added in v0.2.0
func (am *AlertManager) EmitSummary(messageID string, method string)
func (*AlertManager) GetCounts ¶ added in v0.2.0
func (am *AlertManager) GetCounts() map[string]int
func (*AlertManager) RecordRedaction ¶ added in v0.2.0
func (am *AlertManager) RecordRedaction(event RedactionEvent)
func (*AlertManager) SetEnabled ¶ added in v0.2.0
func (am *AlertManager) SetEnabled(enabled bool)
func (*AlertManager) SetVerbose ¶ added in v0.2.0
func (am *AlertManager) SetVerbose(verbose bool)
type BoilerplateProcessor ¶ added in v0.2.0
type BoilerplateProcessor interface {
Process(content []byte, language string) ([]byte, BoilerplateReport, error)
}
type BoilerplatePruner ¶ added in v0.2.0
type BoilerplatePruner struct {
// contains filtered or unexported fields
}
func NewBoilerplatePruner ¶ added in v0.2.0
func NewBoilerplatePruner(enabled, redactImports, redactLicenses bool, customPatterns []PatternDef) *BoilerplatePruner
func (*BoilerplatePruner) Process ¶ added in v0.2.0
func (p *BoilerplatePruner) Process(content []byte, language string) ([]byte, BoilerplateReport, error)
func (*BoilerplatePruner) ProcessStream ¶ added in v0.2.0
func (p *BoilerplatePruner) ProcessStream(reader io.Reader, writer io.Writer, language string) (BoilerplateReport, error)
type BoilerplateReport ¶ added in v0.2.0
type Bouncer ¶
type Bouncer struct{}
Bouncer is a token validation and secret detection service.
func (*Bouncer) ExtractClaims ¶
ExtractClaims extracts claims from a validated token.
type Claims ¶
type Claims map[string]interface{}
Claims represents the claims extracted from a validated token.
type Config ¶ added in v0.2.0
type Config struct {
// Enabled defaults to true when omitted; set `enabled: false` to turn the
// regex redactor off explicitly.
Enabled *bool `yaml:"enabled,omitempty"`
// Patterns is the documented key for custom patterns; CustomPatterns is
// accepted as an alias for older configs. Both lists are compiled.
Patterns []PatternDef `yaml:"patterns,omitempty"`
CustomPatterns []PatternDef `yaml:"custom_patterns,omitempty"`
// SidecarAlwaysCall opts in to the per-request sidecar LLM behavior
// introduced (and then reverted) around #274. When false (the default),
// the sidecar is consulted only when the regex layer matched zero
// secrets, which keeps the per-request cost to one regex pass for the
// common case.
SidecarAlwaysCall *bool `yaml:"sidecar_always_call,omitempty"`
}
Config defines the bouncer configuration, including whether the bouncer is enabled and any custom regex patterns for secret detection.
func LoadConfig ¶ added in v0.2.0
LoadConfig reads and parses a bouncer YAML configuration from the provided reader.
func LoadConfigFile ¶ added in v0.2.0
LoadConfigFile reads and parses a bouncer YAML configuration from the given file path.
func (*Config) AllPatternDefs ¶ added in v0.9.2
func (c *Config) AllPatternDefs() []PatternDef
AllPatternDefs returns the documented and legacy custom pattern lists in a single slice (Patterns followed by CustomPatterns). Exported so external packages (e.g. migrate.Config.Validate) can walk the configured patterns without re-implementing the merge logic.
func (*Config) CompilePatterns ¶ added in v0.2.0
func (c *Config) CompilePatterns() (*LoadedPatterns, error)
CompilePatterns compiles built-in and custom patterns into a LoadedPatterns struct.
func (*Config) IsEnabled ¶ added in v0.9.2
IsEnabled reports whether the redactor should run. A nil Config or an omitted `enabled` key means enabled.
func (*Config) ShouldAlwaysCallSidecar ¶ added in v0.9.2
ShouldAlwaysCallSidecar reports whether the sidecar LLM must run on every request, even when the regex layer already matched a secret. Defaults to false (the regex-cleaned payload is forwarded without the LLM round-trip).
type EmbedOutcome ¶ added in v0.8.0
type EmbedOutcome struct {
Request EmbedRequest `json:"request"`
Provider string `json:"provider"`
Model string `json:"model,omitempty"`
Vector []float32 `json:"vector,omitempty"`
Err string `json:"error,omitempty"`
}
type EmbedRequest ¶ added in v0.8.0
type EmbedRequest struct {
ToolName string
Args json.RawMessage
}
type LoadedPatterns ¶ added in v0.2.0
type LoadedPatterns struct {
BuiltIn []SecretPattern
Custom []SecretPattern
All []*regexp.Regexp
}
LoadedPatterns holds the compiled built-in and custom regex patterns.
type PatternConfig ¶ added in v0.2.0
func (PatternConfig) Validate ¶ added in v0.2.0
func (pc PatternConfig) Validate() error
type PatternDef ¶ added in v0.2.0
PatternDef defines a custom regex pattern for secret detection.
type RedactionEvent ¶ added in v0.2.0
type RedactionMeta ¶ added in v0.2.0
type Redactor ¶ added in v0.1.2
type Redactor struct {
// contains filtered or unexported fields
}
func NewRedactor ¶ added in v0.1.2
func NewRedactorFromLoaded ¶ added in v0.2.0
func NewRedactorFromLoaded(loaded *LoadedPatterns) *Redactor
func NewRedactorWithAlerts ¶ added in v0.2.0
func NewRedactorWithAlerts(patterns []*regexp.Regexp, alertManager *AlertManager) *Redactor
func (*Redactor) Patterns ¶ added in v0.9.2
Patterns returns the compiled patterns this redactor applies.
func (*Redactor) RedactJSON ¶ added in v0.1.2
RedactJSON redacts every string value in a JSON document. If the input is not valid JSON it falls back to a byte-level scan of the raw input rather than passing it through unchanged, so a malformed or truncated payload can never be used to smuggle a secret past the redactor.
func (*Redactor) RedactStream ¶ added in v0.1.2
RedactStream copies reader to writer, replacing secrets as they pass through. Input is accumulated before scanning and a tail of up to maxOverlap bytes after the last match is held back until more input (or EOF) arrives, so a secret split across arbitrary read boundaries — including very small reads from a pipe — is still redacted as a whole.
All per-request scratch buffers (carry, output, scan spans) come from package-level sync.Pools; the defer block returns each one with its backing array zeroed (where it holds sensitive bytes) before this function returns. See issue #277 for the allocation profile this is meant to flatten.
type SecretPattern ¶ added in v0.2.0
func GetBuiltInPatterns ¶ added in v0.2.0
func GetBuiltInPatterns() []SecretPattern
func GetPatternByName ¶ added in v0.2.0
func GetPatternByName(name string) *SecretPattern
type SidecarClient ¶ added in v0.8.0
type Snapshot ¶ added in v0.8.0
type Snapshot struct {
ServerName string
Transport string
EstimatedTools int
NativeTokens int
LeanProxyTokens int
SavedTokens int
SavingsPercent float64
HasRegistryBudget bool
RegistryBudgetNote string
}
Snapshot describes the token impact of adding a server to a LeanProxy deployment. It is computed from a registry feed entry before the server is installed so the user can preview the cost.
func ComputeSnapshot ¶ added in v0.8.0
ComputeSnapshot returns a token-cost preview for the given server definition.
The estimate is derived from the entry's declared tool count when available (TokensPerTurn is treated as a per-turn override), otherwise from a default heuristic. The LeanProxy cost assumes a single gateway schema (invoke_tool + list_tools) regardless of how many real tools the server exposes.
The returned Snapshot is safe for direct fmt.Stringer-style printing via FormatSnapshot.
type StreamingRedactor ¶ added in v0.2.0
type StreamingRedactor struct {
// contains filtered or unexported fields
}
StreamingRedactor is a thin wrapper kept for API compatibility. All streaming redaction is implemented by Redactor.RedactStream, which carries state across read boundaries; this type used to redact each read independently and leaked any secret that straddled a boundary.
func NewStreamingRedactor ¶ added in v0.2.0
func NewStreamingRedactor(patterns []*regexp.Regexp) *StreamingRedactor
func NewStreamingRedactorWithAlerts ¶ added in v0.2.0
func NewStreamingRedactorWithAlerts(patterns []*regexp.Regexp, am *AlertManager) *StreamingRedactor
func (*StreamingRedactor) RedactStream ¶ added in v0.2.0
func (sr *StreamingRedactor) RedactStream(r io.Reader, w io.Writer, meta ...*RedactionMeta) error
func (*StreamingRedactor) RedactToWriter ¶ added in v0.2.0
RedactToWriter is an alias for RedactStream kept for backwards compatibility with callers that pre-date the RedactStream rename. Both methods share the same code path — fixing one fixes both — so any future change to streaming redaction must update the single implementation in Redactor.RedactStream. See issue #277 for the rationale.