Documentation
¶
Overview ¶
Package redactor provides multi-stage secret redaction pipeline for diagnostic bundles. It implements a defense-in-depth approach combining structured parsing (YAML/JSON field-aware), regex pattern matching (PEM keys, JWT tokens), and Shannon entropy detection to prevent credential leakage in diagnostic output.
Pipeline stages: 1. Canary injection: Synthetic test tokens injected to verify redaction completeness 2. Structured redaction: YAML/JSON field-aware redaction (highest precision) 3. Regex patterns: Format-specific secret detection (PEM, JWT, base64) 4. Entropy scanning: High-randomness string detection (catch-all verification) 5. Canary verification: MUST pass - fails hard if canaries leak
Failure modes: - Structured parsing errors: Best-effort (continue with regex-only, emit warning) - Entropy detection: Auto-redact suspicious values + emit warnings - Canary verification failure: FATAL (return error, abort bundle generation)
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type EntropyDetector ¶
type EntropyDetector struct {
// contains filtered or unexported fields
}
EntropyDetector scans for high-randomness strings using Shannon entropy. This is a verification/catch-all stage to detect secrets missed by structured and regex redaction.
func NewEntropyDetector ¶
func NewEntropyDetector() *EntropyDetector
NewEntropyDetector creates a detector with industry-standard thresholds for base64-encoded secrets (4.5 bits/char, 16+ chars).
func (*EntropyDetector) Scan ¶
func (d *EntropyDetector) Scan(data []byte) ([]SuspiciousValue, error)
Scan analyzes input for high-entropy strings that might be secrets. Returns slice of suspicious values with entropy and line numbers for warnings.
type PatternRedactor ¶
type PatternRedactor struct {
// contains filtered or unexported fields
}
PatternRedactor performs regex-based secret detection using known formats
func NewPatternRedactor ¶
func NewPatternRedactor() *PatternRedactor
NewPatternRedactor creates a redactor with Neuwerk-specific secret patterns
func (*PatternRedactor) Redact ¶
func (r *PatternRedactor) Redact(input []byte) []byte
Redact applies all regex patterns to input using line-by-line processing to preserve exact newline structure and avoid binary data corruption.
type Redactor ¶
type Redactor struct {
// contains filtered or unexported fields
}
Redactor orchestrates multi-stage secret redaction pipeline
func New ¶
func New() *Redactor
New creates a redactor with default configuration for Neuwerk secret patterns
func (*Redactor) Redact ¶
Redact runs full multi-stage redaction pipeline with mandatory canary verification.
Pipeline: 1. Inject canaries: Add synthetic test tokens that MUST be redacted 2. Structured redaction: Parse YAML/JSON and redact known secret fields 3. Regex patterns: Match PEM keys, JWT tokens, base64 secrets 4. Entropy scan: Detect high-randomness strings (auto-redact + warn) 5. Verify canaries: Fail hard if any canary tokens leak
Returns:
- []byte: Redacted output with placeholders replacing secrets
- []string: Accumulated warnings from all stages (parsing failures, entropy detections)
- error: Only on canary verification failure (fatal - secrets leaked)
type StructuredRedactor ¶
type StructuredRedactor struct {
// contains filtered or unexported fields
}
StructuredRedactor performs field-aware redaction on parsed YAML/JSON structures. It walks the data structure recursively, matching field names (case-insensitive) against known secret patterns and replacing values with type-specific placeholders.
func NewStructuredRedactor ¶
func NewStructuredRedactor() *StructuredRedactor
NewStructuredRedactor creates a redactor with Neuwerk-specific secret field patterns
func (*StructuredRedactor) Redact ¶
func (r *StructuredRedactor) Redact(input []byte) ([]byte, error)
Redact attempts to parse input as JSON/YAML and redact known secret fields. Preserves the original format (JSON in, JSON out). Also removes the _canary_verification field injected by the verifier. Returns error if input is not parseable structured data (caller should fall back to regex-only).
type SuspiciousValue ¶
type SuspiciousValue struct {
Value string // The suspicious string value
Entropy float64 // Shannon entropy in bits/char
Line int // Line number where found (1-indexed)
}
SuspiciousValue represents a high-entropy string that might be a secret
type Verifier ¶
type Verifier struct {
// contains filtered or unexported fields
}
Verifier implements canary token injection and verification for redaction testing. It injects synthetic test secrets that MUST be redacted, then verifies they don't leak through the pipeline. This is a safety check - if canaries leak, real secrets might too.
func NewVerifier ¶
func NewVerifier() *Verifier
NewVerifier creates a verifier with synthetic test tokens for all Neuwerk secret types
func (*Verifier) InjectCanaries ¶
InjectCanaries adds synthetic test tokens to input for verification. For JSON input, canaries are injected as a nested object to preserve valid JSON structure. For non-JSON input, canaries are appended as key-value lines. All canaries MUST be redacted by the pipeline or verification will fail.
Source Files