Documentation
¶
Overview ¶
Package secretscan is vigolium's native, binary-free secret detector. It replaces the former external kingfisher subprocess with an in-process engine driven by the normalized kingfisher rule catalog (see pkg/secretscan/catalog and pkg/secretscan/secretgen). Detection is regex-based with a keyword prefilter, Shannon-entropy and character-class gating, and a benign-placeholder safelist — faithfully mirroring kingfisher's matcher semantics. Live secret VERIFICATION is intentionally not performed.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func LoadCatalog ¶
LoadCatalog parses the embedded rule catalog and appends the vigolium-authored supplemental rules (see supplementalRules). The embedded JSON is the generated kingfisher set (regenerated by `make update-secret-rules`); the supplemental rules live in Go so they survive regeneration.
Types ¶
type Detector ¶
type Detector struct {
// contains filtered or unexported fields
}
Detector holds the compiled rule set and keyword prefilter.
func Default ¶
Default returns a process-wide Detector built from the embedded catalog with default options (visible rules only). The catalog is parsed and all rule regexes compiled exactly once, then reused across scans. The per-body result cache is enabled (see defaultBodyCacheSize); the experimental Set prefilter is not (opt-in via New with Options.UseRegexSet).
func New ¶
New compiles a Detector from a catalog. Regex compilation happens once here; callers should reuse the Detector across scans (see Default).
func (*Detector) Detect ¶
Detect scans data and returns all secrets found. Safe for concurrent use.
When the body-SHA cache is enabled, an identical body short-circuits to its previously computed matches — Detect is a pure function of (rule set, body), so a cached result is always valid. The returned slice is treated as read-only by all callers; it may be shared with concurrent callers of the same body.
type Engine ¶
type Engine int
Engine selects the regex backend used to evaluate rule patterns.
const ( // EngineRE2 (default) uses github.com/wasilibs/go-re2 — Google's RE2 (C++) // compiled to WebAssembly (pure Go, no cgo, works on arm64). Identical RE2 // semantics to stdlib (so it is correct — zero false negatives), and ~10x // faster on bodies >~1KB with this complex rule set. A one-time ~350ms // compile cost is amortized by the process-wide Default() detector. EngineRE2 Engine = iota // EngineStdlib uses Go's standard library regexp (RE2). Correctness reference // used by tests; ~10x slower than go-re2 here, so not the default. EngineStdlib )
type Match ¶
type Match struct {
RuleID string
RuleName string
Source string // rule source; always "kingfisher"
Confidence string // "high" | "medium" | "low"
Pattern string // the rule's normalized RE2 pattern that fired (for evidence)
Secret string // the selected secret capture
Start int // byte offset of the secret in the scanned input
End int // exclusive end offset
Entropy float64 // computed Shannon entropy (bits/byte) of the secret
}
Match is a single detected secret.
type Options ¶
type Options struct {
// IncludeInvisible enables rules marked visible:false in the catalog
// (kingfisher's noisy/experimental set). Off by default.
IncludeInvisible bool
// MaxMatchesPerRule caps ACCEPTED matches per rule per body (<=0 → default).
MaxMatchesPerRule int
// Engine selects the regex backend (zero value = EngineRE2 / go-re2).
Engine Engine
// BodyCacheSize bounds the per-body result cache (keyed by body SHA-256).
// A value >0 enables caching; 0 leaves the caller default (Default enables
// it, New leaves it off); <0 disables it explicitly. Hashing a body is far
// cheaper than scanning it, so repeated identical bodies (SPA shells, records
// re-seen across scan passes) are served from cache. Safe: Detect is a pure
// function of (rule set, body).
BodyCacheSize int
// UseRegexSet enables the experimental compiled-regex Set prefilter
// (setprefilter.go): the body is scanned once by a sharded go-re2 Set to learn
// which rule patterns can match at all, and only those rules' capture regexes
// are run. Off by default — it layers on top of the keyword prefilter and its
// benefit is workload-dependent, so it ships benchmarked and opt-in. If the
// set fails to compile, detection falls back to the keyword prefilter alone.
UseRegexSet bool
// RegexSetShardSize is the number of patterns per Set shard when UseRegexSet is
// on (<=0 → default). A large single set may exceed go-re2's program-size
// limit, so patterns are compiled into shards.
RegexSetShardSize int
}
Options configures a Detector.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package catalog defines the on-disk schema for the native secret-detection rule catalog.
|
Package catalog defines the on-disk schema for the native secret-detection rule catalog. |
|
Command secretgen normalizes the kingfisher secret-detection rule set into a single RE2-compatible catalog embedded by pkg/secretscan.
|
Command secretgen normalizes the kingfisher secret-detection rule set into a single RE2-compatible catalog embedded by pkg/secretscan. |