Documentation
¶
Overview ¶
Package perception holds the deterministic awareness detectors: personal data, secrets and topics, plus the placeholders that redact what they find.
Detectors run on the text with its invisible runes removed (see textnorm.Strip), so a zero-width space cannot split a token, and report original byte offsets that cover any such runes. A Match never holds the text it matched: it carries the detector, a placeholder type, offsets, a score and an opaque fingerprint that lets Placeholders give a repeated value the same number.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Apply ¶
Apply replaces every span of text with its placeholder. Offsets are clamped to text and empty spans are skipped. Overlapping spans merge into one, and the earliest-starting span's placeholder wins (the first given on a tie).
func BuiltinSecrets ¶
func BuiltinSecrets() []string
BuiltinSecrets returns the built-in secret detector names.
func Entropy ¶
Entropy returns the Shannon entropy of s in bits per character, counting runes. It is 0 for an empty string or one repeated character.
Types ¶
type Match ¶
type Match struct {
Detector string `json:"detector"`
Type string `json:"type"`
Start int `json:"start"`
End int `json:"end"`
Score float64 `json:"score"`
// contains filtered or unexported fields
}
Match is one detection. Start and End are original byte offsets; for a pattern with a named group "secret" they cover that group only. Type is the upper-case placeholder stem, such as "EMAIL" or "GITHUB_TOKEN".
func Custom ¶
Custom runs caller patterns over text and reports every non-empty match as detector name, type name upper-cased with other runes as '_', score 1. A pattern with a named group "secret" reports that group only.
func PII ¶
PII runs the named built-in PII detectors over text. Empty detectors runs every built-in except ip_address; unknown names are ignored. Every match scores 1, and overlapping matches keep the longest.
func Secrets ¶
Secrets runs the named built-in secret detectors over text. Empty detectors runs them all; unknown names are ignored. Known formats score 1, generic_assignment 0.8 and high_entropy 0.6. Where matches overlap only the highest-scoring one is kept, so one secret is one match.
type Span ¶
type Span struct {
Start int `json:"start"`
End int `json:"end"`
Placeholder string `json:"placeholder"`
}
Span is a byte range of the original text and the placeholder that replaces it.
func Placeholders ¶
Placeholders turns matches into redaction spans "[TYPE_n]", sorted by Start. n counts per Type from 1 in reading order, and every match of the same value and Type within one call shares its n. A Match built by hand carries no value and gets its own n.
Source Files