Documentation
¶
Overview ¶
Package unicodecheck provides consolidated Unicode validation for security-sensitive input. It detects zero-width characters, bidirectional overrides, Hangul fillers, control characters, excessive combining marks (Zalgo text), and other problematic Unicode that can be used in attacks.
This package consolidates validation previously scattered across api and auth packages to ensure consistent character detection across all code paths.
Index ¶
- func ContainsAnyCombiningMarks(s string) bool
- func ContainsBidiOverrides(s string) bool
- func ContainsControlChars(s string) bool
- func ContainsDangerousZeroWidthChars(s string) bool
- func ContainsFullwidthStructuralChars(s string) bool
- func ContainsHangulFillers(s string) bool
- func ContainsProblematicCategories(s string) bool
- func ContainsZeroWidthChars(s string) bool
- func HasExcessiveCombiningMarks(s string, maxConsecutive int) bool
- func IsCombiningMark(r rune) bool
- func IsNFCNormalized(s string) bool
- func SanitizeForLogging(s string) string
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ContainsAnyCombiningMarks ¶
ContainsAnyCombiningMarks checks whether the string contains any combining diacritical marks in the basic range (U+0300-U+036F). This is stricter than HasExcessiveCombiningMarks and is used for fields where combining marks are not expected at all. SEM@99fb5e219721394dfd5b0ad63c85d3f01d469306: validate a string for any combining diacritical marks in the basic Unicode range (pure)
func ContainsBidiOverrides ¶
ContainsBidiOverrides checks for bidirectional text override characters that can reorder displayed text to disguise malicious content. SEM@3d0d5a8cf02fa74fad102f0f99c2b936a164bbea: validate a string for bidirectional text override characters that can disguise malicious content (pure)
func ContainsControlChars ¶
ContainsControlChars checks for control characters except common whitespace (\n, \r, \t). SEM@99fb5e219721394dfd5b0ad63c85d3f01d469306: validate a string for control characters excluding common whitespace (pure)
func ContainsDangerousZeroWidthChars ¶
ContainsDangerousZeroWidthChars performs context-aware checking of zero-width characters. Unlike ContainsZeroWidthChars which blocks all zero-width chars, this function allows:
- ZWNJ (U+200C) when appearing between Indic script characters
- ZWJ (U+200D) when appearing between emoji codepoints
It always blocks:
- ZWS (U+200B) — truly invisible, no legitimate use in JSON string values
- BOM (U+FEFF) — no legitimate use mid-string
- LRM (U+200E) and RLM (U+200F) — directional marks
SEM@445f237f7a35ca185cf03ef25426c1a97b1a1917: validate a string for dangerous zero-width chars, allowing legitimate ZWNJ in Indic and ZWJ in emoji contexts (pure)
func ContainsFullwidthStructuralChars ¶
ContainsFullwidthStructuralChars checks for fullwidth characters that could be used for visual spoofing of JSON structural characters (brackets, quotes, etc.). Fullwidth forms are legitimate in CJK text but not in JSON structure. SEM@99fb5e219721394dfd5b0ad63c85d3f01d469306: validate a string for fullwidth JSON structural characters that could enable visual spoofing (pure)
func ContainsHangulFillers ¶
ContainsHangulFillers checks for Hangul filler characters used in fuzzing attacks. SEM@99fb5e219721394dfd5b0ad63c85d3f01d469306: validate a string for Hangul filler characters used in fuzzing attacks (pure)
func ContainsProblematicCategories ¶
ContainsProblematicCategories checks for characters in problematic Unicode categories: Private Use Area, Surrogates, and Non-character codepoints. SEM@99fb5e219721394dfd5b0ad63c85d3f01d469306: validate a string for private-use, surrogate, and non-character Unicode codepoints (pure)
func ContainsZeroWidthChars ¶
ContainsZeroWidthChars checks for zero-width Unicode characters that can be used for spoofing. This includes zero-width spaces, joiners, directional marks, and the byte order mark. SEM@3d0d5a8cf02fa74fad102f0f99c2b936a164bbea: validate a string for zero-width spoofing characters including BOM and directional marks (pure)
func HasExcessiveCombiningMarks ¶
HasExcessiveCombiningMarks detects "Zalgo text" abuse by checking for sequences of consecutive combining marks exceeding the given threshold. A threshold of 3 is typical for middleware; use 1 to reject any combining marks in the basic Combining Diacritical Marks range (U+0300-U+036F). SEM@99fb5e219721394dfd5b0ad63c85d3f01d469306: validate a string for Zalgo-style abuse by detecting consecutive combining marks exceeding a threshold (pure)
func IsCombiningMark ¶
IsCombiningMark returns true if the rune is a Unicode combining character. Covers Combining Diacritical Marks (U+0300-U+036F) and extended ranges for Cyrillic, Hebrew, Arabic, Thai, and Indic scripts. SEM@445f237f7a35ca185cf03ef25426c1a97b1a1917: validate a rune as a Unicode combining character across Cyrillic, Hebrew, Arabic, and Indic ranges (pure)
func IsNFCNormalized ¶
IsNFCNormalized checks whether the string is in NFC (Canonical Composition) form. Non-normalized Unicode can cause storage and comparison issues. SEM@99fb5e219721394dfd5b0ad63c85d3f01d469306: validate that a string is in NFC canonical composition form (pure)
func SanitizeForLogging ¶
SanitizeForLogging removes potentially dangerous characters from strings before logging. Replaces control characters with [CTRL] and zero-width characters with [ZW]. SEM@9745b416c50726fc3ca5d4637364ba55d6ba0699: convert dangerous control and zero-width characters to safe placeholder tokens for log output (pure)
Types ¶
This section is empty.
Source Files