Documentation
¶
Overview ¶
Package weburl is the one URL library of the web attack surface: the sensor, the SDK and the platform parse, normalise and template web URLs the same way, so an endpoint found by one tool and one found by another land on the same row.
- Parse is strict: an absolute http(s) URL, no user info, at most MaxURLBytes, valid UTF-8, no control or white-space characters. It returns the normalised URL: lowercase scheme and ASCII host (Punycode for internationalized labels, trailing dot removed, IPv6 in canonical form), the default port dropped, the path normalised (unreserved percent-escapes decoded and the others upper-cased, dot segments resolved, empty segments collapsed, no trailing slash except the root, ";jsessionid=" removed; case kept: paths are case-sensitive), the query reduced to its sorted parameter NAMES (values are never kept) and no fragment.
- Template replaces the path segments that are identifiers with a typed variable ({int}, {uuid}, {date}, {email}, {hex}, {token}, {id}; see TemplatePath), so /orders/42 and /orders/43 are one endpoint.
- RedactURL removes what a URL must not carry into a report, a log or a database: the user info, every query value and the fragment. It never fails.
- PathHash is the dedup key of an endpoint: a hash of its method and template.
Every function is deterministic, uses no network and no regular expression that can backtrack (Go's regexp is RE2).
Index ¶
- Constants
- Variables
- func Normalize(raw string) (string, error)
- func NormalizeMethod(m string) (string, bool)
- func NormalizePath(p string) (string, error)
- func PathHash(method, template string) string
- func QueryNames(rawQuery string) []string
- func RedactURL(raw string) string
- func Template(raw string) (origin, template string, params []string, err error)
- func TemplatePath(path string) string
- func TemplateSegment(seg string) string
- type URL
Constants ¶
const ( VarInt = "{int}" VarUUID = "{uuid}" VarDate = "{date}" VarEmail = "{email}" VarHex = "{hex}" VarToken = "{token}" VarID = "{id}" )
Typed variables of a path template.
const ( // MaxURLBytes bounds a URL, a path and a template. MaxURLBytes = 2048 // MaxParamNameBytes bounds a query parameter name; a longer one is // dropped. MaxParamNameBytes = 128 // MaxParams bounds the parameter names kept from one query. MaxParams = 100 )
Limits of a web URL.
const MethodAny = "ANY"
Methods of an endpoint. MethodAny is an endpoint seen without a method (a link of unknown use is GET; ANY is "not known").
Variables ¶
var ErrInvalid = errors.New("weburl: invalid URL")
ErrInvalid is the error of every URL Parse refuses.
Functions ¶
func NormalizeMethod ¶
NormalizeMethod returns the upper-case method, MethodAny for "", and ok false for a method outside the closed set.
func NormalizePath ¶
NormalizePath normalises an escaped URL path: unreserved percent-escapes decoded and the others upper-cased (a malformed escape is an error), ";jsessionid=..." removed from each segment, dot segments resolved, empty segments collapsed, the trailing slash removed except for the root. An empty path is "/".
func PathHash ¶
PathHash is the dedup key of an endpoint within its origin: the hex SHA-256 of the normalised method, a newline and the path template.
func QueryNames ¶
QueryNames returns the sorted, unique parameter names of a raw query ("a=1&b=2&a=3" gives [a b]). Values are never read. A name that does not unescape is kept escaped; empty names and names longer than MaxParamNameBytes are dropped; at most MaxParams names are kept.
func RedactURL ¶
RedactURL returns raw without what must not be stored: the user info, every query value (the names are kept, "?a=&b="), and the fragment. A value that does not parse as a URL is cut at its first '?' or '#' and its user info removed. It never fails and never adds content.
func Template ¶
Template parses raw and returns its origin, its path template and its query parameter names.
func TemplatePath ¶
TemplatePath replaces the identifier segments of a normalised path (TemplateSegment).
func TemplateSegment ¶
TemplateSegment returns the typed variable an identifier segment becomes, or the segment itself:
- {int}: digits only ("42");
- {uuid}: an RFC 4122 UUID;
- {date}: an ISO date ("2026-10-07");
- {email}: an e-mail address;
- {hex}: 16 or more hexadecimal digits (hashes, object ids);
- {token}: a JWT-like value (three base64url parts) or 20 or more base64url characters with an upper-case letter, a lower-case letter and a digit (random tokens; lowercase slugs never match);
- {id}: 12 or more letters and digits with at least one of each ("a1b2c3d4e5f6").
The segment is matched unescaped, so "user%40example.com" is an e-mail.
Types ¶
type URL ¶
type URL struct {
// Scheme is "http" or "https".
Scheme string
// Host is the ASCII host: a lowercase DNS name or an IP address
// (IPv6 without brackets).
Host string
// Port is the port when it is not the scheme's default, else "".
Port string
// Path is the normalised path, "/" for the root.
Path string
// Params are the sorted, unique query parameter names.
Params []string
}
URL is a normalised web URL.
Source Files