Documentation
¶
Index ¶
- func AddListToSetString(set map[string]struct{}, list []string) map[string]struct{}
- func CaptureStaticAssetReference(urls map[string]struct{}, scopeURL string, fullURL string, ...) bool
- func ExtractDomain(rawURL string, maxDomainLevel int) string
- func IsAbsoluteURL(u string) bool
- func IsSubdomain(baseURL string, targetURL string) bool
- func IsURLAllowed(scopeURL string, targetURL string, ignoreCrossDomain bool, ...) bool
- func MergeBodyParams(params1 []*discover.RouteBodyParam, params2 []*discover.RouteBodyParam) []*discover.RouteBodyParam
- func MergeQueryParams(params1 []*discover.RouteQueryParam, params2 []*discover.RouteQueryParam) []*discover.RouteQueryParam
- func MergeStaticAssets(staticAssets []string) []string
- func MergeWebRoutes(routes []*discover.RouteDetails) []*discover.RouteDetails
- func NormalizeBaseURLForIdentity(rawURL string) string
- func NormalizeURLForIdentity(rawURL string) string
- func ParseBodyParams(postData string) ([]*discover.RouteBodyParam, error)
- func ParseQueryParams(reqURL *url.URL) []*discover.RouteQueryParam
- func ResolveURL(base, ref string) string
- func SetToListString(set map[string]struct{}) []string
- func SplitURLBaseAndPath(rawURL string) (string, string, error)
- func URLRemoveQueryParams(rawURL string) (string, error)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func AddListToSetString ¶
AddListToSetString adds elements from a list of strings to a set of strings.
func CaptureStaticAssetReference ¶ added in v0.0.279
func CaptureStaticAssetReference(urls map[string]struct{}, scopeURL string, fullURL string, ignoreCrossDomain bool, captureStaticAssets bool) bool
CaptureStaticAssetReference centralizes the static-asset-vs-route decision so every route extractor behaves identically. It reports whether fullURL refers to a static asset (e.g. a PDF, image, or archive), in which case the caller must NOT record it as a route. When it is a static asset and static-asset collection is enabled (and the asset is in scope), the asset is added to the urls set so it surfaces in the StaticAssets output instead. A non-asset URL returns false and is left for the caller to handle as a normal route. scopeURL is the scope anchor (config.Target) used for the in-scope check.
func ExtractDomain ¶
ExtractDomain extracts the domain from a URL up to a specified domain level.
func IsAbsoluteURL ¶
IsAbsoluteURL returns true if the given URL is absolute.
func IsSubdomain ¶
IsSubdomain checks if 'sub' is a subdomain of 'base'.
func IsURLAllowed ¶
func IsURLAllowed(scopeURL string, targetURL string, ignoreCrossDomain bool, captureStaticAssets bool) bool
IsURLAllowed checks if a target URL is allowed based on the scope anchor, host matching, and static-asset policy. When captureStaticAssets is false, static asset URLs (e.g. PDFs, images, archives, and JS bundles) are rejected so they are not recorded as routes or queued for spidering. When ignoreCrossDomain is true, scope is anchored on the full host of scopeURL — which callers set to the original discovery target (config.Target), NOT the per-page post-redirect host — so scope stays tight to the target the user requested: the target host and its subdomains (children) are in scope, while the apex domain and sibling subdomains (e.g. careers.example.com when the target is www.example.com) are treated as out of scope. JS bundle fetching for route discovery is gated separately by MaxBundles in the bundle extractors and intentionally bypasses this allowlist; endpoints discovered inside a bundle are still filtered through IsURLAllowed against the target host.
func MergeBodyParams ¶
func MergeBodyParams(params1 []*discover.RouteBodyParam, params2 []*discover.RouteBodyParam) []*discover.RouteBodyParam
MergeBodyParams merges two slices of RouteBodyParam, retaining unique params and merging example values.
func MergeQueryParams ¶
func MergeQueryParams(params1 []*discover.RouteQueryParam, params2 []*discover.RouteQueryParam) []*discover.RouteQueryParam
MergeQueryParams merges two slices of RouteQueryParam, retaining unique params and merging example values.
func MergeStaticAssets ¶
MergeStaticAssets merges static asset URLs, retaining only unique static assets.
func MergeWebRoutes ¶
func MergeWebRoutes(routes []*discover.RouteDetails) []*discover.RouteDetails
MergeWebRoutes merges WebRoutes, retaining only unique routes (by method and URL).
func NormalizeBaseURLForIdentity ¶ added in v0.12.0
NormalizeBaseURLForIdentity returns a stable origin key for route comparison and result bucketing without changing the URL used for outbound requests. HTTP(S) origins include their effective port so implicit and explicit default ports collapse into the same identity. Userinfo is intentionally excluded.
func NormalizeURLForIdentity ¶ added in v0.12.0
NormalizeURLForIdentity returns a stable request URL key for visited-route deduplication while preserving the original URL for the actual request.
func ParseBodyParams ¶
func ParseBodyParams(postData string) ([]*discover.RouteBodyParam, error)
ParseBodyParams parses body parameters from a JSON or form-urlencoded string into RouteBodyParam structs.
func ParseQueryParams ¶
func ParseQueryParams(reqURL *url.URL) []*discover.RouteQueryParam
ParseQueryParams parses query parameters from a URL into RouteQueryParam structs.
func ResolveURL ¶
ResolveURL resolves a reference URL relative to a base URL.
func SetToListString ¶
SetToListString converts a set of strings to a list of strings.
func SplitURLBaseAndPath ¶ added in v0.0.237
SplitURLBaseAndPath returns the URL origin and escaped path as separate route fields.
func URLRemoveQueryParams ¶
URLRemoveQueryParams removes query parameters from a URL string.
Types ¶
This section is empty.
Source Files
Directories