discoverroute

func AddListToSetString ¶

func AddListToSetString(set map[string]struct{}, list []string) map[string]struct{}

AddListToSetString adds elements from a list of strings to a set of strings.

func ApplyDeclaredRouteTemplates ¶ added in v0.18.0

func ApplyDeclaredRouteTemplates(routes []*discover.RouteDetails) []*discover.RouteDetails

ApplyDeclaredRouteTemplates folds observed literals into templates the application declares.

func CaptureStaticAssetReference ¶ added in v0.0.279

func CaptureStaticAssetReference(urls map[string]struct{}, scopeURL string, fullURL string, ignoreCrossDomain bool, captureStaticAssets bool) bool

CaptureStaticAssetReference centralizes the static-asset-vs-route decision so every route extractor behaves identically. It reports whether fullURL refers to a static asset (e.g. a PDF, image, or archive), in which case the caller must NOT record it as a route. When it is a static asset and static-asset collection is enabled (and the asset is in scope), the asset is added to the urls set so it surfaces in the StaticAssets output instead. A non-asset URL returns false and is left for the caller to handle as a normal route. scopeURL is the scope anchor (config.Target) used for the in-scope check.

func DeclaredParamName ¶ added in v0.18.0

func DeclaredParamName(segment string) (string, bool)

DeclaredParamName returns the parameter a declaration segment names, if it names one.

func EvidenceRank ¶ added in v0.18.0

func EvidenceRank(evidence *string) int

EvidenceRank orders a declaration above an interpolation above where the content was found.

func ExtractDomain ¶

func ExtractDomain(rawURL string, maxDomainLevel int) string

ExtractDomain extracts the domain from a URL up to a specified domain level.

func HasDeclaredParamSyntax ¶ added in v0.18.0

func HasDeclaredParamSyntax(path string) bool

HasDeclaredParamSyntax reports parameter syntax even in forms that yield no usable template.

func HasLiteralSegment ¶ added in v0.20.0

func HasLiteralSegment(segments []string) bool

HasLiteralSegment reports whether any segment is fixed rather than a placeholder.

func InterpolatedParamName ¶ added in v0.18.0

func InterpolatedParamName(expression string, segmentPrefix string, segmentIndex int) string

InterpolatedParamName derives a name; unlike a declaration it is a best effort, not authoritative.

The preceding path segment is deliberately not used. It is as often a verb or a generic token as a resource, which produced names like lockId and idId, and it varies with the call site so one route family fragments into differently named duplicates. A positional name is stable and does not claim a meaning the bundle never carried.

func IsAbsoluteURL ¶

func IsAbsoluteURL(u string) bool

IsAbsoluteURL returns true if the given URL is absolute.

func IsKnownProviderNoiseURL ¶ added in v0.17.0

func IsKnownProviderNoiseURL(rawURL string) bool

IsKnownProviderNoiseURL reports URL-only provider infrastructure endpoints that should never be returned as application routes.

func IsKnownProviderRouteNoise ¶ added in v0.17.0

func IsKnownProviderRouteNoise(route *discover.RouteDetails) bool

IsKnownProviderRouteNoise reports provider-generated routes that may need request context beyond the URL itself to identify safely.

func IsProvenanceEvidence ¶ added in v0.18.0

func IsProvenanceEvidence(evidence string) bool

IsProvenanceEvidence reports evidence recording how a route was derived, not where it was found.

func IsSubdomain ¶

func IsSubdomain(baseURL string, targetURL string) bool

IsSubdomain checks if 'sub' is a subdomain of 'base'.

func IsTemplatedPath ¶ added in v0.18.0

func IsTemplatedPath(path string) bool

IsTemplatedPath reports a declaration rather than a URL that can be requested.

func IsURLAllowed ¶

func IsURLAllowed(scopeURL string, targetURL string, ignoreCrossDomain bool, captureStaticAssets bool) bool

IsURLAllowed checks if a target URL is allowed based on the scope anchor, host matching, and static-asset policy. When captureStaticAssets is false, static asset URLs (e.g. PDFs, images, archives, and JS bundles) are rejected so they are not recorded as routes or queued for spidering. When ignoreCrossDomain is true, scope is anchored on the full host of scopeURL — which callers set to the original discovery target (config.Target), NOT the per-page post-redirect host — so scope stays tight to the target the user requested: the target host and its subdomains (children) are in scope, while the apex domain and sibling subdomains (e.g. careers.example.com when the target is www.example.com) are treated as out of scope. JS bundle fetching for route discovery is gated separately by MaxBundles in the bundle extractors and intentionally bypasses this allowlist; endpoints discovered inside a bundle are still filtered through IsURLAllowed against the target host.

func IsUnrootedEvidence ¶ added in v0.20.0

func IsUnrootedEvidence(evidence *string) bool

IsUnrootedEvidence reports a candidate awaiting rooting.

func MergeBodyParams ¶

func MergeBodyParams(params1 []*discover.RouteBodyParam, params2 []*discover.RouteBodyParam) []*discover.RouteBodyParam

MergeBodyParams merges two slices of RouteBodyParam, retaining unique params and merging example values.

func MergePathParams ¶ added in v0.18.0

func MergePathParams(params1 []*discover.RoutePathParam, params2 []*discover.RoutePathParam) []*discover.RoutePathParam

MergePathParams merges two slices of RoutePathParam, retaining unique params and merging example values.

func MergeQueryParams ¶

func MergeQueryParams(params1 []*discover.RouteQueryParam, params2 []*discover.RouteQueryParam) []*discover.RouteQueryParam

MergeQueryParams merges two slices of RouteQueryParam, retaining unique params and merging example values.

func MergeStaticAssets ¶

func MergeStaticAssets(staticAssets []string) []string

MergeStaticAssets merges static asset URLs, retaining only unique static assets.

func MergeWebRoutes ¶

func MergeWebRoutes(routes []*discover.RouteDetails) []*discover.RouteDetails

MergeWebRoutes merges WebRoutes, retaining only unique routes (by method and URL).

func NormalizeBaseURLForIdentity ¶ added in v0.12.0

func NormalizeBaseURLForIdentity(rawURL string) string

NormalizeBaseURLForIdentity returns a stable origin key for route comparison and result bucketing without changing the URL used for outbound requests. HTTP(S) origins include their effective port so implicit and explicit default ports collapse into the same identity. Userinfo is intentionally excluded.

func NormalizeDeclaredTemplate ¶ added in v0.18.0

func NormalizeDeclaredTemplate(path string) (string, []*discover.RoutePathParam, bool)

NormalizeDeclaredTemplate rewrites a declaration into the `{name}` form ConstructURL substitutes.

func NormalizeURLForIdentity ¶ added in v0.12.0

func NormalizeURLForIdentity(rawURL string) string

NormalizeURLForIdentity returns a stable request URL key for visited-route deduplication while preserving the original URL for the actual request.

func ParseBodyParams ¶

func ParseBodyParams(postData string) ([]*discover.RouteBodyParam, error)

ParseBodyParams parses body parameters from a JSON or form-urlencoded string into RouteBodyParam structs.

func ParseQueryParams ¶

func ParseQueryParams(reqURL *url.URL) []*discover.RouteQueryParam

ParseQueryParams parses query parameters from a URL into RouteQueryParam structs.

func RenumberPositionalParams ¶ added in v0.20.0

func RenumberPositionalParams(template string, params []*discover.RoutePathParam) (string, []*discover.RoutePathParam)

RenumberPositionalParams renumbers synthesized names against the final path, so the same endpoint written with and without an interpolated base yields the same names instead of splitting in two.

func ResolveURL ¶

func ResolveURL(base, ref string) string

ResolveURL resolves a reference URL relative to a base URL.

func ResolveUnrootedInterpolatedRoutes ¶ added in v0.20.0

func ResolveUnrootedInterpolatedRoutes(routes []*discover.RouteDetails) []*discover.RouteDetails

ResolveUnrootedInterpolatedRoutes roots interpolated candidates whose base was unresolvable, using the paths the application is already known to serve.

A candidate carries the path that followed the interpolated base, so `${base}/basket/{id}` arrives as `/basket/{id}`. Whether that is the real path depends entirely on what the base held, which the bundle may never state. Corroborating against observed routes answers it without assuming: if the application serves paths under `/basket` the candidate is already rooted, and if it instead serves `/api/v2/basket` the candidate is missing that prefix.

func SetToListString ¶

func SetToListString(set map[string]struct{}) []string

SetToListString converts a set of strings to a list of strings.

func SplitURLBaseAndPath ¶ added in v0.0.237

func SplitURLBaseAndPath(rawURL string) (string, string, error)

SplitURLBaseAndPath returns the URL origin and escaped path as separate route fields.

func URLRemoveQueryParams ¶

func URLRemoveQueryParams(rawURL string) (string, error)

URLRemoveQueryParams removes query parameters from a URL string.

func UniqueParamName ¶ added in v0.18.0

func UniqueParamName(name string, taken map[string]struct{}) string

Names must be unique per route: the ontology keys on (parameter_name, parameter_location).

func UnrootedEvidenceFor ¶ added in v0.20.0

func UnrootedEvidenceFor(baseName string) string

UnrootedEvidenceFor tags a candidate with the base it was built from, so candidates sharing a base can share whatever prefix corroboration establishes for it.