modkit

func AnnotatePathBypassFinding ¶

func AnnotatePathBypassFinding(res *output.ResultEvent, bypassPath string)

AnnotatePathBypassFinding marks an already-confirmed finding as reached through a reverse-proxy path-normalization bypass: it records the bypass path in the evidence, appends the path-normalization / acl-bypass tags and a reference, and notes the bypass in the name and description. The module's own marker match and FP guards already ran on the bypass response; this only annotates.

func BasePathClaimKey ¶

func BasePathClaimKey(host, base string) string

BasePathClaimKey builds the per-scan dedup key for a (host, base path) pair so a base prefix shared by many observed requests is probed exactly once by a given module. The NUL separator cannot appear in a host or path, so distinct pairs never collide. Pass the result to a module's dedup DiskSet.IsSeen, which atomically claims the pair (returns false and marks it on first sight, true thereafter) — this both de-duplicates across requests and prevents two concurrent workers from probing the same base.

func BodiesSimilar ¶

func BodiesSimilar(a, b string) bool

BodiesSimilar reports whether two response bodies are textually equivalent (QuickRatio >= UpperRatioBound), ignoring status — callers that also care about status gate on it separately. Two empty bodies are treated as similar. It is the body-only counterpart to RatioSimilar (which additionally requires an equal status) and centralizes the "same page?" check shared by the re-confirmation gates (jwt/csrf accepted-as-baseline, forbidden/nginx/firebase stability).

func BodiesSimilarSig ¶

func BodiesSimilarSig(aSig ResponseSignature, b string) bool

BodiesSimilarSig is the signature-reuse form of BodiesSimilar: it compares a body whose signature was precomputed (via BodySignature) against a fresh body b, tokenizing only b.

func BodyHasSQLError ¶

func BodyHasSQLError(body string) bool

BodyHasSQLError is the boolean form of MatchSQLError for callers that only need to know whether the body leaked a database error.

func BodyHasStackTrace ¶

func BodyHasStackTrace(body string) bool

BodyHasStackTrace reports whether body carries any recognized stack trace. It is the gate-only counterpart to iterating StackTracePatterns, mirroring BodyHasSQLError beside MatchSQLError.

Two screens run before the regexes: the body is capped to its leading stackTraceBodyScanLimit bytes, and it must contain at least one cheap literal hint. Matching stops at the first pattern that hits, so a caller wanting only the yes/no answer never pays for the rest.

func CandidateBasePaths ¶

func CandidateBasePaths(p string) []string

CandidateBasePaths returns the base path prefixes under which a context-path- mounted application may expose a known endpoint. The result is the web root ("") first, then ancestor directories of the observed path shallowest-first, then the observed path itself when it looks like a directory (no file extension), e.g.:

"/api/v1/users" -> ["", "/api", "/api/v1", "/api/v1/users"]
"/myapp"        -> ["", "/myapp"]          (single-segment context root)
"/index.html"   -> [""]                     (a file, not a mount point)
"/assets/app.js"-> [""]                     (static-asset dirs are skipped)

Callers append their endpoint suffix (e.g. "/actuator/env") to each base; the "" root yields the original root-anchored probe unchanged, so adopting this helper never loses the root coverage a module already had. Static-asset directories (/assets, /static, /css, ...) are skipped because known endpoints are never mounted under them, and the list is capped at MaxBasePathDepth to bound fan-out. Results are de-duplicated and order-stable (shallowest first).

func CandidateBasePathsIncludingStatic ¶

func CandidateBasePathsIncludingStatic(p string) []string

CandidateBasePathsIncludingStatic is CandidateBasePaths but keeps static-asset directory prefixes (/assets, /static, a CDN/object-storage path, ...). Use it for sensitive-file discovery, where a misconfigured static or CDN-fronted directory is exactly where an exposed .env / .git / backup is found — the one place a known-endpoint probe would never look. Known-endpoint modules should use CandidateBasePaths instead, which skips these.

func CapSeverity ¶

func CapSeverity(s, max severity.Severity) severity.Severity

CapSeverity returns s clamped to at most max (severity is an ordered int, so Critical/High collapse to a Medium ceiling while anything already at or below max passes through unchanged). It is the shared knob the heuristic, FP-prone path-probe families use to report at a fixed ceiling regardless of how sensitive the underlying artifact would be if the match were genuine.

func CollapseFindings ¶

func CollapseFindings(results []*output.ResultEvent, spec CollapseSpec) []*output.ResultEvent

CollapseFindings folds findings that share a Key into ONE ResultEvent per group, so a probe/fuzz module that confirms the same logical issue many ways writes a single http_records row per issue instead of one per probe. (The executor records one row per emitted finding's Request; folded findings ride along in the primary's AdditionalEvidence, which is stored inline on the finding and creates no row.)

Group order follows first appearance, so output ordering is deterministic. Within a group the highest-Rank finding stays primary; the rest are folded into its AdditionalEvidence — labeled, ordered best-rank-first, and capped at MaxEvidencePairs (so the cap drops the least informative pairs). nil findings are skipped. With a nil Key (or empty input) the input is returned unchanged.

func ConfirmMatchReproduces ¶

func ConfirmMatchReproduces(
	ctx *httpmsg.HttpRequestResponse,
	ip httpmsg.InsertionPoint,
	httpClient *http.Requester,
	payloadRaw []byte,
	re *regexp.Regexp,
) bool

ConfirmMatchReproduces re-confirms that a regex-matched server-side error/leak signature was genuinely introduced by an injected payload rather than ambient page noise (a per-request random token, a static error string, or a signature a stale baseline happened to miss). It is the shared confirmation gate behind the error-based injection modules (sqli-error-based, nosqli-error-based, …).

re must (1) reproduce when payloadRaw is re-sent on a genuine application error surface — non-blocked and not a 404/redirect, so a coincidental match against a token that differs on the re-send fails this — and (2) be ABSENT from a fresh, non-blocked control fetch of the insertion point's original value, so an error the endpoint returns for ANY input is rejected even when the captured baseline missed it.

It fails open (returns true) on a transport error so a transient failure never suppresses a true positive, and drops the finding (returns false) when the re-fetch is blocked or not an error surface, or the control is blocked: such a page can neither reproduce the leak nor serve as a clean control. A nil re is treated as "nothing to re-confirm" and passes.

func ConfirmNotSoft404 ¶

func ConfirmNotSoft404(
	sc *ScanContext,
	client *http.Requester,
	probeReq *httpmsg.HttpRequestResponse,
	statusCode int,
	body []byte,
	location string,
) bool

ConfirmNotSoft404 reports whether a marker-matched response is a genuine hit rather than a soft-404 / SPA wildcard shell or a redirect to an auth page. It returns true when the response looks like a real, specific resource.

• probeReq: any request to the same host (used to fetch and cache the host's wildcard fingerprint via WildcardProbe).
• statusCode/body: the response that matched the module's content marker.
• location: the matched response's Location header (may be empty).

On a wildcard-probe error it fails open (returns true) so a transient probe failure does not suppress a real finding.

func ConfirmReflection ¶

func ConfirmReflection(rounds int, probe ReflectionProbe) (bool, error)

ConfirmReflection runs probe across `rounds` (minimum 2) with a fresh random alphanumeric canary (FreshCanary) each round, and returns true only if EVERY round observed the canary reflected. Using a fresh random canary per round makes a coincidental static match astronomically unlikely, so this doubles as a payload-applied-vs-not differential: a value that genuinely flows from input to output reflects every time, while a page that merely happens to contain a fixed string does not track the changing canary.

func ConfirmReflectionWithValue ¶

func ConfirmReflectionWithValue(rounds int, genValue func() string, probe ReflectionProbe) (bool, error)

ConfirmReflectionWithValue is ConfirmReflection with a caller-supplied value generator instead of the built-in alphanumeric FreshCanary: each round it draws a fresh value from genValue, passes it to probe, and returns true only if EVERY round observed it reflected. Use this when the probed field constrains the value SHAPE — a port that must stay numeric, a structured token — so a generic FreshCanary would not exercise the same code path. genValue must return a distinct value each call (e.g. a random port / canary) for the tracking guarantee to hold.

func ContentClassAllows ¶

func ContentClassAllows(required []string, class ContentClass) bool

ContentClassAllows reports whether a module requiring the given content classes may run against a response of the observed class. It fails open: an unknown or plain-text response always runs (text/plain can carry mislabeled markup that a browser still treats as a document only when sniffed, so it is never hard-excluded). A confirmed structured class (html/json/xml/binary) that is not in the required set is the only case that skips.

func CtxRequestRaw ¶

func CtxRequestRaw(ctx *httpmsg.HttpRequestResponse) string

CtxRequestRaw renders ctx's request as a raw string for evidence capture, returning "" when the request is absent. Pair with CtxResponseRaw to record the original (clean) request/response a module compared its probe against.

func CtxResponseRaw ¶

func CtxResponseRaw(ctx *httpmsg.HttpRequestResponse) string

CtxResponseRaw renders ctx's response as a full raw response string for evidence capture, returning "" when no response is present.

func DecoyFileBaseline ¶

func DecoyFileBaseline(
	sc *ScanContext,
	ctx *httpmsg.HttpRequestResponse,
	client *http.Requester,
	probePath string,
) (status int, body string, ok bool)

DecoyFileBaseline fetches a guaranteed-nonexistent file that shares probePath's parent directory AND file extension, and returns its response status and body for baseline subtraction. It is the file-discovery baseline companion to SiblingPathCatchAll: a directory (or extension-specific handler) that answers every <dir>/<anything>.<ext> request with the same content — a SPA fallback, a logging proxy, a catch-all object store — returns that content for the decoy too, so an exposed-file candidate whose body is textually equivalent to the decoy (BodiesSimilar) is a false positive. This is exactly the "/orders/run.log and /orders/<random>.log return the same body" case.

The decoy keeps the candidate's extension because catch-alls are frequently extension-scoped (a *.log handler, a static *.json fallback); a no-extension probe would miss them. ok is false on any build/transport error or a missing response, so a flaky decoy fetch never suppresses a real finding (fail-open). The probe runs with NoClustering so the host's per-request response cache does not alias it to the candidate's own cached entry.

func DetectDirectoryListingServer ¶

func DetectDirectoryListingServer(body string) string

DetectDirectoryListingServer classifies body as an auto-generated directory listing and returns the emitting server ("Jetty", "IIS", "Apache", "Nginx", "Generic"), or "" when body is not a listing. It is the shared classifier behind the active and passive directory-listing modules.

Two guards keep it from firing on ordinary pages:

• LooksLikeAppPage rejects rendered application/CMS/SPA content up front, so a framework page with a heading + table + links is never mistaken for a listing.
• the Generic (title-only) branch additionally requires a listing STRUCTURE (hasListingStructure), so a content page merely titled "Directory of X" / "Index of X" does not false-positive. The server-specific branches already require two corroborating markers (title + heading / <pre> / css) and are left as-is.

func DifferentialSurfaceUnreliable ¶

func DifferentialSurfaceUnreliable(resp *httpmsg.HttpResponse) bool

DifferentialSurfaceUnreliable reports whether resp is a surface on which a boolean / true-false body differential cannot be trusted:

• a cache / CDN layer fronts the origin (infra.CacheState().Layer), so a cached body may answer one probe and an origin render the next; or
• the body is a large rendered text/html content page (>= LargeDynamicHTMLBytes), whose per-request dynamic content dwarfs any real true/false difference.

Differential-oracle modules — boolean-based SQLi / NoSQLi / LDAPi — should skip such surfaces before probing. It reads only response headers and the captured body and sends no traffic. A nil response is treated as reliable (fail open) so a caller without a captured baseline still runs.

func DocrootFetchInfo ¶

func DocrootFetchInfo(targetFile string) output.Info

DocrootFetchInfo builds the informational Info block for a file that turned out to be published under the web root and fetchable at its own URL, rather than read through an inclusion sink. targetFile names the file that was retrieved (".htaccess", "WEB-INF/web.xml", …).

Modules use this so the direct-fetch case is neither silently dropped nor reported at file-inclusion severity — it is a real, if minor, observation, and one that a High/Critical label badly misrepresents.

func DocrootFetchNotInclusion ¶

func DocrootFetchNotInclusion(ipType httpmsg.InsertionPointType, payload string) bool

DocrootFetchNotInclusion reports whether a file-read payload can only ever produce a direct HTTP fetch of a published file rather than evidence of server-side inclusion, given the insertion point it would be injected into.

Replacing a URL path segment with `.htaccess` yields `GET /.htaccess`; the `../` variants normalize back to exactly the same request. When the origin genuinely publishes that file the server answers 200 with real file content, and a body-only confirm step cannot tell a filesystem read apart from an ordinary static fetch. Callers must therefore treat the result as *exposure*, not inclusion: report it with DocrootFetchInfo instead of the module's own file-inclusion severity.

func DocrootPublishableTarget ¶

func DocrootPublishableTarget(payload string) bool

DocrootPublishableTarget reports whether a file-read payload names a file a web root can legitimately publish (see docrootPublishableTargets), ignoring any leading traversal. It is a positive allowlist keyed on the target file, so an unrecognised payload — a stream wrapper such as `php://filter/convert.base64-encode/resource=index.php`, or a target added later — falls through as *not* publishable and keeps the module's own severity. That is the safe default: the cost of a miss is the old false positive returning for one payload, never a real inclusion silently downgraded.

func DriveProbesWithBypass ¶

func DriveProbesWithBypass[P any](
	bases []string,
	probes []P,
	observedPath string,
	name func(P) string,
	endpointPath func(P) string,
	bypassEligible func(P) bool,
	probe func(p P, probePath string) (*output.ResultEvent, int),
) []*output.ResultEvent

DriveProbesWithBypass runs the standard base-walk probe loop plus the reverse- proxy path-normalization bypass fallback shared by the marker-based path-probe modules (Tomcat, the Spring family). It collapses the identical bookkeeping each of those modules' ScanPerRequest otherwise hand-rolls: for every (base, probe) it calls probe(); then, once per host (the web root "" is present and first only on a host's first scan), it tries the bypass forms for each bypass-eligible probe whose direct ROOT call returned a proxy-block status (401/403/405) and was not already served. Behaviour matches the per-module loops it replaces, including the once-per-host gate and the "skip the bypass when the endpoint is already directly reachable" rule (a root hit records rootHit, suppressing the redundant bypass).

P is the module's own probe type; the accessor funcs read its fields and probe issues the request (the module's probeEndpoint, returning the finding-or-nil and the HTTP status). endpointPath is the probe's path; name keys the per-probe root-status/hit maps; bypassEligible selects the admin endpoints worth bypassing.

func EndpointVolatile ¶

func EndpointVolatile(sc *ScanContext, ctx *httpmsg.HttpRequestResponse, client *http.Requester) bool

EndpointVolatile reports whether the endpoint answers the SAME unmodified request with materially different responses across repeated fetches.

It exists because every body-differential oracle — boolean-blind SQLi/NoSQLi, path-normalization's "the traversal reached a different resource than the clean path", cross-ID comparisons — implicitly assumes the endpoint is a function of the request. A host whose load balancer round-robins between mismatched backends breaks that assumption outright: the observed false positive was a pool of parked hosts where an identical `GET /` returned either a 5.9 KB "Red Hat Enterprise Linux Test Page" or an 11.3 KB "Apache2 Ubuntu Default Page" at random. Every pairwise guard those modules run — TRUE vs FALSE, probe vs baseline, and each single-retry stability check — is then a coin flip, and across a scan's payload × insertion-point fan-out some sequence of flips inevitably lines up into a "confirmed" High finding.

No amount of pairwise re-comparison fixes that; the endpoint has to be proven deterministic once, from samples of the unmodified request alone. Callers should invoke this as a LAST gate, at the moment they are about to report, rather than up front: candidates are rare, the verdict is memoized per record (and shared across modules processing that record), so a scan pays the extra round-trips only on hosts that actually produced something.

It fails OPEN — a nil context/client, a build error, or any transport failure returns false ("not proven volatile") — so a flaky probe never suppresses a genuine finding. Replays carry the observed request verbatim, including its method and body; callers on non-idempotent requests inherit that, which is the same trade the differential modules already make when they replay a request dozens of times with mutated values.

func ExecuteRaw ¶

func ExecuteRaw(client *http.Requester, service *httpmsg.Service, rawReq []byte, opts http.Options) (status int, body string, ok bool)

ExecuteRaw parses a raw request, binds it to service (when non-nil), executes it with the given options, and returns the response status and body. ok is false on any build/transport error or a nil response. It centralizes the re-confirmation fetch idiom — notably the NoClustering discipline — shared by the active modules' confirmation helpers, so each call site no longer re-derives (and risks forgetting) it.

func ExecuteRawWithResponse ¶

func ExecuteRawWithResponse(client *http.Requester, service *httpmsg.Service, rawReq []byte, opts http.Options) (status int, body, fullResponse string, ok bool)

ExecuteRawWithResponse is ExecuteRaw that ALSO returns the full raw response (status line + headers + body) alongside the body, from a single request. A module uses the body for its content/similarity checks and the full response as the finding's proving evidence, so it no longer has to issue the request twice (and no longer depends on the executor backfilling a — possibly mismatched — baseline response).

func FetchPath ¶

func FetchPath(ctx *httpmsg.HttpRequestResponse, client *http.Requester, probePath string) (status int, body string, ok bool)

FetchPath issues a fresh GET to probePath carrying the observed request's headers and service, with the response cache bypassed (NoClustering) so a reproduce/re-confirmation probe is never aliased to a cached entry. It returns the response status and body; ok is false on any build/transport error so a flaky fetch never suppresses a finding (fail-open). It is the shared re-fetch primitive behind the file-discovery confirmation rounds.

func FreshCanary ¶

func FreshCanary() string

FreshCanary returns a short random alphanumeric token (alpha-leading, safe to embed in URLs, headers, and HTML) for reflection probes.

func FreshMultExpr ¶

func FreshMultExpr() (expr, product string)

FreshMultExpr returns a fresh random multiplication expression ("A*B") and its product, both as decimal strings, for evaluation-oracle confirmation (SSTI / OGNL): re-inject the expression and confirm the product appears, so the result tracks the changing operands and a fixed number already in the page cannot match. Operands are 4 digits so the product stays well under Java's 32-bit Integer.MAX_VALUE — OGNL evaluates int*int with 32-bit semantics, and an overflowed product would render wrapped and never match.

func HasJSExtension ¶

func HasJSExtension(pathLower string) bool

HasJSExtension returns true if the URL path ends with a JS/TS extension.

func HasParentDirLink ¶

func HasParentDirLink(lowerBody string) bool

HasParentDirLink reports whether lowerBody links to a parent directory ("../" or a "Parent Directory" label) — the near-universal entry of a real autoindex and a strong structural signal that a <table>/<pre>/<ul> of links is a generated file index rather than page content. lowerBody MUST already be lowercased (see LooksLikeAppPage for the rationale).

func HasSalesforceConsumerKeyPrefix ¶

func HasSalesforceConsumerKeyPrefix(s string) bool

HasSalesforceConsumerKeyPrefix reports whether s is a Salesforce Connected App Consumer Key — the `3MVG9…` OAuth client_id.

A Consumer Key is the PUBLIC half of a Connected App: it is placed in every /services/oauth2/authorize URL the browser is redirected to, so a front end that starts a Salesforce login flow must ship it to the client. Surrounding whitespace is trimmed, since callers pass values sliced out of a page.

func HasStaticAssetExtension ¶

func HasStaticAssetExtension(path string) bool

HasStaticAssetExtension reports whether a URL path ends in a known static-asset file extension (.js, .css, .map, fonts, images, archives, ...). Unlike IsStaticAssetPath it does NOT match by directory segment, so it won't classify an HTML document served from /assets/ or /public/ as static. Use it where precision matters — e.g. deciding whether to truncate a stored response body.

func HasSubstantialBodyDifference ¶

func HasSubstantialBodyDifference(a, b ResponseSignature) bool

HasSubstantialBodyDifference reports a large, content-driven length gap: the bodies must differ by both an absolute (>SubstantialBodyDeltaBytes) and a relative (>=SubstantialBodyDeltaRatio) margin.

func HostReflectedAsAuthority ¶

func HostReflectedAsAuthority(s, host string) bool

HostReflectedAsAuthority reports whether host appears as the authority (host component) of a URL inside s — the position that actually controls where a generated redirect or link points — rather than as an incidental substring, such as a value inside a query parameter (an OAuth redirect_uri=, a next=/continue= target) or a URL-encoded %2F%2Fhost echo.

Host-injection vulns (password-reset link poisoning, cache poisoning, open redirect) require the client-supplied host to BECOME the authority of a generated URL. A recurring false-positive class is the inverse: behind a CDN, an OAuth login flow reflects X-Forwarded-Host / Host into the redirect_uri= query parameter of a 3xx whose authority stays the trusted IdP — the real destination never changes with or without the spoofed header, so there is nothing to poison, yet a bare strings.Contains over the Location/header dump flagged it. Requiring authority position drops that echo while still catching a genuine "Location: https://<host>/..." or an absolute link in the body.

A match qualifies only when host is immediately preceded by "//" (after a "scheme://" or as a protocol-relative "//host") and is the whole authority label — the byte after it ends the authority (a port ':', path '/', query '?', fragment '#', end of string, or a quote/angle/backslash/whitespace boundary in a body), not a "." that would make host merely a prefix of <host>.evil.com. URL-encoded slashes (%2F%2F) deliberately do NOT qualify: an encoded host sits inside another URL's query, not in authority position.

func IsDifferent ¶

func IsDifferent(a, b ResponseSignature) bool

IsDifferent returns true if two signatures are meaningfully different by the fast length/hash/status pre-filter (a different status, or a >100 byte / >20% body-length gap). It does not consider token similarity — callers wanting the dynamic-content-robust check should also use QuickRatio.

func IsEdgeBlockedResponse ¶

func IsEdgeBlockedResponse(resp *httpmsg.HttpResponse) bool

IsEdgeBlockedResponse reports whether resp came from a WAF/CDN edge block or interstitial challenge rather than the application. It is the passive-module counterpart to infra.IsBlockedResponse, which requires a *httputil.ResponseChain that passive modules do not hold: this works against the raw httpmsg response (status + Server/X-CDN/Cf-Mitigated headers + body markers).

A content/value-leak detector that flags a secret, token, version string, stack trace, or sensitive field by matching the RESPONSE body or headers MUST skip such a response: a denied/challenged page is the edge talking, not the application, so any token it carries can only be a false positive (the motivating case: an "Authorization" finding on a CloudFront "Request blocked" 403, and the broader Cloudflare-challenge-token-as-SQLi class).

It deliberately recognizes only vendor-identified edge blocks (Server/CDN headers, challenge markers, 429). It does NOT treat a generic application 401/403 as a block, so a module whose finding legitimately rides a protected or app-rendered error response keeps it. Do NOT use it in a module whose finding IS a blocking status (an access-control 403 check) — it would suppress the very response that module looks for.

func IsJSOrTSContentType ¶

func IsJSOrTSContentType(contentType string) bool

IsJSOrTSContentType returns true if the content type indicates JavaScript or TypeScript.

func IsNonHTMLReflectionContext ¶

func IsNonHTMLReflectionContext(contentType string) bool

IsNonHTMLReflectionContext reports whether a response Content-Type declares a concrete non-HTML type, in which case an injected HTML/template marker is only a data value (a JSON string, an XML text node), never rendered markup — so a reflected-marker match is NOT evidence of HTML/CSTI/XSS injection. HTML-family types (text/html, application/xhtml+xml) and an empty/unknown Content-Type return false so genuine HTML reflections and header-less responses still qualify. This is the canonical FastAPI/Pydantic 422 application/json validation-error echo false positive: the bad parameter (markup intact) is echoed back inside the JSON `input` field but never rendered as HTML.

Use this to gate markup-REFLECTION oracles (a raw Contains(body, payload)). Do NOT use it for evaluated-result oracles (SSTI/OGNL computed products, SQL error strings, command-output needles) — those legitimately appear in JSON.

func IsPlaceholderValue ¶

func IsPlaceholderValue(v string) bool

IsPlaceholderValue reports whether v is empty, whitespace-only, or a well-known null-ish placeholder/sentinel (null, undefined, nil, none, nan, [object Object], {}, []) rather than a real value. Comparison is trimmed and case-insensitive.

func IsProxyBlockedStatus ¶

func IsProxyBlockedStatus(status int) bool

IsProxyBlockedStatus reports whether a direct-probe HTTP status looks like a fronting proxy's access-control block (401/403/405) rather than a clean not-present (404) or a server/transport error. It is the gate for whether a path-normalization bypass is worth attempting: a bypass only pays off when something IS in front of the endpoint denying the direct path.

func IsReCaptchaSiteKeyShape ¶

func IsReCaptchaSiteKeyShape(s string) bool

IsReCaptchaSiteKeyShape reports whether s has the Google reCAPTCHA site-key shape: the literal prefix "6L" followed by 38 URL-safe-base64 characters (40 total). Both reCAPTCHA v2 and v3 site keys use this format.

The site key is rendered into the widget on every protected page — it is the half the browser must have; the secret key is the server-side half.

Written as a byte scan rather than a regexp because secret_detect calls it per surviving match while grading, and this keeps it allocation-free.

func IsStaticAssetContentType ¶

func IsStaticAssetContentType(contentType string) bool

IsStaticAssetContentType reports whether the Content-Type header value belongs to a static asset / binary payload (script, style, font, image, media, archive) — content where a substring/regex match of an application-level signal is not trustworthy evidence. An empty/unknown content type returns false, so callers that depend on body inspection still run when the server omits the header. Parameters (e.g. "; charset=utf-8") are ignored.

func IsStaticAssetPath ¶

func IsStaticAssetPath(path string) bool

IsStaticAssetPath reports whether a URL path is a static-asset route, by file-extension suffix OR by containing a known static directory segment (e.g. /css/images has no extension but both segments are static). The segment check catches extensionless asset routes that a suffix-only test misses. This is the URL-path companion to IsStaticAssetContentType (header-based).

func IsURLPathInsertionPoint ¶

func IsURLPathInsertionPoint(t httpmsg.InsertionPointType) bool

IsURLPathInsertionPoint reports whether t addresses a segment of the URL path (a folder or the filename) rather than a parameter, header, cookie or body value.

The distinction matters to every file-read module, because a payload injected into a URL path segment is not injected into anything the application interprets — it simply becomes the request target. `AllParamTypes` and `URLParamTypes` both include the two path types, so a module that declares either is asked to test path segments whether or not its payloads make sense there. See DocrootFetchNotInclusion.

func LikelySessionCookie ¶

func LikelySessionCookie(name string) bool

LikelySessionCookie reports whether a cookie name carries authentication or session state. CSRF tokens are explicitly excluded.

func LooksLikeAppPage ¶

func LooksLikeAppPage(lowerBody string) bool

LooksLikeAppPage reports whether lowerBody is a rendered application / CMS / SPA content page rather than a bare, machine-emitted response such as a directory listing. Directory-listing detectors call it as a negative guard: a real serve-index / Nginx / Apache / IIS autoindex never carries framework, static-site-generator, or SEO/social markers, so their presence rules a listing out — killing the class of false positive where a content page (a Gatsby/Next.js route, a "Directory of X" landing page) has an <h1>, a <table>, and <a href=> links yet is not a listing.

lowerBody MUST already be lowercased (via strings.ToLower). Callers process the same response body through several of these classifiers, so lowercasing is hoisted to the caller to avoid re-copying a large body per check.

func MatchAllGroups ¶

func MatchAllGroups(body string, requireAll [][]string) (matched []string, ok bool)

MatchAllGroups reports whether body satisfies every group in requireAll — it must contain at least one substring from EACH group (an AND across groups, OR within each group). This is the structural-confirmation primitive shared by the path-probing exposure modules: instead of firing on any single weak substring (which catch-all / SPA / i18n handlers trivially contain — e.g. a bare "status", "name", or "filter"), a probe lists one group of strong anchor synonyms plus one or more corroborating groups, so a finding requires the co-occurrence that only the real endpoint emits.

matched returns the specific substring hit from each group (suitable for ExtractedResults / finding evidence). ok is false — and matched nil — if any group has no hit, or requireAll is empty (a probe with no confirmation tokens never fires).

func MatchAndConfirmSibling ¶

func MatchAndConfirmSibling(
	ctx *httpmsg.HttpRequestResponse,
	client *http.Requester,
	probePath, body string,
	markers [][]string,
) (matched []string, ok bool)

MatchAndConfirmSibling is the combined marker-match + catch-all guard used by the marker-based path-probing exposure modules. It confirms body satisfies the marker groups (MatchAllGroups), then drops the finding if a guaranteed- nonexistent sibling under the same parent directory returns the same markers (SiblingPathCatchAll) — a catch-all handler that 200s every child path. Root- level probe paths are already covered by the caller's root soft-404 fingerprint, so the sibling probe is a no-op for them. Finally, when the framework anchor is merely the probe's own last path segment echoed back, it applies a slug-reflection control (PathSegmentReflected) so a content route that renders the requested slug (/topic/<slug> -> "<slug> …") does not self-confirm on the reflected word.

matched carries the evidence substrings for ExtractedResults; ok is false (and matched nil) when the body doesn't satisfy the groups, the sibling reveals a catch-all, or the anchor is only a reflected path slug.

func MatchSQLError ¶

func MatchSQLError(body string) (string, *regexp.Regexp, bool)

MatchSQLError reports whether body leaked a DBMS error and, if so, returns the backend name and the matching pattern. It accepts only a plausibly compact match (span-bounded) so an "X.*?Y" pattern that bridged unrelated content in a large page body is rejected as noise. Map iteration order is unspecified, so the returned name is arbitrary when several backends match — callers use it only as a best-effort backend hint, never as an identity.

func MatchWithinSpan ¶

func MatchWithinSpan(re *regexp.Regexp, body string, maxSpan int) bool

MatchWithinSpan reports whether re has SOME match in body whose span is at most maxSpan bytes. It first runs the cheap first-match MatchString (which short-circuits and scans capture-free), so the common no-match path over a large body stays a single lightweight scan, and only enumerates matches (FindAllStringIndex) once at least one exists. Enumerating ALL matches — rather than checking only the leftmost — means a genuine short signature elsewhere in the body is still detected even when the leftmost match happens to be an over-long, blob-spanning coincidence. It is the shared "is this match a plausibly compact error signature, not page noise?" gate for the error-based injection modules.

func MultiRoundExtDecoyCatchAll ¶

func MultiRoundExtDecoyCatchAll(
	ctx *httpmsg.HttpRequestResponse,
	client *http.Requester,
	probePath, candidateBody string,
	candidateStatus, rounds int,
	markerMatch func(body string) bool,
) bool

MultiRoundExtDecoyCatchAll probes `rounds` distinct guaranteed-nonexistent siblings that share probePath's parent directory AND file extension (e.g. /WEB-INF/vigolium-decoy-<rand>.xml for /WEB-INF/web.xml) and reports whether the path is served by an extension-scoped catch-all rather than a real file.

It is the multi-round companion to DecoyFileBaseline, built for the application-shell host that answers EVERY /<dir>/<anything>.<ext> with the same 200 page (a SPA fallback, a reverse-proxy wildcard, a Salesforce-style framework shell). The root soft-404 fingerprint these modules take cannot see such a catch-all — it probes a no-extension path at the web root, and a shell that embeds the request path / a session token differs enough per path to dodge the fingerprint's hash and length checks — so a weak content marker that merely appears somewhere in the shell forges a finding. Requesting /WEB-INF/thisisclearly404.xml and observing the same shell come back is the direct disproof.

A round trips the catch-all when its decoy returns the SAME status as the candidate AND either the decoy body satisfies markerMatch (the same markers the candidate matched) or is textually ~equal (BodiesSimilar) to candidateBody. Returns true (catch-all → drop the candidate) as soon as any round trips; returns false when every round 404s / errors / serves a clearly different body. Each round uses a fresh canary (DecoyFileBaseline) so a per-path response cache cannot alias the decoys to one another. rounds < 1 is treated as 1. Fails OPEN per round (a transport error or missing decoy response cannot prove a catch-all) so a flaky probe never suppresses a real finding.

func NormalizeForRatio ¶

func NormalizeForRatio(body, reflect string) string

NormalizeForRatio lowercases the body, removes reflected input, and collapses dynamic-looking runs (long hex/digit sequences) so they don't add noise to the token multiset.

func NormalizedBodyHash ¶

func NormalizedBodyHash(body string, reflects ...string) string

NormalizedBodyHash returns a stable hex SHA-256 over a response body that has been normalized for dedup: lowercased, with each reflect value (e.g. the request URL and path that an error/echo page mirrors back into the body) and dynamic-looking runs (long hex/digit sequences — timestamps, ids, hashes) collapsed to a space. Two bodies that differ ONLY by the reflected request target or per-request dynamic tokens therefore hash identically, so the single byte differential an echoed URL introduces no longer defeats record dedup (the classic "404/400 page mirrors the requested URI" case where every probed path yields a distinct response_hash + content_length).

An empty/whitespace-only body yields "" — callers skip those: they carry no signal and are already collapsed by exact response-hash dedup.

func PathBypassPrefixes ¶

func PathBypassPrefixes(observedPath string) []string

PathBypassPrefixes returns a deduplicated, depth-scaled set of reverse-proxy path-normalization bypass prefixes. The caller appends the target endpoint suffix (WITHOUT a leading slash, e.g. "manager/html" or "actuator/env") to each prefix to form a candidate request path. The result combines:

• root-anchored single-level encoding variants (pathBypassSingles), which defeat proxies that block by raw prefix without normalizing;
• root-anchored multi-level climbs (pathBypassChainable repeated 2..N), where N scales with the observed path depth — "do multiple /..;/..;/ when the URL is deep";
• a launch from the observed app directory: climb out of the deepest path the proxy already forwarded for the legitimate request (its allow-rule most likely matches that prefix) back to root, then descend to the target. This covers whitelist proxies that only forward known prefixes.

Order is most-likely-first and stable.

func PathSegmentReflected ¶

func PathSegmentReflected(
	ctx *httpmsg.HttpRequestResponse,
	client *http.Requester,
	probePath string,
) bool

PathSegmentReflected reports whether the application echoes an arbitrary requested path segment back into a 200 response body — a slug-reflecting content route (the /topic/<slug> -> "<slug> …" SEO pattern) OR a path-reflecting SPA/CMS shell (a Frontify/brand app whose router renders one 200 shell for every route and embeds the requested slug as a {"view":"<slug>"} router context, a canonical link, a title). It probes a guaranteed-nonexistent canary path under probePath's BASE directory — the parent directory for a multi-segment path, the web root "/" for a single-segment (root-level) path — and reports whether the app served 200 AND reflected that exact canary. Requiring a 200 canary reflection is what separates a real endpoint (whose random siblings 404) from a reflecting route/shell (whose every path renders): if the canary 404s, redirects, or errors, this is NOT a reflecting host and the finding stands.

Root-level paths are probed at the web root rather than skipped: a path-reflecting shell defeats the root soft-404 fingerprint (the reflected slug + a per-request token vary the body per path), so a root-level slug-equal marker self-matches the reflected request path with no endpoint behind it (the branding.acme.com /healthchecks-ui, /redoc false positives). Requiring an EXACT 200 canary reflection keeps this false-negative-safe: a genuine root-level endpoint (/redoc, /h2-console) whose random web-root siblings 404 is never dropped. Runs with NoRedirects (a 30x is not a reflected body) and NoClustering (the control must be a distinct fetch, never aliased to the candidate's cached entry). Fails open (false) on any build/transport error so a flaky control probe never suppresses a real finding.

func ProbePathBypass ¶

func ProbePathBypass(
	observedPath, endpointSuffix string,
	probe func(bypassPath string) *output.ResultEvent,
) *output.ResultEvent

ProbePathBypass drives reverse-proxy path-normalization bypass probing for a single endpoint. For each PathBypassPrefixes(observedPath) it assembles prefix+endpointSuffix and calls probe — reuse the module's own marker-confirming probe so the soft-404 fingerprint, shell guard, marker match and catch-all guard all run on the bypass response. The first non-nil finding is annotated as a path-normalization bypass and returned; nil when no form reaches the endpoint.

endpointSuffix is the target admin path; a leading slash is tolerated.

func QuickRatio ¶

func QuickRatio(a, b ResponseSignature) float64

QuickRatio returns a 0..1 textual similarity between two responses, following difflib.SequenceMatcher.quick_ratio: 2*M / (Ta+Tb) where M is the number of matching tokens (sum of per-token min counts) and Ta/Tb the token totals. Order-independent and cheap. Two empty bodies are treated as identical.

func RandomDirCatchAll ¶

func RandomDirCatchAll(
	sc *ScanContext,
	ctx *httpmsg.HttpRequestResponse,
	client *http.Requester,
	match func(body string) bool,
) bool

RandomDirCatchAll GETs a guaranteed-nonexistent directory (a random path with a trailing slash, at the web root) and reports whether its 2xx body already satisfies match. It is the host-wide catch-all guard for directory-listing probes: a real autoindex/serve-index server returns a listing only for directories that exist on disk and 404s a random non-existent dir, so if a random dir already "lists" (or matches the detector at all), the host renders that body for ANY path — an SPA shell, a wildcard rewrite, a templated soft-404 — and every per-directory finding is spurious. Drop the whole host's findings when this returns true.

The trailing slash matters: directory handlers key on it. Runs with NoRedirects (a 30x is not a served listing) and NoClustering (so the probe is not aliased to a cached entry). Returns false on any build/transport error, a missing/non-2xx response, or a nil match (fail-open — a flaky probe never suppresses a real finding).

func RandomSameDepthPath ¶

func RandomSameDepthPath(p string) string

RandomSameDepthPath returns a guaranteed-nonexistent absolute path whose segment count equals p's, each segment a fresh random canary token. It is the base for a path-mangling "shape catch-all" control: re-applying a module's payload (a collapsing traversal, a prefix/suffix decoration) to this base lands at the same relative depth as the real probe — and, for an upward `../`-family traversal, collapses to the same ancestor/root — so a response that matches the real probe proves the leading segments were irrelevant and the host is serving one body for the mangled SHAPE rather than exposing a resource the clean URL gated. A root or empty p yields a single random segment. Complements the clean-path guards (RandomDirCatchAll, RootPageCatchAll), which cannot see a shape-keyed catch-all.

func RatioSimilar ¶

func RatioSimilar(a, b ResponseSignature) bool

RatioSimilar reports whether two responses are effectively the same page by textual similarity. Used to confirm a response is stable across retries even when the body carries per-request dynamic content.

func RegistryHostKey ¶

func RegistryHostKey(item *httpmsg.HttpRequestResponse) string

RegistryHostKey returns the per-host key used by every registry in this package — TechRegistry, ContentClassRegistry, WAFRegistry. It is the URL host: the bare hostname plus ":port" for non-default ports.

Read and write paths MUST agree on this key or detections silently go missing, so it lives here rather than being re-derived per caller. The port is part of the key on purpose: keying on the bare hostname let a stack detected on :443 gate modules on :8443 and dropped every non-default-port detection outright. Falls back to the bare service host only when the URL cannot be parsed.

func RequestCookieNames ¶

func RequestCookieNames(header string) []string

RequestCookieNames parses the names from a Cookie request header.

func ResemblesCatchAllShell ¶

func ResemblesCatchAllShell(
	scanCtx *ScanContext,
	ctx *httpmsg.HttpRequestResponse,
	client *http.Requester,
	hitBody string,
) bool

ResemblesCatchAllShell reports whether hitBody is the host's catch-all application shell rather than a distinct resource produced by a path probe. It is the shared, WAF-flake-robust guard against the wildcard-SPA false positive that path-mangling modules (off-by-slash alias traversal, path normalization, proxy path confusion) share: a wildcard SPA / reverse proxy serves one index.html for EVERY unknown path, so a probe's "successful" 2xx body is just that shell. A lone control probe cannot see this reliably on a WAF/CDN-fronted host — each control is one request that fails (open or closed) the moment it is throttled to a non-2xx — so this confirms the shell POSITIVELY from several INDEPENDENT samples and OR-s them, and no single flaky probe can hide the catch-all:

• the page originally observed on ctx (ResemblesObservedPage — zero traffic),
• a random nonexistent directory at the web root (RandomDirCatchAll), and
• the site root "/" (RootPageCatchAll).

All three compare with the dynamic-content-robust BodiesSimilar (QuickRatio), so per-request tokens/timestamps in the shell do not defeat the match. A genuinely distinct resource — an escaped source/config file, a real admin page — is never ~equal to the homepage or a random-directory response, so the guard costs no true positives. The two directory/root probes are memoized per observed record (via decoyProbe), so calling this once per candidate in a probe loop issues at most two extra requests for the whole record.

CAUTION: the site-root sample makes this unsuitable for a detector whose genuine hit can legitimately equal the home page (e.g. an index.php PATH_INFO misconfig where /index.php renders the homepage) — such a module should call ResemblesObservedPage + RandomDirCatchAll directly and skip RootPageCatchAll.

func ResemblesObservedPage ¶

func ResemblesObservedPage(ctx *httpmsg.HttpRequestResponse, probeBody string) bool

ResemblesObservedPage reports whether probeBody is textually equivalent (BodiesSimilar, QuickRatio >= UpperRatioBound) to the page originally observed on ctx — the request/response the module was handed before it began probing. A catch-all / SPA application serves that same application shell for an arbitrary path, so a path-probe whose response matches the observed page is a false positive: the body is "the same with or without the probe". It is the shared catch-all-shell guard for path-probing exposure modules — place it after the soft-404 fingerprint and before marker matching, dropping the candidate when it returns true.

Returns false (fail-open) when ctx carries no baseline response or an empty observed body, so a module invoked without a captured baseline never has a real finding suppressed. A genuinely exposed file (source, config, log, directory listing) is never ~95% similar to the homepage shell, so the guard does not cost true positives.

func ResemblesObservedPageSig ¶

func ResemblesObservedPageSig(ctx *httpmsg.HttpRequestResponse, probeSig ResponseSignature) bool

ResemblesObservedPageSig is the precomputed-signature form of ResemblesObservedPage: a caller that already holds the probe body's ratio signature (e.g. ResemblesCatchAllShell, which reuses it for the shell probes) passes it here to avoid tokenizing the probe body a second time.

func RootPageCatchAll ¶

func RootPageCatchAll(
	sc *ScanContext,
	ctx *httpmsg.HttpRequestResponse,
	client *http.Requester,
	match func(body string) bool,
) bool

RootPageCatchAll GETs the site root ("/") — memoized per observed record — and reports whether its 2xx body satisfies match. On a single-page application the root serves the same client-side shell for every unknown route, so a path probe whose body matches the root is that shell rather than a distinct resource. It is the site-root companion to RandomDirCatchAll (which probes a random nonexistent directory): the two are INDEPENDENT shell samples, so a caller that OR-s them tolerates a WAF/CDN that intermittently blocks or throttles a single probe — the failure mode that lets a wildcard-shell response slip past a lone control. Runs with NoRedirects (a 30x root is not a served shell) and NoClustering (so the probe is not aliased to a cached entry). Returns false on any build/transport error or a non-2xx root, so a flaky root fetch never suppresses a real finding (fail-open toward "not a catch-all").

func ServiceBaseURL ¶

func ServiceBaseURL(service *httpmsg.Service) string

ServiceBaseURL builds the scheme://host[:port] origin for service, omitting the port for the protocol default (443 for https, 80 for http, or an unset port) so probed/matched URLs read cleanly. It gives path-probing modules a correct base to append their probe path to, instead of concatenating onto the observed request's full URL (which carries an unrelated path and query string).

func SiblingPathCatchAll ¶

func SiblingPathCatchAll(
	sc *ScanContext,
	ctx *httpmsg.HttpRequestResponse,
	client *http.Requester,
	probePath string,
	match func(body string) bool,
) bool

SiblingPathCatchAll probes a guaranteed-nonexistent sibling under the SAME parent directory as probePath and reports whether its 200 response satisfies match. It is the catch-all guard for path-probing modules: a genuinely exposed endpoint serves its content only at its own path, whereas a catch-all handler (SPA fallback, i18n/static blob server, reverse-proxy wildcard) returns the same body for an arbitrary sibling too. The root-scoped soft-404 fingerprint these modules already take cannot see a catch-all scoped to a sub-directory prefix (e.g. /admin/*, /actuator/gateway/*, /jolokia/*); this sibling probe can.

Root-level probe paths (a single path segment, e.g. /admin or /jolokia) are a no-op here and return false: their sibling IS a random root path, which the caller's existing root 404 fingerprint already covers. Returns false on any build/transport error or a non-200 sibling so a flaky probe never suppresses a real finding.

func SiblingServesAnyMarker ¶

func SiblingServesAnyMarker(
	sc *ScanContext,
	ctx *httpmsg.HttpRequestResponse,
	client *http.Requester,
	probePath string,
	markers []string,
) bool

SiblingServesAnyMarker reports whether a guaranteed-nonexistent sibling under probePath's parent directory returns a body containing any of markers — i.e. a sub-directory catch-all that 200s every child with the same content. It is the flat-marker ([]string) convenience over SiblingPathCatchAll, replacing the "for marker := range markers { strings.Contains }" closure the path-probing modules each hand-rolled. Returns false (no catch-all detected) for empty markers, a root-level probe path, or any probe error.

func SizeShiftGap ¶

func SizeShiftGap(baselineLen, controlLen, probeLen, probe2Len int) (gap int, ok bool)

SizeShiftGap reports whether a payload reproducibly shifts the response size outside the page's natural variance. Given the lengths of two no-payload samples (baselineLen, controlLen) and two with-payload samples (probeLen, probe2Len), it returns the gap by which the with-payload band sits entirely on ONE side of, and clear of, the no-payload band — and whether that gap exceeds the variance threshold max(30% of baselineLen, 256 bytes). When the two bands overlap (jitter) or the gap is within the threshold, gap is 0 and ok is false.

Two samples per band describe "natural variance" only on an endpoint that is a function of the request. A host round-robining between mismatched backends has a BIMODAL size distribution for the unmodified request alone — a parked pool answering `GET /` with either a 5.9 KB or an 11.3 KB Apache default page — so the two bands land on opposite modes often enough (always, against a strict round robin) for this to report a shift the payload had nothing to do with. Callers must therefore pair an ok verdict with EndpointVolatile; run it AFTER this function, which is free and rejects the common case.

func SlugReflectionFP ¶

func SlugReflectionFP(
	ctx *httpmsg.HttpRequestResponse,
	client *http.Requester,
	probePath string,
	matchedMarkers []string,
) bool

SlugReflectionFP reports whether a flat-marker path-probe match is merely the probe's own last path segment reflected into a content route or a path-reflecting shell — the false positive PathSegmentReflected guards. It is the flat-[]string companion to the grouped slug-reflection guard inside MatchAndConfirmSibling, for modules that confirm with a plain "any marker in the list matched" loop (the SiblingServesAnyMarker callers): pass the markers that ACTUALLY matched the body.

It fires (returns true → drop the finding) ONLY when EVERY passed marker is a case-insensitive substring of probePath's last segment — so no structural marker (a hyphenated tag, an asset URL, a quoted JSON key) is holding the finding up — AND PathSegmentReflected confirms the host echoes an arbitrary slug. Returning early on the first non-segment marker keeps the network control probe off the common path and preserves any finding backed by real endpoint content. The reflection control covers root-level probe paths too (it probes the web root), so a single-segment slug like /redoc reflected by a wildcard shell is caught. Returns false for empty markers or a segmentless path.

Flat-marker callers pass every marker that matched (pure-OR: a single reflected slug is the whole finding, so all must be segment-explained to drop). The grouped MatchAndConfirmSibling passes only the anchor (matched[:1]): its later groups already supplied real corroboration, so only the anchor hit needs the reflection check.

func StripCredentialHeaders ¶

func StripCredentialHeaders(raw []byte) ([]byte, error)

StripCredentialHeaders returns a copy of a raw HTTP request without headers that commonly identify a user or carry an API/session credential. Pair it with http.Requester.CloneWithoutCredentials: the raw copy removes credentials captured in traffic, while the isolated requester prevents configured headers or cookie-jar state from being re-applied during transmission.

func StripReflected ¶

func StripReflected(body, payload string) string

StripReflected removes occurrences of an injected payload from body so a signature or marker that is itself part of that payload cannot self-match when a validation/error response echoes the rejected input verbatim. It is the request-body analogue of StripReflectedProbePath (which strips a reflected URL path): an endpoint that merely REJECTS a request and quotes it back — a 400 "invalid input: <payload>", a gRPC 415 echoing the Content-Type — would otherwise trip any detector whose marker the payload contains. Examples this defends: the PHP unserialize probe O:8:"stdClass":0:{} matching an O:\d+:"..." error pattern, an XXE internal-entity probe carrying its own success marker as the entity value, a NoSQL operator payload echoed back into a would-be error string.

A genuine server-side signal (a real deserialization error, an expanded entity, an evaluated expression) emits its OWN text that is not part of the literal payload, so it survives the strip while a bare reflection does not. Returns body unchanged for an empty payload or body. Run it ONLY for the marker/signature match — keep the original body for stored response evidence.

func StripReflectedProbePath ¶

func StripReflectedProbePath(body, probePath string) string

StripReflectedProbePath removes occurrences of the probe path from body so a marker that merely echoes the request path cannot satisfy a marker check. A path-routing app reflects the requested URL straight into the page — a <form action="/.../admin">, an href, a breadcrumb, a JSON "path":"/x" field — so a generic-word marker that is also a segment of the probe path (e.g. "admin" for /admin, "profiler" for /_profiler/, "restart" for /.~~spring-boot!~/restart) matches our OWN request, not the endpoint's content. This is the self-reflection FP class. Strip the reflected path before marker matching so a marker only counts when it comes from the endpoint's own body.

Both the full path and its query-stripped prefix are removed (a reflected <form action> commonly drops the query string). Returns body unchanged for an empty probePath or body. Run it ONLY for the marker match — keep the original body for the shell/soft-404 guards and for the stored response evidence.

func Tokenize ¶

func Tokenize(normalized string) (map[string]int, int)

Tokenize splits normalized text into a token-count multiset and total count.

func Truncate ¶

func Truncate(s string, maxLen int) string

Truncate shortens a string to maxLen, appending "..." if truncated.

func UnclaimedBasePaths ¶

func UnclaimedBasePaths(diskSet *dedup.DiskSet, host string, candidates []string) []string

UnclaimedBasePaths returns the subset of candidates this scan has not yet probed for host, atomically claiming each returned base via diskSet (IsSeen is test-and-set: false-and-claim on first sight, true thereafter). It collapses the per-(host,base) dedup loop every context-path-walking module runs into one call; pass the output of CandidateBasePaths / CandidateBasePathsIncludingStatic. Order is preserved (so the web root "" stays first when unclaimed). A nil diskSet returns candidates unchanged.

func WindowBody ¶

func WindowBody(body []byte, locators []string, matchLine int, opts ResponseWindowOpts) string

WindowBody returns a view of body suitable for storing as finding evidence. Bodies at or below opts.FullThreshold are returned unchanged. Larger bodies are reduced to a window centered on the first entry of locators found in the body (each tried verbatim, then on the tail after its last ": " so a wrapped label like "Matched: <x>" or "MD5 hash near 'k': <x>" still resolves); failing that, on the 1-indexed matchLine when > 1; failing that, on the first opts.FallbackLines lines. The kept window spans opts.ContextLines lines on each side, clamped to opts.ContextChars bytes, and dropped edges are marked with a "... [N bytes truncated] ..." note.

func WindowBodyAt ¶

func WindowBodyAt(body []byte, anchorStart, anchorEnd int, locators []string, matchLine int, opts ResponseWindowOpts) string

WindowBodyAt is WindowBody anchored on an explicit [anchorStart, anchorEnd) byte span — the offsets the secret detector reports for a match — instead of re-locating the match text in body. Anchoring pins the evidence window to the exact occurrence that fired (a value that also appears elsewhere in the body no longer drags the window to the wrong spot) and skips a full-body substring search. Bodies at or below opts.FullThreshold are still returned whole. When the anchor is out of range it falls back to WindowBody's locator/line/head resolution, so callers can always pass the match text as a safety-net locator.