botwall

Overview ¶

Package botwall detects when a crawl was served a CAPTCHA or bot-protection challenge instead of the real page — reCAPTCHA, hCaptcha, Cloudflare Turnstile, and the full-page interstitials of Cloudflare, DataDome, AWS WAF, PerimeterX/HUMAN, and Imperva Incapsula.

This matters because a challenge page looks like a successful (often HTTP 200) fetch, so every downstream analyzer then audits the challenge HTML — producing a page's worth of misleading "missing title / thin content / no structured data" findings. Surfacing the block tells the reader those findings are artefacts of being blocked, not real problems.

It is a pure analyzer: it reads each page's status, response headers, body, title, and (in headless mode) the outbound request URLs the renderer captured, and emits Issues. It never fetches or solves anything.