Directories
¶
| Path | Synopsis |
|---|---|
|
Package analyze defines the analyzer pipeline: the Issue/Severity types, the Analyzer interface every check implements, and a Registry to hold them.
|
Package analyze defines the analyzer pipeline: the Issue/Severity types, the Analyzer interface every check implements, and a Registry to hold them. |
|
aeo
Package aeo assesses Answer Engine Optimization: how well a page is structured to be surfaced as a direct answer in featured snippets, "People Also Ask", and voice results.
|
Package aeo assesses Answer Engine Optimization: how well a page is structured to be surfaced as a direct answer in featured snippets, "People Also Ask", and voice results. |
|
amp
Package amp implements AMP page detection, required-markup checks, and amphtml link validation against the crawled page set.
|
Package amp implements AMP page detection, required-markup checks, and amphtml link validation against the crawled page set. |
|
botwall
Package botwall detects when a crawl was served a CAPTCHA or bot-protection challenge instead of the real page — reCAPTCHA, hCaptcha, Cloudflare Turnstile, and the full-page interstitials of Cloudflare, DataDome, AWS WAF, PerimeterX/HUMAN, and Imperva Incapsula.
|
Package botwall detects when a crawl was served a CAPTCHA or bot-protection challenge instead of the real page — reCAPTCHA, hCaptcha, Cloudflare Turnstile, and the full-page interstitials of Cloudflare, DataDome, AWS WAF, PerimeterX/HUMAN, and Imperva Incapsula. |
|
consent
Package consent audits how a site asks for, and behaves before, visitor consent to tracking.
|
Package consent audits how a site asks for, and behaves before, visitor consent to tracking. |
|
content
Package content implements cross-page content analysis: thin content and pages whose word count falls well below the crawl-wide average.
|
Package content implements cross-page content analysis: thin content and pages whose word count falls well below the crawl-wide average. |
|
datalayer
Package datalayer audits Google Tag Manager / gtag wiring and the dataLayer event stream.
|
Package datalayer audits Google Tag Manager / gtag wiring and the dataLayer event stream. |
|
duplicates
Package duplicates implements cross-page duplicate detection: identical body content, titles, and meta descriptions across the crawl.
|
Package duplicates implements cross-page duplicate detection: identical body content, titles, and meta descriptions across the crawl. |
|
geo
Package geo assesses Generative Engine Optimization: whether AI answer engines (ChatGPT, Perplexity, Google AI Overviews, Gemini, Claude) can access the site and trust its content enough to cite it.
|
Package geo assesses Generative Engine Optimization: whether AI answer engines (ChatGPT, Perplexity, Google AI Overviews, Gemini, Claude) can access the site and trust its content enough to cite it. |
|
hreflang
Package hreflang implements hreflang annotation validity, self-reference, x-default, and return-link reciprocity checks across the crawled page set.
|
Package hreflang implements hreflang annotation validity, self-reference, x-default, and return-link reciprocity checks across the crawled page set. |
|
httpx
Package httpx implements HTTP-level checks: status codes, redirect chains and loops, slow responses, and mixed content.
|
Package httpx implements HTTP-level checks: status codes, redirect chains and loops, slow responses, and mixed content. |
|
images
Package images implements image checks: missing alt text and missing width/height dimension attributes.
|
Package images implements image checks: missing alt text and missing width/height dimension attributes. |
|
landing
Package landing assesses paid-campaign landing pages: how well the page's title and headings reflect the campaign keywords (utm_term / utm_campaign / utm_content), plus the quality-score signals ads care about (title, H1, meta description, indexability, HTTPS).
|
Package landing assesses paid-campaign landing pages: how well the page's title and headings reflect the campaign keywords (utm_term / utm_campaign / utm_content), plus the quality-score signals ads care about (title, H1, meta description, indexability, HTTPS). |
|
links
Package links implements link analysis: broken internal links, links to redirects, nofollow and external link reporting, and empty anchor detection.
|
Package links implements link analysis: broken internal links, links to redirects, nofollow and external link reporting, and empty anchor detection. |
|
pagination
Package pagination implements rel=next/prev pagination sequence detection and broken-target checks against the crawled page set.
|
Package pagination implements rel=next/prev pagination sequence detection and broken-target checks against the crawled page set. |
|
perf
Package perf reports Core Web Vitals from headless-rendered pages.
|
Package perf reports Core Web Vitals from headless-rendered pages. |
|
robotscheck
Package robotscheck reports on robots.txt: presence, declared sitemaps, and crawled URLs that violate disallow rules.
|
Package robotscheck reports on robots.txt: presence, declared sitemaps, and crawled URLs that violate disallow rules. |
|
seaurl
Package seaurl parses UTM campaign parameters from URLs.
|
Package seaurl parses UTM campaign parameters from URLs. |
|
security
Package security implements security checks.
|
Package security implements security checks. |
|
seo
Package seo implements technical on-page SEO checks (title, meta, canonical, headings, language, viewport, charset, social tags).
|
Package seo implements technical on-page SEO checks (title, meta, canonical, headings, language, viewport, charset, social tags). |
|
sitemap
Package sitemap discovers and parses sitemap.xml (including sitemap indexes) and cross-checks declared URLs against what was actually crawled.
|
Package sitemap discovers and parses sitemap.xml (including sitemap indexes) and cross-checks declared URLs against what was actually crawled. |
|
structured
Package structured extracts and reports on JSON-LD structured data (schema.org).
|
Package structured extracts and reports on JSON-LD structured data (schema.org). |
|
tracking
Package tracking detects marketing and analytics tags on a page — Google Tag Manager, GA4, Universal Analytics, Google Ads, Meta (Facebook) Pixel, and a few others — and flags missing or duplicated installs.
|
Package tracking detects marketing and analytics tags on a page — Google Tag Manager, GA4, Universal Analytics, Google Ads, Meta (Facebook) Pixel, and a few others — and flags missing or duplicated installs. |
|
urls
Package urls implements URL hygiene checks: uppercase letters, underscores, non-ASCII characters, and excessive length.
|
Package urls implements URL hygiene checks: uppercase letters, underscores, non-ASCII characters, and excessive length. |
|
utm
Package utm audits UTM campaign tagging on a page's outbound links: partial tagging, empty or duplicated parameters, inconsistent key casing, and a per-page rollup.
|
Package utm audits UTM campaign tagging on a page's outbound links: partial tagging, empty or duplicated parameters, inconsistent key casing, and a per-page rollup. |
|
wordpress
Package wordpress fingerprints WordPress sites and runs WordPress-specific checks that the generic analyzers do not name.
|
Package wordpress fingerprints WordPress sites and runs WordPress-specific checks that the generic analyzers do not name. |
|
Package atomicfile writes files atomically: content is written to a temporary file in the same directory as the destination, then renamed into place.
|
Package atomicfile writes files atomically: content is written to a temporary file in the same directory as the destination, then renamed into place. |
|
Package config defines gocrawl's layered configuration (defaults -> YAML file -> env -> flags) and maps it onto crawler.Options.
|
Package config defines gocrawl's layered configuration (defaults -> YAML file -> env -> flags) and maps it onto crawler.Options. |
|
Package crawler implements gocrawl's concurrent crawl engine.
|
Package crawler implements gocrawl's concurrent crawl engine. |
|
Package crawlrequest maps a user-facing crawl request onto a config.Config.
|
Package crawlrequest maps a user-facing crawl request onto a config.Config. |
|
Package diff compares two crawl reports and describes what changed between them: which issues are new, which were resolved, which persist, plus summary and page-set deltas.
|
Package diff compares two crawl reports and describes what changed between them: which issues are new, which were resolved, which persist, plus summary and page-set deltas. |
|
Package mcpserver exposes gocrawl over the Model Context Protocol so agentic tools such as Conductor or Claude Code can drive crawls directly.
|
Package mcpserver exposes gocrawl over the Model Context Protocol so agentic tools such as Conductor or Claude Code can drive crawls directly. |
|
Package redirectcheck verifies a redirect-rule CSV export (HubSpot's URL Redirects tool schema) against a live site: whether each rule's source still redirects correctly, whether its target is a live page, and whether both sides agree with the site's current sitemap.xml.
|
Package redirectcheck verifies a redirect-rule CSV export (HubSpot's URL Redirects tool schema) against a live site: whether each rule's source still redirects correctly, whether its target is a live page, and whether both sides agree with the site's current sitemap.xml. |
|
Package render provides the headless-rendering Fetcher backed by chromedp.
|
Package render provides the headless-rendering Fetcher backed by chromedp. |
|
Package report builds and serializes crawl reports (JSON, CSV, and HTML).
|
Package report builds and serializes crawl reports (JSON, CSV, and HTML). |
|
Package runner wires the crawl engine, analyzer registry, and report builder into a single entry point used by both the CLI and the MCP server.
|
Package runner wires the crawl engine, analyzer registry, and report builder into a single entry point used by both the CLI and the MCP server. |
|
Package sitemapgen turns a finished crawl into shareable site-map artifacts: a standard sitemap.xml (the sitemaps.org urlset) and a tree the HTML report draws as a node-link (org-chart-style) diagram, annotated with the analyzer issues found on each page.
|
Package sitemapgen turns a finished crawl into shareable site-map artifacts: a standard sitemap.xml (the sitemaps.org urlset) and a tree the HTML report draws as a node-link (org-chart-style) diagram, annotated with the analyzer issues found on each page. |
|
Package store persists crawl reports to disk and reads them back, so crawls can be listed, reopened, and compared over time.
|
Package store persists crawl reports to disk and reads them back, so crawls can be listed, reopened, and compared over time. |
|
Package webserver exposes gocrawl over HTTP: an API to start and inspect crawls, backed by the same runner.Run seam the CLI and MCP server use, plus the embedded single-page app (see assets.go) that drives it from a browser.
|
Package webserver exposes gocrawl over HTTP: an API to start and inspect crawls, backed by the same runner.Run seam the CLI and MCP server use, plus the embedded single-page app (see assets.go) that drives it from a browser. |
Click to show internal directories.
Click to hide internal directories.