Documentation
¶
Overview ¶
Package sitemapgen turns a finished crawl into shareable site-map artifacts: a standard sitemap.xml (the sitemaps.org urlset) and a tree the HTML report draws as a node-link (org-chart-style) diagram, annotated with the analyzer issues found on each page.
It is a pure transform over a crawler.Result and the analyzer issues — it never fetches or mutates anything. This keeps it outside the analyzer seam (analyzers emit Issues and must not write files); the runner invokes it as an optional side output of a crawl.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func WriteXMLFile ¶
WriteXMLFile writes the sitemap.xml to path, creating parent directories as needed.
Types ¶
type Entry ¶
type Entry struct {
Loc string `json:"loc"` // canonical (post-redirect) URL
LastMod string `json:"lastmod,omitempty"` // YYYY-MM-DD from the Last-Modified header, or "" if unknown
}
Entry is one indexable URL destined for sitemap.xml.
type Map ¶
type Map struct {
Seed string `json:"seed"`
Host string `json:"host"`
Generated time.Time `json:"generated"`
Entries []Entry `json:"entries,omitempty"`
Root *Node `json:"root,omitempty"`
// SiteWide holds issues whose URL didn't resolve to a crawled page in the tree (e.g. the
// robots analyzer's per-host findings, keyed "host <hostname>").
SiteWide []PageIssue `json:"site_wide,omitempty"`
// Totals counts every issue placed on the tree plus the site-wide ones.
Totals SevCounts `json:"totals,omitempty"`
}
Map is the generated site map: a flat list of entries for sitemap.xml and a tree for the HTML visualization, plus any findings that don't belong to a single crawled page.
func Generate ¶
Generate builds a site map from a crawl result and its analyzer issues. Only successful (HTTP 200) HTML pages on the seed host (or its subdomains) are included in the tree and sitemap.xml; redirects, errors, assets, and off-site pages are excluded, matching what belongs in a sitemap. Issues are attached to the page they name (matched on URL); issues that name no crawled page land in SiteWide. The generated timestamp is passed in so callers stay deterministic and testable.
type Node ¶
type Node struct {
Label string `json:"label"` // path segment, e.g. "blog" or "/" for the root
URL string `json:"url,omitempty"` // crawled page URL, or "" for a synthetic intermediate node
Title string `json:"title,omitempty"` // <title> of the page, when available
Status int `json:"status,omitempty"` // HTTP status, 0 for synthetic nodes
Depth int `json:"depth,omitempty"` // crawl depth from the seed
Issues []PageIssue `json:"issues,omitempty"` // findings on this exact page, worst-first
Counts SevCounts `json:"counts,omitempty"` // issue counts for this page only
Subtotal SevCounts `json:"subtotal,omitempty"` // issue counts for this node and all descendants
Children []*Node `json:"children,omitempty"` // sorted by Label
}
Node is a node in the site tree. Intermediate path segments that were never crawled directly (URL == "") still appear so the hierarchy stays connected.
type PageIssue ¶
type PageIssue struct {
Severity string `json:"severity"`
Code string `json:"code"`
Analyzer string `json:"analyzer"`
Message string `json:"message"`
}
PageIssue is an analyzer finding attached to a node, flattened to plain strings for the HTML template (decoupled from analyze.Issue).
Source Files