Documentation
¶
Overview ¶
Package store persists crawl reports to disk and reads them back, so crawls can be listed, reopened, and compared over time. A report is already a complete, self-contained JSON artifact (see internal/report), so the store is a thin filesystem layer over it: one JSON file per crawl, laid out by host and timestamp.
Layout:
<root>/<host>/<timestamp>.json
e.g. ~/.gocrawl/crawls/example.com/20260630T120000Z.json. The crawl ID is the "<host>/<timestamp>" stem — sortable (newest = lexicographically greatest) and readable.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DefaultRoot ¶
func DefaultRoot() string
DefaultRoot is the store directory used when none is configured: ~/.gocrawl/crawls, or ./.gocrawl/crawls if the home directory can't be determined.
Types ¶
type Entry ¶
type Entry struct {
ID string `json:"id"`
Host string `json:"host"`
Seed string `json:"seed"`
FinishedAt string `json:"finished_at"`
PagesCrawled int `json:"pages_crawled"`
BySeverity map[string]int `json:"by_severity"`
Path string `json:"path"`
}
Entry is the metadata of one saved crawl, cheap to list without rendering the full report.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store reads and writes crawl reports under a root directory.
func (*Store) List ¶
List returns every saved crawl, newest first. An optional host filters to one site (matched against the report seed's host); empty lists all.
func (*Store) Resolve ¶
Resolve turns a user-supplied reference into a report. A ref may be:
- a path to an existing JSON file (loaded directly, not from the store);
- "latest" — the newest crawl in the store;
- a bare host like "example.com" — that host's newest crawl;
- a crawl ID "<host>/<timestamp>".
It returns the resolved report and the ID it resolved to (empty for a direct file path).
Source Files