ecommerce

Overview ¶

Example: E-commerce product scraper

Demonstrates how to scrape product listings from books.toscrape.com, a public scraping sandbox. Covers:

• Generating a consistent identity profile (UA + headers match)
• Creating a StealthFetcher with that identity
• Seeding an in-memory queue with the first catalogue page
• Writing a ProcessorFunc that parses product name, price, and URL with goquery via parse.Document
• Building a pipeline: Validate required fields → Clean whitespace → CSV export
• Creating and running a Hunt that processes all seed jobs
• Printing a stats summary on completion

Run:

go run ./examples/ecommerce/