google-maps-scraper

Google maps scraper

A command line google maps parser build using

scrapemate web crawling framework.

You can use this repository either as is, or you can use it's code as a base and customize it to your needs

Quickstart

Using docker:

touch results.csv && docker run -v $PWD/example-queries.txt:/example-queries -v $PWD/results.csv:/results.csv gosom/google-maps-scraper -depth 1 -input /example-queries -results /results.csv

file results.csv will contain the parsed results.

On your host

(tested only on Ubuntu 22.04)

git clone https://github.com/gosom/google-maps-scraper.git
cd google-maps-scraper
go mod download
go build
./google-maps-scraper -input example-queries.txt -results restaurants-in-cyprus.csv

Be a little bit patient. In the first run it downloads required libraries.

In general perfomance can be better. Right, now it requires between 5-15 seconds to fetch and parse a place page.

For the initial searches it requires between 15-30 seconds.

The results are written when they arrive in the results file you specified

Command line options

try ./google-maps-scraper -h to see the command line options available:

  -c int
        concurrency (default 4)
  -cache string
        cache directory (default "cache")
  -debug
        debug
  -depth int
        max depth (default 10)
  -input string
        input file (default "stdin")
  -lang string
        language code (default "en")
  -results string
        results file (default "stdout")

-c: sets the concurrency. By default it uses half of the number of the CPUs detected

-cache: sets the cache directory (no effect for the moment)

-debug: Uses this to perform a headfull crawl (it will open a browser in your host)

-depth: is how much you allow the scraper to scroll in the search results. Experiment with that value a bit

-input: the input file with the keywords to search (see example-queries.txt)

-lang: is the language code to use for google (the hl urlparam). Default is en. For example use de for German or el for Greek.

-results: is the path to write the results

Extracted Data

• Title: the title of the business
• Category: the category of the business
• Address: the address of the business
• OpenHours: the opening hours of the business
• WebSite: the website of the business
• Phone: the phone number of the business
• PlusCode: the plus code of the business
• ReviewCount: the number of reviews
• ReviewRating: the rating of the results

Licence

This code is licenced under the MIT Licence

Contributing

Please open an ISSUE or make a Pull Request

Notes

Please use this scraper responsibly