GoogleScraper

Open source

Python module that scrapes search engines such as Google, Yandex, Bing and DuckDuckGo, with asynchronous networking support; no longer maintained.

Open-source alternative to

GitHub stars
2.9k
Last 30 days
+7(+0%)
Last commit
5 yr ago
Repository age
12 years
Licence
Apache-2.0
Self-hosted
Yes

About GoogleScraper

GoogleScraper is a Python module for pulling results out of several search engines, including Google, Yandex, Bing and DuckDuckGo. It can use asynchronous networking, and it can drive a real browser through Selenium with either Chrome and ChromeDriver or Firefox and geckodriver.

It is written in Python 3, and the README asks for at least Python 3.6. It can be installed with pip or straight from the GitHub sources, though the README discourages the pip route and says some features and examples may not work as expected when installing from source.

The author has stopped maintaining the project. The README, last updated in February 2019, points to a JavaScript successor built on puppeteer, the se-scraper package on npm, and says all new development goes there. The reasons given include that the project is very buggy and that it still supports raw HTTP requests, which anti-bot mechanisms fend off easily.

The author also runs scrapeulous.com, a paid web service for people who want search engine data without running a scraper themselves. GoogleScraper itself is licensed under Apache 2.0.

Key features

  • Scrapes several search engines, including Google, Yandex, Bing and DuckDuckGo
  • Asynchronous networking support
  • Selenium mode with Chrome and ChromeDriver
  • Selenium mode also works with Firefox and geckodriver
  • Driver paths set in scrape_config.py
  • Command line usage and an asynchronous mode

Good fit for

  • Scraping Google search results
  • Collecting results from several search engines
  • SEO data gathering
Tags
python
scraper
search-engine
seo
selenium
crawler
unmaintained

GoogleScraper: questions and answers

What is GoogleScraper used for?
GoogleScraper is a Python module that scrapes search engines such as Google, Yandex, Bing and DuckDuckGo, with asynchronous networking support; no longer maintained. It is a good fit for scraping Google search results, collecting results from several search engines and SEO data gathering.
Is GoogleScraper open source?
Yes. GoogleScraper is open source under the Apache-2.0 licence. Its source code is on GitHub at NikolaiT/GoogleScraper.
Is GoogleScraper free?
Yes. GoogleScraper is open source, so the software itself is free to use. A managed cloud version is also available.
What is GoogleScraper an alternative to?
GoogleScraper is an open-source alternative to SerpApi. Other open-source alternatives to SerpApi include SearXNG, OpenSERP and Wigolo.
Is GoogleScraper actively maintained?
The most recent commit to GoogleScraper was on 3 July 2021. The project has 2.9k stars on GitHub, with 7 added in the last 30 days.

Open-source alternatives to GoogleScraper

See all

SaaS alternatives to GoogleScraper

See all