About GoogleScraper
GoogleScraper is a Python module for pulling results out of several search engines, including Google, Yandex, Bing and DuckDuckGo. It can use asynchronous networking, and it can drive a real browser through Selenium with either Chrome and ChromeDriver or Firefox and geckodriver.
It is written in Python 3, and the README asks for at least Python 3.6. It can be installed with pip or straight from the GitHub sources, though the README discourages the pip route and says some features and examples may not work as expected when installing from source.
The author has stopped maintaining the project. The README, last updated in February 2019, points to a JavaScript successor built on puppeteer, the se-scraper package on npm, and says all new development goes there. The reasons given include that the project is very buggy and that it still supports raw HTTP requests, which anti-bot mechanisms fend off easily.
The author also runs scrapeulous.com, a paid web service for people who want search engine data without running a scraper themselves. GoogleScraper itself is licensed under Apache 2.0.
Key features
- Scrapes several search engines, including Google, Yandex, Bing and DuckDuckGo
- Asynchronous networking support
- Selenium mode with Chrome and ChromeDriver
- Selenium mode also works with Firefox and geckodriver
- Driver paths set in scrape_config.py
- Command line usage and an asynchronous mode
Good fit for
- Scraping Google search results
- Collecting results from several search engines
- SEO data gathering
GoogleScraper: questions and answers
- What is GoogleScraper used for?
- GoogleScraper is a Python module that scrapes search engines such as Google, Yandex, Bing and DuckDuckGo, with asynchronous networking support; no longer maintained. It is a good fit for scraping Google search results, collecting results from several search engines and SEO data gathering.
- Is GoogleScraper open source?
- Yes. GoogleScraper is open source under the Apache-2.0 licence. Its source code is on GitHub at NikolaiT/GoogleScraper.
- Is GoogleScraper free?
- Yes. GoogleScraper is open source, so the software itself is free to use. A managed cloud version is also available.
- What is GoogleScraper an alternative to?
- GoogleScraper is an open-source alternative to SerpApi. Other open-source alternatives to SerpApi include SearXNG, OpenSERP and Wigolo.
- Is GoogleScraper actively maintained?
- The most recent commit to GoogleScraper was on 3 July 2021. The project has 2.9k stars on GitHub, with 7 added in the last 30 days.
Open-source alternatives to GoogleScraper
See all
SearXNG
Search
SearXNG is a free internet metasearch engine which aggregates results from various search
AGPL-3.0vs DuckDuckGo★ 39k
OpenSERP
Search
Self-hosted SERP API for AI, SEO & automation. Browser-rendered Google, Bing, Yandex, Baid
MITvs SerpApi★ 1.6k
Wigolo
Search
The go-to web for your AI coding agent — local-first search, fetch, crawl & research over
OSSvs Tavily★ 5.5k
kitty
Developer Tools
If you live in the terminal, kitty is made for you! Cross-platform, fast, feature-rich, GP
GPL-3.0vs Warp★ 36k
Marimo
Developer Tools
A reactive notebook for Python — run reproducible experiments, query with SQL, execute as
Apache-2.0vs Hex★ 24k
JupyterLab
Developer Tools
JupyterLab computational environment.
BSD-3-Clausevs MATLAB★ 15k
SaaS alternatives to GoogleScraper
See all
SerpApi
Search
API that returns structured results from Google and other search engines
SaaS
Stophy
API Tools
API for AI agents to search YouTube, get transcripts, read comments and inspect channels
SaaS
DataSpell
Developer Tools
JetBrains IDE for data science with notebook and Python support
SaaS
Directory Launch
Developer Tools
Next.js code base for local business directories with claimed listings, quote requests, maps and SEO pages
SaaS
Dirstarter
Developer Tools
Next.js directory template with built-in payments, SEO and AI-generated content
SaaS
Djast
Developer Tools
Django 5 SaaS boilerplate with persona-based agent instructions, slash commands and custom AI agents
SaaS