Repolode
Submit
omkarcloud

omkarcloud/botasaurus

A Python framework for building web scrapers that aim to get past bot detection, with browser and HTTP request modes, UI generation, and desktop app support.

View on GitHubWebsite
Stars
5,783
Forks
507
Open issues
59
Last commit
3 months ago

What is it?

Botasaurus is a Python framework for web scraping. It provides decorators for running scraping functions in a humane Chrome browser (@browser), with lightweight HTTP requests (@request), or with third-party libraries like Playwright and Selenium (@task). Scraping functions can take a list of inputs, run in parallel, and save their return values to JSON output files.

The project focuses on avoiding bot detection. It includes browser configuration options such as proxy rotation, persistent Chrome profiles, extensions, headless mode, and Cloudflare bypass via bypass_cloudflare. It also offers browser-based fetch requests as a way to reduce proxy bandwidth costs, along with utilities for caching, sitemaps, data cleaning, and IP information.

Beyond scripts, Botasaurus can generate a web UI with input controls, result tables, and a REST API from a scraper, and it supports building cross-platform desktop applications for Windows, macOS, and Linux using JavaScript.

Key features

  • 01Three decorators (@browser, @request, @task) for browser, HTTP, and third-party-library scraping.
  • 02Built-in anti-detection options such as Cloudflare bypass, human-like mouse movement, and fingerprint control.
  • 03Proxy rotation, persistent Chrome profiles, and Chrome extension support.
  • 04Parallel execution over lists of inputs, with results saved to JSON.
  • 05Generates a web UI, REST API, and desktop app from a scraper.

Use cases

  • Scraping sites protected by Cloudflare or other bot-detection services.
  • Running large batches of page visits with proxy rotation and parallel workers.
  • Turning an internal scraper into a web or desktop tool for non-developer users.

Topics

#web-scraping#python#anti-bot#browser-automation#selenium-alternative#cloudflare-bypass#proxy#scraping-framework#anti-detect#anti-detect-browser

FAQ

What is botasaurus?
A Python framework for building web scrapers that aim to get past bot detection, with browser and HTTP request modes, UI generation, and desktop app support. Botasaurus is a Python framework for web scraping. It provides decorators for running scraping functions in a humane Chrome browser (@browser), with lightweight HTTP requests (@request), or with third-party libraries like Playwright and Selenium (@task). Scraping functions can take a list of inputs, run in parallel, and save their return values to JSON output files.
Is botasaurus free to use? What license does it have?
botasaurus is open source under the MIT license. Check the license terms before using it in commercial products.
Is botasaurus still maintained?
Updated within 6 months. The last commit to botasaurus was on Jul 26, 2026.
What can I use botasaurus for?
Scraping sites protected by Cloudflare or other bot-detection services. Running large batches of page visits with proxy rotation and parallel workers. Turning an internal scraper into a web or desktop tool for non-developer users.
Where can I get botasaurus?
The source code and installation instructions are on GitHub: https://github.com/omkarcloud/botasaurus

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects