Repolode
Предложить
omkarcloud

omkarcloud/botasaurus

Python framework for building web scrapers that use a humane browser driver and HTTP requests to get past bot detection, with UI and desktop app options.

Открыть на GitHubСайт
Звёзды
5,784
Форки
506
Открытые issues
59
Последний коммит
3 месяца назад

Что это?

Botasaurus is a Python web scraping framework that bundles the pieces scrapers usually need: a browser driver, lightweight HTTP requests, proxy and profile handling, caching, and parallel execution. Its three decorators, @browser, @request and @task, wrap a scraping function and handle configuration such as proxies, headless mode, and how many tasks run in parallel. Results are saved as JSON, and utilities for Excel and CSV export, sitemaps and IP lookup are included.

The browser driver is designed to avoid detection by systems such as Cloudflare, with options for fingerprint control, tiny persistent profiles, Chrome extensions and CAPTCHA solving through CapSolver. The @request decorator makes browser-like HTTP calls for pages that do not need JavaScript, and the README describes running fetch requests through the browser to reduce proxy bandwidth.

Beyond scripts, the project can generate a web UI with input controls and a REST API from a scraper, and it supports building cross-platform desktop apps for Mac, Windows and Linux using JavaScript.

Ключевые возможности

  • 01Three decorators (@browser, @request, @task) wrap scraping functions with configuration for proxies, parallelism and retries.
  • 02Browser driver with options for headless mode, Chrome profiles, extensions and fingerprint settings.
  • 03Lightweight HTTP request module with BeautifulSoup parsing via soupify().
  • 04Built-in helpers for caching, sitemaps, data cleaning and JSON, Excel and CSV output.
  • 05Generates a web UI and REST API from a scraper, and supports building desktop apps.

Сценарии использования

  • Scraping sites protected by Cloudflare or other bot detection services.
  • Running large numbers of page visits through proxies while controlling bandwidth costs.
  • Giving non-technical users a web or desktop interface to run a scraper.
  • Building data-processing tasks that need a shared UI with input controls.

Темы

#web-scraping#python#anti-bot#browser-automation#cloudflare#headless-browser#proxy#anti-detect#anti-detect-browser#anti-detection

Частые вопросы

Что такое botasaurus?
Python framework for building web scrapers that use a humane browser driver and HTTP requests to get past bot detection, with UI and desktop app options. Botasaurus is a Python web scraping framework that bundles the pieces scrapers usually need: a browser driver, lightweight HTTP requests, proxy and profile handling, caching, and parallel execution. Its three decorators, @browser, @request and @task, wrap a scraping function and handle configuration such as proxies, headless mode, and how many tasks run in parallel. Results are saved as JSON, and utilities for Excel and CSV export, sitemaps and IP lookup are included.
botasaurus можно использовать бесплатно? Какая у него лицензия?
botasaurus распространяется с открытым исходным кодом под лицензией MIT. Перед использованием в коммерческих продуктах проверьте условия лицензии.
botasaurus всё ещё поддерживается?
Обновлялся за последние 6 месяцев. Последний коммит в botasaurus — 26 июл. 2026 г..
Для чего можно использовать botasaurus?
Scraping sites protected by Cloudflare or other bot detection services. Running large numbers of page visits through proxies while controlling bandwidth costs. Giving non-technical users a web or desktop interface to run a scraper. Building data-processing tasks that need a shared UI with input controls.
Где скачать botasaurus?
Исходный код и инструкции по установке — на GitHub: https://github.com/omkarcloud/botasaurus

Описание создано ИИ на основе README проекта. Статистика GitHub обновляется ежедневно.

Похожие проекты