Open-source projects tagged “web-scraping”

Open-source Python web crawler that turns any website into clean, LLM-ready Markdown for RAG, AI agents and data pipelines.

Self-hosted tool that monitors web pages for content, price, and stock changes and sends alerts through many notification channels.

A browser for AI agents where agents run automation in isolated Spaces while you keep browsing your own tabs and logins.

A Python framework for building web scrapers that aim to get past bot detection, with browser and HTTP request modes, UI generation, and desktop app support.

A Rust command-line tool that fetches data from dozens of websites such as Bilibili, Reddit and Hacker News with a single command.

A browser automation platform that turns recorded browser actions into MCP commands or Claude Skills so AI agents can drive browsers directly.

A Python tool with a WebUI and CLI for batch-downloading PEP (People's Education Press) electronic textbooks and merging them into PDFs.

A lightweight Python package that searches Google News and returns article results as JSON, with optional full-text extraction.

A tool that extracts reusable visual style templates from WeChat Official Account articles and previews them as rich text ready to paste into the WeChat editor.

Analyzes any website with Playwright and extracts its design system (colors, typography, spacing, components) as Tailwind, React, token, or Markdown exports.