Open-source Python web crawler that turns any website into clean, LLM-ready Markdown for RAG, AI agents and data pipelines.
85.7K8.9KPythonApache-2.0Updated 3 days ago
#web-scraping
Open-source projects tagged “web-scraping”
Curated GitHub repositories related to web-scraping, with AI summaries and key stats.
Sort:Most stars
2 repositories
A Rust command-line tool that fetches data from dozens of websites such as Bilibili, Reddit and Hacker News with a single command.
3K273RustApache-2.0Updated 6 months ago