m92vyas/llm-reader

What is it?

Key features

• 01Converts webpage HTML into LLM-ready input text.
• 02Preserves website and image links for extraction tasks.
• 03Handles tables, useful for scraping structured data.
• 04Uses Playwright for async, concurrent page fetching.
• 05Lets you supply page HTML from your own scraper or service.

Use cases

• Feeding web content into a RAG pipeline.
• Extracting product names, links and prices from e-commerce pages with an LLM.
• Building AI agents that need readable text from URLs.

Quick start

1. 1

   Install the library from GitHub with pip

   pip install git+https://github.com/m92vyas/llm-reader.git
2. 2

   Download the Playwright browser

   playwright install
3. 3

   Install Playwright system dependencies

   playwright install-deps
4. 4

   Import get_page_source and get_processed_text, then fetch a page and convert it to LLM-ready text in your code

Things to know

• !Fetching pages relies on Playwright, which needs a browser download and system dependencies.
• !Sites that block scrapers may require a paid proxy or scraping service.

In collections

Topics

FAQ

What is llm-reader?

Python library that converts a webpage's HTML into LLM-ready text, with links and tables preserved for RAG and scraping. llm-reader is a Python library that takes a webpage and turns it into clean text suited to LLM input. Pre-processing the page before sending it to a model is meant to improve extraction accuracy, especially when the task involves pulling out website links, image links, or tables, as in scraping an e-commerce catalog.

Is llm-reader free to use? What license does it have?

llm-reader is open source under the MIT license. Check the license terms before using it in commercial products.

Is llm-reader still maintained?

Actively maintained. The last commit to llm-reader was on Oct 20, 2026.

What can I use llm-reader for?

Feeding web content into a RAG pipeline. Extracting product names, links and prices from e-commerce pages with an LLM. Building AI agents that need readable text from URLs.

How do I install llm-reader?

Install the library from GitHub with pip: pip install git+https://github.com/m92vyas/llm-reader.git Download the Playwright browser: playwright install Install Playwright system dependencies: playwright install-deps Import get_page_source and get_processed_text, then fetch a page and convert it to LLM-ready text in your code

Where can I get llm-reader?

The source code and installation instructions are on GitHub: https://github.com/m92vyas/llm-reader