Repolode
Submit
laiso

laiso/site2pdf

CLI tool that crawls a website and merges all its pages into one PDF, aimed at feeding RAG and QA pipelines.

View on GitHub
Stars
321
Forks
34
Open issues
7
Last commit
3 months ago

What is it?

site2pdf starts from a main URL, collects the links on that page that match an optional regex or string pattern (defaulting to the same domain), renders each page to PDF in headless Chrome, and merges the results into a single document with pdf-lib. The output is saved under ./out/ named after the site's domain.

The main use case is preparing website content for AI tools that accept file uploads, such as Google NotebookLM or custom GPTs in ChatGPT. Because pages are rendered in a browser, images and formatting are kept, which helps multimodal models. Pages are processed concurrently to speed up generation.

The tool runs on Node.js 22.12 or later and uses Puppeteer, so it needs a Chrome or Chromium binary, which can be supplied via the --executablePath flag or the CHROME_PATH variable.

Key features

  • 01Crawls same-domain links from a starting URL, with optional regex or string filtering.
  • 02Renders each page to PDF in headless Chrome, keeping images and formatting.
  • 03Merges all pages into a single PDF using pdf-lib.
  • 04Processes pages concurrently to speed up generation.
  • 05Accepts a custom Chrome path via flag or environment variable.

Use cases

  • Turning product or API documentation sites into one file for NotebookLM or a custom GPT.
  • Building a single offline snapshot of a documentation section for archiving.
  • Supplying website content as a document source for a RAG pipeline.

Quick start

  1. 1

    Run the tool directly with npx (needs Node.js v22.12 or later)

    npx site2pdf-cli https://example.com
  2. 2

    Find the generated PDF in the out folder

Things to know

  • Requires Node.js v22.12 or later and, on Linux, extra system libraries for Puppeteer.
  • Some systems, such as ARM64 Linux, may fail to launch Chrome and need a manually installed browser.

In collections

CLI & terminal toolsPDF, OCR & document toolsStatic site generators & documentation tools

Topics

#pdf#web-scraping#rag#puppeteer#cli#typescript#documentation

FAQ

What is site2pdf?
CLI tool that crawls a website and merges all its pages into one PDF, aimed at feeding RAG and QA pipelines. site2pdf starts from a main URL, collects the links on that page that match an optional regex or string pattern (defaulting to the same domain), renders each page to PDF in headless Chrome, and merges the results into a single document with pdf-lib. The output is saved under ./out/ named after the site's domain.
Is site2pdf free to use? What license does it have?
site2pdf is open source under the MIT license. Check the license terms before using it in commercial products.
Is site2pdf still maintained?
Updated within 6 months. The last commit to site2pdf was on Aug 21, 2026.
What can I use site2pdf for?
Turning product or API documentation sites into one file for NotebookLM or a custom GPT. Building a single offline snapshot of a documentation section for archiving. Supplying website content as a document source for a RAG pipeline.
How do I install site2pdf?
Run the tool directly with npx (needs Node.js v22.12 or later): npx site2pdf-cli https://example.com Find the generated PDF in the out folder
Where can I get site2pdf?
The source code and installation instructions are on GitHub: https://github.com/laiso/site2pdf

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects