Repolode
Submit
finic-ai

finic-ai/rag-stack

Self-hosted RAG stack for deploying a private ChatGPT-style assistant over your own PDFs and documents with open-source LLMs and Qdrant.

Open-source alternative to:ChatGPT

View on GitHubWebsite
Stars
1,581
Forks
137
Open issues
27
Last commit
26 days ago

What is it?

RAGstack is a deployable retrieval-augmented generation stack meant to run inside your own infrastructure as a private alternative to hosted ChatGPT-style tools. It combines an open-source LLM, the Qdrant vector database, and a server with a simple web UI that accepts PDF uploads, so answers can be grounded in your organization's documents.

The project supports GPT4All for local CPU runs, with a one-script setup that downloads the model and starts the server, LLM, and Qdrant locally. For cloud deployment it provides Terraform-based scripts for Google Cloud (GKE with GPUs), AWS (ECS on EC2), and Azure (AKS), deploying Falcon-7B. Llama 2 deployment is listed as supported on the cloud path, while Llama-2-40b support is still on the roadmap.

The project is marked as legacy and is no longer maintained by its authors. Local setup also depends on a Supabase project for user records.

Key features

  • 01Deploys a private, self-hosted chat interface over your own documents.
  • 02Uses Qdrant, a Rust-based open-source vector database, for retrieval.
  • 03Supports GPT4All for local CPU inference and Falcon-7B or Llama 2 on GPUs.
  • 04Includes a server and web UI with PDF upload.
  • 05Provides Terraform-based deployment scripts for GCP, AWS, and Azure.

Use cases

  • Building an internal question-answering assistant over company PDFs and contracts.
  • Trying out a retrieval-augmented chat setup locally without sending data to hosted APIs.
  • Prototyping a RAG deployment on a cloud GPU cluster before adopting a production design.

Quick start

  1. 1

    Copy the example env files and add your Supabase project URL and keys

  2. 2

    In Supabase, create the ragstack_users table with the columns listed in the README

  3. 3

    Run the local dev script to start the server, LLM and Qdrant

    scripts/local/run-dev

Things to know

  • The README says this legacy project is no longer maintained.
  • Local setup requires a Supabase project and manual table configuration.
  • Cloud deployments of Falcon-7B need GPU-enabled clusters and cloud accounts.

In collections

Private AI knowledge bases and RAGSelf-hosted apps

Topics

#rag#llm#vector-database#qdrant#self-hosted#terraform#gpt4all#falcon

FAQ

What is rag-stack?
Self-hosted RAG stack for deploying a private ChatGPT-style assistant over your own PDFs and documents with open-source LLMs and Qdrant. RAGstack is a deployable retrieval-augmented generation stack meant to run inside your own infrastructure as a private alternative to hosted ChatGPT-style tools. It combines an open-source LLM, the Qdrant vector database, and a server with a simple web UI that accepts PDF uploads, so answers can be grounded in your organization's documents.
Is rag-stack free to use? What license does it have?
rag-stack is open source under the MIT license. Check the license terms before using it in commercial products.
Is rag-stack still maintained?
Actively maintained. The last commit to rag-stack was on Oct 12, 2026.
What can I use rag-stack for?
Building an internal question-answering assistant over company PDFs and contracts. Trying out a retrieval-augmented chat setup locally without sending data to hosted APIs. Prototyping a RAG deployment on a cloud GPU cluster before adopting a production design.
How do I install rag-stack?
Copy the example env files and add your Supabase project URL and keys In Supabase, create the ragstack_users table with the columns listed in the README Run the local dev script to start the server, LLM and Qdrant: scripts/local/run-dev
Where can I get rag-stack?
The source code and installation instructions are on GitHub: https://github.com/finic-ai/rag-stack

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects