Repolode
Submit
NovaSearch-Team

NovaSearch-Team/RAG-Retrieval

End-to-end training, inference and distillation code for RAG retrieval models, covering embedding, ColBERT and reranker models.

View on GitHub
Stars
1,137
Forks
88
Open issues
3
Last commit
2 months ago

What is it?

RAG-Retrieval is a codebase for fine-tuning, serving and distilling the retrieval components used in retrieval-augmented generation. It supports fine-tuning of embedding models (BERT-based and LLM-based), late-interaction ColBERT models, and reranker models (BERT-based and LLM-based), and is compatible with open-source models such as bge, bce and gte families.

For inference, the project publishes a lightweight Python package, rag-retrieval, that provides one interface for calling different reranker models. It handles long documents either by truncation or by splitting and taking the maximum score, and new ranking models can be added by subclassing a base reranker class.

The repository also includes distillation for embedding and reranker models, transferring knowledge from a larger LLM-based model into a smaller one such as a 0.5B LLM or a BERT-base model. Multi-GPU training is available through DeepSpeed and FSDP, and the embedding code implements MRL for reduced-dimension vectors.

Key features

  • 01Fine-tunes BERT-based and LLM-based embedding models, ColBERT models and rerankers.
  • 02Provides a unified Python package for calling different reranker models.
  • 03Handles long documents by truncation or by splitting and taking the maximum score.
  • 04Distills larger LLM-based embedding or reranker models into smaller ones.
  • 05Implements the MRL loss for reduced-dimension embeddings.
  • 06Supports multi-GPU training with DeepSpeed and FSDP.

Use cases

  • Fine-tuning an open-source embedding model on domain-specific retrieval data.
  • Adding a reranker stage to a RAG pipeline through a single Python interface.
  • Distilling a large LLM reranker into a smaller BERT model for lower-latency serving.
  • Experimenting with ColBERT-style late-interaction retrieval on custom datasets.

Quick start

  1. 1

    Create a conda environment with Python 3.8

    conda create -n rag-retrieval python=3.8 && conda activate rag-retrieval
  2. 2

    Install the PyPI package for reranker inference

    pip install rag-retrieval

Things to know

  • Training setup requires a local CUDA-compatible PyTorch installed manually before the requirements.
  • Training code is organized per model type in subdirectories, so each workflow needs its own README.

In collections

Natural language processing tools & libraries

Topics

#rag#retrieval#embedding#reranker#colbert#fine-tuning#distillation#nlp#ai#llm

FAQ

What is RAG-Retrieval?
End-to-end training, inference and distillation code for RAG retrieval models, covering embedding, ColBERT and reranker models. RAG-Retrieval is a codebase for fine-tuning, serving and distilling the retrieval components used in retrieval-augmented generation. It supports fine-tuning of embedding models (BERT-based and LLM-based), late-interaction ColBERT models, and reranker models (BERT-based and LLM-based), and is compatible with open-source models such as bge, bce and gte families.
Is RAG-Retrieval free to use? What license does it have?
RAG-Retrieval is open source under the MIT license. Check the license terms before using it in commercial products.
Is RAG-Retrieval still maintained?
Updated within 6 months. The last commit to RAG-Retrieval was on Aug 28, 2026.
What can I use RAG-Retrieval for?
Fine-tuning an open-source embedding model on domain-specific retrieval data. Adding a reranker stage to a RAG pipeline through a single Python interface. Distilling a large LLM reranker into a smaller BERT model for lower-latency serving. Experimenting with ColBERT-style late-interaction retrieval on custom datasets.
How do I install RAG-Retrieval?
Create a conda environment with Python 3.8: conda create -n rag-retrieval python=3.8 && conda activate rag-retrieval Install the PyPI package for reranker inference: pip install rag-retrieval
Where can I get RAG-Retrieval?
The source code and installation instructions are on GitHub: https://github.com/NovaSearch-Team/RAG-Retrieval

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects