Repolode
Submit
verl-project

verl-project/verl

verl is an open-source reinforcement learning training library for post-training large language models, built on the HybridFlow paper.

View on GitHubWebsite
Stars
23,942
Forks
4,779
Open issues
1,366
Last commit
yesterday

What is it?

verl is an RL training library for large language models, originally released by the ByteDance Seed team and now maintained by a community. It targets post-training workflows such as PPO and GRPO, and is designed to plug into existing training and inference engines rather than replace them.

The framework separates computation from data dependencies through a hybrid-controller programming model, which lets users express RL dataflows in a few lines of code. It supports FSDP, FSDP2 and Megatron-LM for training, and vLLM, SGLang and HF Transformers for rollout generation. Models can be placed flexibly across different sets of GPUs, and a 3D-HybridEngine reshards actor models between training and generation phases.

Beyond core RL algorithms, verl includes supervised fine-tuning, LoRA, multi-turn tool calling, multi-modal RL for vision-language models, and experiment tracking with wandb, swanlab, mlflow and tensorboard. It runs on NVIDIA, AMD ROCm and Ascend hardware.

Key features

  • 01Supports PPO, GRPO, GSPO, REINFORCE++, RLOO, DAPO and other RL algorithms.
  • 02Integrates FSDP, Megatron-LM for training and vLLM, SGLang for rollout.
  • 03Works with Hugging Face models such as Qwen, Llama and Gemma.
  • 04Supports LoRA, sequence packing and expert parallelism for large MoE models.
  • 05Includes multi-turn tool calling and multi-modal RL for vision-language models.
  • 06Runs on NVIDIA, AMD ROCm and Ascend hardware.

Use cases

  • Post-training LLMs for math or coding reasoning with verifiable rewards.
  • Running GRPO or PPO at scale across multi-GPU clusters.
  • Training LLM agents that call tools over multiple turns.
  • Applying RL to vision-language models with multi-modal inputs.

Things to know

  • Training large models requires GPU hardware, and the headline features target multi-GPU clusters.
  • The README excerpt provided does not include an install or run command, so setup steps must be taken from the full documentation.

Topics

#reinforcement-learning#llm#rlhf#post-training#ppo#grpo#distributed-training#vllm

FAQ

What is verl?
verl is an open-source reinforcement learning training library for post-training large language models, built on the HybridFlow paper. verl is an RL training library for large language models, originally released by the ByteDance Seed team and now maintained by a community. It targets post-training workflows such as PPO and GRPO, and is designed to plug into existing training and inference engines rather than replace them.
Is verl free to use? What license does it have?
verl is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is verl still maintained?
Actively maintained. The last commit to verl was on Oct 20, 2026.
What can I use verl for?
Post-training LLMs for math or coding reasoning with verifiable rewards. Running GRPO or PPO at scale across multi-GPU clusters. Training LLM agents that call tools over multiple turns. Applying RL to vision-language models with multi-modal inputs.
Where can I get verl?
The source code and installation instructions are on GitHub: https://github.com/verl-project/verl

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects