Repolode
Submit
hiyouga

hiyouga/LlamaFactory

Unified framework for efficiently fine-tuning 100+ LLMs and VLMs with a zero-code CLI and Web UI.

View on GitHubWebsite
Stars
75,601
Forks
9,275
Open issues
1,197
Last commit
19 hours ago

What is it?

LlamaFactory is a framework for fine-tuning large language models and vision-language models. It covers a wide range of model families, including LLaMA, Qwen, Gemma, DeepSeek, GLM, Mistral and many multimodal variants, and lets users run training without writing code through either a command-line interface or a browser-based UI called LLaMA Board.

The project supports full-parameter tuning, freeze-tuning, LoRA and quantized QLoRA training, along with pre-training, supervised fine-tuning, reward modeling, PPO, DPO, KTO, ORPO and SimPO. It also integrates several memory and speed techniques such as FlashAttention-2, Unsloth and Liger Kernel, and includes optimizers like GaLore, BAdam and APOLLO.

After training, models can be served through an OpenAI-style API, with vLLM or SGLang as optional inference backends. Experiment tracking works with TensorBoard, Wandb, MLflow and SwanLab.

Key features

  • 01Zero-code fine-tuning through a CLI or the LLaMA Board web UI.
  • 02Supports LoRA, QLoRA and full-parameter training across 100+ model families.
  • 03Covers SFT, reward modeling, PPO, DPO, KTO, ORPO and SimPO.
  • 04Supports multimodal models for image, video and audio understanding.
  • 05Serves trained models via an OpenAI-style API with vLLM or SGLang backends.

Use cases

  • Adapting an open-weight model like Qwen or Llama to a domain-specific dataset without writing training code.
  • Comparing LoRA and QLoRA runs on limited GPU memory.
  • Training preference-alignment models with DPO or ORPO.
  • Fine-tuning a vision-language model on custom image-text data.

Topics

#fine-tuning#llm#lora#qlora#peft#rlhf#multimodal#transformers#agent#ai

FAQ

What is LlamaFactory?
Unified framework for efficiently fine-tuning 100+ LLMs and VLMs with a zero-code CLI and Web UI. LlamaFactory is a framework for fine-tuning large language models and vision-language models. It covers a wide range of model families, including LLaMA, Qwen, Gemma, DeepSeek, GLM, Mistral and many multimodal variants, and lets users run training without writing code through either a command-line interface or a browser-based UI called LLaMA Board.
Is LlamaFactory free to use? What license does it have?
LlamaFactory is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is LlamaFactory still maintained?
Actively maintained. The last commit to LlamaFactory was on Oct 19, 2026.
What can I use LlamaFactory for?
Adapting an open-weight model like Qwen or Llama to a domain-specific dataset without writing training code. Comparing LoRA and QLoRA runs on limited GPU memory. Training preference-alignment models with DPO or ORPO. Fine-tuning a vision-language model on custom image-text data.
Where can I get LlamaFactory?
The source code and installation instructions are on GitHub: https://github.com/hiyouga/LlamaFactory

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects