Repolode
Submit
linkedin

linkedin/Liger-Kernel

Efficient Triton GPU kernels for LLM training that fuse common layers and losses to cut memory use and raise throughput.

View on GitHubWebsite
Stars
6,665
Forks
627
Open issues
197
Last commit
10 hours ago

What is it?

Liger Kernel is a collection of Triton kernels for training large language models. It provides drop-in replacements for common operations such as RMSNorm, RoPE, SwiGLU, CrossEntropy and a fused linear cross-entropy, which reduce memory traffic and allow longer context lengths or larger batches on the same GPUs.

Users can patch supported Hugging Face models with a single call, such as apply_liger_kernel_to_llama, or compose individual kernel modules into their own models. The project also includes post-training losses like DPO, ORPO, SimPO, KTO and CPO that avoid materializing full logits, plus distillation losses such as JSD and KL divergence. It works with PyTorch FSDP, DeepSpeed and DDP, and requires only Torch and Triton as core dependencies.

The repository lists support for many model families including Llama, Qwen, Gemma, Mistral, Phi3 and DeepSeek, and includes backends for CUDA, ROCm, Intel GPU and Ascend NPU, with optional cuTile and CuTe DSL implementations on NVIDIA hardware.

Key features

  • 01Drop-in Triton kernels for RMSNorm, RoPE, SwiGLU, CrossEntropy and fused linear cross-entropy.
  • 02One-line patching for many supported Hugging Face model families.
  • 03Memory-efficient DPO, ORPO, SimPO, KTO and CPO losses for post-training.
  • 04Distillation losses including JSD, KL divergence and TVD.
  • 05Compatible with PyTorch FSDP, DeepSpeed and DDP with minimal dependencies.
  • 06Backends for NVIDIA CUDA, AMD ROCm, Intel GPU and Ascend NPU.

Use cases

  • Speeding up full fine-tuning of Llama or Qwen models on multi-GPU clusters.
  • Reducing memory to train with longer context lengths or larger vocabularies.
  • Running memory-heavy preference optimization such as DPO or ORPO on limited GPUs.
  • Adding efficient distillation losses to a training pipeline.

Topics

#llm-training#triton#gpu-kernels#fine-tuning#pytorch#deep-learning#memory-optimization#finetuning#gemma2#hacktoberfest

FAQ

What is Liger-Kernel?
Efficient Triton GPU kernels for LLM training that fuse common layers and losses to cut memory use and raise throughput. Liger Kernel is a collection of Triton kernels for training large language models. It provides drop-in replacements for common operations such as RMSNorm, RoPE, SwiGLU, CrossEntropy and a fused linear cross-entropy, which reduce memory traffic and allow longer context lengths or larger batches on the same GPUs.
Is Liger-Kernel free to use? What license does it have?
Liger-Kernel is open source under the BSD-2-Clause license. Check the license terms before using it in commercial products.
Is Liger-Kernel still maintained?
Actively maintained. The last commit to Liger-Kernel was on Oct 22, 2026.
What can I use Liger-Kernel for?
Speeding up full fine-tuning of Llama or Qwen models on multi-GPU clusters. Reducing memory to train with longer context lengths or larger vocabularies. Running memory-heavy preference optimization such as DPO or ORPO on limited GPUs. Adding efficient distillation losses to a training pipeline.
Where can I get Liger-Kernel?
The source code and installation instructions are on GitHub: https://github.com/linkedin/Liger-Kernel

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects