Repolode
Submit
huggingface

huggingface/peft

Hugging Face library for Parameter-Efficient Fine-Tuning (PEFT), adapting large pretrained models by training only a small set of extra parameters.

View on GitHubWebsite
Stars
21,824
Forks
2,610
Open issues
97
Last commit
17 hours ago

What is it?

Fine-tuning large pretrained models is costly in compute and storage. PEFT provides methods such as LoRA, which train only a small number of added parameters while keeping the base model frozen. The resulting adapters are typically only a few megabytes, and the library reports performance comparable to full fine-tuning in many cases.

Usage centers on wrapping a base model with a configuration object via get_peft_model, then saving or loading the adapter with save_pretrained and PeftModel.from_pretrained. The library integrates with Hugging Face Transformers for training and inference, Diffusers for managing multiple adapters, and Accelerate for distributed training.

The project is in Python and built on PyTorch, and its documentation covers the supported methods along with conceptual guides on adapters, soft prompts and IA3.

Key features

  • 01Implements LoRA and other parameter-efficient methods such as adapters, soft prompts and IA3.
  • 02Wraps any compatible model with get_peft_model and a config object.
  • 03Saves adapters as small checkpoints that can be loaded onto the base model.
  • 04Integrates with Transformers, Diffusers and Accelerate.
  • 05Supports quantized base models for lower memory use during training.

Use cases

  • Fine-tuning a multi-billion-parameter LLM on a single consumer GPU.
  • Maintaining many task-specific adapters for one shared base model.
  • Customizing Stable Diffusion with LoRA for style or subject training.
  • Training large models in distributed setups with Accelerate.

Quick start

  1. 1

    Install PEFT from pip

    pip install peft
  2. 2

    Wrap a base model with a LoRA config using get_peft_model, then train and save it

Things to know

  • Running the README examples requires a PyTorch-compatible GPU or accelerator and downloading large pretrained models.

Topics

#fine-tuning#lora#llm#pytorch#transformers#adapter#diffusion#parameter-efficient-learning#peft#python

FAQ

What is peft?
Hugging Face library for Parameter-Efficient Fine-Tuning (PEFT), adapting large pretrained models by training only a small set of extra parameters. Fine-tuning large pretrained models is costly in compute and storage. PEFT provides methods such as LoRA, which train only a small number of added parameters while keeping the base model frozen. The resulting adapters are typically only a few megabytes, and the library reports performance comparable to full fine-tuning in many cases.
Is peft free to use? What license does it have?
peft is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is peft still maintained?
Actively maintained. The last commit to peft was on Oct 21, 2026.
What can I use peft for?
Fine-tuning a multi-billion-parameter LLM on a single consumer GPU. Maintaining many task-specific adapters for one shared base model. Customizing Stable Diffusion with LoRA for style or subject training. Training large models in distributed setups with Accelerate.
How do I install peft?
Install PEFT from pip: pip install peft Wrap a base model with a LoRA config using get_peft_model, then train and save it
Where can I get peft?
The source code and installation instructions are on GitHub: https://github.com/huggingface/peft

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects