Repolode
Submit
Lightning-AI

Lightning-AI/litgpt

A PyTorch-based toolkit of from-scratch LLM implementations for pretraining, finetuning, evaluating and serving 20+ open-weight models.

View on GitHubWebsite
Stars
13,724
Forks
1,527
Open issues
315
Last commit
9 days ago

What is it?

LitGPT is a Python toolkit for working with large language models. Each supported model is implemented from scratch in a single file with few internal abstraction layers, so the code is easy to read and modify. It covers 20+ model families, including Llama, Gemma, Qwen, Phi, Mistral and Falcon, and provides a CLI and Python API to download weights, chat, finetune, pretrain, evaluate and serve them.

The project targets both experimentation and scaled training. It supports LoRA, QLoRA and adapter-based parameter-efficient finetuning, low-precision and quantized settings (including 4-bit and 8-bit), Flash Attention, and multi-GPU training through fully-sharded data parallelism. A set of YAML training recipes is included for common model and hardware configurations, and checkpoints can be exported to other model weight formats.

LitGPT is built on Lightning Fabric and extends the Lit-LLaMA and nanoGPT codebases. It is released under the Apache 2.0 license.

Key features

  • 01Single-file, from-scratch model implementations with minimal abstraction layers.
  • 02Finetuning with LoRA, QLoRA, Adapter and Adapter v2.
  • 03Pretraining and continued pretraining on custom text data.
  • 04Quantization to 4-bit and 8-bit, plus FP16/BF16 mixed precision.
  • 05Multi-GPU training via fully-sharded data parallelism and TPU/XLA support.
  • 06Serving models over a local web API and interactive chat from the CLI.

Use cases

  • Finetuning an open-weight model like Phi or Llama on a domain-specific instruction dataset.
  • Studying or modifying LLM architecture code without framework indirection.
  • Running a 4-bit quantized model on a GPU with limited memory.
  • Deploying a finetuned checkpoint as an HTTP inference endpoint.

Quick start

  1. 1

    Install LitGPT with pip

    pip install 'litgpt[extra]'
  2. 2

    Load a model with the Python API and generate text (from the README example)

Things to know

  • Training and finetuning larger models realistically require GPU hardware.

Topics

#llm#pytorch#finetuning#pretraining#lora#quantization#llm-inference#deep-learning#ai#artificial-intelligence

FAQ

What is litgpt?
A PyTorch-based toolkit of from-scratch LLM implementations for pretraining, finetuning, evaluating and serving 20+ open-weight models. LitGPT is a Python toolkit for working with large language models. Each supported model is implemented from scratch in a single file with few internal abstraction layers, so the code is easy to read and modify. It covers 20+ model families, including Llama, Gemma, Qwen, Phi, Mistral and Falcon, and provides a CLI and Python API to download weights, chat, finetune, pretrain, evaluate and serve them.
Is litgpt free to use? What license does it have?
litgpt is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is litgpt still maintained?
Actively maintained. The last commit to litgpt was on Oct 13, 2026.
What can I use litgpt for?
Finetuning an open-weight model like Phi or Llama on a domain-specific instruction dataset. Studying or modifying LLM architecture code without framework indirection. Running a 4-bit quantized model on a GPU with limited memory. Deploying a finetuned checkpoint as an HTTP inference endpoint.
How do I install litgpt?
Install LitGPT with pip: pip install 'litgpt[extra]' Load a model with the Python API and generate text (from the README example)
Where can I get litgpt?
The source code and installation instructions are on GitHub: https://github.com/Lightning-AI/litgpt

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects