Repolode
Submit
lucidrains

lucidrains/vit-pytorch

A PyTorch implementation of the Vision Transformer (ViT) and many of its variants for image classification and related vision tasks.

View on GitHub
Stars
25,550
Forks
3,497
Open issues
150
Last commit
8 days ago

What is it?

vit-pytorch is a Python library that implements the Vision Transformer, which classifies images using a single transformer encoder applied to image patches. Beyond the original model, the repository collects a large number of follow-up architectures, including Simple ViT, NaViT, DeepViT, CaiT, CCT, CrossViT, MaxViT, MobileViT and XCiT, each usable through a short import and a constructor with configurable depth, heads and dimensions.

The library also includes self-supervised and pretraining modules such as Masked Autoencoder (MAE), SimMIM, masked patch prediction and DINO, along with distillation from a teacher network into a ViT. Video variants are covered by 3D ViT and ViViT, which add frame and frame-patch parameters.

The code is intended as a readable reference for building and experimenting with transformer-based vision models. It provides architecture implementations rather than pretrained weights, and the repository points users to other projects for pretrained checkpoints.

Key features

  • 01Implements the original Vision Transformer with a configurable constructor for image size, patch size and depth.
  • 02Provides many ViT variants such as Simple ViT, DeepViT, CaiT, CCT, MaxViT and XCiT.
  • 03Includes masked-image pretraining modules such as MAE, SimMIM and masked position prediction.
  • 04Offers knowledge distillation from a teacher network into a ViT.
  • 05Supports video inputs through 3D ViT, ViViT and 3D CCT.

Use cases

  • Experimenting with transformer architectures for image classification in PyTorch.
  • Studying how different ViT variants change attention and downsampling designs.
  • Pretraining a vision transformer on unlabelled images with masked autoencoding.
  • Prototyping video classification models with factorized spatio-temporal attention.

Quick start

  1. 1

    Install the package from PyPI

    pip install vit-pytorch
  2. 2

    Import ViT, create a model, and run a forward pass on a random image

Things to know

  • The README provides no pretrained model weights; you must train the models yourself or use an external repository for checkpoints.

Topics

#vision-transformer#pytorch#image-classification#computer-vision#transformers#deep-learning#artificial-intelligence#attention-mechanism

FAQ

What is vit-pytorch?
A PyTorch implementation of the Vision Transformer (ViT) and many of its variants for image classification and related vision tasks. vit-pytorch is a Python library that implements the Vision Transformer, which classifies images using a single transformer encoder applied to image patches. Beyond the original model, the repository collects a large number of follow-up architectures, including Simple ViT, NaViT, DeepViT, CaiT, CCT, CrossViT, MaxViT, MobileViT and XCiT, each usable through a short import and a constructor with configurable depth, heads and dimensions.
Is vit-pytorch free to use? What license does it have?
vit-pytorch is open source under the MIT license. Check the license terms before using it in commercial products.
Is vit-pytorch still maintained?
Actively maintained. The last commit to vit-pytorch was on Oct 13, 2026.
What can I use vit-pytorch for?
Experimenting with transformer architectures for image classification in PyTorch. Studying how different ViT variants change attention and downsampling designs. Pretraining a vision transformer on unlabelled images with masked autoencoding. Prototyping video classification models with factorized spatio-temporal attention.
How do I install vit-pytorch?
Install the package from PyPI: pip install vit-pytorch Import ViT, create a model, and run a forward pass on a random image
Where can I get vit-pytorch?
The source code and installation instructions are on GitHub: https://github.com/lucidrains/vit-pytorch

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects