Repolode
Submit
meta-pytorch

meta-pytorch/captum

A model interpretability library for PyTorch that explains which input features, neurons and training examples drive a model's predictions.

View on GitHubWebsite
Stars
5,720
Forks
574
Open issues
91
Last commit
7 days ago

What is it?

Captum is a model interpretability and understanding library for PyTorch. It provides general-purpose implementations of attribution algorithms such as Integrated Gradients, saliency maps, SmoothGrad, DeepLift and GradCAM, which assign an importance score to each input element for a given prediction.

Beyond input attribution, Captum can measure how much individual neurons and layers contribute to an output, and includes TracIn-based methods that trace predictions back to influential training examples. It also offers adversarial attacks and minimal input perturbation tools for generating counterfactual explanations. Integration with models built on torchvision and torchtext is described as straightforward.

The library requires Python 3.10 or newer and PyTorch 2.3 or newer, and is installable with pip or conda.

Key features

  • 01Attribution algorithms including Integrated Gradients, DeepLift, GradientShap and Saliency
  • 02Neuron and layer importance analysis via conductance methods
  • 03Training data influence estimation with TracIn variants
  • 04NoiseTunnel for smoothing attributions with SmoothGrad and VarGrad
  • 05Adversarial attacks and input perturbation for counterfactual explanations

Use cases

  • Checking which input pixels or tokens a PyTorch image or text classifier relies on
  • Debugging unexpected model outputs by tracing them to influential training examples
  • Benchmarking new interpretability algorithms against existing implementations
  • Providing feature-level explanations for recommendations or predictions in production

Quick start

  1. 1

    Install Captum from PyPI

    pip install captum
  2. 2

    Clone the repository for a manual dev install

    git clone https://github.com/pytorch/captum.git
  3. 3

    Enter the project directory and install in editable mode

    cd captum && pip install -e .

Things to know

  • Requires Python 3.10 or newer and PyTorch 2.3 or newer.
  • DeepLift currently supports only the Rescale rule and not all non-linear activation types.

Topics

#pytorch#interpretability#explainable-ai#feature-attribution#deep-learning#model-debugging#feature-importance#interpretable-ai#interpretable-ml

FAQ

What is captum?
A model interpretability library for PyTorch that explains which input features, neurons and training examples drive a model's predictions. Captum is a model interpretability and understanding library for PyTorch. It provides general-purpose implementations of attribution algorithms such as Integrated Gradients, saliency maps, SmoothGrad, DeepLift and GradCAM, which assign an importance score to each input element for a given prediction.
Is captum free to use? What license does it have?
captum is open source under the BSD-3-Clause license. Check the license terms before using it in commercial products.
Is captum still maintained?
Actively maintained. The last commit to captum was on Nov 1, 2026.
What can I use captum for?
Checking which input pixels or tokens a PyTorch image or text classifier relies on Debugging unexpected model outputs by tracing them to influential training examples Benchmarking new interpretability algorithms against existing implementations Providing feature-level explanations for recommendations or predictions in production
How do I install captum?
Install Captum from PyPI: pip install captum Clone the repository for a manual dev install: git clone https://github.com/pytorch/captum.git Enter the project directory and install in editable mode: cd captum && pip install -e .
Where can I get captum?
The source code and installation instructions are on GitHub: https://github.com/meta-pytorch/captum

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects