Repolode
Submit
understandable-machine-intelligence-lab

understandable-machine-intelligence-lab/Quantus

An open-source Python toolkit for quantitatively evaluating neural network explanations with 30+ XAI metrics across PyTorch and TensorFlow.

View on GitHubWebsite
Stars
680
Forks
94
Open issues
66
Last commit
12 days ago

What is it?

Quantus is a toolkit for evaluating explanations produced by explainable AI (XAI) methods for neural networks. Visually comparing saliency maps is often not enough to tell which method works best, and ground truth is usually unavailable, so Quantus provides quantitative metrics to compare explanation methods against each other.

The metrics are grouped into six categories: faithfulness, robustness, localisation, complexity, randomisation (sensitivity), and axiomatic properties. The library supports image, tabular and time-series data, and models built in PyTorch or TensorFlow. Users can pass pre-computed attributions or an explanation function, and can run several metrics across several XAI methods with a single call to quantus.evaluate().

Explanation methods from Captum, Zennit and tf-explain are supported directly. The project is installable from PyPI and was published in the Journal of Machine Learning Research (MLOSS track).

Key features

  • 01Offers more than 30 evaluation metrics across six categories, including faithfulness, robustness and localisation.
  • 02Works with PyTorch and TensorFlow models.
  • 03Supports image, tabular and time-series data.
  • 04Accepts pre-computed attributions or any explanation function.
  • 05Provides quantus.evaluate() for running many metrics across many XAI methods at once.
  • 06Integrates with Captum, Zennit and tf-explain.

Use cases

  • Benchmarking which explanation method best fits a trained image classifier.
  • Checking whether explanations stay stable under small input perturbations.
  • Comparing explanation quality for tabular models in a research pipeline.
  • Reproducing published XAI evaluation metrics in a consistent implementation.

Things to know

  • The license is not clearly identified (listed as NOASSERTION), so check the repository's license terms before use.
  • The README says the project is under active development, so pin a specific release version to keep results reproducible.

Topics

#explainable-ai#interpretability#deep-learning#pytorch#tensorflow#model-evaluation#machine-learning#quantification-evaluation-methods#reproducibility#xai

FAQ

What is Quantus?
An open-source Python toolkit for quantitatively evaluating neural network explanations with 30+ XAI metrics across PyTorch and TensorFlow. Quantus is a toolkit for evaluating explanations produced by explainable AI (XAI) methods for neural networks. Visually comparing saliency maps is often not enough to tell which method works best, and ground truth is usually unavailable, so Quantus provides quantitative metrics to compare explanation methods against each other.
Is Quantus free to use? What license does it have?
Quantus is open source under the Other license. Check the license terms before using it in commercial products.
Is Quantus still maintained?
Actively maintained. The last commit to Quantus was on Oct 26, 2026.
What can I use Quantus for?
Benchmarking which explanation method best fits a trained image classifier. Checking whether explanations stay stable under small input perturbations. Comparing explanation quality for tabular models in a research pipeline. Reproducing published XAI evaluation metrics in a consistent implementation.
Where can I get Quantus?
The source code and installation instructions are on GitHub: https://github.com/understandable-machine-intelligence-lab/Quantus

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects