Repolode
Submit
huggingface

huggingface/optimum

Hugging Face's toolkit for exporting and running Transformers, Diffusers, TIMM and Sentence-Transformers models on specific hardware backends with optimized inference and training.

View on GitHubWebsite
Stars
3,519
Forks
719
Open issues
313
Last commit
5 days ago

What is it?

Optimum extends Hugging Face's Transformers, Diffusers, TIMM and Sentence-Transformers libraries with tools for running and training models efficiently on targeted hardware. It keeps the same high-level API, so models loaded through these libraries can be exported and optimized with a few extra steps.

For inference, Optimum covers export and runtime paths for ONNX and ONNX Runtime, OpenVINO for Intel hardware, ExecuTorch for mobile and edge devices, AWS Inferentia, NVIDIA TensorRT-LLM, and others. Export and optimization can be run from Python code or from a command line. Quantization is also supported through the separate Quanto backend for PyTorch.

For training, Optimum provides wrappers around the Transformers Trainer for Intel Gaudi accelerators and AWS Trainium, along with ONNX Runtime acceleration for GPUs. Several backends have moved into their own companion packages such as optimum-onnx and optimum-intel, so installation depends on the target accelerator.

Key features

  • 01Exports Transformers, Diffusers, TIMM and Sentence-Transformers models to ONNX format.
  • 02Runs exported models through ONNX Runtime via ORTModel classes.
  • 03Supports OpenVINO and NNCF optimization for Intel hardware.
  • 04Provides Trainer wrappers for Intel Gaudi and AWS Trainium training.
  • 05Offers PyTorch quantization through the Quanto backend.

Use cases

  • Deploying a Transformers model on CPUs or Intel hardware with OpenVINO.
  • Exporting a Hugging Face model to ONNX for production inference.
  • Fine-tuning models on Intel Gaudi or AWS Trainium instances.
  • Running quantized PyTorch models with reduced memory use.

Quick start

  1. 1

    Install Optimum with pip

    python -m pip install optimum
  2. 2

    Install accelerator-specific extras for your hardware (e.g. ONNX Runtime)

    pip install --upgrade --upgrade-strategy eager optimum[onnxruntime]

Things to know

  • Accelerator features require extra, backend-specific installs and hardware.
  • Some backends have moved to separate packages, such as optimum-onnx and optimum-executorch.

Topics

#inference-optimization#onnx#onnxruntime#transformers#quantization#pytorch#model-export#hardware-acceleration#graphcore#habana

FAQ

What is optimum?
Hugging Face's toolkit for exporting and running Transformers, Diffusers, TIMM and Sentence-Transformers models on specific hardware backends with optimized inference and training. Optimum extends Hugging Face's Transformers, Diffusers, TIMM and Sentence-Transformers libraries with tools for running and training models efficiently on targeted hardware. It keeps the same high-level API, so models loaded through these libraries can be exported and optimized with a few extra steps.
Is optimum free to use? What license does it have?
optimum is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is optimum still maintained?
Actively maintained. The last commit to optimum was on Nov 2, 2026.
What can I use optimum for?
Deploying a Transformers model on CPUs or Intel hardware with OpenVINO. Exporting a Hugging Face model to ONNX for production inference. Fine-tuning models on Intel Gaudi or AWS Trainium instances. Running quantized PyTorch models with reduced memory use.
How do I install optimum?
Install Optimum with pip: python -m pip install optimum Install accelerator-specific extras for your hardware (e.g. ONNX Runtime): pip install --upgrade --upgrade-strategy eager optimum[onnxruntime]
Where can I get optimum?
The source code and installation instructions are on GitHub: https://github.com/huggingface/optimum

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects