Repolode
Submit
NVIDIA

NVIDIA/TensorRT

NVIDIA's open-source components of TensorRT, a SDK for high-performance deep learning inference on NVIDIA GPUs, including plugins, the ONNX parser and samples.

View on GitHubWebsite
Stars
13,411
Forks
2,412
Open issues
668
Last commit
13 days ago

What is it?

TensorRT is NVIDIA's SDK for running deep learning models for inference on NVIDIA GPUs. This repository holds the open-source subset of the TensorRT release: the source code for its plugins and ONNX parser, along with sample applications that show how to use the platform. The closed-source runtime and library come from the general availability release, which this repository builds against.

The project supports building for Linux on x86-64 and aarch64 (including Jetson and DriveOS cross-compilation) and for Windows, with CMake as the build system. A containerized build workflow with Docker is provided for Linux. A pip package, tensorrt, is available for installing the Python bindings without building from source.

The repository also documents import paths such as ONNX, Torch-TensorRT, HuggingFace/Optimum and the Network Definition API, along with a per-model support matrix covering LLMs, NLP encoders, vision, audio, diffusion and multimodal models.

Key features

  • 01Open-source plugins and ONNX parser sources for TensorRT.
  • 02Sample applications demonstrating TensorRT usage.
  • 03Builds for Linux (x86-64, aarch64) and Windows using CMake.
  • 04Docker-based build containers for reproducible Linux builds.
  • 05Python package installable with pip.
  • 06Cross-compilation toolchains for Jetson and DriveOS targets.

Use cases

  • Deploying deep learning models for low-latency inference on NVIDIA GPUs.
  • Converting ONNX or PyTorch models into optimized TensorRT engines.
  • Building custom TensorRT plugins for layers not supported natively.
  • Running inference on Jetson or DriveOS embedded devices.

Quick start

  1. 1

    Install the prebuilt TensorRT Python package

    pip install tensorrt

Things to know

  • Requires an NVIDIA GPU with CUDA installed.
  • The open-source repo is a subset of the TensorRT GA release and needs a separate TensorRT build for a full native build.
  • Building from source needs large system prerequisites such as CUDA, cuDNN and TensorRT GA libraries.

Topics

#deep-learning#inference#gpu-acceleration#onnx#cuda#model-optimization#nvidia#tensorrt

FAQ

What is TensorRT?
NVIDIA's open-source components of TensorRT, a SDK for high-performance deep learning inference on NVIDIA GPUs, including plugins, the ONNX parser and samples. TensorRT is NVIDIA's SDK for running deep learning models for inference on NVIDIA GPUs. This repository holds the open-source subset of the TensorRT release: the source code for its plugins and ONNX parser, along with sample applications that show how to use the platform. The closed-source runtime and library come from the general availability release, which this repository builds against.
Is TensorRT free to use? What license does it have?
TensorRT is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is TensorRT still maintained?
Actively maintained. The last commit to TensorRT was on Oct 9, 2026.
What can I use TensorRT for?
Deploying deep learning models for low-latency inference on NVIDIA GPUs. Converting ONNX or PyTorch models into optimized TensorRT engines. Building custom TensorRT plugins for layers not supported natively. Running inference on Jetson or DriveOS embedded devices.
How do I install TensorRT?
Install the prebuilt TensorRT Python package: pip install tensorrt
Where can I get TensorRT?
The source code and installation instructions are on GitHub: https://github.com/NVIDIA/TensorRT

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects