Repolode
Submit
openvinotoolkit

openvinotoolkit/openvino

Open-source toolkit for optimizing and deploying deep learning models for inference on Intel CPUs, GPUs and NPUs.

View on GitHubWebsite
Stars
11,028
Forks
3,472
Open issues
883
Last commit
6 hours ago

What is it?

OpenVINO is an open-source toolkit from the OpenVINO project for optimizing and deploying AI inference. It takes models trained in frameworks such as PyTorch, TensorFlow, ONNX, Keras, PaddlePaddle and JAX/Flax, converts them without needing the original framework at deploy time, and runs them efficiently on supported hardware.

Inference targets include x86 and ARM CPUs, Intel integrated and discrete GPUs, and Intel NPUs. The project provides APIs in C++, Python, C and NodeJS, plus a GenAI API for LLM and diffusion model pipelines. Models from the Hugging Face Hub can be used through Optimum Intel.

The ecosystem includes Neural Network Compression Framework (NNCF) for quantization and sparsity, OpenVINO Model Server for serving, and integrations with vLLM, ONNX Runtime, LangChain, LlamaIndex and torch.compile.

Key features

  • 01Converts PyTorch, TensorFlow, ONNX, Keras, PaddlePaddle and JAX/Flax models for inference.
  • 02Runs inference on Intel CPUs, GPUs and NPUs, including x86 and ARM CPUs.
  • 03Provides C++, Python, C and NodeJS APIs.
  • 04Includes a GenAI API for LLM and diffusion model pipelines.
  • 05Offers model compression techniques such as quantization and sparsity via NNCF.

Use cases

  • Deploying computer vision models such as YOLO on Intel edge or server hardware.
  • Running local LLM chatbots or text-to-image pipelines on Intel CPUs or GPUs.
  • Accelerating speech recognition models such as Whisper on supported devices.
  • Integrating optimized inference into existing ONNX Runtime or vLLM workflows.

Quick start

  1. 1

    Install OpenVINO with pip

    pip install -U openvino
  2. 2

    Verify the installation by printing the version

    python -c "import openvino as ov; print(ov.__version__)"

Things to know

  • Targets Intel hardware (CPU, GPU, NPU) for best results, with limited benefit on other accelerators.

Topics

#inference#deep-learning#model-optimization#llm-inference#computer-vision#edge-ai#intel#ai#deploy-ai#diffusion-models

FAQ

What is openvino?
Open-source toolkit for optimizing and deploying deep learning models for inference on Intel CPUs, GPUs and NPUs. OpenVINO is an open-source toolkit from the OpenVINO project for optimizing and deploying AI inference. It takes models trained in frameworks such as PyTorch, TensorFlow, ONNX, Keras, PaddlePaddle and JAX/Flax, converts them without needing the original framework at deploy time, and runs them efficiently on supported hardware.
Is openvino free to use? What license does it have?
openvino is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is openvino still maintained?
Actively maintained. The last commit to openvino was on Oct 20, 2026.
What can I use openvino for?
Deploying computer vision models such as YOLO on Intel edge or server hardware. Running local LLM chatbots or text-to-image pipelines on Intel CPUs or GPUs. Accelerating speech recognition models such as Whisper on supported devices. Integrating optimized inference into existing ONNX Runtime or vLLM workflows.
How do I install openvino?
Install OpenVINO with pip: pip install -U openvino Verify the installation by printing the version: python -c "import openvino as ov; print(ov.__version__)"
Where can I get openvino?
The source code and installation instructions are on GitHub: https://github.com/openvinotoolkit/openvino

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects