Repolode
Submit
NVIDIA-NeMo

NVIDIA-NeMo/Speech

NVIDIA's open-source toolkit for building, fine-tuning and deploying speech AI models, covering speech recognition (ASR), text-to-speech (TTS) and speech LLMs.

View on GitHubWebsite
Stars
18,607
Forks
3,654
Open issues
368
Last commit
8 hours ago

What is it?

NeMo Speech is a PyTorch-based framework from NVIDIA for researchers and developers working on speech models. It covers automatic speech recognition, text-to-speech, and speech LLMs, and is intended to help users create, customize and deploy models by reusing existing code and pre-trained checkpoints.

The project provides pre-trained models such as streaming and offline ASR models, multilingual TTS models, and a full-duplex voice chat model, published on Hugging Face. It was split out from the broader NeMo toolkit and now focuses on audio, speech and multimodal LLMs.

Installation is supported through uv from source, through Docker with a prebuilt NGC container, or through pip on top of an existing PyTorch environment. It requires Python 3.12 or newer, PyTorch 2.7 or newer, and an NVIDIA GPU with CUDA for training.

Key features

  • 01Automatic speech recognition models, including streaming and offline modes
  • 02Text-to-speech models with multilingual support
  • 03Speech LLM and full-duplex voice conversation models
  • 04Pre-trained checkpoints published on Hugging Face
  • 05Installation via uv, Docker, or pip alongside an existing PyTorch setup

Use cases

  • Building or fine-tuning a speech recognition system for a specific language or domain
  • Deploying low-latency streaming transcription on NVIDIA GPUs
  • Creating multilingual text-to-speech voices for applications
  • Researching speech LLMs and conversational voice agents

Quick start

  1. 1

    Clone the repository

    git clone https://github.com/NVIDIA-NeMo/Speech.git
  2. 2

    Enter the project directory

    cd Speech
  3. 3

    Install the supported stack with uv (CUDA 13.x)

    uv sync --extra all --extra cu13
  4. 4

    Or install via pip into your existing PyTorch environment

    uv pip install 'nemo-toolkit[asr,tts]'

Things to know

  • Training requires an NVIDIA GPU with CUDA.
  • Requires Python 3.12 or above and PyTorch 2.7 or above.

Topics

#speech-recognition#text-to-speech#asr#tts#pytorch#speech-llm#nvidia#deeplearning#generative-ai#machine-translation

FAQ

What is Speech?
NVIDIA's open-source toolkit for building, fine-tuning and deploying speech AI models, covering speech recognition (ASR), text-to-speech (TTS) and speech LLMs. NeMo Speech is a PyTorch-based framework from NVIDIA for researchers and developers working on speech models. It covers automatic speech recognition, text-to-speech, and speech LLMs, and is intended to help users create, customize and deploy models by reusing existing code and pre-trained checkpoints.
Is Speech free to use? What license does it have?
Speech is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is Speech still maintained?
Actively maintained. The last commit to Speech was on Oct 20, 2026.
What can I use Speech for?
Building or fine-tuning a speech recognition system for a specific language or domain Deploying low-latency streaming transcription on NVIDIA GPUs Creating multilingual text-to-speech voices for applications Researching speech LLMs and conversational voice agents
How do I install Speech?
Clone the repository: git clone https://github.com/NVIDIA-NeMo/Speech.git Enter the project directory: cd Speech Install the supported stack with uv (CUDA 13.x): uv sync --extra all --extra cu13 Or install via pip into your existing PyTorch environment: uv pip install 'nemo-toolkit[asr,tts]'
Where can I get Speech?
The source code and installation instructions are on GitHub: https://github.com/NVIDIA-NeMo/Speech

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects