Repolode
Submit
m-bain

m-bain/whisperX

Open-source speech-to-text built on Whisper that adds word-level timestamps via forced alignment and optional speaker diarization.

Open-source alternative to:Otter.aiDescript

View on GitHub
Stars
24,693
Forks
2,495
Open issues
229
Last commit
2 days ago

What is it?

WhisperX is a Python tool for automatic speech recognition that wraps OpenAI's Whisper with batched inference, using the faster-whisper backend. Whisper alone gives timestamps per utterance, often off by several seconds; WhisperX adds a wav2vec2-based forced alignment step to produce accurate timing for each word.

The pipeline also uses voice activity detection to split audio before transcription, which the project says reduces hallucination and enables batching. Speaker diarization via pyannote-audio labels each segment or word with a speaker ID, and the number of speakers can be constrained when known.

It is available on PyPI and works from a command line (`whisperx audio.wav`) or through a Python API. Alignment models are chosen automatically for languages such as English, French, German, Spanish and Italian, and CPU or GPU execution is supported.

Key features

  • 01Batched Whisper inference with the faster-whisper backend for faster transcription.
  • 02Word-level timestamps produced by wav2vec2 forced alignment.
  • 03Speaker diarization with speaker ID labels via pyannote-audio.
  • 04Voice activity detection preprocessing to reduce hallucination.
  • 05Command-line and Python interfaces, with CPU and CUDA GPU support.

Use cases

  • Generating subtitles (SRT) with accurate word timing from recorded video or audio.
  • Transcribing multi-speaker meetings or interviews with speaker labels.
  • Building speech datasets or research pipelines that need aligned transcripts.
  • Transcribing non-English audio using language-specific alignment models.

Quick start

  1. 1

    Install CUDA toolkit 12.8 first (skip if CPU only)

  2. 2

    Install WhisperX from PyPI

    pip install whisperx
  3. 3

    Transcribe an audio file

    whisperx path/to/audio.wav

Things to know

  • Needs an NVIDIA GPU with CUDA 12.8 for fast GPU use; CPU runs need int8 compute type.
  • Diarization requires a Hugging Face token and accepting the pyannote model user agreement.
  • Words with characters missing from the alignment model dictionary (e.g. numbers with symbols) get no timing.
  • Other languages need a language-specific phoneme alignment model.

Topics

#speech-recognition#whisper#speech-to-text#transcription#speaker-diarization#word-timestamps#subtitles#asr#speech

FAQ

What is whisperX?
Open-source speech-to-text built on Whisper that adds word-level timestamps via forced alignment and optional speaker diarization. WhisperX is a Python tool for automatic speech recognition that wraps OpenAI's Whisper with batched inference, using the faster-whisper backend. Whisper alone gives timestamps per utterance, often off by several seconds; WhisperX adds a wav2vec2-based forced alignment step to produce accurate timing for each word.
Is whisperX free to use? What license does it have?
whisperX is open source under the BSD-2-Clause license. Check the license terms before using it in commercial products.
Is whisperX still maintained?
Actively maintained. The last commit to whisperX was on Oct 18, 2026.
What can I use whisperX for?
Generating subtitles (SRT) with accurate word timing from recorded video or audio. Transcribing multi-speaker meetings or interviews with speaker labels. Building speech datasets or research pipelines that need aligned transcripts. Transcribing non-English audio using language-specific alignment models.
How do I install whisperX?
Install CUDA toolkit 12.8 first (skip if CPU only) Install WhisperX from PyPI: pip install whisperx Transcribe an audio file: whisperx path/to/audio.wav
Where can I get whisperX?
The source code and installation instructions are on GitHub: https://github.com/m-bain/whisperX

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects