Repolode
Submit
QuentinFuxa

QuentinFuxa/WhisperLiveKit

Self-hosted real-time speech-to-text server with streaming ASR, speaker diarization, live translation and OpenAI/Deepgram-compatible APIs.

Open-source alternative to:Otter.aiDeepgram

View on GitHub
Stars
11,155
Forks
1,149
Open issues
17
Last commit
19 days ago

What is it?

WhisperLiveKit (WLK) is a self-hosted speech-to-text pipeline built for low-latency transcription. Instead of running Whisper on fixed audio batches, it uses streaming policies such as AlignAtt SimulStreaming and LocalAgreement, with voice activity detection, to emit partial and stable text while a speaker is still talking.

The package provides a CLI with a web UI served over WebSocket at /asr, plus an OpenAI-compatible REST endpoint for file transcription and a Deepgram-compatible WebSocket. It can also transcribe files directly, generate SRT subtitles, and manage models. Backends include Faster-Whisper, MLX Whisper for Apple Silicon, Voxtral, Qwen3-ASR, NVIDIA Canary and FunASR SenseVoiceSmall, and optional components add speaker diarization (Sortformer or Diart) and translation through NLLB or LLM-based translation.

The server handles multiple concurrent connections, and the project supports HTTPS, API tokens, CORS settings and deployment behind Nginx or Gunicorn. Docker images are provided for GPU and CPU setups.

Key features

  • 01Streaming transcription with partial and stable text output over WebSocket.
  • 02Speaker diarization using Sortformer or Diart.
  • 03Simultaneous translation via NLLB or LLM-based backends.
  • 04OpenAI-compatible transcription endpoint and Deepgram-compatible WebSocket.
  • 05Command-line tools for live serving, file transcription, SRT subtitles and model management.
  • 06Multiple ASR backends, including Apple Silicon MLX and NVIDIA GPU options.

Use cases

  • Real-time meeting transcription with speaker labels.
  • Accessibility tools that help hearing-impaired users follow conversations.
  • Generating subtitles for podcasts or videos locally.
  • Transcribing customer support calls with speaker identification.

Quick start

  1. 1

    Install the package from PyPI

    pip install whisperlivekit
  2. 2

    Start the server, then open http://localhost:8000 and speak

    wlk --model base --language en
  3. 3

    Or auto-pull a model and start the server in one step

    wlk run whisper:tiny

Things to know

  • Some optional backends require specific hardware such as NVIDIA CUDA GPUs or Apple Silicon.
  • Some heavy extras conflict with each other and must be installed in separate environments.
  • The causal Qwen3-ASR mode is English-only.

Topics

#speech-to-text#whisper#real-time#speaker-diarization#streaming#translation#websocket#self-hosted#automatic-speech-recognition#fastapi

FAQ

What is WhisperLiveKit?
Self-hosted real-time speech-to-text server with streaming ASR, speaker diarization, live translation and OpenAI/Deepgram-compatible APIs. WhisperLiveKit (WLK) is a self-hosted speech-to-text pipeline built for low-latency transcription. Instead of running Whisper on fixed audio batches, it uses streaming policies such as AlignAtt SimulStreaming and LocalAgreement, with voice activity detection, to emit partial and stable text while a speaker is still talking.
Is WhisperLiveKit free to use? What license does it have?
WhisperLiveKit is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is WhisperLiveKit still maintained?
Actively maintained. The last commit to WhisperLiveKit was on Oct 1, 2026.
What can I use WhisperLiveKit for?
Real-time meeting transcription with speaker labels. Accessibility tools that help hearing-impaired users follow conversations. Generating subtitles for podcasts or videos locally. Transcribing customer support calls with speaker identification.
How do I install WhisperLiveKit?
Install the package from PyPI: pip install whisperlivekit Start the server, then open http://localhost:8000 and speak: wlk --model base --language en Or auto-pull a model and start the server in one step: wlk run whisper:tiny
Where can I get WhisperLiveKit?
The source code and installation instructions are on GitHub: https://github.com/QuentinFuxa/WhisperLiveKit

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects