Repolode
Submit
microsoft

microsoft/VibeVoice

Open-source voice AI family from Microsoft, with long-form speech recognition (ASR), multi-speaker text-to-speech (TTS), and real-time streaming TTS models.

View on GitHubWebsite
Stars
54,889
Forks
6,175
Open issues
201
Last commit
13 days ago

What is it?

VibeVoice is an open-source family of voice AI models from Microsoft that covers both speech-to-text and text-to-speech. Its ASR model, VibeVoice-ASR, transcribes up to 60 minutes of audio in a single pass and outputs who spoke, when, and what was said, with support for custom hotwords and streaming recognition.

On the generation side, VibeVoice-TTS synthesizes conversational audio up to 90 minutes long with up to four distinct speakers, and VibeVoice-Realtime-0.5B is a lightweight model that accepts streaming text input for low-latency speech output. The models use continuous speech tokenizers running at 7.5 Hz, combined with a language model and a diffusion head.

The repository also includes a finetuning path for the ASR model, vLLM-based serving, and a CPU-oriented BitNet ASR variant. Weights are published on Hugging Face, and the project notes that it is intended for research and development use.

Key features

  • 01Transcribes up to 60 minutes of audio in one pass with speaker labels and timestamps.
  • 02Supports custom hotwords to improve recognition of domain-specific terms.
  • 03Offers streaming speech recognition that emits text per audio chunk.
  • 04Generates multi-speaker conversational speech up to 90 minutes with up to four speakers.
  • 05Provides a 0.5B real-time TTS model that accepts streaming text input.
  • 06Includes ASR finetuning code and vLLM inference support.

Use cases

  • Transcribing long meetings, interviews, or podcasts with speaker attribution.
  • Generating multi-speaker podcast-style audio from scripts.
  • Building low-latency voice assistants that stream text into speech.
  • Running speech recognition experiments with finetuning on custom data.

Quick start

  1. 1

    Open the Colab notebook for the real-time streaming TTS model

  2. 2

    Try the ASR model in the online Playground

Things to know

  • The README says the models are intended for research and development only and are not recommended for commercial use without further testing.
  • The VibeVoice-TTS code was removed from this repository after misuse was discovered.
  • The README warns that outputs may be inaccurate or biased and that the models can be misused to create deepfakes.

Topics

#speech-recognition#text-to-speech#asr#tts#speech-synthesis#diarization#streaming#pytorch

FAQ

What is VibeVoice?
Open-source voice AI family from Microsoft, with long-form speech recognition (ASR), multi-speaker text-to-speech (TTS), and real-time streaming TTS models. VibeVoice is an open-source family of voice AI models from Microsoft that covers both speech-to-text and text-to-speech. Its ASR model, VibeVoice-ASR, transcribes up to 60 minutes of audio in a single pass and outputs who spoke, when, and what was said, with support for custom hotwords and streaming recognition.
Is VibeVoice free to use? What license does it have?
VibeVoice is open source under the MIT license. Check the license terms before using it in commercial products.
Is VibeVoice still maintained?
Actively maintained. The last commit to VibeVoice was on Oct 8, 2026.
What can I use VibeVoice for?
Transcribing long meetings, interviews, or podcasts with speaker attribution. Generating multi-speaker podcast-style audio from scripts. Building low-latency voice assistants that stream text into speech. Running speech recognition experiments with finetuning on custom data.
How do I install VibeVoice?
Open the Colab notebook for the real-time streaming TTS model Try the ASR model in the online Playground
Where can I get VibeVoice?
The source code and installation instructions are on GitHub: https://github.com/microsoft/VibeVoice

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects