Repolode
Submit
kyutai-labs

kyutai-labs/moshi

A speech-text foundation model and full-duplex spoken dialogue framework that talks and listens at the same time, with a streaming neural audio codec called Mimi.

View on GitHub
Stars
11,263
Forks
1,044
Open issues
97
Last commit
last month

What is it?

Moshi is a speech-text foundation model and full-duplex spoken dialogue framework from Kyutai. It models two audio streams at once, one for the user and one for Moshi itself, so it can speak and listen simultaneously rather than waiting for turns. Moshi also predicts text tokens for its own speech as an inner monologue, which the project credits for better generation quality.

The project includes Mimi, a streaming neural audio codec that encodes 24 kHz audio down to a 12.5 Hz representation at 1.1 kbps with an 80 ms frame size. The repository provides three inference stacks: PyTorch for research, MLX for on-device use on Mac and iPhone, and Rust for production, which also includes Mimi bindings exposed to Python as rustymimi. A web UI client, a command-line client and a Gradio demo are included, and pretrained weights are published on Hugging Face.

Running the PyTorch version requires a GPU with about 24 GB of memory, since quantization is not yet supported there. Windows is not officially supported.

Key features

  • 01Full-duplex dialogue that models user and model audio streams simultaneously.
  • 02Mimi streaming audio codec at 12.5 Hz and 1.1 kbps with about 80 ms frame size.
  • 03PyTorch, MLX and Rust inference backends.
  • 04MLX version runs locally on macOS with 4-bit or 8-bit quantized weights.
  • 05Web UI, command-line and Gradio clients, plus Docker Compose for CUDA.

Use cases

  • Building real-time voice assistants that can be interrupted and respond without turn-taking delays.
  • Running a speech dialogue model locally on a Mac or iPhone with MLX.
  • Researching streaming neural audio codecs and speech-text models.
  • Using Mimi as a low-bitrate streaming audio tokenizer in Python or Rust.

Quick start

  1. 1

    Install the PyTorch client from PyPI

    pip install -U moshi
  2. 2

    Start the server (downloads the default model)

    python -m moshi.server
  3. 3

    Open the web UI in your browser at localhost:8998

Things to know

  • The PyTorch version does not support quantization and needs a GPU with about 24GB of memory.
  • Windows is not officially supported.
  • Model weights are released under CC-BY 4.0, not an open software license.

Topics

#speech#audio#dialogue#speech-to-speech#codec#pytorch#rust#mlx

FAQ

What is moshi?
A speech-text foundation model and full-duplex spoken dialogue framework that talks and listens at the same time, with a streaming neural audio codec called Mimi. Moshi is a speech-text foundation model and full-duplex spoken dialogue framework from Kyutai. It models two audio streams at once, one for the user and one for Moshi itself, so it can speak and listen simultaneously rather than waiting for turns. Moshi also predicts text tokens for its own speech as an inner monologue, which the project credits for better generation quality.
Is moshi free to use? What license does it have?
moshi is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is moshi still maintained?
Updated within 6 months. The last commit to moshi was on Sep 9, 2026.
What can I use moshi for?
Building real-time voice assistants that can be interrupted and respond without turn-taking delays. Running a speech dialogue model locally on a Mac or iPhone with MLX. Researching streaming neural audio codecs and speech-text models. Using Mimi as a low-bitrate streaming audio tokenizer in Python or Rust.
How do I install moshi?
Install the PyTorch client from PyPI: pip install -U moshi Start the server (downloads the default model): python -m moshi.server Open the web UI in your browser at localhost:8998
Where can I get moshi?
The source code and installation instructions are on GitHub: https://github.com/kyutai-labs/moshi

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects