Repolode
Submit
netease-youdao

netease-youdao/Confucius4-R2T2

A low-latency streaming speech recognition model for Chinese and English, built on Qwen3-ASR-1.7B, that commits text monotonically without revising earlier words.

Open-source alternative to:Google Cloud Speech-to-TextAssemblyAI

View on GitHub
Stars
858
Forks
80
Open issues
6
Last commit
24 hours ago

What is it?

Confucius4-R2T2 is a streaming automatic speech recognition model from NetEase Youdao. It processes audio in configurable chunks from 80 ms to 2 s and operates in a monotonic output mode, so committed transcript text is never revised. This suits live captioning, simultaneous subtitling, and pipelines where downstream code acts on text as soon as it appears.

The model adapts Qwen3-ASR-1.7B using Longest Stable Prefix learning on about 1,500 hours of Chinese and English audio. Training updates only the language decoder, keeping the audio encoder and projector frozen, and adds an explicit Wait action so the model knows when it needs more audio. A provisional-token budget lets users trade recognition quality against emission lag without retraining.

The same checkpoint supports offline full-context transcription and streaming. Inference runs on vLLM, through a Hugging Face transformers route, or with optional llama.cpp decoding. Topic and vocabulary hints are supported, and a WebSocket server with a reference Python client is included.

Key features

  • 01Streaming chunk sizes configurable from 80 ms to 2 s.
  • 02Committed transcript text is never revised, reducing on-screen flicker.
  • 03Provisional-token budget trades recognition quality against emission lag.
  • 04Offline full-context and streaming inference from one checkpoint.
  • 05Inference through vLLM, Hugging Face transformers, or llama.cpp.
  • 06Context and hotword hints for topics and vocabulary.
  • 07WebSocket streaming server with a reference Python client.

Use cases

  • Live captioning for meetings, streams, or lectures.
  • Simultaneous speech translation where unstable text would cause errors.
  • Feeding real-time transcripts into LLM agents or NLP pipelines.
  • Chinese and English voice input in latency-sensitive applications.

Quick start

  1. 1

    Clone the repository (see the Installation section of the README)

  2. 2

    Start the recommended Docker container and run the example, as described in the README's Docker section

Things to know

  • The repository license metadata is unclear, though the README badge links to Apache 2.0 for the code.
  • Reported emission lag excludes serving and network time, so real-world latency may be higher.

In collections

AI video & audio toolsStreaming tools

Topics

#speech-recognition#asr#streaming#real-time-transcription#speech-to-text#chinese#vllm#llm

FAQ

What is Confucius4-R2T2?
A low-latency streaming speech recognition model for Chinese and English, built on Qwen3-ASR-1.7B, that commits text monotonically without revising earlier words. Confucius4-R2T2 is a streaming automatic speech recognition model from NetEase Youdao. It processes audio in configurable chunks from 80 ms to 2 s and operates in a monotonic output mode, so committed transcript text is never revised. This suits live captioning, simultaneous subtitling, and pipelines where downstream code acts on text as soon as it appears.
Is Confucius4-R2T2 free to use? What license does it have?
Confucius4-R2T2 is open source under the Other license. Check the license terms before using it in commercial products.
Is Confucius4-R2T2 still maintained?
Actively maintained. The last commit to Confucius4-R2T2 was on Nov 6, 2026.
What can I use Confucius4-R2T2 for?
Live captioning for meetings, streams, or lectures. Simultaneous speech translation where unstable text would cause errors. Feeding real-time transcripts into LLM agents or NLP pipelines. Chinese and English voice input in latency-sensitive applications.
How do I install Confucius4-R2T2?
Clone the repository (see the Installation section of the README) Start the recommended Docker container and run the example, as described in the README's Docker section
Where can I get Confucius4-R2T2?
The source code and installation instructions are on GitHub: https://github.com/netease-youdao/Confucius4-R2T2

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects