Repolode
Submit
TEN-framework

TEN-framework/ten-framework

Open-source framework for real-time multimodal conversational voice AI agents, with prebuilt agent examples and a visual designer.

View on GitHubWebsite
Stars
11,183
Forks
1,366
Open issues
249
Last commit
7 hours ago

What is it?

TEN is an open-source framework for building real-time, multimodal conversational AI agents. It connects speech-to-text, language models, and text-to-speech into a pipeline built from modular extensions, so developers can assemble a voice agent and swap individual components without rewriting the whole system.

The repository includes a set of agent examples: a multi-purpose voice assistant that supports RTC and WebSocket connections, a Live2D and avatar-based lip-sync assistant, a SIP extension for phone calls, a speaker diarization demo, a transcription tool, and an ESP32-S3 hardware integration. The TMAN Designer lets you configure extensions and their API keys in a browser, and the examples can run locally with Docker or in GitHub Codespaces.

For deployment, the example backend ships as a Docker image that can run on any container platform, while the frontend can be hosted separately on services such as Vercel or Netlify.

Key features

  • 01Modular extension pipeline connecting speech recognition, LLMs, and speech synthesis.
  • 02Voice assistant supporting both RTC and WebSocket connections.
  • 03Visual TMAN Designer for configuring extensions and API keys in the browser.
  • 04Live2D and avatar lip-sync examples, plus a SIP extension for phone calls.
  • 05Docker-based deployment, with the frontend deployable separately to Vercel or Netlify.

Use cases

  • Building a voice assistant that answers user speech with an LLM and synthesized voice.
  • Adding an AI voice agent to phone calls through the SIP extension.
  • Prototyping an animated avatar or Live2D character that speaks with lip sync.
  • Connecting a voice agent to ESP32-S3 hardware for embedded conversational features.

Topics

#voice-ai#conversational-ai#real-time#multimodal#speech-to-text#text-to-speech#agent-framework#ai#multi-modal#video

FAQ

What is ten-framework?
Open-source framework for real-time multimodal conversational voice AI agents, with prebuilt agent examples and a visual designer. TEN is an open-source framework for building real-time, multimodal conversational AI agents. It connects speech-to-text, language models, and text-to-speech into a pipeline built from modular extensions, so developers can assemble a voice agent and swap individual components without rewriting the whole system.
Is ten-framework free to use? What license does it have?
ten-framework is open source under the Other license. Check the license terms before using it in commercial products.
Is ten-framework still maintained?
Actively maintained. The last commit to ten-framework was on Oct 21, 2026.
What can I use ten-framework for?
Building a voice assistant that answers user speech with an LLM and synthesized voice. Adding an AI voice agent to phone calls through the SIP extension. Prototyping an animated avatar or Live2D character that speaks with lip sync. Connecting a voice agent to ESP32-S3 hardware for embedded conversational features.
Where can I get ten-framework?
The source code and installation instructions are on GitHub: https://github.com/TEN-framework/ten-framework

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects