Repolode
Submit
xinnan-tech

xinnan-tech/xiaozhi-esp32-server

Python/Java/Vue backend server for xiaozhi-esp32 ESP32 voice devices, supporting MQTT+UDP, WebSocket, MCP, voiceprint recognition and knowledge base.

View on GitHubWebsite
Stars
10,940
Forks
3,716
Open issues
732
Last commit
5 hours ago

What is it?

xiaozhi-esp32-server is a self-hosted backend for the xiaozhi-esp32 open-source smart hardware project. It implements the xiaozhi communication protocol in Python, Java and Vue, so ESP32 voice devices can connect to a server you run yourself instead of a hosted service.

The server handles the voice pipeline: streaming speech recognition, streaming speech synthesis and voice activity detection, with a choice of local models (such as FunASR and SherpaASR) or cloud APIs for ASR, LLM, VLLM and TTS. It also supports intent recognition through function calling, several memory backends, RAGFlow knowledge base retrieval, and tool calls over IoT, MCP and custom function protocols.

A full-module deployment adds a web console for user, device and agent management, and a configuration-file mode runs without a database. Deployment is available via Docker or from source.

Key features

  • 01Supports MQTT+UDP gateway and WebSocket connections for ESP32 devices.
  • 02Streaming ASR, TTS and voice activity detection with many cloud and local provider options.
  • 03Voiceprint registration and speaker identification passed to the LLM.
  • 04Tool calling through IoT, MCP client/server and custom function protocols.
  • 05Web console for user, device and agent management, with Docker or source deployment.

Use cases

  • Running a private backend for a self-built xiaozhi-esp32 voice assistant.
  • Connecting ESP32 devices to OpenAI-compatible or domestic LLM APIs.
  • Controlling home appliances or sending MCP commands to devices from a server.
  • Prototyping voice agents with knowledge base retrieval on low-spec hardware.

Quick start

  1. 1

    Choose a deployment mode (minimal or full module) from the README table

  2. 2

    Follow the Docker or source deployment guide for the chosen mode

Things to know

  • The README says the project is not finished and has not passed security testing, so it should not be used in production.
  • It is designed to work with ESP32 hardware devices and is only useful to those who own compatible hardware.
  • It depends on third-party API services for speech, LLM and TTS, and some of these require paid keys.
  • Running it with local models such as FunASR needs 2 cores and 4 GB RAM, or 4 cores and 8 GB for the full module setup.

Topics

#esp32#voice-assistant#mcp-server#websocket#mqtt#speech-recognition#tts#llm#dify#xiaozhi

FAQ

What is xiaozhi-esp32-server?
Python/Java/Vue backend server for xiaozhi-esp32 ESP32 voice devices, supporting MQTT+UDP, WebSocket, MCP, voiceprint recognition and knowledge base. xiaozhi-esp32-server is a self-hosted backend for the xiaozhi-esp32 open-source smart hardware project. It implements the xiaozhi communication protocol in Python, Java and Vue, so ESP32 voice devices can connect to a server you run yourself instead of a hosted service.
Is xiaozhi-esp32-server free to use? What license does it have?
xiaozhi-esp32-server is open source under the MIT license. Check the license terms before using it in commercial products.
Is xiaozhi-esp32-server still maintained?
Actively maintained. The last commit to xiaozhi-esp32-server was on Oct 20, 2026.
What can I use xiaozhi-esp32-server for?
Running a private backend for a self-built xiaozhi-esp32 voice assistant. Connecting ESP32 devices to OpenAI-compatible or domestic LLM APIs. Controlling home appliances or sending MCP commands to devices from a server. Prototyping voice agents with knowledge base retrieval on low-spec hardware.
How do I install xiaozhi-esp32-server?
Choose a deployment mode (minimal or full module) from the README table Follow the Docker or source deployment guide for the chosen mode
Where can I get xiaozhi-esp32-server?
The source code and installation instructions are on GitHub: https://github.com/xinnan-tech/xiaozhi-esp32-server

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects