An agent framework that connects digital humans (2.5D, 3D, mobile, PC, web) and LLMs (OpenAI-compatible, DeepSeek) to business systems.
joey-zhou/xiaozhi-esp32-server-java
Java server and admin platform for Xiaozhi ESP32 voice devices, covering device monitoring, voice and persona management, and conversation logs.
- Stars
- 1,381
- Forks
- 505
- Open issues
- 2
- Last commit
- 23 days ago
What is it?
Xiaozhi ESP32 Server Java is a Java backend for ESP32-based voice hardware compatible with the Xiaozhi ESP32 firmware. It ships with a web admin console for managing users, devices, roles and OTA upgrades, and a separate dialogue service that handles WebSocket and MQTT audio streams and the AI conversation pipeline.
The system is split into two processes, xiaozhi-server for administration and xiaozhi-dialogue for real-time voice, which share MySQL and Redis and can be scaled independently. Speech recognition and synthesis can use local models such as sherpa-onnx and Vosk or cloud providers, and language models can come from OpenAI, Zhipu, Ollama, Dify, Coze and others. MCP tool support, function calling, a RAG knowledge base and voiceprint recognition extend the assistant's abilities.
Deployment is available through Docker Compose or from source with JDK 21, Maven, Node 22, MySQL 8 and Redis 7. Models are downloaded by a provided script on first run.
Key features
- 01Separate admin and dialogue processes that share MySQL and Redis and scale independently
- 02Local and cloud speech recognition and synthesis, including voice cloning and streaming interaction
- 03Integration with OpenAI, Zhipu, Ollama, Dify, Coze and other LLM providers
- 04MCP tool protocol support and function calling for smart-home control
- 05RAG knowledge base for PDF, Office and image documents
- 06Long-term memory and voiceprint recognition for multiple household members
Use cases
- Running a self-hosted voice assistant backend for ESP32 devices in a home or office
- Managing a fleet of Xiaozhi-compatible devices with OTA upgrades and monitoring
- Building a voice assistant that controls smart-home devices through MCP or function calls
- Prototyping an AI voice product on ESP32 hardware with swappable speech and LLM providers
Quick start
- 1
Create a folder and enter it
mkdir xiaozhi && cd xiaozhi - 2
Download the docker-compose.yml file
curl -O https://raw.githubusercontent.com/joey-zhou/xiaozhi-esp32-server-java/main/docker-compose.yml - 3
Start the containers
docker compose up -d - 4
Once the containers are healthy, open http://localhost:8084 and log in as admin / 123456, then configure an LLM API key
Things to know
- You must supply your own LLM API key before the system can hold conversations; no keys are included.
- Some features are not open source and are only available in the commercial version.
Topics
FAQ
- What is xiaozhi-esp32-server-java?
- Java server and admin platform for Xiaozhi ESP32 voice devices, covering device monitoring, voice and persona management, and conversation logs. Xiaozhi ESP32 Server Java is a Java backend for ESP32-based voice hardware compatible with the Xiaozhi ESP32 firmware. It ships with a web admin console for managing users, devices, roles and OTA upgrades, and a separate dialogue service that handles WebSocket and MQTT audio streams and the AI conversation pipeline.
- Is xiaozhi-esp32-server-java free to use? What license does it have?
- xiaozhi-esp32-server-java is open source under the MIT license. Check the license terms before using it in commercial products.
- Is xiaozhi-esp32-server-java still maintained?
- Actively maintained. The last commit to xiaozhi-esp32-server-java was on Sep 27, 2026.
- What can I use xiaozhi-esp32-server-java for?
- Running a self-hosted voice assistant backend for ESP32 devices in a home or office Managing a fleet of Xiaozhi-compatible devices with OTA upgrades and monitoring Building a voice assistant that controls smart-home devices through MCP or function calls Prototyping an AI voice product on ESP32 hardware with swappable speech and LLM providers
- How do I install xiaozhi-esp32-server-java?
- Create a folder and enter it: mkdir xiaozhi && cd xiaozhi Download the docker-compose.yml file: curl -O https://raw.githubusercontent.com/joey-zhou/xiaozhi-esp32-server-java/main/docker-compose.yml Start the containers: docker compose up -d Once the containers are healthy, open http://localhost:8084 and log in as admin / 123456, then configure an LLM API key
- Where can I get xiaozhi-esp32-server-java?
- The source code and installation instructions are on GitHub: https://github.com/joey-zhou/xiaozhi-esp32-server-java
Summary generated by AI from the project's README. GitHub stats refresh daily.
Related projects
Python/Java/Vue backend server for xiaozhi-esp32 ESP32 voice devices, supporting MQTT+UDP, WebSocket, MCP, voiceprint recognition and knowledge base.
Official Python SDK for building Model Context Protocol servers and clients that connect LLM applications to tools, data and prompts.
The official TypeScript SDK for building Model Context Protocol servers and clients that give LLM applications standardized context.
A cross-platform desktop AI assistant that connects to major LLM providers and acts as an MCP client with a local knowledge base.
An MCP server that exposes Ghidra's reverse engineering functions so LLM clients can decompile and analyze binaries.