Audio
VoiceMem
VoiceMem — streaming dual-brain memory for voice agents: a left brain for facts, a right brain for emotion
★ 2,4 Tsd. SterneApache-2.0
Anleitung ansehen →xiaozhi-esp32-server ist ein Open-Source-Projekt aus der Kategorie Audio von xinnan-tech (xinnan-tech/xiaozhi-esp32-server auf GitHub). Es hat 10.756 Sterne auf GitHub und 3.670 Forks, wird unter der Lizenz MIT veröffentlicht und wurde zuletzt am 8. Oktober 2026 auf GitHub aktualisiert.
Self-hosted backend that gives ESP32 'Xiaozhi' devices streaming voice chat, memory and tool control.
Flash or buy a Xiaozhi ESP32 device and confirm it works against a public test backend, so you know the hardware side is fine before self-hosting.
Pick the minimal install (single agent, config-file storage, no database) or the full-module install (multi-user, multi-agent, web control console, database) - each is offered as Docker or source deployment in the docs.
Select ASR, LLM, vision and TTS providers in the config; the documented all-free starter setup uses local FunASR, EdgeTTS and free-tier cloud models.
Set your firmware to your server's OTA and WebSocket endpoints, then watch it answer, recognize speakers and run tools through the control console.
Streaming ASR, streaming TTS and voice-activity detection give low-latency, interruptible conversations in multiple languages.
Multiple users can register voiceprints; the server identifies who is speaking in parallel with recognition and passes that to the LLM for personalized replies.
LLMs connect via OpenAI-compatible APIs, Ollama, Dify, FastGPT, Coze, Xinference or Home Assistant; ASR, TTS and vision models include local options like FunASR, FishSpeech and GPT-SoVITS as well as major cloud providers.
Supports client IoT protocols, client and server MCP, and MCP access points, and can push MCP commands from the console down to ESP32 devices over MQTT.
Choose local short-term memory, mem0ai or PowerMem for conversation memory, and plug in a RAGFlow knowledge base the model can consult before answering.
The full-module install adds a web admin for users, system configuration and devices (in Simplified/Traditional Chinese and English), plus performance and audio test tools.
Get the server source; deployment itself is covered by the project's step-by-step deployment docs.
git clone https://github.com/xinnan-tech/xiaozhi-esp32-server.git
cd xiaozhi-esp32-serverFollow the deployment document for your tier - Docker or source, minimal or full-module - linked from the README's deployment section.
# see the deployment docs linked from the README for the exact commandsPoint your ESP32 firmware at the server's OTA/WebSocket endpoints and use the bundled test tools to check ASR, LLM and TTS latency.
# verify with the performance tester and audio test tool described in the READMEThe xinnan-tech team maintains it; the project is spearheaded by Professor Siyuan Liu's team at South China University of Technology and a large contributor community.
Yes. The server is MIT-licensed, and the README documents an all-free configuration (local FunASR recognition, EdgeTTS, free-tier models) for home use.
Not necessarily. You can run every module with local or free-tier components; paid cloud ASR, LLM or TTS providers are optional upgrades for speed and quality.
The public service is zero-setup but shared and third-party-run; self-hosting gives you your own data, your own model choices, multi-user management and no dependence on someone else's server.
Clone the repo, follow the deployment doc for the minimal Docker install, then point one ESP32 device at your server's WebSocket endpoint.
VoiceMem — streaming dual-brain memory for voice agents: a left brain for facts, a right brain for emotion
Self-host Chatterbox TTS with web UI, API, and voice cloning.
Open source voice AI platform. Self-hosted alternative to Vapi and Retell. On Prem, BYOK across Speech to Speech or LLM/STT/TTS, with a visual workflow build…
Self-hosted AI video analysis: summaries, chapters, transcripts.
Browse WhatsApp chat exports offline, with AI voice transcription.
CosyVoice: multilingual voice generation with zero-shot cloning, streaming and fine-tuning
Die anderen Projekte der Kategorie Audio mit den meisten Sternen im LibreAI-Katalog — die realistische Auswahl, falls xiaozhi-esp32-server nicht das Richtige ist.
利用 AI 大模型和自动化工作流，根据主题或关键词一键生成高清短视频。Generate HD short videos from a topic or keyword with an automated AI workflow.
Robust Speech Recognition via Large-Scale Weak Supervision
1 min voice data can also be used to train a good TTS model! (few shot voice cloning)
Clone a voice from 5 seconds of audio and generate arbitrary speech in real time
Port of OpenAI's Whisper model in C/C++
Verwandte Projekte der Kategorie Audio im Katalog, die xiaozhi-esp32-server bei den Sternen am nächsten stehen — zum Weiterstöbern auf beiden Seiten.
A PyTorch-based Speech Toolkit
Neural building blocks for speaker diarization: speech activity detection, speaker change detection, overlapped speech detection, speaker embedding
Use Microsoft Edge's online text-to-speech service from Python WITHOUT needing Microsoft Edge or Windows or an API key
Silero VAD: pre-trained enterprise-grade Voice Activity Detector
Easy-to-use Speech Toolkit including Self-Supervised Learning model, SOTA/Streaming ASR with punctuation, Streaming TTS with text…
Open research toolkit for text-to-speech, voice conversion, singing voice and text-to-audio generation.
Kostenlos und offen — hol dir den Quellcode, gib dem Repository einen Stern und leg los.