Audio
hifi-gan
GAN vocoder that turns mel-spectrograms into high-fidelity speech audio, fast enough for real-time use.
★ 2.4k starsMIT
View guide →FastRTC turns any Python function into a real-time audio or video stream over WebRTC or WebSockets
fastrtc is an open-source audio project by gradio-app (gradio-app/fastrtc on GitHub). It has 4,631 GitHub stars and 436 forks, is released under the MIT license, and was last updated on GitHub on 12 January 2026.
Write a Python generator that takes audio (or video) frames and yields responses.
Install from PyPI, adding the vad/tts extras if you want pause detection and text-to-speech.
Create a Stream with ReplyOnPause for audio conversations, and pick a delivery mode: built-in UI, FastAPI mount, or a phone number.
Start the stream and speak to your function in real time.
Built-in voice activity detection (the ReplyOnPause handler) runs your function when the user stops speaking.
Stream real-time audio or video over either transport, in send, receive, or send-receive modes.
Every stream can launch with a test interface via stream.ui.launch().
Mount a stream into an existing FastAPI application with stream.mount(app).
Get a temporary phone number to call your stream, using a Hugging Face token.
Optional vad and tts extras add pause detection and text-to-speech support.
Clone the project and enter its directory (library users can install from PyPI instead).
git clone https://github.com/gradio-app/fastrtc.git
cd fastrtcInstall the package, with the speech extras if you need them.
pip install fastrtc
pip install "fastrtc[vad, tts]"Wrap a reply function in a Stream and launch the built-in UI.
from fastrtc import Stream, ReplyOnPause
stream = Stream(handler=ReplyOnPause(fn), modality="audio", mode="send-receive")
stream.ui.launch()The Gradio team (gradio-app) — the same group behind the Gradio ML interface library.
Yes, it is MIT-licensed open source. FastPhone numbers are provided through Hugging Face and require an HF token.
No — the library is self-contained; you supply your own speech/LLM models inside your handler function.
FastRTC abstracts the transport and adds the conversational layer — VAD, turn-taking, a test UI, and FastAPI mounting — so you write a plain function instead of managing peers and tracks.
`pip install fastrtc`, wrap your function with `Stream(handler=ReplyOnPause(fn), ...)`, and call `stream.ui.launch()`.
GAN vocoder that turns mel-spectrograms into high-fidelity speech audio, fast enough for real-time use.
MCP server that turns any video — YouTube.
gTTS — Python library and CLI that turns text into spoken mp3 via Google Translate's speech API
Clone a voice from 5 seconds of audio and generate arbitrary speech in real time
Vietnamese TTS with instant voice cloning — on-device, real-time CPU inference, 48 kHz audio
Python library for audio and music analysis
The most-starred other Audio projects in the LibreAI catalogue — the realistic shortlist if fastrtc is not the right fit.
利用 AI 大模型和自动化工作流，根据主题或关键词一键生成高清短视频。Generate HD short videos from a topic or keyword with an automated AI workflow.
Robust Speech Recognition via Large-Scale Weak Supervision
1 min voice data can also be used to train a good TTS model! (few shot voice cloning)
Clone a voice from 5 seconds of audio and generate arbitrary speech in real time
Port of OpenAI's Whisper model in C/C++
Free and open — grab the source, star the repo, and start creating.