Audio
Real-Time-Voice-Cloning
Clone a voice from 5 seconds of audio and generate arbitrary speech in real time
★ 60.2k starsNOASSERTION
View guide →MARS5 speech model from CAMB.AI — clone a voice from 5 seconds of audio with striking prosody
MARS5-TTS is an open-source audio project by Camb-ai (Camb-ai/MARS5-TTS on GitHub). It has 2,821 GitHub stars and 246 forks, is released under the AGPL-3.0 license, and was last updated on GitHub on 1 August 2024.
Gather the text you want spoken and a clean reference recording of the target voice (1–12 seconds; around 6 seconds is optimal) — plus its transcript for best quality.
Install the dependencies with pip; the model itself loads through torch.hub, so cloning is optional (clone if you want the Hugging Face/local path).
Create an InferenceConfig: choose deep_clone=True with the reference transcript for quality, or False for fast shallow cloning, and tune temperature/top_k if needed.
Call mars5.tts(...) with your text, reference audio and config, and save the returned 24 kHz audio tensor.
An autoregressive transformer produces coarse EnCodec features, then a multinomial DDPM model refines the remaining codebooks before vocoding.
Speaker identity comes from a reference clip of 2–12 seconds; no transcript needed for a shallow clone.
Providing the reference transcript enables deep cloning — slower, but typically higher quality.
Trained on raw audio with byte-pair-encoded text, so commas add pauses and capitalization adds emphasis — a natural way to steer delivery.
Load AR and NAR models in two lines with torch.hub.load('Camb-ai/mars5-tts', 'mars5_english', trust_repo=True).
InferenceConfig exposes deep_clone, temperature, top_k, top_p, repetition/frequency penalties and more.
Clone MARS5-TTS if you want the local/Hugging Face path; torch.hub loading works without cloning.
git clone https://github.com/Camb-ai/MARS5-TTS.git
cd MARS5-TTSInstall the packages listed in the README (Python >= 3.10, Torch >= 2.0).
pip install --upgrade torch torchaudio librosa vocos encodec safetensors regexLoad the model from torch.hub and run the tts() call from the README example.
import torch, librosa
mars5, config_class = torch.hub.load('Camb-ai/mars5-tts', 'mars5_english', trust_repo=True)CAMB.AI maintains the model and repo; CAMB.AI also offers commercial dubbing/TTS products on its website.
The repo is AGPL-3.0 licensed, so it is free to use under AGPL terms (including copyleft obligations for network use). Check the license before commercial use.
No — inference runs locally in PyTorch; model weights load from torch.hub or Hugging Face.
gTTS calls a hosted service for standard voices. MARS5 is a local model focused on voice cloning and difficult prosody from a few seconds of reference audio.
pip install the listed dependencies, load the model with torch.hub, prepare a 2–12s reference clip (plus transcript for deep clone), and call mars5.tts().
Clone a voice from 5 seconds of audio and generate arbitrary speech in real time
🚀Clone a voice in 5 seconds to generate arbitrary speech in real-time
GPA — one tiny model that does ASR, TTS and voice conversion (General Purpose Audio).
Fish Speech — Fish Audio's expressive open text-to-speech and voice cloning: SOTA-quality TTS you can self-host, under the Fish Audio Community License (check terms)
A mental-model-first guide from waveforms to voice systems.
MOSS-TTS-Nano: a 0.1B open multilingual speech model that clones voices and streams in real time on CPU
The most-starred other Audio projects in the LibreAI catalogue — the realistic shortlist if MARS5-TTS is not the right fit.
利用 AI 大模型和自动化工作流，根据主题或关键词一键生成高清短视频。Generate HD short videos from a topic or keyword with an automated AI workflow.
Robust Speech Recognition via Large-Scale Weak Supervision
1 min voice data can also be used to train a good TTS model! (few shot voice cloning)
Clone a voice from 5 seconds of audio and generate arbitrary speech in real time
Port of OpenAI's Whisper model in C/C++
Free and open — grab the source, star the repo, and start creating.