Аудио
Real-Time-Voice-Cloning
Clone a voice from 5 seconds of audio and generate arbitrary speech in real time
★ 60,2 тыс. звёздNOASSERTION
Смотреть руководство →MARS5-TTS — проект с открытым исходным кодом из категории «Аудио» от Camb-ai (Camb-ai/MARS5-TTS на GitHub). У него 2 821 звезда на GitHub и 246 форков, он распространяется по лицензии AGPL-3.0 и последний раз обновлялся на GitHub 1 августа 2024.
MARS5 speech model from CAMB.AI — clone a voice from 5 seconds of audio with striking prosody
Gather the text you want spoken and a clean reference recording of the target voice (1–12 seconds; around 6 seconds is optimal) — plus its transcript for best quality.
Install the dependencies with pip; the model itself loads through torch.hub, so cloning is optional (clone if you want the Hugging Face/local path).
Create an InferenceConfig: choose deep_clone=True with the reference transcript for quality, or False for fast shallow cloning, and tune temperature/top_k if needed.
Call mars5.tts(...) with your text, reference audio and config, and save the returned 24 kHz audio tensor.
An autoregressive transformer produces coarse EnCodec features, then a multinomial DDPM model refines the remaining codebooks before vocoding.
Speaker identity comes from a reference clip of 2–12 seconds; no transcript needed for a shallow clone.
Providing the reference transcript enables deep cloning — slower, but typically higher quality.
Trained on raw audio with byte-pair-encoded text, so commas add pauses and capitalization adds emphasis — a natural way to steer delivery.
Load AR and NAR models in two lines with torch.hub.load('Camb-ai/mars5-tts', 'mars5_english', trust_repo=True).
InferenceConfig exposes deep_clone, temperature, top_k, top_p, repetition/frequency penalties and more.
Clone MARS5-TTS if you want the local/Hugging Face path; torch.hub loading works without cloning.
git clone https://github.com/Camb-ai/MARS5-TTS.git
cd MARS5-TTSInstall the packages listed in the README (Python >= 3.10, Torch >= 2.0).
pip install --upgrade torch torchaudio librosa vocos encodec safetensors regexLoad the model from torch.hub and run the tts() call from the README example.
import torch, librosa
mars5, config_class = torch.hub.load('Camb-ai/mars5-tts', 'mars5_english', trust_repo=True)CAMB.AI maintains the model and repo; CAMB.AI also offers commercial dubbing/TTS products on its website.
The repo is AGPL-3.0 licensed, so it is free to use under AGPL terms (including copyleft obligations for network use). Check the license before commercial use.
No — inference runs locally in PyTorch; model weights load from torch.hub or Hugging Face.
gTTS calls a hosted service for standard voices. MARS5 is a local model focused on voice cloning and difficult prosody from a few seconds of reference audio.
pip install the listed dependencies, load the model with torch.hub, prepare a 2–12s reference clip (plus transcript for deep clone), and call mars5.tts().
Clone a voice from 5 seconds of audio and generate arbitrary speech in real time
🚀Clone a voice in 5 seconds to generate arbitrary speech in real-time
GPA — one tiny model that does ASR, TTS and voice conversion (General Purpose Audio).
Fish Speech — Fish Audio's expressive open text-to-speech and voice cloning: SOTA-quality TTS you can self-host, under the Fish Audio Community License (check terms)
A mental-model-first guide from waveforms to voice systems.
MOSS-TTS-Nano: a 0.1B open multilingual speech model that clones voices and streams in real time on CPU
Другие проекты из категории «Аудио» с наибольшим числом звёзд в каталоге LibreAI — реалистичный список альтернатив, если MARS5-TTS вам не подходит.
利用 AI 大模型和自动化工作流，根据主题或关键词一键生成高清短视频。Generate HD short videos from a topic or keyword with an automated AI workflow.
Robust Speech Recognition via Large-Scale Weak Supervision
1 min voice data can also be used to train a good TTS model! (few shot voice cloning)
Clone a voice from 5 seconds of audio and generate arbitrary speech in real time
Port of OpenAI's Whisper model in C/C++
Соседние проекты из категории «Аудио» в каталоге, ближайшие по числу звёзд к MARS5-TTS — продолжайте изучать полку по обе стороны от него.
Unofficial PyTorch implementation of VALL-E, the neural codec language model for zero-shot TTS
Vietnamese TTS with instant voice cloning — on-device, real-time CPU inference, 48 kHz audio
GPA — one tiny model that does ASR, TTS and voice conversion (General Purpose Audio).
gTTS — Python library and CLI that turns text into spoken mp3 via Google Translate's speech API
One Gradio + React WebUI for dozens of TTS, voice-cloning and music models — Bark to Kokoro.
MaryTTS — open-source, multilingual text-to-speech platform written in pure Java
Бесплатно и открыто — забирайте исходный код, ставьте звезду репозиторию и начинайте создавать.