Аудио
WaveRNN
WaveRNN vocoder plus Tacotron TTS in PyTorch: train your own neural speech synthesis on LJSpeech.
★ 2,2 тыс. звёздMIT
Смотреть руководство →vall-e — проект с открытым исходным кодом из категории «Аудио» от enhuiz (enhuiz/vall-e на GitHub). У него 2 970 звёзд на GitHub и 399 форков, он распространяется по лицензии MIT и последний раз обновлялся на GitHub 10 мая 2023.
Unofficial PyTorch implementation of VALL-E, the neural codec language model for zero-shot TTS
Understand this is a research implementation to train yourself: you will need your own audio/text data and a GPU setup DeepSpeed supports.
Clone with submodules, or pip install directly from GitHub as the README shows.
Install the package (tested under Python 3.10.7), put .wav + .normalized.txt files in a data folder, quantize them and generate phonemes.
Create AR/NAR configs, train both models, export checkpoints, then run the synthesis CLI with text and a reference audio path.
Implements the autoregressive model for the first quantizer and the non-autoregressive model for the remaining quantizers.
Built on Facebook's EnCodec for audio tokenization and decoding from tokens.
Data quantization (vall_e.emb.qnt), phoneme generation (vall_e.emb.g2p) and DeepSpeed-based trainers for both models.
After exporting AR and NAR checkpoints, synthesize speech from text plus a reference recording from the command line.
YAML configs per dataset with different model presets defined in the code.
A linked Google Colab notebook demonstrates the flow on a single utterance (the README notes it overfits and is not usable as a model).
Clone VALL-E with submodules as the README instructs.
git clone https://github.com/enhuiz/vall-e.git
cd vall-eAlternatively install directly from GitHub with pip (Python 3.10.7 tested; a DeepSpeed-supported GPU is needed for training).
pip install git+https://github.com/enhuiz/vall-eExport your trained AR/NAR checkpoints, then run the synthesis CLI with text, reference and output paths.
python -m vall_e <text> <ref_path> <out_path> --ar-ckpt zoo/ar.pt --nar-ckpt zoo/nar.ptenhuiz maintains this unofficial implementation; VALL-E itself is a Microsoft Research model described in the cited arXiv paper.
The code is MIT-licensed. Note the README's notice that EnCodec is CC-BY-NC 4.0, so non-commercial terms apply to the EnCodec parts.
No API key — it runs locally with PyTorch. But there is no pretrained checkpoint provided, so you must train the models yourself on GPU hardware.
Those synthesize speech immediately. This repo is a research training implementation of a specific paper, with no shipped pretrained model.
Clone with --recurse-submodules, install, prepare a dataset of .wav and .normalized.txt files, then follow the quantize → phoneme → train → export flow in the README.
WaveRNN vocoder plus Tacotron TTS in PyTorch: train your own neural speech synthesis on LJSpeech.
MetaVoice-1B: a 1.2B-parameter foundation model for emotional, zero-shot text-to-speech
1 min voice data can also be used to train a good TTS model! (few shot voice cloning)
IndexTTS: industrial-level, controllable zero-shot text-to-speech with voice cloning
Gradio WebUI for creators and developers, featuring key TTS (Edge-TTS, kokoro) and zero-shot Voice Cloning (E2 & F5-TTS, CosyVoice), with Whisper audio proce…
A tiny conversational TTS model with cloning, streaming, and CPU speed.
Другие проекты из категории «Аудио» с наибольшим числом звёзд в каталоге LibreAI — реалистичный список альтернатив, если vall-e вам не подходит.
利用 AI 大模型和自动化工作流，根据主题或关键词一键生成高清短视频。Generate HD short videos from a topic or keyword with an automated AI workflow.
Robust Speech Recognition via Large-Scale Weak Supervision
1 min voice data can also be used to train a good TTS model! (few shot voice cloning)
Clone a voice from 5 seconds of audio and generate arbitrary speech in real time
Port of OpenAI's Whisper model in C/C++
Соседние проекты из категории «Аудио» в каталоге, ближайшие по числу звёзд к vall-e — продолжайте изучать полку по обе стороны от него.
GPA — one tiny model that does ASR, TTS and voice conversion (General Purpose Audio).
MARS5 speech model from CAMB.AI — clone a voice from 5 seconds of audio with striking prosody
One Gradio + React WebUI for dozens of TTS, voice-cloning and music models — Bark to Kokoro.
Vietnamese TTS with instant voice cloning — on-device, real-time CPU inference, 48 kHz audio
A pure C++ ggml inference engine for audio models — TTS, STT, VAD, voice conversion and music.
gTTS — Python library and CLI that turns text into spoken mp3 via Google Translate's speech API
Бесплатно и открыто — забирайте исходный код, ставьте звезду репозиторию и начинайте создавать.