Audio
WaveRNN
WaveRNN vocoder plus Tacotron TTS in PyTorch: train your own neural speech synthesis on LJSpeech.
★ 2,2 Tsd. SterneMIT
Anleitung ansehen →vall-e ist ein Open-Source-Projekt aus der Kategorie Audio von enhuiz (enhuiz/vall-e auf GitHub). Es hat 2.970 Sterne auf GitHub und 399 Forks, wird unter der Lizenz MIT veröffentlicht und wurde zuletzt am 10. Mai 2023 auf GitHub aktualisiert.
Unofficial PyTorch implementation of VALL-E, the neural codec language model for zero-shot TTS
Understand this is a research implementation to train yourself: you will need your own audio/text data and a GPU setup DeepSpeed supports.
Clone with submodules, or pip install directly from GitHub as the README shows.
Install the package (tested under Python 3.10.7), put .wav + .normalized.txt files in a data folder, quantize them and generate phonemes.
Create AR/NAR configs, train both models, export checkpoints, then run the synthesis CLI with text and a reference audio path.
Implements the autoregressive model for the first quantizer and the non-autoregressive model for the remaining quantizers.
Built on Facebook's EnCodec for audio tokenization and decoding from tokens.
Data quantization (vall_e.emb.qnt), phoneme generation (vall_e.emb.g2p) and DeepSpeed-based trainers for both models.
After exporting AR and NAR checkpoints, synthesize speech from text plus a reference recording from the command line.
YAML configs per dataset with different model presets defined in the code.
A linked Google Colab notebook demonstrates the flow on a single utterance (the README notes it overfits and is not usable as a model).
Clone VALL-E with submodules as the README instructs.
git clone https://github.com/enhuiz/vall-e.git
cd vall-eAlternatively install directly from GitHub with pip (Python 3.10.7 tested; a DeepSpeed-supported GPU is needed for training).
pip install git+https://github.com/enhuiz/vall-eExport your trained AR/NAR checkpoints, then run the synthesis CLI with text, reference and output paths.
python -m vall_e <text> <ref_path> <out_path> --ar-ckpt zoo/ar.pt --nar-ckpt zoo/nar.ptenhuiz maintains this unofficial implementation; VALL-E itself is a Microsoft Research model described in the cited arXiv paper.
The code is MIT-licensed. Note the README's notice that EnCodec is CC-BY-NC 4.0, so non-commercial terms apply to the EnCodec parts.
No API key — it runs locally with PyTorch. But there is no pretrained checkpoint provided, so you must train the models yourself on GPU hardware.
Those synthesize speech immediately. This repo is a research training implementation of a specific paper, with no shipped pretrained model.
Clone with --recurse-submodules, install, prepare a dataset of .wav and .normalized.txt files, then follow the quantize → phoneme → train → export flow in the README.
WaveRNN vocoder plus Tacotron TTS in PyTorch: train your own neural speech synthesis on LJSpeech.
MetaVoice-1B: a 1.2B-parameter foundation model for emotional, zero-shot text-to-speech
1 min voice data can also be used to train a good TTS model! (few shot voice cloning)
IndexTTS: industrial-level, controllable zero-shot text-to-speech with voice cloning
Gradio WebUI for creators and developers, featuring key TTS (Edge-TTS, kokoro) and zero-shot Voice Cloning (E2 & F5-TTS, CosyVoice), with Whisper audio proce…
A tiny conversational TTS model with cloning, streaming, and CPU speed.
Die anderen Projekte der Kategorie Audio mit den meisten Sternen im LibreAI-Katalog — die realistische Auswahl, falls vall-e nicht das Richtige ist.
利用 AI 大模型和自动化工作流，根据主题或关键词一键生成高清短视频。Generate HD short videos from a topic or keyword with an automated AI workflow.
Robust Speech Recognition via Large-Scale Weak Supervision
1 min voice data can also be used to train a good TTS model! (few shot voice cloning)
Clone a voice from 5 seconds of audio and generate arbitrary speech in real time
Port of OpenAI's Whisper model in C/C++
Verwandte Projekte der Kategorie Audio im Katalog, die vall-e bei den Sternen am nächsten stehen — zum Weiterstöbern auf beiden Seiten.
GPA — one tiny model that does ASR, TTS and voice conversion (General Purpose Audio).
MARS5 speech model from CAMB.AI — clone a voice from 5 seconds of audio with striking prosody
One Gradio + React WebUI for dozens of TTS, voice-cloning and music models — Bark to Kokoro.
Vietnamese TTS with instant voice cloning — on-device, real-time CPU inference, 48 kHz audio
A pure C++ ggml inference engine for audio models — TTS, STT, VAD, voice conversion and music.
gTTS — Python library and CLI that turns text into spoken mp3 via Google Translate's speech API
Kostenlos und offen — hol dir den Quellcode, gib dem Repository einen Stern und leg los.