Audio
kubeai
The easiest way to serve AI models, including speech-to-text, on Kubernetes.
★ 1.3k starsApache-2.0
View guide →Human-level text-to-speech through style diffusion and adversarial training with speech language models.
StyleTTS2 is an open-source audio project by yl4579 (yl4579/StyleTTS2 on GitHub). It has 6,365 GitHub stars and 709 forks, is released under the MIT license, and was last updated on GitHub on 10 August 2024.
Decide whether you will use a pretrained voice model or train/fine-tune on your own dataset and language.
Clone the repo, install PyTorch and the requirements, plus phonemizer and espeak-ng for text processing.
Download a pretrained StyleTTS 2 model (LJSpeech or LibriTTS) or prepare your training config.
Run inference with a reference audio clip to generate speech in that style, or start a training run.
Models speech style with diffusion, producing natural, expressive prosody rather than flat monotone output.
Uses large speech language models such as WavLM in adversarial training to push quality towards human level.
Adapts to a new speaker's style from a reference recording using the pretrained text aligner and pitch extractor.
Ready models trained on LJSpeech (24 kHz) and LibriTTS are downloadable from Hugging Face.
Includes first-stage, second-stage and fine-tuning training scripts with Accelerate support for your own datasets.
Pretrained components cover English, Japanese and Chinese corpora, and a multilingual PL-BERT supports training other languages.
Clone StyleTTS 2 and enter the directory.
git clone https://github.com/yl4579/StyleTTS2.git
cd StyleTTS2Install the requirements, plus phonemizer and espeak-ng used for text processing.
pip install -r requirements.txt
pip install phonemizer
sudo apt-get install espeak-ngFor your own data, launch the documented two-stage training pipeline; for inference, download a pretrained model from Hugging Face and follow the README's inference notebook.
accelerate launch train_first.py --config_path ./Configs/config.yml
python train_second.py --config_path ./Configs/config.ymlStyleTTS 2 is maintained by yl4579 and accompanies the StyleTTS 2 research paper.
The code is MIT licensed. Pretrained models have their own usage conditions described in the README, especially for voices not in the training set.
No — inference and training run locally on your own GPU with downloaded models.
All are open voice-cloning TTS systems; StyleTTS 2's distinction is its style-diffusion design plus adversarial training with speech language models, and its two-stage training pipeline.
Clone the repo, install requirements and espeak-ng, download the LJSpeech or LibriTTS pretrained model from Hugging Face, and run the inference example.
The easiest way to serve AI models, including speech-to-text, on Kubernetes.
Silero Models: pre-trained text-to-speech models made embarrassingly simple
VITS — the original end-to-end text-to-speech model: a conditional VAE with adversarial learning, in PyTorch.
IndexTTS: industrial-level, controllable zero-shot text-to-speech with voice cloning
A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recogni…
Chatterbox — Resemble AI's open-source TTS family: expressive text-to-speech and voice cloning, multilingual V3, low-latency Turbo and tiny Nano models, MIT
The most-starred other Audio projects in the LibreAI catalogue — the realistic shortlist if StyleTTS2 is not the right fit.
利用 AI 大模型和自动化工作流，根据主题或关键词一键生成高清短视频。Generate HD short videos from a topic or keyword with an automated AI workflow.
Robust Speech Recognition via Large-Scale Weak Supervision
1 min voice data can also be used to train a good TTS model! (few shot voice cloning)
Clone a voice from 5 seconds of audio and generate arbitrary speech in real time
Port of OpenAI's Whisper model in C/C++
Free and open — grab the source, star the repo, and start creating.