Audio
PaddleSpeech
Easy-to-use Speech Toolkit including Self-Supervised Learning model, SOTA/Streaming ASR with punctuation, Streaming TTS with text…
★ 12.7k starsApache-2.0
View guide →VITS — the original end-to-end text-to-speech model: a conditional VAE with adversarial learning, in PyTorch.
vits is an open-source audio project by jaywalnut310 (jaywalnut310/vits on GitHub). It has 7,893 GitHub stars and 1,378 forks, is released under the MIT license, and was last updated on GitHub on 6 December 2023.
Decide whether you will train on LJ Speech, multi-speaker VCTK, or your own dataset (which needs the preprocessing/g2p step first).
Clone jaywalnut310/vits from GitHub (or use the install path in the run steps below) and skim its README for the parts relevant to your goal.
Clone the repo, install its Python requirements, and build the Cython monotonic alignment search extension as the README's prerequisites describe.
Run the provided inference example with a pretrained setup to hear output, then start a short training run on one dataset.
A single conditional VAE with adversarial learning generates speech from text end to end, the approach the VITS paper introduced.
Includes the alignment search component (with a Cython build) that aligns text and speech during training.
Training examples cover LJ Speech (single speaker) and VCTK (multi-speaker with speaker IDs).
A preprocess script converts your file lists to phonemes (g2p), with pre-processed phonemes already provided for LJ Speech and VCTK.
The README provides concrete training commands per dataset and an inference example to synthesise speech.
The canonical implementation researchers cite and fork as the base for later TTS work.
Get the source from the official GitHub repository for vits.
git clone https://github.com/jaywalnut310/vits.git
cd vitsAfter cloning and installing requirements, build the Cython monotonic alignment search per the README prerequisites.
# cd monotonic_align && python setup.py build_ext --inplace — see README pre-requisitesPreprocess your file lists, then run the training example for your dataset (LJ Speech or VCTK) from the README.
python preprocess.py --text_index 1 --filelists filelists/ljs_audio_text_train_filelist.txt # then run the dataset's training example — see READMEjaywalnut310 and co-authors, through the jaywalnut310/vits repository, with an online demo page linked from the repo.
Yes — the project is open source under the MIT license as recorded for this listing. You can use, modify and distribute it under the MIT terms.
No API key or paid service is involved — it is local PyTorch training and inference; you need a GPU and training data for good results.
A hosted API gives instant voices with no training, billed per character. VITS gives you the model itself to train on your own data and run locally — full control and no per-use fees, but you do the training work.
Clone the repository (or use the documented installer), follow the install and configure steps on this page, and run the smallest real task first — run the provided inference example with a pretrained setup to hear output, then start a short training run on one dataset.
Easy-to-use Speech Toolkit including Self-Supervised Learning model, SOTA/Streaming ASR with punctuation, Streaming TTS with text…
Human-level text-to-speech through style diffusion and adversarial training with speech language models.
MetaVoice-1B: a 1.2B-parameter foundation model for emotional, zero-shot text-to-speech
A generative speech model built for daily dialogue — natural, conversational text-to-speech
Suno's text-prompted generative audio model: speech, music and sound effects from text prompts
Mozilla's deep-learning text-to-speech library with Tacotron, Glow-TTS and neural vocoders.
The most-starred other Audio projects in the LibreAI catalogue — the realistic shortlist if vits is not the right fit.
利用 AI 大模型和自动化工作流，根据主题或关键词一键生成高清短视频。Generate HD short videos from a topic or keyword with an automated AI workflow.
Robust Speech Recognition via Large-Scale Weak Supervision
1 min voice data can also be used to train a good TTS model! (few shot voice cloning)
Clone a voice from 5 seconds of audio and generate arbitrary speech in real time
Port of OpenAI's Whisper model in C/C++
Free and open — grab the source, star the repo, and start creating.