AI Music
instruct-MusicGen
Research code for instruction-based text-to-music editing of music models.
★ 111 starsApache-2.0
View guide →Official DiffSinger code: diffusion-based singing voice synthesis and text-to-speech (AAAI 2022)
DiffSinger is an open-source ai music project by MoonInTheRiver (MoonInTheRiver/DiffSinger on GitHub). It has 4,875 GitHub stars and 830 forks, is released under the MIT license, and was last updated on GitHub on 24 July 2026.
Decide whether you want singing synthesis (lyrics + F0 or MIDI input) or the DiffSpeech text-to-speech variant.
Clone the repo and create the documented Conda or venv environment for your GPU/CUDA version.
Install the requirements file matching your hardware, as listed in the README.
Download a checkpoint, follow the SVS or TTS document, and synthesize your first utterance or song line.
The paper's core contribution: diffusion applied at a shallow stage for faster, high-quality mel-spectrogram generation.
Generate singing from lyrics with ground-truth F0, or from lyrics + MIDI with explicit F0 prediction, on PopCS/OpenCpop pipelines.
A text-to-speech variant in the same framework, trained on LJSpeech with a HiFiGAN vocoder.
Optional DiffSinger-PN adds the PNDM sampler to speed up diffusion sampling.
NSF-HiFiGAN for singing pipelines and HiFiGAN for TTS, per the README's pipeline table.
Full official code with documented SVS/TTS run guides and TensorBoard logging, not just a demo.
Clone the project and enter its directory.
git clone https://github.com/MoonInTheRiver/DiffSinger.git
cd DiffSingerCreate and activate a Python 3.8 Conda environment as documented.
conda create -n your_env_name python=3.8
conda activate your_env_namePick the requirements file matching your GPU/CUDA (2080Ti/CUDA 10.2 or 3090/CUDA 11.4 examples in the README), then follow the SVS/TTS docs to run inference.
pip install -r requirements_3090.txt # or requirements_2080.txt — see READMEIt is the authors' official repository (MoonInTheRiver), with later maintenance help noted from Team Openvpi; the paper appeared at AAAI 2022.
Yes, the code is MIT-licensed. Pretrained checkpoints and datasets (PopCS, OpenCpop, LJSpeech) have their own terms — check each before commercial use.
No. Everything runs locally on your own GPU; there is no hosted API involved.
This is the original research implementation the community forks build on; forks such as OpenVPI's add newer editors and workflows, while this repo remains the reference for the paper's method.
Clone the repo, create the Python 3.8 environment from the README, install the requirements file for your GPU, then follow the linked 'Run DiffSinger (SVS)' or 'Run DiffSpeech (TTS)' document.
Research code for instruction-based text-to-music editing of music models.
Emotion-aware music recommender that reads text, speech, and faces.
Text-to-speech for Home Assistant from OpenAI, Mistral, Groq, Lemonfox, Kokoro, Chatterbox or any server that implements the OpenAI speech API, with announce…
Open-source local AI SDK - run AI on-device with no cloud, no API keys. Supports GGUF, RAG, image, music, and video generation, speech-to-text, P2P inference…
AI based tool to convert vocals lyrics and pitch from music to autogenerate Ultrastar Deluxe, Midi and notes. It automatic tapping, adding text, pitch vocals…
Rust voice framework: TTS, recognition, and sound in one fast stack.
The most-starred other AI Music projects in the LibreAI catalogue — the realistic shortlist if DiffSinger is not the right fit.
YuE2: frontier music generation with symbolic planning, zero-shot covers, and agentic music editing.
🎵 The Ultimate Open Source Suno Alternative - Professional UI for ACE-Step 1.5 AI Music Generation. Free, local, unlimited. Stop paying for Suno!
ACE-Step — the open foundation model for music generation: full songs with vocals and lyrics in 19 languages, up to 4 minutes in about 20 seconds, Apache-2.0
Use API to call the music generation AI of suno.ai, and easily integrate it into agents like GPTs.
DiffRhythm — blazingly fast, embarrassingly simple full-length song generation with latent diffusion: complete songs with vocals from lyrics and a style prompt, Apache-2.0
Free and open — grab the source, star the repo, and start creating.