Аудио
fastrtc
FastRTC turns any Python function into a real-time audio or video stream over WebRTC or WebSockets
★ 4,6 тыс. звёздMIT
Смотреть руководство →hifi-gan — проект с открытым исходным кодом из категории «Аудио» от jik876 (jik876/hifi-gan на GitHub). У него 2 373 звезды на GitHub и 560 форков, он распространяется по лицензии MIT и последний раз обновлялся на GitHub 27 июля 2024.
GAN vocoder that turns mel-spectrograms into high-fidelity speech audio, fast enough for real-time use.
Decide whether you need inference with a pretrained checkpoint, fine-tuning on your own voice data, or training from scratch.
Clone the repository and install the Python requirements; for training, download the LJ Speech dataset and place its wavs as documented.
Download a pretrained model folder that matches your use case, such as a universal model or one fine-tuned for Tacotron2 mel-spectrograms.
Run inference from wav files or mel-spectrograms, or start training with one of the bundled V1, V2, or V3 configurations.
A GAN generator designed around periodic patterns in audio, producing 22.05 kHz speech the paper reports as close to human quality in listening tests.
Checkpoints for LJSpeech, VCTK, and a universal model, including versions fine-tuned on Tacotron2-generated mel-spectrograms.
inference.py reconstructs audio from wav-derived features, while inference_e2e.py synthesises speech directly from mel-spectrogram files for end-to-end TTS.
Generate mel-spectrograms with a model such as Tacotron2, place them in the fine-tuning dataset folder, and continue training with the fine-tuning flag.
Bundled config_v1, config_v2, and config_v3 JSON files trade model size and speed against quality.
The universal model ships with discriminator weights intended as a starting point for adapting to new datasets.
Python 3.6 or newer is required; install the dependencies from requirements.txt.
git clone https://github.com/jik876/hifi-gan.git
cd hifi-gan
pip install -r requirements.txtTrain with a bundled config, or fine-tune a checkpoint on your own mel-spectrograms.
python train.py --config config_v1.json
python train.py --fine_tuning True --config config_v1.jsonPut wav files in test_files (or mel files in test_mel_files) and generate audio with a checkpoint.
python inference.py --checkpoint_file [generator checkpoint file path]
python inference_e2e.py --checkpoint_file [generator checkpoint file path]It is maintained by jik876 (Jungil Kong) and collaborators, as the official open-source release accompanying the HiFi-GAN paper.
Yes. It is MIT-licensed, including the provided pretrained models, so research and commercial use are permitted.
No. HiFi-GAN runs entirely locally in PyTorch; there are no API keys or hosted services involved.
WaveGlow and MelGAN are earlier neural vocoders the authors drew on; HiFi-GAN focuses on modelling periodic audio patterns to reach higher fidelity at much faster than real-time generation.
Clone the repo, pip install -r requirements.txt, download a pretrained model, and run inference.py or inference_e2e.py with its checkpoint.
FastRTC turns any Python function into a real-time audio or video stream over WebRTC or WebSockets
Clone a voice from 5 seconds of audio and generate arbitrary speech in real time
On-device streaming speech-to-text engine for real-time apps.
A macOS voice assistant that turns speech into actions and notes.
MOSS-TTS-Nano: a 0.1B open multilingual speech model that clones voices and streams in real time on CPU
gTTS — Python library and CLI that turns text into spoken mp3 via Google Translate's speech API
Другие проекты из категории «Аудио» с наибольшим числом звёзд в каталоге LibreAI — реалистичный список альтернатив, если hifi-gan вам не подходит.
利用 AI 大模型和自动化工作流，根据主题或关键词一键生成高清短视频。Generate HD short videos from a topic or keyword with an automated AI workflow.
Robust Speech Recognition via Large-Scale Weak Supervision
1 min voice data can also be used to train a good TTS model! (few shot voice cloning)
Clone a voice from 5 seconds of audio and generate arbitrary speech in real time
Port of OpenAI's Whisper model in C/C++
Соседние проекты из категории «Аудио» в каталоге, ближайшие по числу звёзд к hifi-gan — продолжайте изучать полку по обе стороны от него.
VoiceMem — streaming dual-brain memory for voice agents: a left brain for facts, a right brain for emotion
ToucanTTS: fast, controllable text-to-speech for over 7,000 languages, from Stuttgart's IMS.
Offline text-to-speech for Python — uses your operating system's own voices, no internet needed
WaveRNN vocoder plus Tacotron TTS in PyTorch: train your own neural speech synthesis on LJSpeech.
MaryTTS — open-source, multilingual text-to-speech platform written in pure Java
Fast local speech-to-text across 16+ model families, powered by ggml.
Бесплатно и открыто — забирайте исходный код, ставьте звезду репозиторию и начинайте создавать.