Image
lite.ai.toolkit
A lite C++ AI toolkit: 100+ models with MNN, ORT and TRT, including Det, Seg, Stable-Diffusion, Face-Fusion.
★ 4.4k starsGPL-3.0
View guide →ControlNet — the original conditional control for diffusion models: steer Stable Diffusion with edges, depth, pose and scribbles instead of prompts alone, Apache-2.0
ControlNet is an open-source image project by lllyasviel (lllyasviel/ControlNet on GitHub). It has 34,123 GitHub stars and 3,019 forks, is released under the Apache-2.0 license, and was last updated on GitHub on 25 February 2024.
Per the README: conda env create -f environment.yaml, then conda activate control.
Get the pretrained models and annotator detectors from the project Hugging Face page — SD models go in ControlNet/models, detectors in ControlNet/annotator/ckpts.
Run the app for your condition, e.g. python gradio_canny2image.py for edge control or gradio_pose2image.py for pose control, and generate in the browser UI.
Edges, depth, pose, segmentation, scribbles and line art become hard constraints the diffusion model must follow.
The base model stays frozen and safe; only the trainable copy learns the condition, so small datasets suffice.
One app per condition type, each with the relevant detector (Canny thresholds, OpenPose, MiDaS depth and others) built in.
Control weights can be transferred to customised Stable Diffusion checkpoints, as documented in the project discussions.
The README install path.
conda env create -f environment.yaml
conda activate controlDownload models and detectors from the Hugging Face page into the folders the README names.
https://huggingface.co/lllyasviel/ControlNetExample: Canny edge control in the browser.
python gradio_canny2image.pyLvmin Zhang (lllyasviel) and co-authors; this is the official implementation of the ControlNet paper (arXiv 2302.05543).
Yes — Apache-2.0 licensed. The base Stable Diffusion models it controls carry their own licences, which you should check separately.
For the original models and research code, this repo. For day-to-day generation, most people use the same ControlNet models through the A1111 extension or ComfyUI nodes, which are also listed on LibreAI.
Against plain Stable Diffusion prompting, ControlNet adds structural guarantees a prompt cannot give. Against IP-Adapter-style methods, ControlNet controls layout and pose from explicit maps, while adapters transfer style/subject from a reference image — many pipelines use both.
conda env create -f environment.yaml, activate control, download the models and detectors from Hugging Face, then run one of the gradio_*.py apps.
A lite C++ AI toolkit: 100+ models with MNN, ORT and TRT, including Det, Seg, Stable-Diffusion, Face-Fusion.
Create 🔥 videos with Stable Diffusion by exploring the latent space and morphing between text prompts
Easily modify and tune a Stable Diffusion VAE
Stable Diffusion with Core ML on Apple Silicon
stable diffusion webui colab
Multi-Platform Package Manager for Stable Diffusion
The most-starred other Image projects in the LibreAI catalogue — the realistic shortlist if ControlNet is not the right fit.
Stable Diffusion web UI
The node-graph powerhouse for precise, professional AI image and video generation.
High performance self-hosted photo and video management solution
Focus on prompting and generating
🆙 Upscayl - #1 Free and Open Source AI Image Upscaler for Linux, MacOS and Windows
Free and open — grab the source, star the repo, and start creating.