English
English
ControlNet v1.0 Model SD 1.5
Author: lllyasviel
Github repository: https://github.com/lllyasviel/ControlNet
Hugging face repository: https://huggingface.co/lllyasviel/ControlNet/tree/main
Compatible with Stable diffusion version: stable diffusion SD1.5 / 2.0
Here is a collection of ControlNet model resources provided by ControlNet author lllyasviel
ControlNet v1.0 corresponding model files and downloads
| file name | size | Update time | illustrate | Download link |
|---|---|---|---|---|
control_sd15_canny.pth | 5.71 GB | February 2023 | Download link | |
control_sd15_depth.pth | 5.71 GB | February 2023 | Download link | |
control_sd15_hed.pth | 5.71 GB | February 2023 | Download link | |
control_sd15_mlsd.pth | 5.71 GB | February 2023 | Download link | |
control_sd15_normal.pth | 5.71 GB | February 2023 | Download link | |
control_sd15_openpose.pth | 5.71 GB | February 2023 | Download link | |
control_sd15_scribble.pth | 5.71 GB | February 2023 | Download link | |
control_sd15_seg.pth | 5.71 GB | February 2023 | Download link |
ControlNet principle
The fundamental principle of ControlNet is to guide the diffusion model to generate images by adding additional control conditions. Specifically, it replicates the original neural network into two versions: a "locked" copy and a "trainable" copy. During use, the trainable replica will be learned and adjusted according to the new control conditions, while the locked replica keeps the original weight unchanged, ensuring that the generation effect of the diffusion model itself is not affected.
ControlNet usage scenarios
Stable Diffusion ControlNet 1.0 is a powerful plug-in that can control image generation through various conditions. In different types of image generation tasks, the plug-in can be flexibly applied to achieve the expected results. Here are a few specific application methods:
Image Precision Control ControlNet can control image generation based on various conditions such as edge detection, sketch processing, or human posture. For example, when a specific part of a character needs to be described in detail, precise image generation can be achieved by defining these conditions.
Fixed composition and pose definition ControlNet allows users to fix composition and pose of images to generate images that meet expectations. This is especially useful for character designs that require specific actions or poses.
Use ControlNet to draw contours and line drawings to generate a full and exquisite illustration based on line drawings alone. This is very practical in artistic creation, especially when it is necessary to build complex images from simple lines.
ControlNet can also be used to convert real avatars into two-dimensional styles, and can accurately restore the hairstyle and hair color of the characters in the original image. This feature is very helpful for anime production and character design.