New technology for coloring line drawings MangaNinja: input line drawings and reference images to accurately color line drawings

Recently, a line drawing coloring method called MangaNinja has attracted widespread attention. Just input the line drawing and reference picture, and you can color the target line drawing based on the reference picture. This technology is based on the diffusion model and focuses on line drawing coloring guided by reference images, which greatly improves the accuracy and interactive control of coloring.

The research team ensured the precise transmission of character details through two innovative designs. First, they introduced a patch rearrangement module to facilitate correspondence learning between reference color images and target line drawings. Secondly, a point-driven control scheme is adopted, allowing users to finely match colors.

In their experiments, the researchers constructed a self-collected benchmark dataset and compared it with existing colorization methods. The results showed that MangaNinja significantly outperformed other methods in colorization accuracy and generated image quality. An important feature of this method is that it does not rely on point guidance in generating results and still achieves high-quality coloring effects.

MangaNinja shows its unique strengths in handling some challenging scenarios. For example, when faced with large changes in character posture or lack of details, point guidance can help solve these problems. Point guidance is also effective at preventing color confusion when multiple objects are involved. In addition, users can colorize multiple reference images by selecting specific areas of multiple reference images, thereby providing guidance for individual elements of the line drawing and effectively resolving conflicts between similar visual elements.

The technology also enables semantic color matching and fine control when using different reference images. Researchers believe that this interactive coloring method can help users find inspiration during the coloring process and provide more creative possibilities.

Project: https://johanan528.github.io/MangaNinjia/

github:https://github.com/ali-vilab/MangaNinjia