Project 32: Digital Artist: Creating ART with Deep Learning
Contact Information:
Prof. Lizhuang Ma
Project Description and Objectives:
Neural style transfer is one of the main techniques in the machine learning area for combining the artistic style of one image with the content of another image. The basic idea is to take the feature representations learned by a pre-trained deep convolutional neural network to obtain separate representations for the style and content of an image. Once these representations are found, we can then try to optimize a generated image to recombine the content and style of different targets. Since 2015, much progress has been made on style transfer to make training and inference faster and also to extend the style transfer technique from still images to videos and even to game scenes.
Besides, one of the most attractive research areas in Computer Vision is colorizing the grey-scale images or sketches. Focusing on clear and plausible colorization of images or sketches to obtain a final realistic result is the main goal of research in this area. Picture colorization has been shown as an inverse problem, and there are final multimodal solutions for these kinds of problems. Numerous studies have been done in image colorization, among them, some teams have been working on presenting the robust colorization of cartoon black-white pictures. However, there are some bugs in the final results of the previously proposed method, such as the inability of their methods in showing a smooth background for some images and some special kinds of reference colors.
In this project, our purpose is to present an efficient and robust solution to the problem of generating a plausible cartoon-style picture and colorization of cartoon sketches, thereby giving more inspiration to the original painting that could be accepted by human vision and perception.
Eligibility Requirements:
An ideal candidate is expected to
have experience in deep learning, as well as excellent programming skills especially in Python or C/C++.
be self-motivated and active.
respect the rules of the laboratory as well as the department.
Main Tasks:
Preparing a final report of the internship.
Providing a presentation.
Proposing a new idea for converting a given a scene photo into the desired 2D style scene through an algorithm.
Proposing a new idea to enhance the reliability of output-colored pictures for automatic coloring a line art image.
Website:
School: http://seiee.sjtu.edu.cn/