Color reproduction in virtual lip makeup using convolutional neural networks

Abstract

Traditionally consumers visited the offline store, examined the suitability of a product by swatching, and finally made a purchase decision. Recently, it is possible to examine the suitability by virtual makeup techniques and buy a product through online shopping instead of visiting the offline store. The virtual makeup can also be utilized at the offline store to prevent possible sanitation problems associated with swatching. Faithful color reproduction is one of the most important factors in the workflow of online purchased cosmetic products using virtual makeup technologies. In other words, the color difference between the virtual and real makeup results should be minimized. However, the most of previous works on the virtual makeup focus on the recommendation of makeup style. In addition, the existing virtual makeup applications and systems require further improvement in color reproduction performance. Therefore, this study proposes an accurate lipstick color reproduction method based on the Convolutional Neural Networks. Among various factors affecting the lipstick color reproduction, color of lips before makeup, the strength of applying lipstick, and color of lipstick are considered in this study. Experimental results indicate that the proposed method with the CIELab color space results in the minimum value of the color difference between the virtual and real makeup. In addition, it results in the best performance when compared with the existing methods.