Recently, it has become possible to examine the suitability of cosmetic products by virtual makeup techniques so that shoppers can buy products online. The virtual makeup can also be utilized at offline stores to prevent possible sanitation problems associated with swatching. Faithful color reproduction is one of the most important factors in virtual makeup applications. Thus, the color difference between the virtual and real makeup results needs to be minimized. However, most previous studies on virtual makeup focus on the recommendation of makeup style rather than on the accuracy of color reproduction. This article proposes an accurate lipstick color reproduction method based on convolutional neural network. This study indicates that the proposed method using a convolutional neural network results in the minimum value of color difference compared with linear regression and multilayer perceptron algorithms.