Multi-energy radiograph and facial expression image synthesis using collaborative generative adversarial network

Abstract

Collaborative generative adversarial network (CollaGAN) is the algorithm recently proposed that is for multi-domain image translation or missing data imputation. This study expands the CollaGAN for the two different tasks with other methods for each applications. First application is the multi-energy chest radiograph bone subtraction. We propose the network of CollaGAN that can synthesize ridograph of unmeasured energy level using multi-energy radiographs. After that, we proposed eigenvalue decomposition method that decompose bone and soft tissue images from multi-energy radiographs. Also, the noise of the image is decreased as a post-processing by using wavelet cycle consistency generative adversarial network. Experimental results and qualitative evaluation verify the method effectively results in bone-removed chest radiographs. Second part of the study is about the facial expression images translation by using registration network and the GANs. For two arbitrary face images, it is difficult to train GANs alone for image translation. In this study, We introduce the registration network for the first step deformation. And then, unnatural deformed images are refined to be more realistic by CollaGAN.