Contrastive representation based image translation and continual learning

Abstract

The primary objective of image-to-image translation tasks is to learn a mapping function from a source domain to a target domain, maintaining the content while converting the appearance similar to target domain. Also, continual learning aims to learn the complete knowledge for a set of tasks when each task is presented sequentially. I propose the unified approach for two distinct tasks based on the contrastve learning, exploring the inherent invariance properties in both tasks. Specifically, for image translation tasks, the input and output image can regarded as two different views of the same instance, as they share the contents but have different appearance. Likewise, the conitnual learning involves two different views of the same data, provided by the current model and the previous model. In both tasks, enhancing shared information between the two different views is crucial. Hence, I propose the contrastive learning based methodologies to guide the model to learn the desired representation. For the image translation tasks, I present three different approaches. Firstly, I focus on the diverse contrastive semantic relation of the image patches. I propose the consistency of the patch-wise semantic relation, and the hard negative mining strategy. Secondly, I suggest the deep metric learning for the CT denoising, focusing on the invarinace property between the low-dose and high-dose CT images. Lastly, I explore the invariance of the topology-aware graph features for patch-wise recognition targetted for image translation tasks. For the conitnual learning, I propose the novel approach for prompt-based continual learning, which enhance the correspondence between the features extracted by the current prompt and the previous prompt.