(A) study on saliency-weighted LDA model for scene analysis

Abstract

The bag-of-visual words (BoW) models have widely been studied for image classification in a computer vision area. However, since the BoW models are mostly based on histograms, they have a limitation in discovering the distributions of visual words within images for semantic scene analysis. Therefore, there has been an attempt to use the Latent Dirichlet Allocation (LDA) model for image scene classification by revealing the latent topic distributions as feature vectors for visual words. Based on the LDA model, each image is represented by word distributions with their latent topics, which can capture semantic regularities in the image. Many previous LDA models, however, are not capable of dealing with spatial information of visual words in images, especially visual saliency which is important in scene classification and understanding. In this dissertation, the LDA model is extended, which is called saliency-weighted LDA (swLDA), by accommodating the visual saliency into the topic distribution inference for visual words in order to capture a human’s perception characteristic that image classification is often performed with focus of attention on salient regions in images. For this, all training images are first divided into image patches which are then grouped into salient and non-salient regions based on saliency maps. Then, the topic distributions of visual words are learned with saliency weights of visual words in the salient and non-salient regions separately. During the training phase, these saliency weights are learned by the swLDA model for image scene classification, which are to be used in the testing phase. While the previous LDA models parameterize the topic distributions of visual words by a single topic distribution, our proposed model incorporates saliency maps to separate the input images into salient and non-salient regions for which their respective topic distributions are computed independently. In order to show the effectiveness of the swLDA model for image scene classification, we present experiment results which reveal that the swLDA model effectively incorporates visual saliency as focus of attention to mimic the human perception behavior and outperforms the previous LDA models in terms of image classification precision.