Scene recognition using improved codebook generation and multiple kernel learning

Abstract

n Scene Recognition, one of the commonly used approach is using the Bag-of-Words(BoW) Model. The BoW approach basically collects local features from different classes/labels and cluster them together to create visual words that best represents all the features gathered. These visual words are used to represent each image through a co-occurence histogram matrix which is used as an observation model for training and testing the classifier. One popular extension done using BoW considers the spatial information of an image. Here, a Spatial Pyramid Matching scheme is used where the co-occurrence matrix are evaluated at every region of different resolutions/pyramid levels, and are cascaded together with fixed weights. This approach has shown great results, and has been considered as state-of-the-art.

In this thesis, we improve Scene Recognition in two ways. First, is to improve codebook generation by creating a novel clustering algorithm which uses label information and an underlying assumption that less entropy clusters result to discriminative codebooks. Second, is having a novel implementation of Multiple Kernel Learning in Scene Recognition by using Spatial Kernels. Two approaches of MKL are investigated particularly Incremental-MKL and Generalized MKL. In Incremental MKL, an Adaboost framework is used where we choose the best pre-defined spatial kernels at every iteration based on the weighted data. Different types of kernel-based weak classifiers were investigated as well, particularly, weakSVM and Dyadic Hypercut. In Generalized MKL, we deal with a non-convex optimization problem which is formed from extending the SVM problem and adding a regularization term. Here, $\ell1$ (sparse) and $\ell2$ (euclidean norm) cases are investigated and optimized. The proposed approach is tested and evaluated using benchmark datasets and is compared with state-of-the-art algorithms. Results shown that Incremental MKL using weak-SVM (SPBoost-SVM) and Generalized M...