Design of a unified face recognition system using multiple features

Abstract

In this dissertation, we present a robust face recognition system using an extended Gabor-based facial feature extraction method and a Markov network-based score fusion scheme. From the viewpoint of feature extraction, the extended curvature Gabor (ECG) kernels are first proposed by adding a spatial curvature term to the kernel and adjusting the width of the Gaussian at the kernel, which leads to numerous feature candidates being extracted from a single low-resolution image. A single ECG classifier is implemented by applying the linear discriminant analysis (LDA) method to the feature vectors selected according to the ECG kernel parameters. Though the single classifier demonstrates better results than the previous face recognition method, there are performance limitations in large face databases such as the Face Recognition Grand Challenge (FRGC) ver 2.0. To overcome the accuracy limitation of a single classifier, we propose an ECG classifier bunch that combines multiple ECG classifiers with the fusion scheme. Moreover, a novel unifying framework using a Markov network is proposed to understand the relationship among multiple classifiers. We assume that we have several complementary classifiers, for example, the bunch of proposed ECG classifiers. Under the proposed unified framework, we assign observation nodes to the features of a query image and hidden nodes to the features of gallery images, and we connect each hidden node to its corresponding observation node and to the hidden nodes of other neighboring classifiers on the Markov assumption. For each observation-hidden node pair, we collect the set of gallery candidates that are most similar to the observation instance, and the relationship between the hidden nodes is captured in terms of the similarity matrix between the collected gallery images. Posterior probabilities in the hidden nodes are computed using the belief propagation algorithm, and we use the marginal probability as the new similarity value of a classifier. We present extensive evaluation results using four publicly available databases: FRGC ver. 2.0, XM2VTS, BANCA, and Multi-PIE. To ensure that the comparison is fair, we create two different test protocols: known variation and unknown variation tests. The first one is designed to evaluate trained facial variations, and the other examines untrained facial variations. For example, we only use the FRGC images as the training face models, which have only frontal face variations under the controlled and uncontrolled variations, and we use the other FRGC images as the test images in the known variation test. But, in the unknown variation test, the FRGC trained face models are evaluated using another database, e.g., Multi-PIE images whose images include pose variations. Through these extensive evaluations, we show that the proposed framework consistently leads to improved recognition rates in various situations.