Extraction of independent discriminant features for improved classification performance

Abstract

In this dissertation, a single stage linear semi-supervised extraction of discriminative independent features is proposed. Discriminant ICA $\It{d}$ICA presents a framework of linearly projecting multivariate data to a lower dimension where the features are maximally discriminant and have minimal redundancy. The optimization problem is formulated as the maximization of linear summation of Negentropy and weighted functional measure of classification. Motivated by independence among features, Fisher linear discriminant is used as the functional measure of classification. Under the unit covariance constraint of extracted $\It{d}$ICA features, maximization of between-class scatter of features matrix is theoretically the same as the maximization of Fisher linear discriminant. It not only reduces the computational complexity of the algorithm but also avoids singularity posed by small within-class scatter. Experimental results show improved classification performance when $\It{d}$ICA features are used for recognition tasks in comparison to unsupervised and supervised feature extraction techniques. $\It{d}$ICA features also give reduced data reconstruction error in comparison to LDA and ICA method based on Negentropy maximization. Prior to the work on $\It{d}$ICA, a new feature selection (filtering) method was proposed to select discriminant features from a set of unsupervised features. Proposed hybrid feature selection (HFS) method is applied to the union of feature subspaces selected by Fisher criterion and univariate feature-class mutual information (MI). It scores each feature as a linear weighted sum of its MI with multinomial class distribution and Fisher criterion score. HFS selects features with higher class discrimination in comparison to feature-class MI, Fisher criterion or unsupervised selection using variance of features; thus, resulting in improved recognition performance. In addition, it was also highlighted that HFS is an optimal feature selector ...