Detection problems in the spiked random matrix models

Abstract

In this dissertation, we study the statistical decision process of detecting the low-rank signal from various signal-plus-noise type data matrices, known as the spiked random matrix models. Various spectral phenomena observed in these types of models have been actively studied in random matrix theory until recently. One of the most remarkable phenomena is that the principal components (the largest eigenvalues and the associated eigenvectors) exhibit the sharp phase transition, named Baik-Ben Arous-P\'{e}ch\'{e} (BBP) transition. In the first part of this dissertation, we first show that the vanilla principal component analysis (PCA) is a sub-optimal way for the non-Gaussian noise. Specifically, we prove that the PCA can be improved by applying the non-linear pre-transformation element-wisely to the data matrix if the noise is non-Gaussian. As an intermediate step, we find out sharp BBP-type phase transition thresholds for the extreme eigenvalues of spiked random matrices. We also prove the central limit theorem for the linear spectral statistics (LSS) for the spiked random matrices and propose a hypothesis test based on it, which does not depend on the distribution of the signal or the noise. When the noise is non-Gaussian, the LSS-based test can be improved with an entrywise transformation to the data matrix with additive noise. We also introduce an algorithm that estimates the rank of the signal when it is not known a prior. After this, in the second part, we consider more structural Gaussian noise matrices such as the Gaussian block matrix, and the Gaussian band matrix. In particular, we consider not only the phase transition phenomenon of the principal components, but the likelihood ratio of the spiked matrix models with such noise structures, and prove its asymptotic normality. This provides the exact error of the likelihood ratio test.