Transceiver design and channel estimation in communication systems with large antenna arrays

Abstract

To support the demand of the high data rate communications, we need to increase the capacity of the communication systems and one key solution is increasing the number of antennas, namely, using a large antenna arrays. The huge improvements in spectral efficiency, compared to the conventional multiple-input multiple-output (MIMO) systems, can be achieved by using the large antenna arrays from an information theoretic perspective. However, the large antenna arrays result in large dimensional MIMO processing and high channel estimation overhead in practical use. Thus, to overcome this challenge, we should turn our attention to transceiver design and channel estimation. Fortunately, compressed sensing (CS) provides an attractive framework for dealing with large dimensional signals efficiently. In this dissertation, we propose and evaluate novel CS-based schemes for the low complexity transceiver design and the low overhead channel estimation. In the first part, the design of a hybrid MIMO transceiver consisting of a radio frequency (RF) beamformer and a baseband MIMO processor for millimeter-wave (mm-wave) communications over multiuser interference channels is considered. Sparse approximation problems are formulated to design hybrid MIMO processors approximating the minimum mean square error (MMSE) transmit/receive processors in MIMO interference channels. They are solved by orthogonal matching pursuit (OMP) based algorithms that successively select RF beamforming vectors from a set of candidate vectors and optimize the corresponding baseband processor in the least squares (LS) sense. It is shown that various beamformers can be designed by considering different types of candidate vector sets. Numerical results demonstrate the advantage of the proposed design over the conventional method that designs the baseband processor after steering the RF beams. In the second part, the channel estimation for massive MIMO systems operating in frequency division duplexing (FDD) mode is considered. By exploiting sparsity of significant propagation paths in massive MIMO channels, we develop a CS-based channel estimator that can reduce the pilot overhead as compared with the conventional LS and MMSE estimators. The proposed scheme is based on oblique matching pursuit (ObMP), an extension of OMP, that can exploit prior information about the sparse signal vector. Given the channel covariance matrix, we derive a regularized oblique operator for the proposed scheme. The pilot sequence is designed by minimizing the total coherence of the equivalent sensing matrix for a given array response matrix. It is observed that the estimation accuracy can be improved significantly by using proposed pilot sequence instead of independent and identically distributed random training sequence which is popular in CS. Numerical results demonstrate the advantage of the proposed grid-based ObMP (G-ObMP) over various existing methods including grid-based OMP (G-OMP) and LS/MMSE estimators. In the third part, the ObMP based channel estimation in massive MIMO is extended to the OFDM systems. A CS problem exploiting common sparse property is formulated for channel estimation in massive MIMO-OFDM systems based on the parametric channel model with quantized angles, called the angle grids. The problem is solved by ObMP based algorithm employing a redundant dictionary consisting of array response vectors with fine angle grids. The use of time/frequency pilot signals which are same in frequency domain but different in time domain and the low complexity scheme are proposed. Numerical results demonstrate that the proposed joint ObMP (J-ObMP) can outperform the existing methods such as joint OMP (J-OMP) and LS/MMSE estimators.