Monaural speech segregation based on pitch track correction using bayesian filters

Abstract

In this work, pitch tracking technique that adopts Bayesian filters and speech/music pitch classification using recurrent neural networks (RNN) for speech segregation from mixtures of speech and competing sounds are proposed. Conventional speech segregation methods use sub-band masking in which the masks are obtained by modulation at the found speech pitch frequency. Segregation performance, therefore, relies heavily on the quality of the pitch estimation. However, pitch estimation is difficult in severe noise environment. In order to improve the accuracy of estimation, we use Bayesian filters which are popularly used in object tracking from noisy videos. Two types of Bayesian filters, particle filter and ensemble Kalman filter, are adopted for tracking the pitch contours. The particle filter uses a simple first-order Markovian process from the past state to the present, and the ensemble Kalman filter adds a linear transition model to the same Markovian model. As speech and music has similar harmonic structures, the conventional speech segregation methods based on sub-band masking perform badly against music interference. Therefore, we propose speech/music pitch classification which adopts RNNs, which are simple recurrent network, long short-term memory (LSTM) and bidirectional LSTM, for modeling the characteristics of the speech pitch and music pitch. The experiment results conducted on mixtures of speech signals and various types of noise and music sound sources show that the proposed methods achieved significantly better segregation performance than the conventional method in most cases. Among all proposed methods, the segregation method with ensemble Kalman filter and bidirectional LSTM achieved the best performance.