(A) speech separation from multi-speaker dialogues under reverberant environment based on enhanced interaural coherence

Abstract

The degenerate unmixing estimation technique (DUET) and model-based expectation-maximization source separation and localization (MESSL) separate the spectrogram based on the histogram. However, accurate histogram separation is difficult because the histogram is distributed around the actual source location and overlap due to the reverberation effect. In addition, since speech recognition performance is lower than that of speech without reverberation, only a direct speech having less reverberation influence should be extracted. In order to solve this problem, the interaural coherence proposed in the previous study is used to isolate spectrogram bins which have a large influence of reverberation. However, it does not apply sufficient ensemble averaging, so we can not exactly see the effect of reverberation. In this research, we tried to apply sufficient ensemble averaging by determining the quasi-steady state interval of speech and the Canny edge detection algorithm, which is used in image processing, is applied to the spectrogram image to determine this interval. Based on the determined interval, the optimal interaural coherence calculation method is applied so that the effect of the reverberation can be seen more accurately for the same resolution. In order to extract only the direct sound source with less effect of reverberation, we proposed a model in which the coherence is applied as a sigmoid function to the MESSL. As a result, we improve the speech separation performance by reducing the distribution of the histogram and extract only the spectrogram bins with less influence of the reverberation, so that the speech recognition performance deteriorates. As a result of this research, it is possible to improve the performance of multiple direct speech separation in a reverberant environment with a small number of microphones and apply it to a mobile device or a companion robot so as to provide better service through improved speech recognition performance.