Neural audio fingerprinting for broadcast monitoring with source separation

Abstract

Audio fingerprinting systems have evolved over time through frequency-analysis techniques, and have recently shown significantly improved performance in noisy environments through deep neural networks. However, these systems work well for identifying music played in specific spaces, but show lower performance in broadcast monitoring tasks. A major problem is that both deep neural network-based and frequency analysis-based systems often fail to detect music segments, mistaking them for non-musical content, primarily due to speech noise overpowering the music in broadcast audio. To address this, our study employed a pre-trained source separation model to remove vocals before feeding the query audio into the fingerprint extraction model, enhancing the performance of the broadcast monitoring system. Furthermore, We fine-tuned the source separation model to optimize it for speech removal. To do this, we customized the training dataset by replacing the vocal source with speech source. As a result, we improved the speech removal performance, boosting the performance of the broadcast monitoring system.