Synthetic speech detection using self-supervised learning and variational information bottleneck

Abstract

Synthetic speech detection is a task that decides whether an input speech is from human or synthetic speech system such as a text-to-speech (TTS) or voice conversion (VC). Recent advances in sophisticated synthetic speech generated from TTS or VC systems cause threats to the existing automatic speaker verification (ASV) systems. Also, there is a possibility for abuse of human-like synthetic speech in crimes such as impersonation or fake news. Since such synthetic speech is generated from the diverse algorithms, generalization ability with using limited training data is indispensable for a robust synthetic speech detection system. In this thesis, we propose a self-supervised learning scheme based on the wav2vec 2.0 pretrained model with variational information bottleneck (VIB) for the synthetic speech detection task to improve the generalization ability. Evaluation on the ASVspoof 2019 logical access (LA) database shows that our method improves the performance of distinguishing unseen synthetic speech and genuine speech, outperforming current state-of-the-art synthetic speech detection systems. Furthermore, we show that the proposed system improves performance in low-resource and cross-dataset settings of the synthetic speech detection task, demonstrating that our system is also robust in terms of data size and data distribution. Finally, we also propose a Korean synthetic speech detection system using self-supervised learning by constructing corresponding dataset.