Test-time adaptation for automatic speech recognition via sequential-level generalized entropy minimization

Abstract

In real-world scenarios, automatic speech recognition (ASR) models often encounter data distribution shifts, leading to erroneous predictions. To tackle this issue, a recent test-time adaptation (TTA) method has been proposed to adapt the pre-trained ASR model to the unlabeled target domain without source data. Despite decent performance gain, this approach relies solely on naive greedy decoding and performs adaptation across timesteps at the frame level, which may not be optimal given the sequential nature of model outputs. Motivated by this limitation, this thesis introduces a novel Sequential-level Generalized Entropy Minimization (SGEM) framework for general ASR models. To handle sequential output, SGEM first exploits beam search to explore candidate output logits and selects the most plausible one. Then, it utilizes generalized entropy minimization and negative sampling as effective unsupervised objectives to adapt the model. Through extensive experiments, SGEM verifies its state-of-the-art performance across three mainstream ASR models under various distribution shifts.