Statistical heterogeneity-aware federated approach and application

Abstract

In this thesis, we propose robust federated learning (FL) approach in a statistical heterogeneous environment and develop FL based automatic music transcription (AMT) system. FL is emerging as a next-generation collaborative computing framework to replace traditional centralized learing (CL) methods. Specifically, the machine learning models of clients are trained through secure model aggregation rather than data aggregation. Due to statistical heterogeneity in the client's local datasets, conventional FL approaches results in degradation of performance in FL. We proposed greedy aggregation to tackle the statistical heterogeneity challenge of FL. With proposed approach, we have achieved higher performance than FedAvg and FedShare with low communication cost based on MNIST dataset. AMT is the process of converting an acoustic musical signal into form of musical notation. Due to data privacy leak problems and high communication costs of transferring audio data to the parameter server, it is a major challenge to build AMT models on traditional machine learning with large scale. In this thesis, we designed FedMusic - automatic music transcription system powered by a cooperative training method. With proposed system, a client can operate AMT locally without uploading its audio data to the parameter server with low communication cost. We have shown that proposed system outperformed CL with low communication cost experimented on the MAESTRO dataset.