Effective task generation in meta-learning based personalized federated learning

Abstract

In Federated Learning, a central server trains a model by communicating the model with edge devices, and edge devices train the model individually on themselves without exchanging data samples with the central server. However, the trained central model obtained from the classic federated learning cannot perform well in diverse environments of edge devices. Therefore, many personalization approaches on federated learning were proposed to increase performance across diverse environments. Among these personalization methods, we focused on the method that utilizes Model-Agnostic Meta-Learning (MAML). We identified the problem that the model performance degrades in MAML-based personalization because the model overfits to the training tasks. To overcome this issue, we propose a task augmentation method, $\textit{label shuffling}$, that alleviates the task overfitting problem. For every epoch, labels are shuffled randomly, while data samples with same labels receive same labels after shuffling. With this approach, we can augment new, yet realistic tasks. We simulated our method on two human activity recognition datasets and one human speech recognition dataset. We found that the classification accuracy was improved by 4.07%p and 0.83%p for human activity recognition datasets, and the classification accuracy was improved by 8.33%p for human speech recognition dataset compared to the existing MAML-based personalization method. We visualize loss functions and conclude that the accuracy improvement is due to reduced task overfitting.