Clustered Federated Learning with Model Integration for Non-IID Data in Wireless Networks

As a typical distributed learning paradigm, federated learning has enabled network edge intelligence by making full use of the local data and the computing resources at edge devices without privacy leakage. However, due to the non-IID characteristics of data samples and the unreliability of transmission circumstances, deployment of federated learning in edge networks cannot be well guaranteed. To tackle these challenges, in this paper, a clustered federated learning paradigm with model integration is proposed. First, the detailed framework of our paradigm is introduced. The key idea is to divide the users into multiple individual user clusters by managing the scale and participants of each cluster, and the distribution divergence can be mitigated via cluster-based federated learning. Then, all the learning models are ensembled by model integration to generalize on various target tasks. Second, an upper bound on the accuracy loss of our proposed paradigm is derived, which provides some insights for the impact of data distributions and channel qualities on model performance. To further improve the accuracy performance in wireless networks, a user clustering algorithm is sophisticatedly designed. Finally, the simulation results are provided to verify the significant performance gains of our proposed framework.