Efficient privacy-preserving ML for IoT: Cluster-based split federated learning scheme for non-IID data

In this paper, we propose a solution to address the challenges of varying client resource capabilities in the IoT environment when using the SplitFed architecture for training models without compromising user privacy. Federated Learning (FL) and Split Learning (SL) are technologies designed to maintain privacy in distributed machine learning training. While FL generally offers faster training, it requires clients to train the entire neural network model, which may not be feasible for resource-limited IoT devices. Additionally, FL's performance is heavily impacted by client data distribution and struggles with non-Independent and Identically Distributed (non-IID) data. In parallel, SL offloads part of the training to a server, enabling weak devices to participate by training only portions of the model. However, SL performs slower due to forced synchronization between the server and clients. Combining FL and SL can mitigate each approach's limitations but also introduce new challenges. For instance, integrating FL's parallelism into SL brings issues such as non-IID data and stragglers, where faster devices must wait for slower ones to complete their tasks. To address these challenges, we propose a novel two-stage clustering scheme: the first stage addresses non-IID clients by grouping them based on their weights, while the second stage clusters clients with similar capabilities to ensure that faster clients do not have to wait excessively for slower ones. To further optimize our approach, we develop a multi-objective client selection solution, which is solved using a genetic algorithm to select the most suitable clients for each training round based on their model contribution and resource availability. Our experimental evaluations demonstrate the superiority of our approach, achieving higher accuracy in less time compared to several benchmarks.