On-Demand Model and Client Deployment in Federated Learning with Deep Reinforcement Learning

In Federated Learning (FL), the limited accessibility of data from diverse locations and user types poses a significant challenge due to restricted user participation. Expanding client access and diversifying data enhance models by incorporating diverse perspectives, thereby improving adaptability. However, in dynamic and mobile environments, the availability of FL clients fluctuates as devices may become inaccessible, leading to inefficient client selection and reduced model performance. Current solutions often fail to adapt quickly to these changes, creating a gap in achieving real-time client availability and efficient data utilization. To address this, we propose a Deep Reinforcement Learning (DRL) On-Demand solution, deploying new clients using Docker Containers on-the-fly. Our On-Demand solution, employing DRL, targets client availability and selection while considering data shifts and container deployment complexities. It employs an autonomous end-to-end approach for handling model deployment and client selection. The DRL strategy leverages a Markov Decision Process (MDP) framework, with a Master Learner and a Joiner Learner to optimize decision-making. The designed cost functions account for the complexity of dynamic client deployment and selection, ensuring effective resource management and service reliability. Simulated tests show that our architecture can easily adapt to changes in the environment and respond to On-Demand requests while reducing the number of learning rounds used by 20-50 % compared with existing approaches. This highlights its ability to improve client availability, capability, accuracy, and learning efficiency, surpassing heuristic and traditional reinforcement learning methods.