Towards On-Demand Model and Client Deployment in Federated Learning with Reinforcement Learning

In Federated Learning (FL) solutions, a significant challenge lies in the limited accessibility of data sourced from diverse locations and user types, primarily due to restricted user participation. Expanding client access and diversifying data enrich models by incorporating diverse perspectives and enhancing adaptability. Increasing the client pool through volunteer devices and diversifying data enables the applicability of FL in previously static or inaccessible areas. While past research has focused on improving client selection techniques, the dynamic nature of the environment may render certain devices inaccessible as FL clients, impacting data availability and the effectiveness of current client selection methods. To overcome this problem, we propose our On-Demand solution, prioritizing the on-the-fly deployment of new clients using Docker Containers. Frequent FL model updates present challenges in environment setup, particularly when FL applications and available volunteer nodes change. Thus, this paper introduces an On-Demand solution targeting client availability and selection in FL by introducing a Reinforcement Learning-based solution driven by the need for swift model updates and the complexities of container deployment. The RL strategy employs a Markov Decision Process (MDP) framework with a Master Learner and a Joiner Learner engaging in offline and online learning based on server log data reflecting client and application demands. Simulated experiments and simulations demonstrate the adaptability of our architecture to environmental changes and On-Demand requests, along with its potential to enhance client availability, capability, accuracy, and learning efficiency.