Adaptive Reinforcement Learning Policies Through Homogeneity and Ensembles for Time-Variant UAV Systems

Abstract

This dissertation presents a comprehensive framework for enhancing the robustness, scalability, and real-world applicability of reinforcement learning (RL) in controlling unmanned aerial vehicles (UAVs) with slung loads, particularly for complex tasks such as load transportation, trajectory tracking, and confined-space placement. Addressing the limitations of traditional RL approaches and conventional gain-scheduled controllers, the proposed methods focus on systematically improving generalization capabilities to accommodate large parameter variations, bridging the gap between simulation and reality, and ensuring reliable performance in diverse operational conditions. To achieve this, the concept of dual-scale homogeneity is introduced, enabling the analytical understanding and predictable scaling of dynamic behaviors in a quadrotor with a slung-load system (QSLS). Leveraging these transformations, parameter-dependent RL policies are designed to inherently homogenize the underlying system dynamics, thereby overcoming the restrictive conditions imposed by training data and ensuring robust performance over a wide range of parameters. Furthermore, this dissertation develops the Fuzzy Ensemble of Reinforcement Learning (FERL) approach, where multiple RL policies, independently trained under different conditions, are adaptively fused into a single control command. Using fuzzy memberships informed by the current system parameters, FERL systematically fuses RL policies to achieve superior generalization and improved success rates without the need for centralized, joint training sessions. The effectiveness of these contributions is validated through extensive simulated experiments and real-world demonstrations. In simulation, the dual-scale homogeneity-based RL controllers exhibit consistent success rates, outperforming conventional controllers and reducing trajectory tracking errors under large parameter variations. The FERL framework further improves these metrics, demonstrating up to 15.6% higher success rates compared to domain randomization and robust ensemble baselines. Real-world quadrotor experiments verify the practical viability of these solutions, achieving notable improvements in tracking accuracy—reducing the root mean square error (RMSE) by 30%—and enabling precise load placement in challenging, confined environments. By integrating theoretical insights with data-driven control methods, this dissertation provides a path toward scalable, adaptable, and high-performance RL controllers for advanced UAV applications. The results pave the way for safer, more efficient, and versatile aerial robotic operations, facilitating a transition from laboratory-scale demonstrations to reliable, real-world deployments.

Date of Award	7 May 2025
Original language	American English
Supervisor	Yahya Zweiri (Supervisor)