Learning-Based Navigation and Collision Avoidance Through Reinforcement for UAVs

Reinforcement learning (RL) has been proven to enable the automation of tasks involving complex sequential decision-making. The simulation to reality (sim2real) gap, however, poses a major challenge in most engineering applications. In this work, we propose a learning approach combining RL based navigation and collision avoidance scheme with low-level advanced control to bridge the sim2real gap for unmanned aerial vehicle (UAV) applications. The proposed approach puts the RL agent at the top of the control hierarchy to focus on behavioral intelligence. We demonstrate the transferability of the RL policy trained in simulation to a real UAV without randomization of the system&#x0027;s dynamic parameters. The direct transfer is enabled by (1) the use of deep neural networks with the modified relay feedback test (DNN-MRFT) to identify the parameters of the UAV, and (2) formulating a reward function to penalize excessive actor actions. Particularly, the RL agent generates high-level velocity actions to achieve the sought task, while the low-level controller minimizes any unwanted disturbances and model discrepancies. The proposed approach has been tested and validated using computer simulations and real-world experiments. The real-world experimental results demonstrated the agent&#x0027;s capability to achieve the navigation task with a <inline-formula><tex-math notation="LaTeX">$90 \&#x0025;$</tex-math></inline-formula> success rate. The experimental results can be found in this video: <uri>https://youtu.be/I1BF4mhJLLs</uri>.