Towards more Robust Autonomous Perception and Navigation Systems in Low-visibility Conditions with Neuromorphic Vision

Abstract

Autonomous robots require the ability to continuously perceive their surroundings to navigate extreme dynamic environments and successfully execute high-level missions safely. Various autonomous platforms of civilian and military applications are illumination dependent and incompatible to fulfil scene understanding in visually impaired conditions (e.g., night missions). Remarkably, the scene perception capabilities of robots are constrained by the onboard vision modalities, which might fail to interpret dynamic scenes when illumination is Insufficient. The neuromorphic vision sensor has the ability to revolutionize robotics perception thanks to its high dynamic range and microsecond-level temporal resolution, low latency, and high bandwidth. Therefore, this technology, coupled with artificial intelligence (AI) approaches, offers dramatic enhancements in how robotic systems operate, perceive, and understand surrounding environments with limited visibility. Nonetheless, the potential of this technology for autonomous navigation in low-light and unknown dynamic environments remains unexplored. This dissertation deals with research into the application of autonomous robots that could efficiently navigate such extreme environmental conditions. The main aim of this dissertation is to investigate, design, and develop novel (AI)-based algorithms for enhancing the situational awareness and navigation capabilities of a neuromorphic vision camera-equipped robot in unexplored environments with limited visibility. In particular, three contributions are proposed towards achieving this aforementioned aim: (1) a novel neuromorphic camera denoising approach using graph neural network-driven transformers is developed to aid perception through low light, (2) a novel neuromorphic event-based motion segmentation model using a graph transformer neural network (GTNN) algorithm is developed to guide autonomous robots navigating in unknown dynamic environments, (3) an event-based autonomous navigation framework is proposed with preliminary assessments under various illumination conditions (e.g., low light). Various publicly available and locally recorded event datasets are used to demonstrate the capability of the proposed contributions through extensive qualitative and quantitative testing and evaluations. Based on the commonly used evaluation metrics, the proposed approaches in (1) and (2) have proven their superiority against their state-of-the-art counterparts. The outcomes of this Ph.D. research unlock the potential of AI-enabled neuromorphic perception to solve some long-standing challenges of autonomous robots in complex and visually impaired environments.

Date of Award	Apr 2023
Original language	American English
Supervisor	Yahya Zweiri (Supervisor)