Towards Robotic Manipulation using Event-Based Camera

Abstract

Robotic vision has a major role in robotic manipulation in industry to assist and serve different applications. The traditional frame-based visual servoing has several limitations in visual feedback for robotic manipulation due to their operation method of capturing scenes continuously even if no changes in the scene is occurring. Alternatively, event cameras detect dynamic changes asynchronously at a high temporal resolution (1μs) with low latency and wide dynamic range hence, it is more of interest. In this thesis, purely event-based visual servoing methods using event-camera in an eye-in-hand configuration were presented to achieve robotic manipulation tasks. The vision-based robot controllers utilize multiple perception algorithms to regulate the robot motion during the manipulation tasks. Event-based multi-view 3D reconstruction is used for the 6 DoF localization of objects/workpiece in the environment by matching the events generated by the camera and the camera poses. In addition, an event-based approach for the detection and tracking of circular objects in the scene is also used. The vision based robot controllers used are Position Based Visual Servoing (PBVS) and Image Based Visual Servoing (IBVS). PBVS guides the end-effector towards alignment with the target point in the scene, using the 6 DoF pose estimate from the multi-view detection. IBVS refines the end-effector alignment to submillimeter accuracy using the circle detection algorithm. The perception algorithms and the event-based controllers are comprehensively studied and validated experimentally for grasping and drilling tasks. The results showed that the event based multi-view localization algorithm is valid with an average distance error of 5.08 mm. The grasping experiments for both 2D and 3D objects in good light conditions shows the effectiveness of the proposed pipelines, where the average grasping error is 0.48 cm and 0.7 cm for both 2D and 3D objects grasping pipelines, respectively. While for low light conditions, the average grasping error is 1.78 cm and 1.03 cm for both 2D and 3D objects grasping pipelines, respectively. 2D objects grasping experimental results show superior performance of the event-based visual servoing method over frame-based vision, especially in high-speed operations and poor lighting conditions. Finally, the proposed perception algorithms and robotic-vision controllers showed superior performance for the robotic drilling experiments performed with a success rate of 0.9 while maintaining a tolerance of

Date of Award	Jul 2021
Original language	American English