Hybrid image registration of endoscopic robotic capsule (ERC) images using vision-inertial sensors fusion

In this paper, a hybrid registration technique, which takes advantage of data fusion of multiple sensors integrated inside a robotic endoscopic capsule, is proposed to enhance the registration accuracy and construct the 3D trajectory of the endoscopic robotic capsule. The proposed hybrid technique extracts motion information of the endoscopic capsule from the image sequences captured by the capsules camera and combines it with the Inertial Measurements Unit readings, to simultaneously localize and map the path traveled by the capsule device. Furthermore, the performance of three different bundle adjustment techniques was evaluated for mapping and registering endoscopic capsule images. The evaluated methods are: (i) global bundle adjustment, (ii) local bundle adjustment and (iii) inertial-integrated local bundle adjustment. The performance of the three bundle adjustment techniques were compared in terms of number of iterations, elapsed time, initial and final errors while varying the temporal distances between images and the sliding window size. Experimental results show that a 3D map can be precisely constructed to represent the position of the capsule inside the colon. The proposed method allows the operator to register several maps of endoscopic images toward the reconstruction of the full colon model.