Multi-spectral image fusion for victim detection in urban search-and-rescue robots

Abstract

This thesis investigates the field of multi-spectral image fusion systems in general, and the fusion of visual and thermal images in particular. The thesis tackles all stages of a complete fusion system from registering captured images till the delivery of the fused image. In addition, it highlights the area of performance evaluation with a special emphasis on blind objective evaluation measures. Image fusion systems start by registering input images. Registration is done in two phases. In the first phase, pairs of corresponding points in both images are identified. The pair-wise point correspondence allows us to model the spatial transformation required to register the two images. In the target platform, the transformation takes the form of a rigid transformation and the pair-wise correspondence is established manually and off-line. The first phase is performed only once for a particular sensor configuration. In the second phase, the thermal image is spatially transformed in order to be aligned with the visual image. Following the registration, input images are enhanced to facilitate a better fusion. In addition, both images are spatially remapped into a log-polar representation. The log-polar transformation allows a reduction in the sampling density in both image dimensions without a loss in perceptual information. Hence, the computational complexity decreases making a real-time implementation feasible. For the proposed method, using log-polar images instead of regular images approximately halved the execution time. The fact that perceptual information is not lost assumes that the images are centred on an area of interest in the scene, as will normally be the case in actively-controlled camera such as in USAR application. This is because we are matching the log-polar sampling pattern of the registered images with the foveated sampling pattern of the retina. Registered input images in the log-polar form are then fused using a multiscale fusion technique. After that, the fused image is transformed back to the normal Cartesian coordinates. In order to ease the detection of victims in USAR environments, the fused image is coloured using a novel false-colouring method. Consequently, the final fused image depicts essential information from both modalities with victims in the scene having a reddish appearance. Finally, the proposed fusion process is objectively evaluated by comparing it with existing fusion techniques using a set of established metrics. The metrics used are the Mutual Information (MI), Xydas & Petrovic (QAB=F ), and Piella & Heijmans (Qw) metrics which are used to compare the proposed system with the Laplacian pyramid, the Discrete Wavelet Transform, and the image averaging fusion methods.

Date of Award	2009
Original language	American English
Supervisor	David Vernon (Supervisor)