Incremental 3D body reconstruction framework for robotic telepresence applications

This research proposes an on-line incremental 3D reconstruction framework that can be used on telepresence robots or human robot interaction (HRI). The aim is a low cost solution that enables users to communicate and interact remotely experiencing the benefits of a face-to-face meeting. By exploring computers graphics techniques and spatial audio we intend to induce sensations of being physical in the presence of other people. There is a wide variety of research opportunities including high performance imaging, multi-view video, virtual view synthesis, etc. One fundamental challenge in geometry reconstruction from traditional cameras array is the lack of accuracy in low-texture or repeated pattern region. Our approach explores virtual view synthesis through motion body estimation and hybrid sensors composed by video cameras and a depth camera based on structured-light or time-of-flight. We present a full 3D body reconstruction system that combines visual features and shape-based alignment. Modeling is based on meshes computed from dense depth maps in order lower the data to be processed and create a 3D mesh representation that is independent of view-point.