Towards human motion capture from a camera mounted on a mobile robot

This article describes a multiple feature data fusion applied to a particle filter for marker-less human motion capture (HMC) by using a single camera devoted to an assistant mobile robot. Particle filters have proved to be well suited to this robotic context. Like numerous approaches, the principle relies on the projection of the model's silhouette of the tracked human limbs and appearance features located on the model surface, to validate the particles (associated configurations) which correspond to the best model-to-image fits. Our particle filter based HMC system is improved and extended in two ways. First, our estimation process is based on the so-called AUXILIARY scheme which has been surprisingly seldom exploited for tracking purpose. This scheme is shown to outperform conventional particle filters as it limits drastically the well-known burst in term of particles when considering high dimensional state-space. The second line of investigation concerns data fusion. Data fusion is considered both in the importance and measurement functions with some degree of adaptability depending on the current human posture and the environmental context encountered by the robot. Implementation and experiments on indoor sequences acquired by an assistant mobile robot highlight the relevance and versatility of our HMC system. Extensions are finally discussed.