User Routine Model Using a Cloud-Connected Social Robot

In this manuscript we propose a distributed classifier to perform inference on a person daily behaviour routine, based on multi-modal input data. The model is implemented on a social robot and allows to efficiently fuse locally perceived information with data classified remotely on a cloud. Unlike the dominant multi-class approaches, where each class is classified separately, the multi-label scheme estimates all classes simultaneously from the available input instances. This method enables a robot to capture user typical behaviour and provides a simple scheme of regulation that allows the identification of abnormal situations. We propose to solve our problem in two steps based on the principles of Binary Relevance and Label Power-set: (1) a label classification is used to filter input instances into independent labels, (2) the algorithm will map the labels into an hyper-label space, where each hyper-label represents the behaviour which maximizes input instance correlations. Results show the proposed multi-label model to achieve a highly accurate comprehension of the user behaviour even within more demanding test scenarios. As for the regulatory experiments, initial results show that the proposed behaviour model allows to identify unexpected events, that can be used to trigger care giver interventions.