ReFuSeAct: Representation fusion using self-supervised learning for activity recognition in next generation networks

Over the years, wearable sensors have gained a lot of attention from the research community due to their non-invasive nature, adoption of sensors by general public, and their applicability in healthcare services. With the advancements in communication networks, machine learning methods, and wearable sensor deployment, it is essential to design a method that could accurately classify human activities while reducing the dependence on annotated data. Traditional machine learning approaches require large-scale annotated data in order to provide a reasonable recognition performance. Recently, self-supervised learning methods are proposed but they are either limited to single sensor devices or fail to model intra-modal correlations within the self-supervised learning paradigm. In this work, we propose Representation Fusion using Self-supervised learning for Activity Recognition (ReFuSeAct) framework that uses modality-specific encoders, attention encoders, and decision-level fusion strategies to address the aforementioned limitations. The self-supervised learning paradigm ensures that the method achieves better performance even with less amount of annotated data. The architecture proposed for modality-specific encoder ensures that extraction of representative features that could help in improving recognition performance. The feature-level fusion performed using the proposed attention encoders enhances the quality of representative features that could be used in supervised learning phase. Finally, the decision-level fusion strategy enhances the activity recognition accuracy in comparison to the single deep learning classifier. Our experimental analysis shows that the proposed approach records 9.1% improvement over semi-supervised learning baselines and more than 2% improvement in comparison to existing self-supervised learning approaches.