Deep learning Approaches for Human Gait Segmentation and Activity Recognition on Lower-Limb Exoskeleton

Abstract

Wearable exoskeletons can improve the quality of life of individuals with lower-limb impairments. However, the use of these devices is limited by their inability to adapt to individual patients’ needs, reducing their effectiveness and increasing the risk of injury. Deep learning-based action segmentation model for enhanced human gait assistance has shown promise in addressing this issue by providing more personalized and adaptive assistance. This study aims to replicate the results obtained from a Multi-Stage Spatial-Temporal Graph Convolutional Neural Network (MS-GCN) algorithm for action segmentation and compare them with those reported by the original developers. Then, a Transformer model was constructed and compared to the MS-GCN. Our reproduction process involves downloading the original code and model and preprocessing four datasets used in the original paper. We train and test the MS-GCN model on each dataset with different down-sampling factors and evaluate the results using accuracy metric. An action segmentation model that is based on Transformer is proposed and trained. Our reproduced results show a slight difference in performance compared to the original results reported by the authors, and close results were noticed for the Transformer in some datasets. We have modified and optimized the MS-GCN model to acquire the Parameter-Optimized MS-GCN model (PO-MS-GCN) which achieved higher results than the state-of-the-art. Then, the PO-MS-GCN and the Transformer were trained and the features from the last layer of each model were combined and fed into a classifier. The findings prove that PO-MS-GCN outperforms state-of-the-art models in human activity recognition. Specifically, HuGaDB achieved an accuracy of 92.7%, TUG achieved an accuracy of 93.2%, while LARa and PKU-MMD achieved lower accuracies of 64.31% and 69%, respectively. Moreover, feature fusion exceeded the PO-MS-GCN’s results in PKU-MMD, LARa, and TUG datasets. In addition to the previously mentioned work, sensory data were converted to images via Fourier and wavelet transforms to generate spectrograms and scalograms. These images were then used to train the pre-trained ResNet50 model with the addition of 3 custom layers to perform Human Activity Recognition task using transfer learning on two datasets, HuGaDB and LARa. ResNet50 achieved remarkable performance, surpassing state-of-the-art methods and even outperforming feature fusion approach. For HuGaDB, ResNet50 achieved 93.31% accuracy and 93.33% F1-score with scalograms, while achieving 84.17% accuracy and 84.20% F1-score with spectrograms. On the LARa dataset, ResNet50 achieved 63.30% accuracy and 62.30% F1-score with scalograms, as well as 66.14% accuracy and 65.65% F1-score with spectrograms. These results highlight the efficacy of utilizing image representations of sensory data alongside deep learning models like ResNet50 for enhanced performance in human activity recognition tasks, showcasing superior accuracy levels compared to existing methodologies.

Date of Award	26 Apr 2024
Original language	American English
Supervisor	Irfan Hussain (Supervisor)