Empowering assisted living: ultra leap motion and deep learning for sign language recognition

Sign language recognition (SLR) is crucial for improving communication and accessibility for people who are deaf or hard of hearing, particularly in our aging population, which increasingly requires healthcare and inclusive environments. Current SLR systems face significant challenges, such as dynamic hand gestures, real-time boundary detection, variable lighting conditions, complex backgrounds, and a lack of diverse real-world datasets. To overcome these challenges, we propose a patch-based network (PBN) that effectively leverages features from various channel patches to handle complex sign language gestures. In addition, a new SLR dataset has been created using ultra leap motion technology that contains 7800 samples related to 26 different classes with a resolution of (224 × 224 pixels). In addition, it offers various contextually relevant information useful in health-oriented domains. Comprehensive experiments are conducted in terms of ablation studies for optimal module selection, showing a remarkable performance of 97% on the ASL-A dataset and 98% on the Massey dataset. These SLR developments not only improve communication for people with disabilities but also improve their overall quality of life and independence, highlighting the critical role of technology in supporting their health and well-being.