Color-Glove-Assisted Recognition of Pakistan Sign Language Alphabets Using Geometrical Features

Sign language is a vital mode of communication for the deaf and hard-of-hearing community, Pakistan Sign Language (PSL) is a natural language; in this study, we focus on the PSL manual alphabet (fingerspelling) used to represent Urdu letters. While several approaches have been proposed for PSL recognition, challenges related to limited publicly available datasets, cost-effective acquisition setups, and efficient feature representation remain insufficiently explored. This paper presents a color-glove-assisted framework for the recognition of 34 static PSL alphabets. The proposed approach involves video-based dataset collection, extraction of geometrical features, and classification using Support Vector Machines (SVM) with an optimized kernel selection strategy. Unlike earlier glove-assisted PSL works that relied mainly on fuzzy inference or pixel-level features, this study introduces geometrical feature extraction (centroid, distances, and angles) combined with optimized SVM kernel selection, offering both accuracy and efficiency. In addition, a new video-based PSL alphabet dataset is introduced, contributing to the limited resources available for PSL research. Experimental results demonstrate that the SVM with a Gaussian RBF kernel achieves the highest performance, attaining an accuracy of 93.62%. To further improve model generalization and reduce reliance on handcrafted features, a deep learning model employing the VGG16 network with fine-tuning was introduced alongside machine learning classifiers, yielding enhanced recognition performance on the custom PSL dataset. Furthermore, a lightweight MobileNet-V2 model was also evaluated to investigate computationally efficient deep learning performance. The key novelties of this work include 1) a new video-based dataset of 34 PSL static alphabets, and 2) a cost-effective recognition framework leveraging color gloves and geometrical features. The findings validate the efficacy of this approach for 34 static alphabets and provide a foundation for future PSL alphabet recognition.