Effect of Arousal and Valence Dimensional Variation on Emotion Recognition Using Peripheral Physiological Signals Acquired from Wearable Sensors

  • Feryal Alskafi

Student thesis: Master's Thesis

Abstract

Wearable sensors have made an impact on healthcare and medicine by enabling out-of clinic health monitoring and prediction of pathological events. Further advancements made in the analysis of multimodal signals have been in emotion recognition which utilizes peripheral physiological signals captured by sensors in wearable devices. There is no universally accepted emotion model, though various multidimensional ones are often used, the most popular of which is Russell's circumplex model of affect which proposes that all affective states arise from valence and arousal. Arousal and valence values are usually measured on different numerical scales. When these ranges are translated to accommodate the use of classifiers, a heuristic assignment of these values is done to create a binary classification problem with low and high labels, with the median value either assigned to the low or high class. In day-to-day life, the neutral emotion class is the most dominant leaving emotion datasets with the inherent problem of class imbalance, which in turn affects the performance of any artificial intelligence algorithm used for classification. This study shows how the choice of classes, and the addition of the neutral class, can improve the recognition of the low valence states as unpleasant emotion awareness is of more importance in emotion recognition because negative feelings are usually indications of mental illnesses or social unrest. For the one-dimensional classification, accuracy results of up to 91.3% for both arousal and valence were achieved when two classes (low/high) were considered, and up to 86.8% for arousal and 86.3% for valence when three classes (low/neutral/high) were considered. Although the accuracy decreased, the TPR for the low classes increased 3% for arousal and about 2% for valence. In the two-dimensional classification problem, first, the intersection of low/high arousal with low/high valence gave accuracy up to 85.4% while the accuracy for the intersection of low/neutral/high arousal with low/neutral/high valence gave accuracy up to 80.1%. In terms of accuracy, the quadrant model showcased higher results, however, in terms of correctly classified instances and TPR for the LALV class that is the least occurring in the dataset, the nine-intersections model performed better, consistent with the behavior observed in the one-dimensional classification problem. The results indicate that the use of decision trees and ensemble classification, in particular ensemble bagged trees and ensemble RUSBoosted trees, combined with multimodal peripheral physiological signals may be a useful model for emotion recognition using imbalanced datasets.
Date of AwardDec 2021
Original languageAmerican English

Keywords

  • Emotion recognition
  • artificial intelligence
  • affective computing
  • wearables
  • physiological signals.

Cite this

'