Emotion Recognition in the Wild from Long-term Heart Rate Recording using Wearable Sensor and Deep Learning Ensemble Classification

Sara A. Nasrat, Uichin Lee, M. Sami Zitouni, Ahsan H. Khandoker, Soowon Kang, Herbert F. Jelinek

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Scopus citations

Abstract

Long-term, continuous physiological recordings are currently being intensely investigated for tracking emotions. Emotional valence has been of more interest due to its relevance to cardiac and neurophysiological disease. In this research, multiple configurable convolutional neural networks (CNNs) were developed for different image-encoding techniques used as their input. Ensemble classification was then used to achieve a combined performance of the multiple CNNs by training a simple support vector machine (SVM) classifier using the last output layers of the CNNs as its input. Valence-labelled signals from the heart rate (HR) recorded using a wearable sensor from a wristband in a daily setting for one week from 80 participants were used for the image transforms. Accuracies of more than 91% were achieved with the classification ensembling, showing an improvement of the binary classification of emotional valence by more than 19% compared to using CNNs on their own.

Original languageBritish English
Title of host publicationProceedings - 2021 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2021
EditorsYufei Huang, Lukasz Kurgan, Feng Luo, Xiaohua Tony Hu, Yidong Chen, Edward Dougherty, Andrzej Kloczkowski, Yaohang Li
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1676-1678
Number of pages3
ISBN (Electronic)9781665401265
DOIs
StatePublished - 2021
Event2021 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2021 - Virtual, Online, United States
Duration: 9 Dec 202112 Dec 2021

Publication series

NameProceedings - 2021 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2021

Conference

Conference2021 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2021
Country/TerritoryUnited States
CityVirtual, Online
Period9/12/2112/12/21

Keywords

  • convolutional neural network
  • deep learning
  • emotion recognition
  • emotional valence
  • ensemble classification
  • heart rate
  • wearable sensor

Fingerprint

Dive into the research topics of 'Emotion Recognition in the Wild from Long-term Heart Rate Recording using Wearable Sensor and Deep Learning Ensemble Classification'. Together they form a unique fingerprint.

Cite this