Emotional Climate Recognition in Interactive Conversational Speech Using Deep Learning

Ghada Alhussein, Mohanad Alkhodari, Ahsan Khandokher, Leontios J. Hadjileontiadis

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

6 Scopus citations

Abstract

Emotions play a pivotal role in the individual's overall physical health. Therefore, there has been a steadily increasing interest towards emotion recognition in conversation (ERC). In this work, we propose bidirectional long short term memory (Bi-LSTM), convolutional neural network (CNN), and CNN-BiLSTM based models to predict the emotional climate established during the conversation by peers. Their speech signals across their conversation are analyzed using Mel frequency cepstral coefficients (MFCCs) that are then fed to the Bi-LSTM, CNN and CNN-BiLSTM models to predict the valence and arousal emotional climate cues. The proposed approach was tested on a publicly available dataset, namely K-EmoCon, that includes emotion labeling and peers' speech signals, during their conversation. The obtained results show that Bi-LSTM, CNN and CNN-BiLSTM models achieved a classification accuracy (arousal/valence) of 67.5%/57.7%, 73.3%/66.9%, and 75.1%/68.3%, respectively. These encouraging results show that a combination of deep learning schemes could increase the classification accuracy and provide efficient emotional climate recognition in naturalistic conversation environments.

Original languageBritish English
Title of host publicationProceedings - 2022 IEEE International Conference on Digital Health, ICDH 2022
EditorsSheikh Iqbal Ahamed, Claudio Augistino Ardagna, Hongyi Bian, Mario Bochicchio, Carl K. Chang, Rong N. Chang, Ernesto Damiani, Lin Liu, Misha Pavel, Corrado Priami, Hossain Shahriar, Robert Ward, Fatos Xhafa, Jia Zhang, Farhana Zulkernine
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages96-103
Number of pages8
ISBN (Electronic)9781665481496
DOIs
StatePublished - 2022
Event2022 IEEE International Conference on Digital Health, ICDH 2022 - Barcelona, Spain
Duration: 10 Jul 202216 Jul 2022

Publication series

NameProceedings - 2022 IEEE International Conference on Digital Health, ICDH 2022

Conference

Conference2022 IEEE International Conference on Digital Health, ICDH 2022
Country/TerritorySpain
CityBarcelona
Period10/07/2216/07/22

Keywords

  • Bi-LSTM
  • CNN
  • CNN-BiLSTM
  • conversational speech signals
  • deep learning
  • Emotion recognition in conversations
  • emotional climate
  • MFCC

Fingerprint

Dive into the research topics of 'Emotional Climate Recognition in Interactive Conversational Speech Using Deep Learning'. Together they form a unique fingerprint.

Cite this