TY - JOUR

T1 - Linguistic based emotion analysis using softmax over time attention mechanism

AU - Roshan, Megha

AU - Rawat, Mukul

AU - Aryan, Karan

AU - Lyakso, Elena

AU - Mekala A., Mary

AU - Ruban, Nersissona

PY - 2024/4/16

Y1 - 2024/4/16

N2 - Recognizing the real emotion of humans is considered the most essential task for any customer feedback or medical applications. There are many methods available to recognize the type of emotion from speech signal by extracting frequency, pitch, and other dominant features. These features are used to train various models to auto-detect various human emotions. We cannot completely rely on the features of speech signals to detect the emotion, for instance, a customer is angry but still, he is speaking at a low voice (frequency components) which will eventually lead to wrong predictions. Even a video-based emotion detection system can be fooled by false facial expressions for various emotions. To rectify this issue, we need to make a parallel model that will train on textual data and make predictions based on the words present in the text. The model will then classify the type of emotions using more comprehensive information, thus making it a more robust model. To address this issue, we have tested four text-based classification models to classify the emotions of a customer. We examined the text-based models and compared their results which showed that the modified Encoder decoder model with attention mechanism trained on textual data achieved an accuracy of 93.5%. This research highlights the pressing need for more robust emotion recognition systems and underscores the potential of transfer models with attention mechanisms to significantly improve feedback management processes and the medical applications.

AB - Recognizing the real emotion of humans is considered the most essential task for any customer feedback or medical applications. There are many methods available to recognize the type of emotion from speech signal by extracting frequency, pitch, and other dominant features. These features are used to train various models to auto-detect various human emotions. We cannot completely rely on the features of speech signals to detect the emotion, for instance, a customer is angry but still, he is speaking at a low voice (frequency components) which will eventually lead to wrong predictions. Even a video-based emotion detection system can be fooled by false facial expressions for various emotions. To rectify this issue, we need to make a parallel model that will train on textual data and make predictions based on the words present in the text. The model will then classify the type of emotions using more comprehensive information, thus making it a more robust model. To address this issue, we have tested four text-based classification models to classify the emotions of a customer. We examined the text-based models and compared their results which showed that the modified Encoder decoder model with attention mechanism trained on textual data achieved an accuracy of 93.5%. This research highlights the pressing need for more robust emotion recognition systems and underscores the potential of transfer models with attention mechanisms to significantly improve feedback management processes and the medical applications.

KW - emotion analysis

KW - Male

KW - Humans

KW - Emotions

KW - Speech

KW - Voice

KW - Linguistics

KW - Recognition, Psychology

UR - https://www.mendeley.com/catalogue/5499e226-c864-3197-bc64-538188af057c/

U2 - 10.1371/journal.pone.0301336

DO - 10.1371/journal.pone.0301336

M3 - Article

C2 - 38625932

VL - 19

JO - PLoS ONE

JF - PLoS ONE

SN - 1932-6203

IS - 4

M1 - e0301336

ER -