TY - JOUR

T1 - Transfer Learning Based Convolution Neural Net for Authentication and Classification of Emotions from Natural and Stimulated Speech Signals

AU - Kumar, Mukula

AU - Katyal, Nipuna

AU - Ruban, Nersissona

AU - Lyakso, Elena

AU - Mekala A., Mary

AU - Noel, Joseph Raj, Alex

AU - Richard G, Maarc

PY - 2021/8/11

Y1 - 2021/8/11

N2 - Over the years the need for differentiating various emotions from oral communication plays an important role in emotion based studies. There have been different algorithms to classify the kinds of emotion. Although there is no measure of fidelity of the emotion under consideration, which is primarily due to the reason that most of the readily available datasets that are annotated are produced by actors and not generated in real-world scenarios. Therefore, the predicted emotion lacks an important aspect called authenticity, which is whether an emotion is actual or stimulated. In this research work, we have developed a transfer learning and style transfer based hybrid convolutional neural network algorithm to classify the emotion as well as the fidelity of the emotion. The model is trained on features extracted from a dataset that contains stimulated as well as actual utterances. We have compared the developed algorithm with conventional machine learning and deep learning techniques by few metrics like accuracy, Precision, Recall and F1 score. The developed model performs much better than the conventional machine learning and deep learning models. The research aims to dive deeper into human emotion and make a model that understands it like humans do with precision, recall, F1 score values of 0.994, 0.996, 0.995 for speech authenticity and 0.992, 0.989, 0.99 for speech emotion classification respectively.

AB - Over the years the need for differentiating various emotions from oral communication plays an important role in emotion based studies. There have been different algorithms to classify the kinds of emotion. Although there is no measure of fidelity of the emotion under consideration, which is primarily due to the reason that most of the readily available datasets that are annotated are produced by actors and not generated in real-world scenarios. Therefore, the predicted emotion lacks an important aspect called authenticity, which is whether an emotion is actual or stimulated. In this research work, we have developed a transfer learning and style transfer based hybrid convolutional neural network algorithm to classify the emotion as well as the fidelity of the emotion. The model is trained on features extracted from a dataset that contains stimulated as well as actual utterances. We have compared the developed algorithm with conventional machine learning and deep learning techniques by few metrics like accuracy, Precision, Recall and F1 score. The developed model performs much better than the conventional machine learning and deep learning models. The research aims to dive deeper into human emotion and make a model that understands it like humans do with precision, recall, F1 score values of 0.994, 0.996, 0.995 for speech authenticity and 0.992, 0.989, 0.99 for speech emotion classification respectively.

KW - Deep learning

KW - linear prediction cepstral coefficients (LPCC)

KW - mel frequency cepstral coefficients (MFCC)

KW - speech emotion recognition

KW - speech fidelity classification

KW - RECOGNITION

KW - FREQUENCY

UR - http://www.scopus.com/inward/record.url?scp=85113260578&partnerID=8YFLogxK

UR - https://www.mendeley.com/catalogue/776f5d4c-3bd8-3475-9472-cb6252564d87/

U2 - 10.3233/jifs-210711

DO - 10.3233/jifs-210711

M3 - Article

VL - 41

SP - 2013

EP - 2024

JO - Journal of Intelligent and Fuzzy Systems

JF - Journal of Intelligent and Fuzzy Systems

SN - 1064-1246

IS - 1

ER -