TY - GEN

T1 - Gestures vs. Prosodic Structure in Laboratory Ironic Speech

AU - Васильева, Полина Евгеньевна

AU - Кочеткова, Ульяна Евгеньевна

AU - Скрелин, Павел Анатольевич

N1 - Conference code: 25

PY - 2023

Y1 - 2023

N2 - The current paper deals with paralinguistic means of irony expression in Russian laboratory speech and their synchronization with different parts of prosodic structure of the utterance. The role that gestures and prosodic characteristics of laboratory speech play in irony perception is also studied. The research is based on the part of the multimodal ironic speech corpus built at the Department of Phonetics of Saint Petersburg State University. The total of 744 audio-visual stimuli were analysed using Praat, Wave Assistant and ELAN software and then corrected manually. The material includes pairs of homonymous ironic and non-ironic target utterances that allowed making a consistent comparison. The results of the acoustic and paralinguistic analyses showed that gestures are synchronized with the nucleus in most of ironic and non-ironic utterances, the head movements prevail in both speech types, the repetitive head movements are more frequent in ironic speech. As opposed to the actors’ speech in films and series, there is no significant correspondence between pitch movement and gesture direction. Series of perceptual experiments, in which single- and multichannel signals were suggested, confirms the hypothesis about prevalence of the visual channel.

AB - The current paper deals with paralinguistic means of irony expression in Russian laboratory speech and their synchronization with different parts of prosodic structure of the utterance. The role that gestures and prosodic characteristics of laboratory speech play in irony perception is also studied. The research is based on the part of the multimodal ironic speech corpus built at the Department of Phonetics of Saint Petersburg State University. The total of 744 audio-visual stimuli were analysed using Praat, Wave Assistant and ELAN software and then corrected manually. The material includes pairs of homonymous ironic and non-ironic target utterances that allowed making a consistent comparison. The results of the acoustic and paralinguistic analyses showed that gestures are synchronized with the nucleus in most of ironic and non-ironic utterances, the head movements prevail in both speech types, the repetitive head movements are more frequent in ironic speech. As opposed to the actors’ speech in films and series, there is no significant correspondence between pitch movement and gesture direction. Series of perceptual experiments, in which single- and multichannel signals were suggested, confirms the hypothesis about prevalence of the visual channel.

KW - Acoustic Analysis

KW - F0 Direction

KW - Gestures

KW - Irony

KW - Perceptual Experiments

KW - Prosodic Structure

UR - https://www.mendeley.com/catalogue/1f627915-e7a5-36d9-be23-5835585768fe/

U2 - 10.1007/978-3-031-48309-7_25

DO - 10.1007/978-3-031-48309-7_25

M3 - Conference contribution

SN - 978-3-031-48308-0

T3 - Lecture Notes in Computer Science

SP - 301

EP - 313

BT - Speech and Computer

PB - Springer Nature

Y2 - 29 November 2023 through 2 December 2023

ER -