TY - GEN

T1 - Transformer Models for Question Answering on Autism Spectrum Disorder QA Dataset

AU - Firsanova, Victoria

N1 - Conference code: 6

PY - 2022

Y1 - 2022

N2 - Question answering (QA) Transformer-based models might become efficient in inclusive education. For example, one can test and tune such models with small closed-domain datasets before the implementation of a new system in an inclusive organization. However, studies in the sociomedical domain show that such models can be unpredictable. They can mislead a user or evoke aversive emotional states. The paper addresses the problem of investigating safety-first QA models that would generate user-friendly outputs. The study aims to analyze the performance of SOTA Transformer-based QA models on a custom dataset collected by the author of the paper. The dataset contains 1 134 question-answer pairs about autism spectrum disorders (ASD) in Russian. The study presents the validation and evaluation of extractive and generative QA models. The author used transfer learning techniques to investigate domain-specific QA properties and suggest solutions that might provide higher QA efficiency in the inclusion. The study shows that although generative QA models can misrepresent facts and generate false tokens, they might bring diversity in the system outputs and make the automated QA more user-friendly for younger people. Although extractive QA is more reliable, according to the metric scores presented in this study, such models might be less efficient than generative ones. The principal conclusion of the study is that a combination of generative and extractive approaches might lead to higher efficiency in building QA systems for inclusion. However, the performance of such combined systems in the inclusion is yet to be investigated.

AB - Question answering (QA) Transformer-based models might become efficient in inclusive education. For example, one can test and tune such models with small closed-domain datasets before the implementation of a new system in an inclusive organization. However, studies in the sociomedical domain show that such models can be unpredictable. They can mislead a user or evoke aversive emotional states. The paper addresses the problem of investigating safety-first QA models that would generate user-friendly outputs. The study aims to analyze the performance of SOTA Transformer-based QA models on a custom dataset collected by the author of the paper. The dataset contains 1 134 question-answer pairs about autism spectrum disorders (ASD) in Russian. The study presents the validation and evaluation of extractive and generative QA models. The author used transfer learning techniques to investigate domain-specific QA properties and suggest solutions that might provide higher QA efficiency in the inclusion. The study shows that although generative QA models can misrepresent facts and generate false tokens, they might bring diversity in the system outputs and make the automated QA more user-friendly for younger people. Although extractive QA is more reliable, according to the metric scores presented in this study, such models might be less efficient than generative ones. The principal conclusion of the study is that a combination of generative and extractive approaches might lead to higher efficiency in building QA systems for inclusion. However, the performance of such combined systems in the inclusion is yet to be investigated.

KW - Question answering

KW - Dialogue system

KW - Transformer

UR - http://dtgs-conference.org/

UR - http://www.scopus.com/inward/record.url?scp=85124659793&partnerID=8YFLogxK

UR - https://www.mendeley.com/catalogue/d7794455-f073-39cf-a5db-a0db82680a43/

U2 - 10.1007/978-3-030-93715-7_9

DO - 10.1007/978-3-030-93715-7_9

M3 - Conference contribution

SN - 978-3-030-93714-0

T3 - Communications in Computer and Information Science

SP - 122

EP - 133

BT - Digital Transformation and Global Society - 6th International Conference, DTGS 2021, Revised Selected Papers

A2 - Alexandrov, Daniel A.

A2 - Chugunov, Andrei V.

A2 - Kabanov, Yury

A2 - Koltsova, Olessia

A2 - Musabirov, Ilya

A2 - Pashakhin, Sergei

A2 - Boukhanovsky, Alexander V.

A2 - Chugunov, Andrei V.

PB - Springer Nature

CY - Cham

T2 - 6th International Conference on Digital Transformation and Global Society, DTGS 2021

Y2 - 23 June 2021 through 25 June 2021

ER -