TY - GEN

T1 - Argument Identification for Neuro-Symbolic Dispute Resolution in Scientific Peer Review

AU - Прокудин, Дмитрий Евгеньевич

AU - Лисанюк, Елена Николаевна

AU - Баймуратов, Ильдар Раисович

AU - Карпович, А.

N1 - Conference code: 24

PY - 2025/3/13

Y1 - 2025/3/13

N2 - Peer review is a cornerstone of the academic editorial decisionmaking process, yet it faces significant challenges. Artificial intelligence can help address these challenges, but its use raises concerns about reliability and the potential for reproducing existing biases. In this research, we employ a formal argumentation-Theoretic framework that allows for explicit analysis of arguments and their interrelations, combined with argument mining techniques to streamline the formalization of peer reviews, and resulting in a neuro-symbolic approach to dispute resolution. Our method involves identifying parties arguments in peer reviews and representing them as abstract argumentation frameworks, which facilitate dispute resolution through logical inference. We annotate these frameworks within a corpus of scientific peer reviews, achieving a high Krippendorff s alpha of 0.81. Having the annotated corpus, we implement an argument mining pipeline that integrates BERT sentence embeddings with an LSTM model, classifying sentences into three categories: Authors arguments, reviewers arguments, and nonarguments. We achieved an accuracy of 0.634 and an F1 score of 0.631, which are comparable to models trained on other datasets. However, our approach stands out by enabling the processing of the extracted argumentation with logical inference.

AB - Peer review is a cornerstone of the academic editorial decisionmaking process, yet it faces significant challenges. Artificial intelligence can help address these challenges, but its use raises concerns about reliability and the potential for reproducing existing biases. In this research, we employ a formal argumentation-Theoretic framework that allows for explicit analysis of arguments and their interrelations, combined with argument mining techniques to streamline the formalization of peer reviews, and resulting in a neuro-symbolic approach to dispute resolution. Our method involves identifying parties arguments in peer reviews and representing them as abstract argumentation frameworks, which facilitate dispute resolution through logical inference. We annotate these frameworks within a corpus of scientific peer reviews, achieving a high Krippendorff s alpha of 0.81. Having the annotated corpus, we implement an argument mining pipeline that integrates BERT sentence embeddings with an LSTM model, classifying sentences into three categories: Authors arguments, reviewers arguments, and nonarguments. We achieved an accuracy of 0.634 and an F1 score of 0.631, which are comparable to models trained on other datasets. However, our approach stands out by enabling the processing of the extracted argumentation with logical inference.

KW - Abstract Argumentation Frameworks

KW - Argumentation Mining

KW - Dispute Resolution

KW - Neuro-Symbolic AI

KW - Scientific Peer Review

KW - Text Annotation

UR - https://www.mendeley.com/catalogue/74b95791-b48a-3f4f-be13-bd5c81cbbb86/

U2 - 10.1145/3677389.3702506

DO - 10.1145/3677389.3702506

M3 - Conference contribution

SN - 9798400710933

T3 - Proceedings of the ACM/IEEE Joint Conference on Digital Libraries

BT - Proceedings of the 24th ACM/IEEE Joint Conference on Digital Libraries (JCDL '24)

PB - Association for Computing Machinery

Y2 - 16 December 2024 through 20 December 2024

ER -