Application of automated tools in researching internet discourses: Experience of using the recurrent neural networks for studying discussions on pension reform

Petr Begen, Yuri Misnikov, Olga Filatova

Результат исследований: Публикации в книгах, отчётах, сборниках, трудах конференцийстатья в сборнике материалов конференциинаучнаярецензирование

Выдержка

The paper presents the results of an experiment that applied the Recurrent Neural Network (RNN) and long short-term memory (LSTM) networks to assess how accurately they can determine the attitude of 998 participants towards the pension reform policy in Russia who posted 10,592 comments on 16 online forums in 11 cities. The training set was assembled and coded according to a proposed conceptual model of a moral discourse based on Jurgen Habermas’s discourse ethics theory. The main conclusion of this experiment is that the discourse-based approach — based on the identification of basic validity claims — can be instrumental in building training datasets for deep machine learning on a socially salient topic. The experiment also shows benefits and limitations of using artificial neural networks for a deeper understanding of the results of public discussions in an online environment. The main benefit was that the built neural networks have proven to be sufficiently accurate in predicting positions of discourse participants towards the pension reform policy, with almost 90% in the case of binary classification (two “For” and “Against” positions). However, the accuracy level drops with the inclusion of a third “Neutral” category (to 78%), which was a major limitation of the research; that is, the variation in the prediction accuracy is due to the uneven distribution of data among categories and an increase of new data. Yet this indicator is still acceptable when working with Internet discourse data.

Язык оригиналаанглийский
Название основной публикации21st Conference on Scientific Services and Internet, SSI 2019
Страницы336-344
Число страниц9
Том2543
СостояниеОпубликовано - 1 янв 2020
Событие21st Conference on Scientific Services and Internet, SSI 2019 - Novorossiysk-Abrau, Российская Федерация
Продолжительность: 23 сен 201928 сен 2019

Серия публикаций

НазваниеCEUR Workshop Proceedings
ИздательRWTH Aahen University
ISSN (печатное издание)1613-0073

Конференция

Конференция21st Conference on Scientific Services and Internet, SSI 2019
СтранаРоссийская Федерация
ГородNovorossiysk-Abrau
Период23/09/1928/09/19

Отпечаток

Recurrent neural networks
Internet
Neural networks
Experiments
Learning systems

Предметные области Scopus

  • Компьютерные науки (все)

Цитировать

Begen, P., Misnikov, Y., & Filatova, O. (2020). Application of automated tools in researching internet discourses: Experience of using the recurrent neural networks for studying discussions on pension reform. В 21st Conference on Scientific Services and Internet, SSI 2019 (Том 2543, стр. 336-344). (CEUR Workshop Proceedings).
Begen, Petr ; Misnikov, Yuri ; Filatova, Olga. / Application of automated tools in researching internet discourses : Experience of using the recurrent neural networks for studying discussions on pension reform. 21st Conference on Scientific Services and Internet, SSI 2019. Том 2543 2020. стр. 336-344 (CEUR Workshop Proceedings).
@inproceedings{ca45a30ce2564920b7eea097f87397e7,
title = "Application of automated tools in researching internet discourses: Experience of using the recurrent neural networks for studying discussions on pension reform",
abstract = "The paper presents the results of an experiment that applied the Recurrent Neural Network (RNN) and long short-term memory (LSTM) networks to assess how accurately they can determine the attitude of 998 participants towards the pension reform policy in Russia who posted 10,592 comments on 16 online forums in 11 cities. The training set was assembled and coded according to a proposed conceptual model of a moral discourse based on Jurgen Habermas’s discourse ethics theory. The main conclusion of this experiment is that the discourse-based approach — based on the identification of basic validity claims — can be instrumental in building training datasets for deep machine learning on a socially salient topic. The experiment also shows benefits and limitations of using artificial neural networks for a deeper understanding of the results of public discussions in an online environment. The main benefit was that the built neural networks have proven to be sufficiently accurate in predicting positions of discourse participants towards the pension reform policy, with almost 90{\%} in the case of binary classification (two “For” and “Against” positions). However, the accuracy level drops with the inclusion of a third “Neutral” category (to 78{\%}), which was a major limitation of the research; that is, the variation in the prediction accuracy is due to the uneven distribution of data among categories and an increase of new data. Yet this indicator is still acceptable when working with Internet discourse data.",
keywords = "Deliberation, E-participation, Internet discourse, Machine learning, Recurrent neural networks, Validity claims",
author = "Petr Begen and Yuri Misnikov and Olga Filatova",
year = "2020",
month = "1",
day = "1",
language = "English",
volume = "2543",
series = "CEUR Workshop Proceedings",
publisher = "RWTH Aahen University",
pages = "336--344",
booktitle = "21st Conference on Scientific Services and Internet, SSI 2019",

}

Begen, P, Misnikov, Y & Filatova, O 2020, Application of automated tools in researching internet discourses: Experience of using the recurrent neural networks for studying discussions on pension reform. в 21st Conference on Scientific Services and Internet, SSI 2019. том. 2543, CEUR Workshop Proceedings, стр. 336-344, Novorossiysk-Abrau, Российская Федерация, 23/09/19.

Application of automated tools in researching internet discourses : Experience of using the recurrent neural networks for studying discussions on pension reform. / Begen, Petr; Misnikov, Yuri; Filatova, Olga.

21st Conference on Scientific Services and Internet, SSI 2019. Том 2543 2020. стр. 336-344 (CEUR Workshop Proceedings).

Результат исследований: Публикации в книгах, отчётах, сборниках, трудах конференцийстатья в сборнике материалов конференциинаучнаярецензирование

TY - GEN

T1 - Application of automated tools in researching internet discourses

T2 - Experience of using the recurrent neural networks for studying discussions on pension reform

AU - Begen, Petr

AU - Misnikov, Yuri

AU - Filatova, Olga

PY - 2020/1/1

Y1 - 2020/1/1

N2 - The paper presents the results of an experiment that applied the Recurrent Neural Network (RNN) and long short-term memory (LSTM) networks to assess how accurately they can determine the attitude of 998 participants towards the pension reform policy in Russia who posted 10,592 comments on 16 online forums in 11 cities. The training set was assembled and coded according to a proposed conceptual model of a moral discourse based on Jurgen Habermas’s discourse ethics theory. The main conclusion of this experiment is that the discourse-based approach — based on the identification of basic validity claims — can be instrumental in building training datasets for deep machine learning on a socially salient topic. The experiment also shows benefits and limitations of using artificial neural networks for a deeper understanding of the results of public discussions in an online environment. The main benefit was that the built neural networks have proven to be sufficiently accurate in predicting positions of discourse participants towards the pension reform policy, with almost 90% in the case of binary classification (two “For” and “Against” positions). However, the accuracy level drops with the inclusion of a third “Neutral” category (to 78%), which was a major limitation of the research; that is, the variation in the prediction accuracy is due to the uneven distribution of data among categories and an increase of new data. Yet this indicator is still acceptable when working with Internet discourse data.

AB - The paper presents the results of an experiment that applied the Recurrent Neural Network (RNN) and long short-term memory (LSTM) networks to assess how accurately they can determine the attitude of 998 participants towards the pension reform policy in Russia who posted 10,592 comments on 16 online forums in 11 cities. The training set was assembled and coded according to a proposed conceptual model of a moral discourse based on Jurgen Habermas’s discourse ethics theory. The main conclusion of this experiment is that the discourse-based approach — based on the identification of basic validity claims — can be instrumental in building training datasets for deep machine learning on a socially salient topic. The experiment also shows benefits and limitations of using artificial neural networks for a deeper understanding of the results of public discussions in an online environment. The main benefit was that the built neural networks have proven to be sufficiently accurate in predicting positions of discourse participants towards the pension reform policy, with almost 90% in the case of binary classification (two “For” and “Against” positions). However, the accuracy level drops with the inclusion of a third “Neutral” category (to 78%), which was a major limitation of the research; that is, the variation in the prediction accuracy is due to the uneven distribution of data among categories and an increase of new data. Yet this indicator is still acceptable when working with Internet discourse data.

KW - Deliberation

KW - E-participation

KW - Internet discourse

KW - Machine learning

KW - Recurrent neural networks

KW - Validity claims

UR - http://www.scopus.com/inward/record.url?scp=85078464907&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85078464907

VL - 2543

T3 - CEUR Workshop Proceedings

SP - 336

EP - 344

BT - 21st Conference on Scientific Services and Internet, SSI 2019

ER -

Begen P, Misnikov Y, Filatova O. Application of automated tools in researching internet discourses: Experience of using the recurrent neural networks for studying discussions on pension reform. В 21st Conference on Scientific Services and Internet, SSI 2019. Том 2543. 2020. стр. 336-344. (CEUR Workshop Proceedings).