TY - GEN

T1 - Application of automated tools in researching internet discourses

T2 - 21st Conference on Scientific Services and Internet, SSI 2019

AU - Begen, Petr

AU - Misnikov, Yuri

AU - Filatova, Olga

PY - 2020/1/1

Y1 - 2020/1/1

N2 - The paper presents the results of an experiment that applied the Recurrent Neural Network (RNN) and long short-term memory (LSTM) networks to assess how accurately they can determine the attitude of 998 participants towards the pension reform policy in Russia who posted 10,592 comments on 16 online forums in 11 cities. The training set was assembled and coded according to a proposed conceptual model of a moral discourse based on Jurgen Habermas’s discourse ethics theory. The main conclusion of this experiment is that the discourse-based approach — based on the identification of basic validity claims — can be instrumental in building training datasets for deep machine learning on a socially salient topic. The experiment also shows benefits and limitations of using artificial neural networks for a deeper understanding of the results of public discussions in an online environment. The main benefit was that the built neural networks have proven to be sufficiently accurate in predicting positions of discourse participants towards the pension reform policy, with almost 90% in the case of binary classification (two “For” and “Against” positions). However, the accuracy level drops with the inclusion of a third “Neutral” category (to 78%), which was a major limitation of the research; that is, the variation in the prediction accuracy is due to the uneven distribution of data among categories and an increase of new data. Yet this indicator is still acceptable when working with Internet discourse data.

AB - The paper presents the results of an experiment that applied the Recurrent Neural Network (RNN) and long short-term memory (LSTM) networks to assess how accurately they can determine the attitude of 998 participants towards the pension reform policy in Russia who posted 10,592 comments on 16 online forums in 11 cities. The training set was assembled and coded according to a proposed conceptual model of a moral discourse based on Jurgen Habermas’s discourse ethics theory. The main conclusion of this experiment is that the discourse-based approach — based on the identification of basic validity claims — can be instrumental in building training datasets for deep machine learning on a socially salient topic. The experiment also shows benefits and limitations of using artificial neural networks for a deeper understanding of the results of public discussions in an online environment. The main benefit was that the built neural networks have proven to be sufficiently accurate in predicting positions of discourse participants towards the pension reform policy, with almost 90% in the case of binary classification (two “For” and “Against” positions). However, the accuracy level drops with the inclusion of a third “Neutral” category (to 78%), which was a major limitation of the research; that is, the variation in the prediction accuracy is due to the uneven distribution of data among categories and an increase of new data. Yet this indicator is still acceptable when working with Internet discourse data.

KW - Deliberation

KW - E-participation

KW - Internet discourse

KW - Machine learning

KW - Recurrent neural networks

KW - Validity claims

UR - http://www.scopus.com/inward/record.url?scp=85078464907&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85078464907

VL - 2543

T3 - CEUR Workshop Proceedings

SP - 336

EP - 344

BT - 21st Conference on Scientific Services and Internet, SSI 2019

Y2 - 23 September 2019 through 28 September 2019

ER -