TY - JOUR

T1 - Automatic collocation extraction

T2 - 2017 International Conference on Internet and Modern Society, IMS 2017

AU - Zakharov, V. P.

N1 - Conference code: XX

PY - 2017

Y1 - 2017

N2 - The paper deals with collocation extraction from corpus data. A collocation is meant as a special type of a set phrase. Many modern authors and most of corpus linguists understand collocations as statistically determined set phrases. The above approach is the basic point of this paper which is aimed at evaluation of various statistical methods of automatic collocation extraction. There are several ways to calculate the degree of coherence of parts of a collocation. A whole number of formulae have been created to integrate different factors that determine the association between the collocation components. Usually, such formulae are called association measures. The experiments are described which objective was to study the method of collocation extraction based on the statistical association measures. We extracted collocations for the word (water) and some others by means of the tool Collocations of the NoSketch Engine system using 7 association measures. It is important to stress that the experiments were conducted using representative corpora, with large amount of the resulting collocations being under study. The data on the measure precision allows to establish to some degree that in cases when collocation extraction is not used for some special purposes such measures as MI.l-og-f, log-Dice, and minimum sensitivity should be used. No measure is ideal, which is why various options of their integration are desirable and useful. And we propose a number of parameters that allow to rank collocates in an integrated list, namely, an average rank, a normalised rank and an optimised rank.

AB - The paper deals with collocation extraction from corpus data. A collocation is meant as a special type of a set phrase. Many modern authors and most of corpus linguists understand collocations as statistically determined set phrases. The above approach is the basic point of this paper which is aimed at evaluation of various statistical methods of automatic collocation extraction. There are several ways to calculate the degree of coherence of parts of a collocation. A whole number of formulae have been created to integrate different factors that determine the association between the collocation components. Usually, such formulae are called association measures. The experiments are described which objective was to study the method of collocation extraction based on the statistical association measures. We extracted collocations for the word (water) and some others by means of the tool Collocations of the NoSketch Engine system using 7 association measures. It is important to stress that the experiments were conducted using representative corpora, with large amount of the resulting collocations being under study. The data on the measure precision allows to establish to some degree that in cases when collocation extraction is not used for some special purposes such measures as MI.l-og-f, log-Dice, and minimum sensitivity should be used. No measure is ideal, which is why various options of their integration are desirable and useful. And we propose a number of parameters that allow to rank collocates in an integrated list, namely, an average rank, a normalised rank and an optimised rank.

KW - Association measures

KW - Average rank

KW - Collocation extraction

KW - Evaluation

KW - Normalised rank

KW - Optimised medium rank

UR - http://www.scopus.com/inward/record.url?scp=85021840994&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:85021840994

VL - 1

SP - 387

EP - 398

JO - Компьютерная лингвистика и интеллектуальные технологии

JF - Компьютерная лингвистика и интеллектуальные технологии

SN - 2221-7932

IS - 16

Y2 - 21 June 2017 through 23 June 2017

ER -