Abstract

As visual representation of information is getting more and more popular nowadays, studies of so-called “polycode” texts integrating both verbal and non-verbal types of information are of high priority. This study investigates how readers integrate text- gure information when reading multimodal texts. Using the eye- tracking method, we compared the processing of different multimodal and verbal texts. Three experiments were carried out. In Exp.1, native speakers of Russian read infographics (graphic visual representation of information). In Exp.2, we studied the processing of different types of visual notes containing a handwritten text and drawings (‘path’ (trajectory), linear, and radial sketchnotes) by native speakers of Russian. In Exp.3, Chinese students learning Russian as a foreign language examined infographics and verbal texts. We measured the total dwell time, the total xation count, average xation duration for each verbal and non-verbal zone of the texts. Text comprehension was controlled by off-line methods (subjective scaling, answers to the after the text questions, key words). We revealed speci c features of verbal text and polycode text perception and proposed a number of recommendations for creating effective polycode texts. The overall results of the study show that readers process the information better and faster while reading multimodal text of any format than a verbal text. The reading patterns in polycode text processing are text directed. The in uence of the text type becomes crucial only to experienced readers: the better the reading skills are, the greater the in uence of the factor “text type” is. Supported by RFBR grant No18-00-00640.
Original languageEnglish
Title of host publicationThe 20th European Conference on Eye Movements, ECEM 2019
Subtitle of host publicationAbstract book
Pages292
Publication statusPublished - Aug 2019
EventThe 20th European Conference on Eye Movements, ECEM 2019 - Аликанте
Duration: 18 Aug 201922 Aug 2019
Conference number: 20

Conference

ConferenceThe 20th European Conference on Eye Movements, ECEM 2019
Abbreviated titleECEM 2019
CountrySpain
CityАликанте
Period18/08/1922/08/19

Cite this

Petrova , T., & Riekhakaynen, E. (2019). Eye-tracking study of processing Russian polycode texts. In The 20th European Conference on Eye Movements, ECEM 2019 : Abstract book (pp. 292)
Petrova , Tatiana ; Riekhakaynen, Elena. / Eye-tracking study of processing Russian polycode texts. The 20th European Conference on Eye Movements, ECEM 2019 : Abstract book. 2019. pp. 292
@inproceedings{a50835845ca748ba89668f8f38203672,
title = "Eye-tracking study of processing Russian polycode texts",
abstract = "As visual representation of information is getting more and more popular nowadays, studies of so-called “polycode” texts integrating both verbal and non-verbal types of information are of high priority. This study investigates how readers integrate text- gure information when reading multimodal texts. Using the eye- tracking method, we compared the processing of different multimodal and verbal texts. Three experiments were carried out. In Exp.1, native speakers of Russian read infographics (graphic visual representation of information). In Exp.2, we studied the processing of different types of visual notes containing a handwritten text and drawings (‘path’ (trajectory), linear, and radial sketchnotes) by native speakers of Russian. In Exp.3, Chinese students learning Russian as a foreign language examined infographics and verbal texts. We measured the total dwell time, the total xation count, average xation duration for each verbal and non-verbal zone of the texts. Text comprehension was controlled by off-line methods (subjective scaling, answers to the after the text questions, key words). We revealed speci c features of verbal text and polycode text perception and proposed a number of recommendations for creating effective polycode texts. The overall results of the study show that readers process the information better and faster while reading multimodal text of any format than a verbal text. The reading patterns in polycode text processing are text directed. The in uence of the text type becomes crucial only to experienced readers: the better the reading skills are, the greater the in uence of the factor “text type” is. Supported by RFBR grant No18-00-00640.",
keywords = "регистрация движений глаз, инфографика, поликодовый текст, русский язык, регистрация движений глаз, инфографика, поликодовый текст, русский язык",
author = "Tatiana Petrova and Elena Riekhakaynen",
year = "2019",
month = "8",
language = "English",
pages = "292",
booktitle = "The 20th European Conference on Eye Movements, ECEM 2019",

}

Petrova , T & Riekhakaynen, E 2019, Eye-tracking study of processing Russian polycode texts. in The 20th European Conference on Eye Movements, ECEM 2019 : Abstract book. pp. 292, Аликанте, 18/08/19.

Eye-tracking study of processing Russian polycode texts. / Petrova , Tatiana; Riekhakaynen, Elena.

The 20th European Conference on Eye Movements, ECEM 2019 : Abstract book. 2019. p. 292.

Research output

TY - GEN

T1 - Eye-tracking study of processing Russian polycode texts

AU - Petrova , Tatiana

AU - Riekhakaynen, Elena

PY - 2019/8

Y1 - 2019/8

N2 - As visual representation of information is getting more and more popular nowadays, studies of so-called “polycode” texts integrating both verbal and non-verbal types of information are of high priority. This study investigates how readers integrate text- gure information when reading multimodal texts. Using the eye- tracking method, we compared the processing of different multimodal and verbal texts. Three experiments were carried out. In Exp.1, native speakers of Russian read infographics (graphic visual representation of information). In Exp.2, we studied the processing of different types of visual notes containing a handwritten text and drawings (‘path’ (trajectory), linear, and radial sketchnotes) by native speakers of Russian. In Exp.3, Chinese students learning Russian as a foreign language examined infographics and verbal texts. We measured the total dwell time, the total xation count, average xation duration for each verbal and non-verbal zone of the texts. Text comprehension was controlled by off-line methods (subjective scaling, answers to the after the text questions, key words). We revealed speci c features of verbal text and polycode text perception and proposed a number of recommendations for creating effective polycode texts. The overall results of the study show that readers process the information better and faster while reading multimodal text of any format than a verbal text. The reading patterns in polycode text processing are text directed. The in uence of the text type becomes crucial only to experienced readers: the better the reading skills are, the greater the in uence of the factor “text type” is. Supported by RFBR grant No18-00-00640.

AB - As visual representation of information is getting more and more popular nowadays, studies of so-called “polycode” texts integrating both verbal and non-verbal types of information are of high priority. This study investigates how readers integrate text- gure information when reading multimodal texts. Using the eye- tracking method, we compared the processing of different multimodal and verbal texts. Three experiments were carried out. In Exp.1, native speakers of Russian read infographics (graphic visual representation of information). In Exp.2, we studied the processing of different types of visual notes containing a handwritten text and drawings (‘path’ (trajectory), linear, and radial sketchnotes) by native speakers of Russian. In Exp.3, Chinese students learning Russian as a foreign language examined infographics and verbal texts. We measured the total dwell time, the total xation count, average xation duration for each verbal and non-verbal zone of the texts. Text comprehension was controlled by off-line methods (subjective scaling, answers to the after the text questions, key words). We revealed speci c features of verbal text and polycode text perception and proposed a number of recommendations for creating effective polycode texts. The overall results of the study show that readers process the information better and faster while reading multimodal text of any format than a verbal text. The reading patterns in polycode text processing are text directed. The in uence of the text type becomes crucial only to experienced readers: the better the reading skills are, the greater the in uence of the factor “text type” is. Supported by RFBR grant No18-00-00640.

KW - регистрация движений глаз

KW - инфографика

KW - поликодовый текст

KW - русский язык

KW - регистрация движений глаз

KW - инфографика

KW - поликодовый текст

KW - русский язык

M3 - Conference contribution

SP - 292

BT - The 20th European Conference on Eye Movements, ECEM 2019

ER -

Petrova T, Riekhakaynen E. Eye-tracking study of processing Russian polycode texts. In The 20th European Conference on Eye Movements, ECEM 2019 : Abstract book. 2019. p. 292