TY - GEN

T1 - Automatic Areas of Interest Detector for Mobile Eye Trackers

AU - Ryabinin, Konstantin

AU - Alexeeva, Svetlana

AU - Petrova, Tatiana

PY - 2022

Y1 - 2022

N2 - Thе paper deals with automatic areas of interest detection in video streams derived from mobile eyetrackers. Defining such areas on a visual stimulus viewed by an informant is an important step in settingup any eye-tracking-based experiment. If the informant’s field of view is stationary, areas of interestcan be selected manually, but when we use mobile eye trackers, the field of view is usually constantlychanging, so automation is badly needed. We propose using computer vision algorithms to automaticallylocate the given 2D stimulus template in a video stream and construct the homography transform thatcan map the undistorted stimulus template to the video frame coordinate system. In parallel to this, thesegmentation of a stimulus template into the areas of interest is performed, and the areas of interest aremapped to the video frame. The considered stimuli are texts typed in specific fonts and the interest areasare individual words in these texts. Optical character recognition leveraged by the Tesseract engine isused for segmentation. The text location relies on a combination of Scale-Invariant Feature Transformand Fast Library for Approximate Nearest Neighbors. The homography is constructed using RandomSample Consensus. All the algorithms are implemented based on the OpenCV library as microserviceswithin the SciVi ontology-driven platform that provides high-level tools to compose pipelines using adata-flow-based visual programming paradigm. The proposed pipeline was tested on real eye trackingdata and proved to be efficient and robust.

AB - Thе paper deals with automatic areas of interest detection in video streams derived from mobile eyetrackers. Defining such areas on a visual stimulus viewed by an informant is an important step in settingup any eye-tracking-based experiment. If the informant’s field of view is stationary, areas of interestcan be selected manually, but when we use mobile eye trackers, the field of view is usually constantlychanging, so automation is badly needed. We propose using computer vision algorithms to automaticallylocate the given 2D stimulus template in a video stream and construct the homography transform thatcan map the undistorted stimulus template to the video frame coordinate system. In parallel to this, thesegmentation of a stimulus template into the areas of interest is performed, and the areas of interest aremapped to the video frame. The considered stimuli are texts typed in specific fonts and the interest areasare individual words in these texts. Optical character recognition leveraged by the Tesseract engine isused for segmentation. The text location relies on a combination of Scale-Invariant Feature Transformand Fast Library for Approximate Nearest Neighbors. The homography is constructed using RandomSample Consensus. All the algorithms are implemented based on the OpenCV library as microserviceswithin the SciVi ontology-driven platform that provides high-level tools to compose pipelines using adata-flow-based visual programming paradigm. The proposed pipeline was tested on real eye trackingdata and proved to be efficient and robust.

KW - Eye Gaze Tracking

KW - Area of Interest

KW - Video Segmentation

KW - Image Template Detection

KW - openCV

KW - scientific visualization

UR - https://www.graphicon.ru/en/node/226

U2 - 10.20948/graphicon-2022-228-239

DO - 10.20948/graphicon-2022-228-239

M3 - Conference contribution

SP - 228

EP - 239

BT - GraphiCon 2022: 32nd International Conference on Computer Graphics and Vision

T2 - 32-я Международная конференция по компьютерной графике и машинному зрению

Y2 - 19 September 2022 through 22 September 2022

ER -