Thе paper deals with automatic areas of interest detection in video streams derived from mobile eye
trackers. Defining such areas on a visual stimulus viewed by an informant is an important step in setting
up any eye-tracking-based experiment. If the informant’s field of view is stationary, areas of interest
can be selected manually, but when we use mobile eye trackers, the field of view is usually constantly
changing, so automation is badly needed. We propose using computer vision algorithms to automatically
locate the given 2D stimulus template in a video stream and construct the homography transform that
can map the undistorted stimulus template to the video frame coordinate system. In parallel to this, the
segmentation of a stimulus template into the areas of interest is performed, and the areas of interest are
mapped to the video frame. The considered stimuli are texts typed in specific fonts and the interest areas
are individual words in these texts. Optical character recognition leveraged by the Tesseract engine is
used for segmentation. The text location relies on a combination of Scale-Invariant Feature Transform
and Fast Library for Approximate Nearest Neighbors. The homography is constructed using Random
Sample Consensus. All the algorithms are implemented based on the OpenCV library as microservices
within the SciVi ontology-driven platform that provides high-level tools to compose pipelines using a
data-flow-based visual programming paradigm. The proposed pipeline was tested on real eye tracking
data and proved to be efficient and robust.
Original languageEnglish
Title of host publicationGraphiCon 2022: 32nd International Conference on Computer Graphics and Vision
Subtitle of host publicationProceedings
Pages228-239
DOIs
StatePublished - 2022
Event32-я Международная конференция по компьютерной графике и машинному зрению -
Duration: 19 Sep 202222 Sep 2022

Conference

Conference32-я Международная конференция по компьютерной графике и машинному зрению
Period19/09/2222/09/22

    Research areas

  • Eye Gaze Tracking, Area of Interest, Video Segmentation, Image Template Detection, openCV, scientific visualization

ID: 100333386