Thе paper deals with automatic areas of interest detection in video streams derived from mobile eye trackers. Defining such areas on a visual stimulus viewed by an informant is an important step in setting up any eye-tracking-based experiment. If the informant’s field of view is stationary, areas of interest can be selected manually, but when we use mobile eye trackers, the field of view is usually constantly changing, so automation is badly needed. We propose using computer vision algorithms to automatically locate the given 2D stimulus template in a video stream and construct the homography transform that can map the undistorted stimulus template to the video frame coordinate system. In parallel to this, the segmentation of a stimulus template into the areas of interest is performed, and the areas of interest are mapped to the video frame. The considered stimuli are texts typed in specific fonts and the interest areas are individual words in these texts. Optical character recognition leveraged by the Tesseract engine is used for segmentation. The text location relies on a combination of Scale-Invariant Feature Transform and Fast Library for Approximate Nearest Neighbors. The homography is constructed using Random Sample Consensus. All the algorithms are implemented based on the OpenCV library as microservices within the SciVi ontology-driven platform that provides high-level tools to compose pipelines using a data-flow-based visual programming paradigm. The proposed pipeline was tested on real eye tracking data and proved to be efficient and robust.

Eye Gaze Tracking, Area of Interest, Video Segmentation, Image Template Detection, OpenCV, Scientific Visualization

Ryabinin Konstantin Valentinovich,  ,  orcid.org/0000-0002-8353-7641,  Saint Petersburg State University; Perm State University

Alexeeva Svetlana Vladimirovna,  ,  orcid.org/0000-0002-8540-6178,  Saint Petersburg State University

Petrova Tatiana Evgenievna,  ,  orcid.org/0000-0002-6711-5385,  Saint Petersburg State University