KIAM Main page Web Library  •  Publication Searh  Русский 

KIAM Preprint № 67, Moscow, 2021
Authors: Sorokin M.I., Zhdanov D.D., Valiev I.V.
Reconstruction Lighting Positions of a Scene for Mixed Reality Systems Using Convolutional Neural Network and Shadow Ray Tracing
The paper examines the causes of visual discomfort in mixed reality systems and algorithmic solutions that eliminate one of the main causes of discomfort, namely, the mismatch between the lighting conditions of objects in the real and virtual worlds. To eliminate this cause of discomfort, the algorithm is proposed, which consists in constructing groups of shadow rays from points on the boundaries of shadows to points on the boundaries of objects. Part of the rays corresponding to the real lighting conditions form caustics in area of the real light source, which makes it possible to determine the source of illumination of virtual objects for their correct embedding into the mixed reality system. Convolutional neural networks and computer vision algorithms were used to classify shadows in the image. Examples of reconstructing the coordinates of a light source from RGBD data are presented.
mixed reality systems, computer vision, convolutional neural networks
Publication language: russian,  pages: 18
Research direction:
Mathematical modelling in actual problems of science and technics
Russian source text:
Export link to publication in format:   RIS    BibTeX
View statistics (updated once a day)
over the last 30 days — 9 (-13), total hit from 30.09.2021 — 42
About authors:
  • Sorokin Maxim Igorevich,,  ITMO University
  • Zhdanov Dmitry Dmitrievich,,  ITMO University
  • Valiev Ildar Vagizovich, RAS