KIAM Main page Web Library  •  Publication Searh   
Publication

Conference material: "Proceedings of the International Conference on Computer Graphics and Vision Graphicon"
Authors: Sorokin M.I., Zhdanov D.D., Potemin I.S., Barladyan B. K., Bogdanov N.N., Zhdanov A.D.
Segmentation of illuminated areas of scene using fully-convolutional neural networks and computer vision algorithms for augmented reality systems
Abstract:
The relevance of this topic is due to the rapid development of virtual and augmented reality systems. The problem lies in the formation of natural conditions for lighting objects of the virtual world in real space. To solve a light sources determination problem and recovering its optical parameters were proposed the fully-convolutional neural network, which allows catching the 'behavior of light' features. The output of FCNN is a segmented image with light levels and its strength. Naturally, the fully-convolutional neural network is well suited for image segmentation, so as an encoder was taken the architecture of VGG-16 with layers that pools and convolves an input image to 1x1 pixel and wisely classifies it to one of a class which characterizes its strength. Neural network training was conducted on 221 train images and 39 validation images with learning rate 1E-2 and 200 epochs, after training the loss was 0,2. As a test was used an intersection over union method, that compares the ground truth area of an input image and output image, comparing its pixels and giving the result of accuracy. The mean IoU is 0.7, almost rightly classifying the first class with a value of 90 percents of accordance and the last class with a probability of 30 percents.
Keywords:
classification, illumination, convolutional neural networks, segmentation
Publication language: russian,  pages: 5 (p. 42-46)
Russian source text:
List of publications citation:
Export link to publication in format:   RIS    BibTeX
About authors:
  • Sorokin Maxim Igorevich,  ,  Saint Petersburg State University of Information Technologies, Mechanics and Optics
  • Zhdanov Dmitry Dmitrievich,  ,  Saint Petersburg State University of Information Technologies, Mechanics and Optics
  • Potemin Igor Stanislavovich,  ,  Saint Petersburg State University of Information Technologies, Mechanics and Optics
  • Barladyan Boris Khaimovich,  orcid.org/0000-0002-2391-2067KIAM RAS
  • Bogdanov Nikolay Nikolaevich,  ,  Saint Petersburg State University of Information Technologies, Mechanics and Optics
  • Zhdanov Andrey Dmitrievich,  ,  Saint Petersburg State University of Information Technologies, Mechanics and Optics