Conference material: "Proceedings of the International Conference on Computer Graphics and Vision “Graphicon” (23-26 September 2019, Bryansk)"
Authors:Sorokin M.I., Zhdanov D.D., Potemin I.S., Barladyan B. K., Bogdanov N.N., Zhdanov A.D.
Segmentation of illuminated areas of scene using fully-convolutional neural networks and computer vision algorithms for augmented reality systems
Abstract:
The relevance of this topic is due to the rapid development of virtual and augmented reality systems. The problem lies in the formation of natural conditions for lighting objects of the virtual world in real space. To solve a light sources determination problem and recovering its optical parameters were proposed the fully-convolutional neural network, which allows catching the 'behavior of light' features. The output of FCNN is a segmented image with light levels and its strength. Naturally, the fully-convolutional neural network is well suited for image segmentation, so as an encoder was taken the architecture of VGG-16 with layers that pools and convolves an input image to 1x1 pixel and wisely classifies it to one of a class which characterizes its strength. Neural network training was conducted on 221 train images and 39 validation images with learning rate 1E-2 and 200 epochs, after training the loss was 0,2. As a test was used an ‘intersection over union’ method, that compares the ground truth area of an input image and output image, comparing its pixels and giving the result of accuracy. The mean IoU is 0.7, almost rightly classifying the first class with a value of 90 percents of accordance and the last class with a probability of 30 percents.
