Authors:
Seán Martin
;
Seán Bruton
;
David Ganter
and
Michael Manzke
Affiliation:
School of Computer Science and Statistics, Trinity College Dublin, College Green, Dublin 2 and Ireland
Keyword(s):
Light Fields, View Synthesis, Convolutional Neural Networks, Volume Rendering, Depth Estimation, Image Warping, Angular Resolution Enhancement.
Related
Ontology
Subjects/Areas/Topics:
Computer Vision, Visualization and Computer Graphics
;
Real-Time Rendering
;
Rendering
;
Rendering Algorithms
;
Volume Rendering
Abstract:
Existing approaches to light field view synthesis assume a unique depth in the scene. This assumption does not hold for an alpha-blended volume rendering. We propose to use a depth heuristic to overcome this limitation and synthesise views from one volume rendered sample view, which we demonstrate for an 8 × 8 grid. Our approach is comprised of a number of stages. Firstly, during direct volume rendering of the sample view, a depth heuristic is applied to estimate a per-pixel depth map. Secondly, this depth map is converted to a disparity map using the known virtual camera parameters. Then, image warping is performed using this disparity map to shift information from the reference view to novel views. Finally, these warped images are passed into a Convolutional Neural Network to improve visual consistency of the synthesised views. We evaluate multiple existing Convolutional Neural Network architectures for this purpose. Our application of depth heuristics is a novel contribution to li
ght field volume rendering, leading to high quality view synthesis which is further improved by a Convolutional Neural Network.
(More)