Optimization of Free Viewpoint Interpolation by Applying Adaptive Depth Plane Distributions in Plane Sweeping - A Histogram-based Approach to a Non-uniform Plane Distribution

Patrik Goorts, Steven Maesen, Maarten Dumont, Sammy Rogmans, Philippe Bekaert

Abstract

In this paper, we present a system to increase performance of plane sweeping for free viewpoint interpolation. Typical plane sweeping approaches incorporate a uniform depth plane distribution to investigate different depth hypotheses to generate a depth map, used in novel camera viewpoint generation. When the scene consists of a sparse number of objects, some depth hypotheses do not contain objects and can cause noise and wasted computational power. Therefore, we propose a method to adapt the plane distribution to increase the quality of the depth map around objects and to reduce computational power waste by reducing the number of planes in empty spaces in the scene. First, we generate the cumulative histogram of the previous frame in a temporal sequence of images. Next, we determine a new normalized depth for every depth plane by analyzing the cumulative histogram. Steep sections of the cumulative histogram will result in a dense local distribution of planes; a flat section will result in a sparse distribution. The results, performed on controlled and on real images, demonstrate the effectiveness of the method over a uniform distribution and allows a lower number of depth planes, and thus a more performant processing, for the same quality.

References

  1. Dumont, M., Rogmans, S., Maesen, S., and Bekaert, P. (2009). Optimized two-party video chat with restored eye contact using graphics hardware. e-Business and Telecommunications, pages 358-372.
  2. Gallup, D., Frahm, J.-M., Mordohai, P., Yang, Q., and Pollefeys, M. (2007). Real-time plane-sweeping stereo with multiple sweeping directions. In Computer Vision and Pattern Recognition, 2007. CVPR'07. IEEE Conference on, pages 1-8. IEEE.
  3. Goorts, P., Ancuti, C., Dumont, M., and Bekaert, P. (2013). Real-time video-based view interpolation of soccer events using depth-selective plane sweeping. In Proceedings of the Eight International Conference on Computer Vision Theory and Applications (VISAPP 2013). INSTICC.
  4. Goorts, P., Dumont, M., Rogmans, S., and Bekaert, P. (2012a). An end-to-end system for free viewpoint video for smooth camera transitions. In Proceedings of the Second International Conference on 3D Imaging (IC3D 2012). 3D Stereo Media.
  5. Goorts, P., Rogmans, S., and Bekaert, P. (2012b). Raw camera image demosaicing using finite impulse response filtering on commodity gpu hardware using cuda. In Proceedings of the Tenth International Conference on Signal Processing and Multimedia Applications (SIGMAP 2012). INSTICC.
  6. Green, S. (2005). Image processing tricks in opengl. Presentation at GDC, 2005:2.
  7. Kutulakos, K. and Seitz, S. (2000). A theory of shape by space carving. Intl. Journal of Computer Vision, 38(3):199-218.
  8. Matusik, W., Buehler, C., Raskar, R., Gortler, S., and McMillan, L. (2000). Image-based visual hulls. In Proc. of the 27th annual conference on Computer graphics and interactive techniques, pages 369-374. ACM Press/Addison-Wesley Publishing Co.
  9. Miller, G., Hilton, A., and Starck, J. (2005). Interactive free-viewpoint video. In IEE Eur. Conf. Vis. Media Prod, pages 50-59. Citeseer.
  10. Rogmans, S., Dumont, M., Cuypers, T., Lafruit, G., and Bekaert, P. (2009). Complexity reduction of real-time depth scanning on graphics hardware. In Proc. of Intl. Conference on Computer Vision Theory and Applications, Lisbon, Portugal (February 2009).
  11. Seitz, S., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, volume 1, pages 519-528. Ieee.
  12. Seitz, S. and Dyer, C. (1999). Photorealistic scene reconstruction by voxel coloring. Intl. Journal of Computer Vision, 35(2):151-173.
  13. Svoboda, T., Martinec, D., and Pajdla, T. (2005). A convenient multicamera self-calibration for virtual environments. Presence: Teleoperators & Virtual Environments, 14(4):407-422.
  14. Yang, R., Pollefeys, M., Yang, H., and Welch, G. (2004). A unified approach to real-time, multi-resolution, multibaseline 2d view synthesis and 3d depth estimation using commodity graphics hardware. Intl. Journal of Image and Graphics, 4(4):627-651.
  15. Yang, R., Welch, G., and Bishop, G. (2003). Real-time consensus-based scene reconstruction using commodity graphics hardware. In Computer Graphics Forum, volume 22, pages 207-216. Wiley Online Library.
  16. Zitnick, C., Kang, S., Uyttendaele, M., Winder, S., and Szeliski, R. (2004). High-quality video view interpolation using a layered representation. In ACM Transactions on Graphics, volume 23, pages 600- 608. ACM.
Download


Paper Citation


in Harvard Style

Goorts P., Maesen S., Dumont M., Rogmans S. and Bekaert P. (2013). Optimization of Free Viewpoint Interpolation by Applying Adaptive Depth Plane Distributions in Plane Sweeping - A Histogram-based Approach to a Non-uniform Plane Distribution . In Proceedings of the 10th International Conference on Signal Processing and Multimedia Applications and 10th International Conference on Wireless Information Networks and Systems - Volume 1: SIGMAP, (ICETE 2013) ISBN 978-989-8565-74-7, pages 7-15. DOI: 10.5220/0004524700070015


in Bibtex Style

@conference{sigmap13,
author={Patrik Goorts and Steven Maesen and Maarten Dumont and Sammy Rogmans and Philippe Bekaert},
title={Optimization of Free Viewpoint Interpolation by Applying Adaptive Depth Plane Distributions in Plane Sweeping - A Histogram-based Approach to a Non-uniform Plane Distribution},
booktitle={Proceedings of the 10th International Conference on Signal Processing and Multimedia Applications and 10th International Conference on Wireless Information Networks and Systems - Volume 1: SIGMAP, (ICETE 2013)},
year={2013},
pages={7-15},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004524700070015},
isbn={978-989-8565-74-7},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 10th International Conference on Signal Processing and Multimedia Applications and 10th International Conference on Wireless Information Networks and Systems - Volume 1: SIGMAP, (ICETE 2013)
TI - Optimization of Free Viewpoint Interpolation by Applying Adaptive Depth Plane Distributions in Plane Sweeping - A Histogram-based Approach to a Non-uniform Plane Distribution
SN - 978-989-8565-74-7
AU - Goorts P.
AU - Maesen S.
AU - Dumont M.
AU - Rogmans S.
AU - Bekaert P.
PY - 2013
SP - 7
EP - 15
DO - 10.5220/0004524700070015