Shape and Reflectance from RGB-D Images using Time Sequential Illumination

Matis Hudon, Adrien Gruson, Paul Kerbiriou, Rémi Cozot, Kadi Bouatouch

2016

Abstract

In this paper we propose a method for recovering the shape (geometry) and the diffuse reflectance from an image (or video) using a hybrid setup consisting of a depth sensor (Kinect), a consumer camera and a partially controlled illumination (using a flash). The objective is to show how combining RGB-D acquisition with a sequential illumination is useful for shape and reflectance recovery. A pair of two images are captured: one non flashed (image under ambient illumination) and a flashed one. A pure flash image is computed by subtracting the non flashed image from the flashed image. We propose an novel and near real-time algorithm, based on a local illumination model of our flash and the pure flash image, to enhance geometry (from the noisy depth map) and recover reflectance information.

References

  1. Bruckstein, A. M. (1988). On shape from shading. Computer Vision, Graphics, and Image Processing, 44(2):139-154.
  2. de Decker, B., Kautz, J., Mertens, T., and Bekaert, P. (2009). Capturing multiple illumination conditions using time and color multiplexing. In 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), 20-25 June 2009, Miami, Florida, USA, pages 2536-2543.
  3. Debevec, P. (2012). The light stages and their applications to photoreal digital actors. SIGGRAPH Asia Technical Briefs.
  4. DiCarlo, J. M., Xiao, F., and Wandell, B. A. (2001). Illuminating illumination. In Color and Imaging Conference, volume 2001, pages 27-34. Society for Imaging Science and Technology.
  5. Diebel, J. and Thrun, S. (2005). An application of markov random fields to range sensing. InAdvances in neural information processing systems, pages 291-298.
  6. Fanello, S. R., Keskin, C., Izadi, S., Kohli, P., Kim, D., Sweeney, D., Criminisi, A., Shotton, J., Kang, S. B., and Paek, T. (2014). Learning to be a depth camera for close-range human capture and interaction. ACM Transactions on Graphics (TOG), 33(4):86.
  7. Hernández, C., Vogiatzis, G., Brostow, G. J., Stenger, B., and Cipolla, R. (2007). Non-rigid photometric stereo with colored lights. In ICCV.
  8. Higo, T., Matsushita, Y., and Ikeuchi, K. (2010). Consensus photometric stereo. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, pages 1157-1164. IEEE.
  9. Horn, B. K. (1970). Shape from shading: A method for obtaining the shape of a smooth opaque object from one view.
  10. Horn, B. K. and Brooks, M. J. (1989). Shape from shading. MIT press.
  11. Kim, H., Wilburn, B., and Ben-Ezra, M. (2010). Photometric stereo for dynamic surface orientations. In Computer Vision - ECCV 2010, 11th European Conference on Computer Vision, Heraklion, Crete, Greece, September 5-11, 2010, Proceedings, Part I, pages 59- 72.
  12. Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. (2005). Efficiently combining positions and normals for precise 3d geometry. ACM transactions on graphics (TOG), 24(3):536-543.
  13. Newcombe, R. A., Fox, D., and Seitz, S. M. (2015). Dynamicfusion: Reconstruction and tracking of nonrigid scenes in real-time. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 343-352.
  14. Or-El, R., Rosman, G., Wetzler, A., Kimmel, R., and Bruckstein, A. M. (2015). Rgbd-fusion: Real-time high precision depth recovery. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5407-5416.
  15. Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., and Toyama, K. (2004). Digital photography with flash and no-flash image pairs. ACM transactions on graphics (TOG), 23(3):664-672.
  16. Prados, E. and Faugeras, O. (2005). Shape from shading: a well-posed problem? In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, volume 2, pages 870-877. IEEE.
  17. Richardt, C., Stoll, C., Dodgson, N. A., Seidel, H.-P., and Theobalt, C. (2012). Coherent spatiotemporal filtering, upsampling and rendering of rgbz videos. In Computer Graphics Forum, volume 31, pages 247- 256. Wiley Online Library.
  18. Shen, L., Tan, P., and Lin, S. (2008). Intrinsic image decomposition with non-local texture cues. In Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on, pages 1-7. IEEE.
  19. Shen, L. and Yeo, C. (2011). Intrinsic images decomposition using a local and global sparse representation of reflectance. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 697- 704. IEEE.
  20. Tunwattanapong, B., Fyffe, G., Graham, P., Busch, J., Yu, X., Ghosh, A., and Debevec, P. (2013). Acquiring reflectance and shape from continuous spherical harmonic illumination. ACM Transactions on graphics (TOG), 32(4):109.
  21. Wenger, A., Gardner, A., Tchou, C., Unger, J., Hawkins, T., and Debevec, P. (2005). Performance relighting and reflectance transformation with time-multiplexed illumination. 24(3):756-764.
  22. Woodham, R. J. (1980). Photometric method for determining surface orientation from multiple images. Optical Engineering, 19(1):191139-191139-.
  23. Wu, C., Zollhöfer, M., Nießner, M., Stamminger, M., Izadi, S., and Theobalt, C. (2014). Real-time shading-based refinement for consumer depth cameras. Proc. SIGGRAPH Asia.
  24. Xiao, F., DiCarlo, J. M., Catrysse, P. B., and Wandell, B. A. (2001). Image analysis using modulated light sources. In Photonics West 2001-Electronic Imaging, pages 22-30. International Society for Optics and Photonics.
Download


Paper Citation


in Harvard Style

Hudon M., Gruson A., Kerbiriou P., Cozot R. and Bouatouch K. (2016). Shape and Reflectance from RGB-D Images using Time Sequential Illumination . In Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2016) ISBN 978-989-758-175-5, pages 532-541. DOI: 10.5220/0005726305320541


in Bibtex Style

@conference{visapp16,
author={Matis Hudon and Adrien Gruson and Paul Kerbiriou and Rémi Cozot and Kadi Bouatouch},
title={Shape and Reflectance from RGB-D Images using Time Sequential Illumination},
booktitle={Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2016)},
year={2016},
pages={532-541},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005726305320541},
isbn={978-989-758-175-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2016)
TI - Shape and Reflectance from RGB-D Images using Time Sequential Illumination
SN - 978-989-758-175-5
AU - Hudon M.
AU - Gruson A.
AU - Kerbiriou P.
AU - Cozot R.
AU - Bouatouch K.
PY - 2016
SP - 532
EP - 541
DO - 10.5220/0005726305320541