Enhanced Depth Estimation using a Combination of Structured Light Sensing and Stereo Reconstruction

Andreas Wittmann, Anas Al-Nuaimi, Eckehard Steinbach, Georg Schroth

2016

Abstract

We present a novel approach for depth sensing that combines structured light scanning and stereo reconstruc- tion. High-resolution disparity maps are derived in an iterative-upsampling process that jointly optimizes measurements from graph cuts based stereo reconstruction and structured light sensing using an accelerated α-expansion algorithm. Different from previously proposed fusion approaches, the disparity estimation is initialized using the low-resolution structured light prior. This results in a dense disparity map that can be computed very efficiently and which serves as an improved prior for subsequent iterations at higher resolu- tions. The advantages of the proposed fusion approach over the sole use of stereo are threefold. First, for pixels that exhibit prior knowledge from structured lighting, a reduction of the disparity search range to the uncertainty interval of the prior allows for a significant reduction of ambiguities. Second, the resulting limited search range greatly reduces the runtime of the algorithm. Third, the structured light prior enables a dynamic tuning of the smoothness constraint to allow for a better depth estimation for inclined surfaces.

References

  1. Ansar, A. I., Huertas, A., Matthies, L. H., and Goldberg, S. (2004). Enhancement of stereo at range discontinuities. In Defense and Security, pages 24-35. International Society for Optics and Photonics.
  2. Birchfield, S. and Tomasi, C. (1998). Depth discontinuities by pixel-to-pixel stereo. In Computer Vision, 1998. Sixth International Conference on, pages 1073-1080.
  3. Boykov, Y., Veksler, O., and Zabih, R. (2001). Fast approximate energy minimization via graph cuts. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 23(11):1222-1239.
  4. Choi, S., Ham, B., Oh, C., gon Choo, H., Kim, J., and Sohn, K. (2012). Hybrid approach for accurate depth acquisition with structured light and stereo camera. In Broadband Multimedia Systems and Broadcasting (BMSB), 2012 IEEE International Symposium on, pages 1-4.
  5. Engel, J., Schöps, T., and Cremers, D. (2014). Lsd-slam: Large-scale direct monocular slam. In Computer Vision-ECCV 2014 , pages 834-849. Springer.
  6. Felzenszwalb, P. F. and Huttenlocher, D. P. (2006). Efficient belief propagation for early vision. International journal of computer vision, 70(1):41-54.
  7. Foix, S., Alenya, G., and Torras, C. (2011). Lock-in timeof-flight (tof) cameras: A survey. 11(9):1917-1926.
  8. Gandhi, V., Cech, J., and Horaud, R. (2012). Highresolution depth maps based on tof-stereo fusion. In Robotics and Automation (ICRA), 2012 IEEE International Conference on, pages 4742-4749.
  9. Goldberg, A. O., S., L., M., D., T., t., J., S., J., D., and T., G. (2014). Google tango teardown. www.ifixit.com/Teardown/Project+Tango+Teardown/23835.
  10. Khoshelham, K. and Elberink, S. O. (2012). Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors, 12(2):1437-1454.
  11. Kolmogorov, V., Monasse, P., and Tan, P. (2014). Kolmogorov and Zabihs Graph Cuts Stereo Matching Algorithm. Image Processing On Line, 4:220-251.
  12. Li, Q., Biswas, M., Pickering, M. R., and Frater, M. R. (2011). Accurate depth estimation using structured light and passive stereo disparity estimation. In Image Processing (ICIP), 2011 18th IEEE International Conference on, pages 969-972.
  13. Liling, Z., Yuhui, Z., Quansen, S., and Deshen, X. (2012). Suppression for luminance difference of stereo imagepair based on improved histogram equalization. Proceedings of the computer science and technology, 6:2.
  14. Newcombe, R. A., Lovegrove, S. J., and Davison, A. J. (2011). Dtam: Dense tracking and mapping in realtime. In Computer Vision (ICCV), 2011 IEEE International Conference on, pages 2320-2327. IEEE.
  15. Occipital (2015). Occipital depth sensors. http://occipital.com/.
  16. Piszczor, M. and Yang, C. (2014). Project tango. https://sites.google.com/a/google.com/project-tangosdk/home.
  17. Scharstein, D. and Szeliski, R. (2002). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1-3):7-42.
  18. Scharstein, D. and Szeliski, R. (2003a). High-accuracy stereo depth maps using structured light. In Computer Vision and Pattern Recognition, 2003. Proceedings. 2003 IEEE Computer Society Conference on, volume 1, pages I-195.
  19. Scharstein, D. and Szeliski, R. (2003b). High-accuracy stereo depth maps using structured light. In Computer Vision and Pattern Recognition, 2003. Proceedings. 2003 IEEE Computer Society Conference on, volume 1.
  20. Scharstein, D. and Szeliski, R. (2015). A taxonomy and evaluation of dense twoframe stereo correspondence algorithms. http://www.vision.middlebury.edu/stereo/.
  21. Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In Computer vision and pattern recognition, 2006 IEEE Computer Society Conference on, volume 1, pages 519- 528. IEEE.
  22. Somanath, G., Cohen, S., Price, B., and Kambhamettu, C. (2013). Stereo+kinect for high resolution stereo correspondences. In 3D Vision - 3DV 2013, 2013 International Conference on, pages 9-16.
  23. Tappen, M. F. and Freeman, W. T. (2003). Comparison of graph cuts with belief propagation for stereo, using identical mrf parameters. In Computer Vision, 2003. Proceedings. Ninth IEEE International Conference on, pages 900-906. IEEE.
  24. Wei-Chen Chiu, U. B. and Fritz, M. (2011). Improving the kinect by cross-modal stereo. In Proceedings of the British Machine Vision Conference, pages 116.1- 116.10.
Download


Paper Citation


in Harvard Style

Wittmann A., Al-Nuaimi A., Steinbach E. and Schroth G. (2016). Enhanced Depth Estimation using a Combination of Structured Light Sensing and Stereo Reconstruction . In Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2016) ISBN 978-989-758-175-5, pages 510-521. DOI: 10.5220/0005724605100521


in Bibtex Style

@conference{visapp16,
author={Andreas Wittmann and Anas Al-Nuaimi and Eckehard Steinbach and Georg Schroth},
title={Enhanced Depth Estimation using a Combination of Structured Light Sensing and Stereo Reconstruction},
booktitle={Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2016)},
year={2016},
pages={510-521},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005724605100521},
isbn={978-989-758-175-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2016)
TI - Enhanced Depth Estimation using a Combination of Structured Light Sensing and Stereo Reconstruction
SN - 978-989-758-175-5
AU - Wittmann A.
AU - Al-Nuaimi A.
AU - Steinbach E.
AU - Schroth G.
PY - 2016
SP - 510
EP - 521
DO - 10.5220/0005724605100521