Temporal Selection of Images for a Fast Algorithm for Depth-map Extraction in Multi-baseline Configurations

Dimitri Bulatov

2015

Abstract

Obtaining accurate depth maps from multi-view configurations is an essential component for dense scene reconstruction from images and videos. In the first part of this paper, a plane sweep algorithm for sampling an energy function for every depth label and a dense set of points is presented. The distinctive features of this algorithm are 1) that despite a flexible model choice for the underlying geometry and radiometry, the energy function is performed by merely image operations instead of pixel-wise computations, and 2) that it can be easily manipulated by different terms, such as triangle-based smoothing term, or post-processed by one of the numerous state-of-the-art non-local energy minimization algorithms. The second contribution of this paper is a search for optimal ways to aggregate multiple observations in order to make the cost function more robust near the image border and in occlusions areas. Experiments with different data sets show the relevance of the proposed research, emphasize the potential of the algorithm, and provide ideas of future work.

References

  1. Belhumeur, P. (1996). A Bayesian approach to binocular stereopsis. International Journal of Computer Vision, 19(3):237-260.
  2. Bodensteiner, C. and Arens, M. (2012). Real-time 2D video/3D LiDAR registration. In International Conference on Pattern Recognition (ICCV), pages 2206- 2209, Tsukuba (Japan).
  3. Boykov, Y., Veksler, O., and Zabih, R. (1998). A Variable Window Approach to Early Vision. Transactions on Pattern Analysis and Machine Intelligence, 20(12):1283-1294.
  4. Bulatov, D., Wernerus, P., and Heipke, C. (2011). Multiview dense matching supported by triangular meshes. ISPRS Journal of Photogrammetry and Remote Sensing, 66(6):907-918.
  5. Delong, A., Osokin, A., Isack, H. N., and Boykov, Y. (2012). Fast approximate energy minimization with label costs. International Journal of Computer Vision, 96(1):1-27.
  6. Furukawa, Y. and Ponce, J. (2010). Accurate, dense, and robust multiview stereopsis. Transactions on Pattern Analysis and Machine Intelligence, 32(8):1362-1376.
  7. Goesele, M., Snavely, N., Curless, B., Hoppe, H., and Seitz, S. M. (2007). Multi-view stereo for community photo collections. In International Conference on Computer Vision (ICCV), pages 1-8.
  8. Hansen, P. C. and O'Leary, D. P. (1993). The use of the L-curve in the regularization of discrete ill-posed problems. SIAM Journal on Scientific Computing, 14(6):1487-1503.
  9. Heinrichs, M., Hellwich, O., and Rodehorst, V. (2007). Efficient semi-global matching for trinocular stereo. Photogrammetrie - Fernerkundung - Geoinformation, 6:405-414.
  10. Hirschmüller, H. (2008). Stereo processing by semiglobal matching and mutual information. Transactions on Pattern Analysis and Machine Intelligence, 30(2):328-341.
  11. Hirschmüller, H. and Scharstein, D. (2009). Evaluation of stereo matching costs on images with radiometric differences. Transactions on Pattern Analysis and Machine Intelligence, 31(9):1582-1599.
  12. Irschara, A., Rumpler, M., Meixner, P., Pock, T., and Bischof, H. (2012). Efficient and globally optimal multi view dense matching for aerial images. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences.
  13. Kang, S. B., Szeliski, R., and Chai, J. (2001). Handling occlusions in dense multi-view stereo. In Computer Vision and Pattern Recognition (CVPR), volume 1.
  14. Koch, R., Pollefeys, M., and Van Gool, L. (1998). Multi viewpoint stereo from uncalibrated video sequences. In European Conference on Computer Vision (ECCV), pages 55-71. Springer.
  15. Kolmogorov, V. (2003). Graph based algorithms for scene reconstruction from two or more views. PhD thesis, Cornell University.
  16. Kolmogorov, V. (2006). Convergent tree-reweighted message passing for energy minimization. Transactions on Pattern Analysis and Machine Intelligence, 28(10):1568-1583.
  17. Nakamura, Y., Matsuura, T., Satoh, K., and Ohta, Y. (1996). Occlusion detectable stereo-occlusion patterns in camera matrix. In Computer Vision and Pattern Recognition (CVPR), pages 371-378.
  18. Okutomi, M. and Kanade, T. (1993). A multiple-baseline stereo. Transactions on Pattern Analysis and Machine Intelligence, 15(4):353-363.
  19. Pollefeys, M., Nistér, D., Frahm, J.-M., Akbarzadeh, A., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Kim, S.-J., Merrell, P., Salmi, C., Sinha, S., Talton, B., Wang, L., Yang, Q., Stewénius, H., Yang, R., Welch, G., and Towles, H. (2008). Detailed real-time urban 3D reconstruction from video. International Journal of Computer Vision, 78(2-3):143-167.
  20. Rothermel, M., Bulatov, D., Haala, N., and Wenzel, K. (2014). Fast and robust generation of semantic urban terrain models from UAV video streams. In International Conference on Pattern Recognition (ICPR), pages 592-597.
  21. Scharstein, D. and Szeliski, R. (2002). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1):7-42.
  22. Sun, J., Zheng, N.-N., and Shum, H.-Y. (2003). Stereo matching using belief propagation. Transactions on Pattern Analysis and Machine Intelligence, 25(7):787-800.
  23. Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., and Rother, C. (2006). A comparative study of energy minimization methods for markov random fields. In European Conference on Computer Vision (ECCV), pages 16- 29. Springer.
  24. Wainwright, M. J., Jaakkola, T. S., and Willsky, A. S. (2005). Map estimation via agreement on trees: message-passing and linear programming. Transactions on Information Theory, 51(11):3697-3717.
  25. Zhang, H., C?ech, J., Wu, F., and Hu, Z. (2003). A linear trinocular rectification method for accurate stereoscopic matching. In British Machine Vision Conf., 2003, pages 281-290.
Download


Paper Citation


in Harvard Style

Bulatov D. (2015). Temporal Selection of Images for a Fast Algorithm for Depth-map Extraction in Multi-baseline Configurations . In Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2015) ISBN 978-989-758-091-8, pages 395-402. DOI: 10.5220/0005239503950402


in Bibtex Style

@conference{visapp15,
author={Dimitri Bulatov},
title={Temporal Selection of Images for a Fast Algorithm for Depth-map Extraction in Multi-baseline Configurations},
booktitle={Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2015)},
year={2015},
pages={395-402},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005239503950402},
isbn={978-989-758-091-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2015)
TI - Temporal Selection of Images for a Fast Algorithm for Depth-map Extraction in Multi-baseline Configurations
SN - 978-989-758-091-8
AU - Bulatov D.
PY - 2015
SP - 395
EP - 402
DO - 10.5220/0005239503950402