AN EVALUATION METHODOLOGY FOR STEREO CORRESPONDENCE ALGORITHMS

Ivan Cabezas, Maria Trujillo, Margaret Florian

2012

Abstract

A comparison of stereo correspondence algorithms can be conducted by a quantitative evaluation of disparity maps. Among the existing evaluation methodologies, the Middlebury’s methodology is commonly used. However, the Middlebury’s methodology has shortcomings in the evaluation model and the error measure. These shortcomings may bias the evaluation results, and make a fair judgment about algorithms accuracy difficult. An alternative, the A* methodology is based on a multiobjective optimisation model that only provides a subset of algorithms with comparable accuracy. In this paper, a quantitative evaluation of disparity maps is proposed. It performs an exhaustive assessment of the entire set of algorithms. As innovative aspect, evaluation results are shown and analysed as disjoint groups of stereo correspondence algorithms with comparable accuracy. This innovation is obtained by a partitioning and grouping algorithm. On the other hand, the used error measure offers advantages over the error measure used in the Middlebury’s methodology. The experimental validation is based on the Middlebury’s test-bed and algorithms repository. The obtained results show seven groups with different accuracies. Moreover, the top-ranked stereo correspondence algorithms by the Middlebury’s methodology are not necessarily the most accurate in the proposed methodology.

References

  1. Barnes, M., Shechtman, C., & Finkelstein, E., 2009. Patchmatch: A randomized correspondence algorithm for structural image editing. In ACM Transactions on Graphics, 28(3). pp. 1-11.
  2. Bleyer, M., & Chambon, S., 2010. Does Color Really Help in Dense Stereo Matching?. In Proceedings of the International Symposium 3D Data Processing, Visualization and Transmission, pp. 1-8.
  3. Bleyer M. & Gelautz M. 2004. A layered stereo algorithm using image segmentation and global visibility constraints. In Proceedings International Conference Image Processing. IEEE Computer Society pp. 2997- 3000.
  4. Bleyer, M., Rhemann, C., & Rother, C., 2011. PatchMatch Stereo - Stereo Matching with Slanted Support Windows. In Proceedings of the British Machine Vision Conference, pp. 1-11.
  5. Cabezas, I., Padilla V., & Trujillo, M., 2011. A measure for accuracy disparity maps evaluation. In Proceedings of the Iberoamerican Congress on Pattern Recognition. Springer-Verlag, pp. 223-231.
  6. Cabezas, I., & Trujillo, M., 2011. A Non-Linear Quantitative Evaluation Approach for Disparity Estimation. In Proceedings of the International Conference on Computer Vision, Theory and Applications, pp. 704-709.
  7. Felzenszwalb, P., & Huttenlocher, D., 2004. Efficient belief propagation for early vision. In Proceedings of Computer Vision and Pattern Recognition. IEEE Computer Society, pp. 261-268.
  8. Gallup, D., Frahm, J., Mordohai, & P., Pollefeys, M., 2008. Variable Baseline/Resolution Stereo. In Proceedings of Computer Vision and Pattern Recognition, IEEE Computer Society, pp. 1-8.
  9. Hirschmüller, H., & Scharstein, D., 2009. Evaluation of Stereo Matching Costs on Images with Radiometric Differences. In IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(9). IEEE Computer Society, pp. 1582-1599.
  10. Kelly, P., O'Connor N., & Smeaton A., 2007. A Framework for Evaluating Stereo-Based Pedestrian Detection Techniques. In IEEE Transactions Circuits and Systems for Video Technology, 18(8). IEEE Computer Society. IEEE, pp. 1163-1167.
  11. Kolmogorov, V., & Zabih, R., 2001. Computing visual correspondence with occlusions using graph cuts. In Proceedings of Eighth International Conference on Computer Vision, IEEE Computer Society, pp. 508- 515.
  12. Kolmogorov, V., & Zabih, R., 2002. Multi-camera scene reconstruction via graph cuts. In Proceedings of European Conference on Computer Vision, Springer Verlag, pp. 82-96.
  13. Kostlivá, J., Cech, J., & Sara, R., 2007. Feasibility Boundary in Dense and Semi-Dense Stereo Matching. In Computer Vision and Pattern Recognition. IEEE Computer Society, pp. 1-8.
  14. Larsen, E., Mordohai, P., Pollefeys, M., & Fuchs, H., 2007. Temporally consistent reconstruction from multiple video streams using enhanced belief propagation. In International Conference on Computer Vision. IEEE Computer Society, pp. 282-298.
  15. Manduchi, R., & Tomasi, C., 1999. Distinctiveness maps for image matching. In International Conference on Image Analysis and Processing. pp. 26-31.
  16. Neilson, D., & Yang, Y., 2008. Evaluation of Constructable Match Cost Measures for Stereo Correspondence using Cluster Ranking. In Computer Vision and Pattern Recognition. IEEE Computer Society, pp. 1-8.
  17. Papadakis, N., & Caselles V., 2010. Multi-label Depth Estimation for Graph Cuts Stereo Problems. In Journal of Mathematical Imaging and Vision, 38(1). Kluwer Academic Publishers, pp. 70-82.
  18. Scharstein, D., & Szeliski, R., 2011. Middlebury Stereo Evaluation - Version 2. Retrieved October 24th, 2011, from: http://vision.middlebury.edu/stereo/eval/.
  19. Scharstein, D., & Szeliski, R., 2002. A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms. In International Journal of Computer Vision, Volume 47, pp. 7-42.
  20. Scharstein, D., & Szeliski, R., 2003. High-accuracy Stereo Depth Maps using Structured Light. In Computer Vision and Pattern Recognition. IEEE Computer Society, pp. I-195-I-202.
  21. Sun J., Li, Y., Kang, S., & Shum H., 2005. Symmetric stereo matching for occlusion handling. In Computer Vision and Pattern Recognition. IEEE Computer Society, pp. 399-406.
  22. Szeliski, R., 1999. Prediction Error as a Quality Metric for Motion and Stereo. In International Conference on Computer Vision, Volume 2. IEEE Computer Society, pp. 781-788.
  23. Szeliski, R., & Zabih, R., 2000. An Experimental Comparison of Stereo Algorithms. In Proceedings of the International Workshop on Vision Algorithms. Springer-Verlag, pp. 1-19.
  24. Tombari, F., Mattoccia, S., & Di Stefano, L., 2010. Stereo for Robots: Quantitative Evaluation of Efficient and Low-memory Dense Stereo Algorithms. In Proceedings of International Conference Control Automation Robotics and Vision. IEEE Computer Society, pp. 1231-1238.
  25. Van der Mark, W., & Gavrila, D., 2006. Real-time Dense Stereo for Intelligent Vehicles. In IEEE Transactions on on Intelligent Transportation Systems, 7(1). IEEE Computer Society, pp. 38-50.
  26. Van Veldhuizen, D., Zydallis, J., & Lamont, G., 2003. Considerations in engineering parallel multiobjective evolutionary algorithms. In IEEE Transactions on Evolutionary Computation, 7(2). IEEE Computer Society, pp. 144-173.
  27. Yang, Q., Wang, L., Yang, R., Stewénius, H., & Nistér, D., 2008. Stereo matching with color-weighted correlation, hierarchical belief propagation and occlusion handling. In Transactions on Pattern Analysis and Machine Intelligence, 31(3). IEEE Computer Society, pp. 492-504.
  28. Yoon, K., & Kweon, I., 2007. Stereo matching with the distinctive similarity measure. In International Conference on Computer Vision. IEEE Computer Society, pp. 1-7.
Download


Paper Citation


in Harvard Style

Cabezas I., Trujillo M. and Florian M. (2012). AN EVALUATION METHODOLOGY FOR STEREO CORRESPONDENCE ALGORITHMS . In Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2012) ISBN 978-989-8565-04-4, pages 154-163. DOI: 10.5220/0003850801540163


in Bibtex Style

@conference{visapp12,
author={Ivan Cabezas and Maria Trujillo and Margaret Florian},
title={AN EVALUATION METHODOLOGY FOR STEREO CORRESPONDENCE ALGORITHMS},
booktitle={Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2012)},
year={2012},
pages={154-163},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003850801540163},
isbn={978-989-8565-04-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2012)
TI - AN EVALUATION METHODOLOGY FOR STEREO CORRESPONDENCE ALGORITHMS
SN - 978-989-8565-04-4
AU - Cabezas I.
AU - Trujillo M.
AU - Florian M.
PY - 2012
SP - 154
EP - 163
DO - 10.5220/0003850801540163