DENSE PIXEL MATCHING BETWEEN UNRECTIFIED AND DISTORTED IMAGES USING DYNAMIC PROGRAMMING

Jerome Thevenon, Jesus Martinez-del-Rincon, Romain Dieny, Jean-Christophe Nebel

2012

Abstract

In this paper, a novel framework for dense pixel matching based on dynamic programming is introduced. Unlike most techniques proposed in the literature, our approach assumes neither known camera geometry nor the availability of rectified images. Under such conditions, the matching task cannot be reduced to finding correspondences between a pair of scanlines. We propose to extend existing dynamic programming methodologies to a larger dimensional space by using a 3D scoring matrix so that correspondences between a line and a whole image can be calculated. After assessing our framework on a standard evaluation dataset of rectified stereo images, experiments are conducted on unrectified and non-linearly distorted images. Results validate our new approach and reveal the versatility of our algorithm.

References

  1. Ayache, N., Hansen, C., 1988. Rectification of images for binocular and trinocular stereovision. In International Conference on Pattern Recognition, pp. 11-16.
  2. Ayache, N., Hansen, C., 1988. Rectification of images for binocular and trinocular stereovision. In International Conference on Pattern Recognition, pp. 11-16.
  3. Baker, H., Binford, T., 1981. Depth from edge and intensity based stereo. In IJCAI81, pp.631-636.
  4. Baker, H., Binford, T., 1981. Depth from edge and intensity based stereo. In IJCAI81, pp.631-636.
  5. Barnard, S. T., Fischler, M. A., 1982. Computational stereo. In ACM Comp. Surveys, 14(4), pp.553-572.
  6. Barnard, S. T., Fischler, M. A., 1982. Computational stereo. In ACM Comp. Surveys, 14(4), pp.553-572.
  7. Barnum, P., Kanade, T., Narasimhan., S.G., 2007. SpatioTemporal Frequency Analysis for Removing Rain and Snow from Videos. In PACV.
  8. Barnum, P., Kanade, T., Narasimhan., S.G., 2007. SpatioTemporal Frequency Analysis for Removing Rain and Snow from Videos. In PACV.
  9. Belhumeur, P. N., 1996. A Bayesian approach to binocular stereopsis. International Journal of Computer Vision, 19(3), pp.237-260.
  10. Belhumeur, P. N., 1996. A Bayesian approach to binocular stereopsis. International Journal of Computer Vision, 19(3), pp.237-260.
  11. Bobick, A.F., Intille, S.S., 1999. Large occlusion stereo. International Journal of Computer Vision, 33(3), pp.181-200.
  12. Bobick, A.F., Intille, S.S., 1999. Large occlusion stereo. International Journal of Computer Vision, 33(3), pp.181-200.
  13. Brown., L.G., 1992. A survey of image registration techniques. In ACM Comp. Surveys, 24(4), pp.325- 376.
  14. Brown., L.G., 1992. A survey of image registration techniques. In ACM Comp. Surveys, 24(4), pp.325- 376.
  15. Brown, M.Z., Burschka, D., Hager, G.D., 2003. Advances in computational stereo, IEEE Transactions on Pattern Analysis and Machine Intelligence, 5 (8), pp. 993- 1008.
  16. Brown, M.Z., Burschka, D., Hager, G.D., 2003. Advances in computational stereo, IEEE Transactions on Pattern Analysis and Machine Intelligence, 5 (8), pp. 993- 1008.
  17. Cox, I. J., Hingorani, S. L., Rao, S. B., Maggs, B. M., 1996. A maximum likelihood stereo algorithm. Computer Vision and Image Understanding, 63(3), pp.542-567.
  18. Cox, I. J., Hingorani, S. L., Rao, S. B., Maggs, B. M., 1996. A maximum likelihood stereo algorithm. Computer Vision and Image Understanding, 63(3), pp.542-567.
  19. Deng, Y., Lin, X., 2006. A fast line segment based dense stereo algorithm using tree dynamic programming. In European Conference on Computer Vision, Graz, Austria, May 7 - 13.
  20. Deng, Y., Lin, X., 2006. A fast line segment based dense stereo algorithm using tree dynamic programming. In European Conference on Computer Vision, Graz, Austria, May 7 - 13.
  21. Dhond, U. R., Aggarwal, J. K., 1989. Structure from stereo: a review. In IEEE Trans. on Systems,Man, and Cybern., 19(6), pp.1489-1510.
  22. Dhond, U. R., Aggarwal, J. K., 1989. Structure from stereo: a review. In IEEE Trans. on Systems,Man, and Cybern., 19(6), pp.1489-1510.
  23. Dieny, R., Thevenon, J., Martinez-del-Rincon, J., Nebel, J.C., 2011. Bioinformatics inspired algorithm for stereo correspondence, In VISAPP, Vilamoura, Portugal, March 5-7.
  24. Dieny, R., Thevenon, J., Martinez-del-Rincon, J., Nebel, J.C., 2011. Bioinformatics inspired algorithm for stereo correspondence, In VISAPP, Vilamoura, Portugal, March 5-7.
  25. Duchaineau, M., Cohen, J., Vaidya, S., 2007. Toward Fast Computation of Dense Image Correspondence on the GPU. In High Performance Embedded Computing Workshop, Lincoln Laboratory, Massachusetts Institute of Technology, pp.91-92.Forstmann.
  26. Duchaineau, M., Cohen, J., Vaidya, S., 2007. Toward Fast Computation of Dense Image Correspondence on the GPU. In High Performance Embedded Computing Workshop, Lincoln Laboratory, Massachusetts Institute of Technology, pp.91-92.Forstmann.
  27. Forstmann, S., Kanou, Y., Ohya, J., Thuering, S., Schmitt, A., 2004. Real-Time Stereo by using Dynamic Programming, In Computer Vision and Pattern Recognition Workshop, Washington, DC, USA, 27 June-2 July 2004.
  28. Forstmann, S., Kanou, Y., Ohya, J., Thuering, S., Schmitt, A., 2004. Real-Time Stereo by using Dynamic Programming, In Computer Vision and Pattern Recognition Workshop, Washington, DC, USA, 27 June-2 July 2004.
  29. Garg, K., Nayar, S. K., 2004. Detection and removal of rain from videos. In CVPR.
  30. Garg, K., Nayar, S. K., 2004. Detection and removal of rain from videos. In CVPR.
  31. Geiger, D., Ladendorf, B., Yuille, A., 1992. Occlusions and binocular stereo. In European Conference on Computer Vision, pp.425-433.
  32. Geiger, D., Ladendorf, B., Yuille, A., 1992. Occlusions and binocular stereo. In European Conference on Computer Vision, pp.425-433.
  33. Gong, M., 2006. Enforcing Temporal Consistency in RealTime Stereo Estimation. In ECCV 2006, Part III, LNCS 3953, pp.564- 577, 2006.
  34. Gong, M., 2006. Enforcing Temporal Consistency in RealTime Stereo Estimation. In ECCV 2006, Part III, LNCS 3953, pp.564- 577, 2006.
  35. Hartley, R. I., 1999. Theory and practice of projective rectification. International Journal of Computer Vision, 35(2), pp.115-127.
  36. Hartley, R. I., 1999. Theory and practice of projective rectification. International Journal of Computer Vision, 35(2), pp.115-127.
  37. Jones. G. A., 1997. Constraint, Optimization, and Hierarchy: Reviewing Stereoscopic Correspondence of Complex Features. In Computer Vision and Image Understanding, 65(1), pp. 57-78.
  38. Jones. G. A., 1997. Constraint, Optimization, and Hierarchy: Reviewing Stereoscopic Correspondence of Complex Features. In Computer Vision and Image Understanding, 65(1), pp. 57-78.
  39. Lazaros, N., Sirakoulis, G. C., Gasteratos A., 2008. Review of Stereo Vision Algorithms: From Software to Hardware. International Journal of Optomechatronics, 2(4), pp.435 - 462.
  40. Lazaros, N., Sirakoulis, G. C., Gasteratos A., 2008. Review of Stereo Vision Algorithms: From Software to Hardware. International Journal of Optomechatronics, 2(4), pp.435 - 462.
  41. Lim, S. N., Mittal, A., Davis, L., Paragios, N., 2004. Uncalibrated stereo rectification for automatic 3D surveillance. In International Conference on Image Processing, 2, pp.1357.
  42. Lim, S. N., Mittal, A., Davis, L., Paragios, N., 2004. Uncalibrated stereo rectification for automatic 3D surveillance. In International Conference on Image Processing, 2, pp.1357.
  43. Lu, J., Ke Zhang, Lafruit, G., Catthoor, F. 2009. Real-time stereo matching: A cross-based local approach. In International Conference on Acoustics, Speech and Signal Processing, pp.733-736, Washington, DC, USA, April 19 - 24, 2009.
  44. Lu, J., Ke Zhang, Lafruit, G., Catthoor, F. 2009. Real-time stereo matching: A cross-based local approach. In International Conference on Acoustics, Speech and Signal Processing, pp.733-736, Washington, DC, USA, April 19 - 24, 2009.
  45. MacLean, W. J., Sabihuddin, S., Islam, J., 2010. Leveraging cost matrix structure for hardware implementation of stereo disparity computation using dynamic programming. Computer Vision and Image Understanding, In Press.
  46. MacLean, W. J., Sabihuddin, S., Islam, J., 2010. Leveraging cost matrix structure for hardware implementation of stereo disparity computation using dynamic programming. Computer Vision and Image Understanding, In Press.
  47. Needleman, S. B., Wunsch, C.D., 1970. A general method applicable to the search for similarities in the aminoacid sequence of two proteins. Journal of Molecular Biology, 48(3), pp.443-53.
  48. Needleman, S. B., Wunsch, C.D., 1970. A general method applicable to the search for similarities in the aminoacid sequence of two proteins. Journal of Molecular Biology, 48(3), pp.443-53.
  49. Note, J. B., Shand, M., Vuillemin, J., 2006. Real-Time Video Pixel Matching. In FPL , pp. 1-6.
  50. Note, J. B., Shand, M., Vuillemin, J., 2006. Real-Time Video Pixel Matching. In FPL , pp. 1-6.
  51. Ohta, Y., Kanade, T., 1985. Stereo by intra- and interscanline search using dynamic programming. IEEE TPAMI, 7(2), pp.139-154.
  52. Ohta, Y., Kanade, T., 1985. Stereo by intra- and interscanline search using dynamic programming. IEEE TPAMI, 7(2), pp.139-154.
  53. Oram., D., 2001. Rectification for Any Epipolar Geometry. In BMVC, pp. 653-662.
  54. Oram., D., 2001. Rectification for Any Epipolar Geometry. In BMVC, pp. 653-662.
  55. Pollefeys, M., Koch, R., Van Gool, L., 1999. A simple and efficient rectification method for general motion. In International Conference on Computer Vision, vol 1, pp. 496-501.
  56. Pollefeys, M., Koch, R., Van Gool, L., 1999. A simple and efficient rectification method for general motion. In International Conference on Computer Vision, vol 1, pp. 496-501.
  57. Roy, S., Meunier, J., Cox, I., 1997. Cylindrical Rectification to Minimize Epipolar Distortion. In Conference on Computer Vision and Pattern Recognition, pp.393-399.
  58. Roy, S., Meunier, J., Cox, I., 1997. Cylindrical Rectification to Minimize Epipolar Distortion. In Conference on Computer Vision and Pattern Recognition, pp.393-399.
  59. Salmen, J., Schlipsing, M., Edelbrunner, J., Hegemann, S., Lueke, S., 2009. Real-time stereo vision: making more out of dynamic programming. In International Conference on Computer Analysis of Images and Patterns, Münster, Germany, Sept. 2-4.
  60. Salmen, J., Schlipsing, M., Edelbrunner, J., Hegemann, S., Lueke, S., 2009. Real-time stereo vision: making more out of dynamic programming. In International Conference on Computer Analysis of Images and Patterns, Münster, Germany, Sept. 2-4.
  61. Scharstein, D., Szeliski, R, 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1), pp.7-42.
  62. Scharstein, D., Szeliski, R, 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1), pp.7-42.
  63. Scharstein, D., Szeliski, R, 2003. High-accuracy stereo depth maps using structured light. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 195-202.
  64. Scharstein, D., Szeliski, R, 2003. High-accuracy stereo depth maps using structured light. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 195-202.
  65. Sun, C., 2002. Fast Stereo Matching Using Rectangular Subregioning and 3D Maximum-Surface Techniques. In International Journal of Computer Vision archive, 47 (1-3), pp. 99-107.
  66. Sun, C., 2002. Fast Stereo Matching Using Rectangular Subregioning and 3D Maximum-Surface Techniques. In International Journal of Computer Vision archive, 47 (1-3), pp. 99-107.
  67. Sun., C., 2002. Fast Optical Flow Using 3D Shortest Path Techniques. In Image and Vision Computing, 20(13/14), pp. 981-991.
  68. Sun., C., 2002. Fast Optical Flow Using 3D Shortest Path Techniques. In Image and Vision Computing, 20(13/14), pp. 981-991.
  69. Tappen, M., Freeman, W., 2003. Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters, in: IEEE International Conference on Computer Vision (ICCV), 2, pp. 900-906.
  70. Tappen, M., Freeman, W., 2003. Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters, in: IEEE International Conference on Computer Vision (ICCV), 2, pp. 900-906.
  71. Tippetts, B., Lee, D. J., Archibald, J., 2010. Fast correspondence of unrectified stereo images using genetic algorithm and spline representation. In Intelligent Robots and Computer Vision XXVII: Algorithms and Techniques, 7539, 17 January 2010.
  72. Tippetts, B., Lee, D. J., Archibald, J., 2010. Fast correspondence of unrectified stereo images using genetic algorithm and spline representation. In Intelligent Robots and Computer Vision XXVII: Algorithms and Techniques, 7539, 17 January 2010.
  73. Torr, P. H. S., Criminisi, A., 2004. Dense stereo using pivoted dynamic programming. Image and Vision Computing, 22(10), pp.795-806.
  74. Torr, P. H. S., Criminisi, A., 2004. Dense stereo using pivoted dynamic programming. Image and Vision Computing, 22(10), pp.795-806.
  75. Vanetti, M., Gallo, I., Binaghi, E., 2009. Dense TwoFrame Stereo Correspondence by Self-organizing Neural Network. In ICIAP 2009, LNCS 5716, pp.1035-1042.
  76. Vanetti, M., Gallo, I., Binaghi, E., 2009. Dense TwoFrame Stereo Correspondence by Self-organizing Neural Network. In ICIAP 2009, LNCS 5716, pp.1035-1042.
  77. Veksler, O., 2005. Stereo correspondence by dynamic programming on a tree. In Computer Vision and Pattern Recognition, San Diego, CA, USA, 20-26.
  78. Veksler, O., 2005. Stereo correspondence by dynamic programming on a tree. In Computer Vision and Pattern Recognition, San Diego, CA, USA, 20-26.
  79. Willson, R. G., Johnson, A. E., Goguen, J. D., 2005. MOC2DIMES: A camera simulator for the mars exploration Rover descent image motion estimation system. In Proc. 8th Int'l. Symp. Artificial Intelligence, Robotics and Automation in Space.
  80. Willson, R. G., Johnson, A. E., Goguen, J. D., 2005. MOC2DIMES: A camera simulator for the mars exploration Rover descent image motion estimation system. In Proc. 8th Int'l. Symp. Artificial Intelligence, Robotics and Automation in Space.
  81. Wan, D., Zhou, J., 2008. Stereo vision using two PTZ cameras. Computer Vision and Image Understanding, 112, pp.184-194.
  82. Wan, D., Zhou, J., 2008. Stereo vision using two PTZ cameras. Computer Vision and Image Understanding, 112, pp.184-194.
  83. Wang, L., Liao, M., Gong, M., Yang, R., Nistér, D., 2006. High-quality real-time stereo using adaptive cost aggregation and dynamic programming. In 3D Data Processing, Visualization and Transmission. Chapel Hill, USA, June 14-16.
  84. Wang, L., Liao, M., Gong, M., Yang, R., Nistér, D., 2006. High-quality real-time stereo using adaptive cost aggregation and dynamic programming. In 3D Data Processing, Visualization and Transmission. Chapel Hill, USA, June 14-16.
  85. Yang, Q., Wang, L., Yang, R., 2006. Real-time Global Stereo Matching Using Hierarchical Belief propagation. In BMVC 2006, Edinburgh, UK.
  86. Yang, Q., Wang, L., Yang, R., 2006. Real-time Global Stereo Matching Using Hierarchical Belief propagation. In BMVC 2006, Edinburgh, UK.
  87. Yaguchi, Y., Iseki, K., Oka, R., 2009. Optimal Pixel Matching between Images. In PSIVT pp. 597-610.
  88. Yaguchi, Y., Iseki, K., Oka, R., 2009. Optimal Pixel Matching between Images. In PSIVT pp. 597-610.
  89. Yin, X. C., Sun, J., 2007. Perspective Rectification for Mobile Phone Camera-Based Documents Using a Hybrid Approach to Vanishing Point Detection, CBDAR'07, pp. 37-44.
  90. Yin, X. C., Sun, J., 2007. Perspective Rectification for Mobile Phone Camera-Based Documents Using a Hybrid Approach to Vanishing Point Detection, CBDAR'07, pp. 37-44.
  91. Zhengping, J., 1988. On the multi-scale iconic representation for low-level computer vision systems. In PhD thesis, The Turing Institute and The University of Strathclyde, 1988.
  92. Zhengping, J., 1988. On the multi-scale iconic representation for low-level computer vision systems. In PhD thesis, The Turing Institute and The University of Strathclyde, 1988.
Download


Paper Citation


in Harvard Style

Thevenon J., Martinez-del-Rincon J., Dieny R. and Nebel J. (2012). DENSE PIXEL MATCHING BETWEEN UNRECTIFIED AND DISTORTED IMAGES USING DYNAMIC PROGRAMMING . In Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2012) ISBN 978-989-8565-04-4, pages 216-224. DOI: 10.5220/0003812602160224


in Harvard Style

Thevenon J., Martinez-del-Rincon J., Dieny R. and Nebel J. (2012). DENSE PIXEL MATCHING BETWEEN UNRECTIFIED AND DISTORTED IMAGES USING DYNAMIC PROGRAMMING . In Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2012) ISBN 978-989-8565-04-4, pages 216-224. DOI: 10.5220/0003812602160224


in Bibtex Style

@conference{visapp12,
author={Jerome Thevenon and Jesus Martinez-del-Rincon and Romain Dieny and Jean-Christophe Nebel},
title={DENSE PIXEL MATCHING BETWEEN UNRECTIFIED AND DISTORTED IMAGES USING DYNAMIC PROGRAMMING},
booktitle={Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2012)},
year={2012},
pages={216-224},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003812602160224},
isbn={978-989-8565-04-4},
}


in Bibtex Style

@conference{visapp12,
author={Jerome Thevenon and Jesus Martinez-del-Rincon and Romain Dieny and Jean-Christophe Nebel},
title={DENSE PIXEL MATCHING BETWEEN UNRECTIFIED AND DISTORTED IMAGES USING DYNAMIC PROGRAMMING},
booktitle={Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2012)},
year={2012},
pages={216-224},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003812602160224},
isbn={978-989-8565-04-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2012)
TI - DENSE PIXEL MATCHING BETWEEN UNRECTIFIED AND DISTORTED IMAGES USING DYNAMIC PROGRAMMING
SN - 978-989-8565-04-4
AU - Thevenon J.
AU - Martinez-del-Rincon J.
AU - Dieny R.
AU - Nebel J.
PY - 2012
SP - 216
EP - 224
DO - 10.5220/0003812602160224


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2012)
TI - DENSE PIXEL MATCHING BETWEEN UNRECTIFIED AND DISTORTED IMAGES USING DYNAMIC PROGRAMMING
SN - 978-989-8565-04-4
AU - Thevenon J.
AU - Martinez-del-Rincon J.
AU - Dieny R.
AU - Nebel J.
PY - 2012
SP - 216
EP - 224
DO - 10.5220/0003812602160224