Fast and Accurate Refinement Method for 3D Reconstruction from Stereo Spherical Images

Marek Solony, Evren Imre, Viorela Ila, Lukas Polok, Hansung Kim, Pavel Zemcik

2015

Abstract

Realistic 3D models of the environment are beneficial in many fields, from natural or man-made structure inspection and volumetric analysis, to movie-making, in particular, special effects integration to natural scenes. Spherical cameras are becoming popular in environment modelling because they capture the full surrounding scene visible from the camera location as a consistent seamless image at once. In this paper, we propose a novel pipeline to obtain fast and accurate 3D reconstructions from spherical images. In order to have a better estimation of the structure, the system integrates a joint camera pose and structure refinement step. This strategy proves to be much faster, yet equally accurate, when compared to the conventional method, registration of a dense point cloud via iterative closest point (ICP). Both methods require an initial estimate for successful convergence. The initial positions of the 3D points are obtained from stereo processing of pair of spherical images with known baseline. The initial positions of the cameras are obtained from a robust wide-baseline matching procedure. The performance and accuracy of the 3D reconstruction pipeline is analysed through extensive tests on several indoor and outdoor datasets.

References

  1. Agarwal, S., Snavely, N., Simon, I., Seitz, S. M., and Szeliski, R. (2009). Building rome in a day.
  2. Arun, K., Huang, T., and Blostein, S. (1987). Least square fitting of two 3-d point sets. IEEE Trans. Pattern Analysis and Machine Intelligence, 9(5):698-700.
  3. Beall, C., Lawrence, B., Ila, V., and Dellaert, F. (2010). 3D Reconstruction of Underwater Structures.
  4. Besl, P. and McKay, N. (1992). A method for registration of 3-D shapes. 14(2).
  5. Chum, O. and Matas, J. (2005). Matching with PROSAC - Progressive Sample Consensus. In Proc. CVPR, pages 220-226.
  6. Chum, O. and Matas, J. (2008). Optimal Randomized RANSAC. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(8):1472-1482.
  7. Chum, O., Matas, J., and Kittler, J. (2003). Locally Optimized RANSAC. In Lecture Notes in Computer Science, volume 2781, pages 236-243. Springer.
  8. Dellaert, F. and Kaess, M. (2006). Square Root SAM: Simultaneous localization and mapping via square root information smoothing. 25(12):1181-1203.
  9. Feldman, D. and Weinshall, D. (2005). Realtime ibr with omnidirectional crossed-slits projection. In Proc. ICCV, pages 839-845.
  10. Fischler, M. A. and Bolles, R. C. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381-395.
  11. Hartley, R. and Zisserman, A. (2003). Multiple View Geometry in Computer Vision. 2nd edition.
  12. Hong, J., Tan, X., Pinette, B., Weiss, R., and E.M., R. (1991). Image-based homing. In Proc. ICRA, pages 620-625.
  13. Horn, B. K. P. (1987). Closed-form Solution of Absolute Orientation Using Unit Quaternions. Journal of the Optical Society of America A, 4(4):629-642.
  14. ?Imre, E., Guillemaut, J.-Y., and Hilton, A. (2010). Moving Camera Registration for Multiple Camera Setups in Dynamic Scenes. In Proc. BMVC, pages 1-12.
  15. Jeong, Y., Nister, D., Steedly, D., Szeliski, R., and Kweon, I.-S. (2012). Pushing the envelope of modern methods for bundle adjustment. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 34(8):1605-1617.
  16. Kaess, M., Ranganathan, A., and Dellaert, F. (2008). iSAM: Incremental smoothing and mapping. 24(6):1365- 1378.
  17. Kim, H. and Hilton, A. (2013). 3d scene reconstruction from multiple spherical stereo pairs. International Journal of Computer Vision, 104(1):94-116.
  18. Kümmerle, R., Grisetti, G., Strasdat, H., Konolige, K., and Burgard, W. (2011). g2o: A general framework for graph optimization. In Proc. of the IEEE Int. Conf. on Robotics and Automation (ICRA), Shanghai, China.
  19. Lhuillier, M. (2008). Automatic scene structure and camera motion using a catadioptric system. Computer Vision and Image Understanding, 109(2):186-203.
  20. Lowe, D. (2004). Distinctive image features from scaleinvariant keypoints. 60(2):91-110.
  21. Nayar, S. (1997). Catadioptric omnidirectional camera. In Proc. CVPR, pages 482-488.
  22. Polok, L., Ila, V., Solony, M., Smrz, P., and Zemcik, P. (2013a). Incremental block cholesky factorization for nonlinear least squares in robotics. In Proceedings of the Robotics: Science and Systems 2013.
  23. Polok, L., Solony, M., Ila, V., Zemcik, P., and Smrz, P. (2013b). Efficient implementation for block matrix operations for nonlinear least squares problems in robotic applications. In Proceedings of the IEEE International Conference on Robotics and Automation. IEEE.
  24. Rusinkiewicz, S. and Levoy, M. (2001). Efficient variants of the icp algorithm.
  25. Rusu, R. B. and Cousins, S. (2011). 3D is here: Point Cloud Library (PCL). In Proc. ICRA.
  26. Scharstein, D. and Szeliski, R. (2002). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1):7-42.
  27. Snavely, N., Seitz, S. M., and Szeliski, R. (2006). Photo tourism: exploring photo collections in 3d. ACM transactions on graphics (TOG), 25(3):835-846.
  28. Torr, P. H. S. and Zisserman, A. (2000). MLESAC: A New Robust Estimator with Application to Estimating Image Geometry. Computer Vision and Image Understanding, 78(1):138-156.
  29. V.Fragoso and Turk, M. (2013). SWIGS: A Swift Guided Sampling Method. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 2770-2777, Portland, Oregon.
  30. Zhang, F. (2005). The Schur complement and its applications, volume 4. Springer.
Download


Paper Citation


in Harvard Style

Solony M., Imre E., Ila V., Polok L., Kim H. and Zemcik P. (2015). Fast and Accurate Refinement Method for 3D Reconstruction from Stereo Spherical Images . In Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2015) ISBN 978-989-758-091-8, pages 575-583. DOI: 10.5220/0005310805750583


in Bibtex Style

@conference{visapp15,
author={Marek Solony and Evren Imre and Viorela Ila and Lukas Polok and Hansung Kim and Pavel Zemcik},
title={Fast and Accurate Refinement Method for 3D Reconstruction from Stereo Spherical Images},
booktitle={Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2015)},
year={2015},
pages={575-583},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005310805750583},
isbn={978-989-758-091-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2015)
TI - Fast and Accurate Refinement Method for 3D Reconstruction from Stereo Spherical Images
SN - 978-989-758-091-8
AU - Solony M.
AU - Imre E.
AU - Ila V.
AU - Polok L.
AU - Kim H.
AU - Zemcik P.
PY - 2015
SP - 575
EP - 583
DO - 10.5220/0005310805750583