REALISTIC 3D SCENE RECONSTRUCTION FROM UNSCONSTRAINED AND UNCALIBRATED IMAGES TAKEN WITH A HANDHELD CAMERA

Minh Hoang Nguyen, Burkhard Wünsche, Patrice Delmas, Christof Lutteroth

2011

Abstract

We address the problem of reconstructing 3D scenes from a set of unconstrained images. These image sequences can be acquired by a video camera or handheld digital camera without requiring calibration. Our approach does not require any a priori information about the cameras being used. The camera's motion and intrinsic parameters are all unknown. We use a novel combination of advanced computer vision algorithms for feature detection, feature matching, and projection matrix estimation in order to reconstruct a 3D point cloud representing the location of geometric features estimated from input images. In a second step a full 3D model is reconstructed using the projection matrix and a triangulation process. We demonstrate with data sets of different structures obtained under different weather conditions that our algorithm is stable and enables inexperienced users to easily create complex 3D content using a simple consumer level camera.

References

  1. Yang. R and Wünsche. B. (2010). Life-Sketch - A Framework for Sketch-Based Modelling and Animation of 3D Objects. AUIC 7810 Proceedings of the Eleventh Australasian Conference on User Interface, Volume 106, pp. 1-9.
  2. Yang. R and Wünsche. B. (2010). Life-Sketch - A Framework for Sketch-Based Modelling and Animation of 3D Objects. AUIC 7810 Proceedings of the Eleventh Australasian Conference on User Interface, Volume 106, pp. 1-9.
  3. Hu. S, Qiao. J, Zhang. A and Huang. Q. (2008). 3D Reconstruction from Image sequence taken with a handheld camera. International Society for Photogrammetry and Remote Sensing, Vol 37, pp. 1-4.
  4. Hu. S, Qiao. J, Zhang. A and Huang. Q. (2008). 3D Reconstruction from Image sequence taken with a handheld camera. International Society for Photogrammetry and Remote Sensing, Vol 37, pp. 1-4.
  5. Zhang. J, Boutin. M, Aliaga. D. G. Robust Bundle Adjustment for structure from motion. Image Processing, 2006 IEEE International Conference, pp. 2185-2188.
  6. Zhang. J, Boutin. M, Aliaga. D. G. Robust Bundle Adjustment for structure from motion. Image Processing, 2006 IEEE International Conference, pp. 2185-2188.
  7. Snavely. N, Seitz. M. S, Szeliski. R. (2006). Photo tourism: Exploring photo collections in 3D. ACM Transactions on Graphics - SIGGRAPH Proceedings, 25(3), 2006, pp. 835-846.
  8. Snavely. N, Seitz. M. S, Szeliski. R. (2006). Photo tourism: Exploring photo collections in 3D. ACM Transactions on Graphics - SIGGRAPH Proceedings, 25(3), 2006, pp. 835-846.
  9. Debevec. P, Taylor. C and Malik. J. Modeling and Rendering Architecture From Photographs: A Hybrid Geometry and Image-Based Approach. In SIGGRAPH, 1996, pp. 11-20.
  10. Debevec. P, Taylor. C and Malik. J. Modeling and Rendering Architecture From Photographs: A Hybrid Geometry and Image-Based Approach. In SIGGRAPH, 1996, pp. 11-20.
  11. Criminisi. A, Reid. I and Zisserman. A. Single View Metrology. International Journal of Computer Vision (2000), pp. 123-148 .
  12. Criminisi. A, Reid. I and Zisserman. A. Single View Metrology. International Journal of Computer Vision (2000), pp. 123-148 .
  13. Hua. S and Liu.T. Realistic 3D Reconstruction from Two Uncalibrated Views. International Journal of Computer Science and Network Security, Volume 7 No.6 (2007), pp. 178-183.
  14. Hua. S and Liu.T. Realistic 3D Reconstruction from Two Uncalibrated Views. International Journal of Computer Science and Network Security, Volume 7 No.6 (2007), pp. 178-183.
  15. Amenta. N, Choi. S, Kolluri. R. K. The Power Crust. International Journal of Computational Geometry: Theory and Applications (2000), Volume 19, pp. 127- 153.
  16. Amenta. N, Choi. S, Kolluri. R. K. The Power Crust. International Journal of Computational Geometry: Theory and Applications (2000), Volume 19, pp. 127- 153.
  17. Kazhdan. M, Bolitho. M, Hoppe. H. Poisson Surface Reconstruction. ACM International Conference Proceeding Series; Vol. 256 ( 2006), pp. 185-192.
  18. Kazhdan. M, Bolitho. M, Hoppe. H. Poisson Surface Reconstruction. ACM International Conference Proceeding Series; Vol. 256 ( 2006), pp. 185-192.
  19. Triggs. B, McLauchlan. P. F, Hartley. R. I, Fitzgibbon. A. W. Bundle Adjustment - A Modern Synthesis. ICCV 7899: Proceedings of the International Workshop on Vision Algorithms. Springer-Verlag. pp. 298-372.
  20. Triggs. B, McLauchlan. P. F, Hartley. R. I, Fitzgibbon. A. W. Bundle Adjustment - A Modern Synthesis. ICCV 7899: Proceedings of the International Workshop on Vision Algorithms. Springer-Verlag. pp. 298-372.
  21. Brown. M, Lowe. D. G. Unsupervised 3D Object Recognition and Reconstruction in Unordered Datasets. 3-D Proceedings of the Fifth International Conference on Digital Imaging and Modelling, 2005. pp. 110-119.
  22. Brown. M, Lowe. D. G. Unsupervised 3D Object Recognition and Reconstruction in Unordered Datasets. 3-D Proceedings of the Fifth International Conference on Digital Imaging and Modelling, 2005. pp. 110-119.
  23. Kolmogorov, V and Zabih. R. Multi-camera Scene Reconstruction via Graph Cuts. In European Conference on Computer Vision (ECCV), May 2002. pp. 82-96.
  24. Kolmogorov, V and Zabih. R. Multi-camera Scene Reconstruction via Graph Cuts. In European Conference on Computer Vision (ECCV), May 2002. pp. 82-96.
  25. Derpanis. G. K. The Harris Corner Detector. 2004. URL: http://www.cse.yorku.ca/kosta/CompVis_ Notes/harris_detector.pdf
  26. Derpanis. G. K. The Harris Corner Detector. 2004. URL: http://www.cse.yorku.ca/kosta/CompVis_ Notes/harris_detector.pdf
  27. International Conference on Innovative Computing, Information and Control - Volume I (ICICIC'06), 2006. pp. 322-325.
  28. International Conference on Innovative Computing, Information and Control - Volume I (ICICIC'06), 2006. pp. 322-325.
  29. Lowe, D. G. Object recognition from local scale-invariant features. Proceedings of the International Conference on Computer Vision. 2. pp. 1150-1157. 1999.
  30. Lowe, D. G. Object recognition from local scale-invariant features. Proceedings of the International Conference on Computer Vision. 2. pp. 1150-1157. 1999.
  31. Choudhury. R. Recognizing pictures at an exhibition using SIFT. Biomedical Infomatics Department, Stanford University, USA. EE 368 Project Report, 2007. URL: http://www.stanford.edu/class/ee368/Project_07/report s/ee368group11.pdf
  32. Choudhury. R. Recognizing pictures at an exhibition using SIFT. Biomedical Infomatics Department, Stanford University, USA. EE 368 Project Report, 2007. URL: http://www.stanford.edu/class/ee368/Project_07/report s/ee368group11.pdf
  33. Brown. M, Szeliski. R and Winder. S. Multi-Image Matching using Multi-Scale Oriented Patches. In Proceedings of the Interational Conference on Computer Vision and Pattern Recognition, San Diego, June 2005. Vol 1, pp. 510-517.
  34. Brown. M, Szeliski. R and Winder. S. Multi-Image Matching using Multi-Scale Oriented Patches. In Proceedings of the Interational Conference on Computer Vision and Pattern Recognition, San Diego, June 2005. Vol 1, pp. 510-517.
  35. Hartley. R and Zissweman. A. MultipleView Geometry in computer vision. Cambridge Press. 2003.
  36. Hartley. R and Zissweman. A. MultipleView Geometry in computer vision. Cambridge Press. 2003.
  37. Kazhdan. M, M. Bolitho, H. Hoppe. Poisson Surface reconstruction. Symposium on Geometry Processing 2006, pp. 61-70.
  38. Kazhdan. M, M. Bolitho, H. Hoppe. Poisson Surface reconstruction. Symposium on Geometry Processing 2006, pp. 61-70.
  39. Hoppe. H, DeRose. T, Duchamp, T. McDonald, J. Stuetzle. W. Surface reconstruction from unorganized points. ACM SIGGRAPH 1992 Conference Proceedings, pp. 71-78.
  40. Hoppe. H, DeRose. T, Duchamp, T. McDonald, J. Stuetzle. W. Surface reconstruction from unorganized points. ACM SIGGRAPH 1992 Conference Proceedings, pp. 71-78.
  41. Hoppe. H, DeRose. T, Duchamp, T. McDonald, J. Stuetzle. W.. Mesh optimization. ACM SIGGRAPH 1993 Conference Proceedings, pp. 19-26.
  42. Hoppe. H, DeRose. T, Duchamp, T. McDonald, J. Stuetzle. W.. Mesh optimization. ACM SIGGRAPH 1993 Conference Proceedings, pp. 19-26.
  43. Edelsbrunner. H and Mucke. E. P. Three-dimensional alpha shapes. ACM Trans. Graphics 13 (1994), pp. 43- 72.
  44. Edelsbrunner. H and Mucke. E. P. Three-dimensional alpha shapes. ACM Trans. Graphics 13 (1994), pp. 43- 72.
  45. Edelsbrunner. H. Surface reconstruction by wrapping finite point sets in space. Discrete and Computational Geometry. The Goodman-Pollack Festschrift, ed. B. Aronov, S. Basu, J. Pach and M. Sharir, SpringerVerlag, 2003, pp. 379-404.
  46. Edelsbrunner. H. Surface reconstruction by wrapping finite point sets in space. Discrete and Computational Geometry. The Goodman-Pollack Festschrift, ed. B. Aronov, S. Basu, J. Pach and M. Sharir, SpringerVerlag, 2003, pp. 379-404.
  47. Cheng. H.L, Dey. T.K, Edelsbrunner. H and Sullivan. J. Dynamic skin triangulation. Discrete Comput. Geom. 25 (2001), pp. 525-568.
  48. Cheng. H.L, Dey. T.K, Edelsbrunner. H and Sullivan. J. Dynamic skin triangulation. Discrete Comput. Geom. 25 (2001), pp. 525-568.
Download


Paper Citation


in Harvard Style

Nguyen M., Wünsche B., Delmas P. and Lutteroth C. (2011). REALISTIC 3D SCENE RECONSTRUCTION FROM UNSCONSTRAINED AND UNCALIBRATED IMAGES TAKEN WITH A HANDHELD CAMERA . In Proceedings of the International Conference on Computer Graphics Theory and Applications - Volume 1: GRAPP, (VISIGRAPP 2011) ISBN 978-989-8425-45-4, pages 67-75. DOI: 10.5220/0003376900670075


in Harvard Style

Nguyen M., Wünsche B., Delmas P. and Lutteroth C. (2011). REALISTIC 3D SCENE RECONSTRUCTION FROM UNSCONSTRAINED AND UNCALIBRATED IMAGES TAKEN WITH A HANDHELD CAMERA . In Proceedings of the International Conference on Computer Graphics Theory and Applications - Volume 1: GRAPP, (VISIGRAPP 2011) ISBN 978-989-8425-45-4, pages 67-75. DOI: 10.5220/0003376900670075


in Bibtex Style

@conference{grapp11,
author={Minh Hoang Nguyen and Burkhard Wünsche and Patrice Delmas and Christof Lutteroth},
title={REALISTIC 3D SCENE RECONSTRUCTION FROM UNSCONSTRAINED AND UNCALIBRATED IMAGES TAKEN WITH A HANDHELD CAMERA},
booktitle={Proceedings of the International Conference on Computer Graphics Theory and Applications - Volume 1: GRAPP, (VISIGRAPP 2011)},
year={2011},
pages={67-75},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003376900670075},
isbn={978-989-8425-45-4},
}


in Bibtex Style

@conference{grapp11,
author={Minh Hoang Nguyen and Burkhard Wünsche and Patrice Delmas and Christof Lutteroth},
title={REALISTIC 3D SCENE RECONSTRUCTION FROM UNSCONSTRAINED AND UNCALIBRATED IMAGES TAKEN WITH A HANDHELD CAMERA},
booktitle={Proceedings of the International Conference on Computer Graphics Theory and Applications - Volume 1: GRAPP, (VISIGRAPP 2011)},
year={2011},
pages={67-75},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003376900670075},
isbn={978-989-8425-45-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Computer Graphics Theory and Applications - Volume 1: GRAPP, (VISIGRAPP 2011)
TI - REALISTIC 3D SCENE RECONSTRUCTION FROM UNSCONSTRAINED AND UNCALIBRATED IMAGES TAKEN WITH A HANDHELD CAMERA
SN - 978-989-8425-45-4
AU - Nguyen M.
AU - Wünsche B.
AU - Delmas P.
AU - Lutteroth C.
PY - 2011
SP - 67
EP - 75
DO - 10.5220/0003376900670075


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Computer Graphics Theory and Applications - Volume 1: GRAPP, (VISIGRAPP 2011)
TI - REALISTIC 3D SCENE RECONSTRUCTION FROM UNSCONSTRAINED AND UNCALIBRATED IMAGES TAKEN WITH A HANDHELD CAMERA
SN - 978-989-8425-45-4
AU - Nguyen M.
AU - Wünsche B.
AU - Delmas P.
AU - Lutteroth C.
PY - 2011
SP - 67
EP - 75
DO - 10.5220/0003376900670075