Structure from Motion - ToF-aided 3D Reconstruction of Isometric Surfaces

S. Jafar Hosseini, Helder Araújo

2014

Abstract

This paper deals with structure-from-motion (SfM) for non-rigid surfaces that undergo isometric motion. Our SfM framework aims at the joint estimation of the 3D surface and the camera motion by combining a ToF range sensor and a monocular RGB camera through a template-based approach. Our goal is to use the 2D low-resolution depth estimates provided by the TOF camera, in order to facilitate the estimation of non-rigid structure using the high-resolution images obtained by means of a RGB camera. In this paper, we model isometric surfaces with a triangular mesh. The ToF sensor is used to obtain the depth of a sparse set of 3D feature points, from which the depth of the mesh vertices can be recovered using a multivariate linear system. Subsequently, we form a non-linear constraint based on the projected length of each edge. A second non-linear constraint is then used for minimizing re-projection errors. These constraints are finally incorporated into an optimization scheme to solve for structure and motion. Experimental results show that the proposed approach has good performance even if only a low-resolution depth image is used.

References

  1. Aans, H. and Kahl, F. (2002). Estimation of deformable structure and motion. Workshop on Vision and Modelling of Dynamic Scenes, ECCV, Denmark.
  2. Akhter, I., Sheikh, Y., and Khan, S. (2009). In defense of orthonormality constraints for nonrigid structure from motion. pages 1534-1541. CVPR.
  3. Dai, Y., Li, H., and He, M. (2012). A simple prior-free method for non-rigid structure-from-motion factorization. pages 2018-2025. CVPR.
  4. Del-Bue, A., Llad, X., and Agapito, L. (2006). Non-rigid metric shape and motion recovery from uncalibrated images using priors. IEEE Conference on Computer Vision and Pattern Recognition, New York.
  5. Diebel, J. and Thrun, S. (2005). An application of markov random fields to range sensing. Proc. NIPS.
  6. Gay-Bellile, V., Perriollat, M., Bartoli, A., and Sayd, P. (2006). Image registration by combining thin-plate splines with a 3d morphable model. International Conference on Image Processing.
  7. Gumerov, N., Zandifar, A., Duraiswami, R., and Davis, L. (2004). Structure of applicable surfaces from single views. European Conference on Computer Vision.
  8. Kim, H., Tai, Y.-W., and Brown, M. (2011). High quality depth map upsampling for 3d-tof cameras. pages 1623 - 1630. Inso Kweon Computer Vision (ICCV), IEEE International Conference, Barcelona.
  9. Kim, Y., Theobalt, C., Diebel, J., Kosecka, J., Miscusik, B., and Thrun, S. (2009). Multi-view image and tof sensor fusion for dense 3d reconstruction. pages 1542-1549. Computer Vision Workshops (ICCV Workshops).
  10. Llado, X., Bue, A., and Agapito, L. (2005). Non-rigid 3d factorization for projective reconstruction. BMVC.
  11. Metaxas, D. and Terzopoulos, D. (1993). Constrained deformable superquadrics and nonrigid motion tracking. PAMI 15, pages 580-591.
  12. Paladini, M., Bue, A., Stosic, M., Dodig, M., Xavier, J., and Agapito, L. (2009). Factorization for non-rigid and articulated structure using metric projections. page 28982905. Proc. IEEE Conf. on Computer Vision and Pattern Recognition.
  13. Perriollat, M., Hartley, R., and Bartoli (2010). Monocular template-based reconstruction of inextensible surfaces. International Journal of Computer Vision.
  14. Prasad, M., Zisserman, A., and Fitzgibbon, A. (2006). Single view reconstruction of curved surfaces. pages 1345-1354. IEEE Conference on Computer Vision and Pattern Recognition.
  15. Salzmann, M. and Fua, P. (2007). Reconstructing sharply folding surfaces: A convex formulation. IEEE Conference on Computer Vision and Pattern Recognition.
  16. Salzmann, M., Hartley, R., and Fua, P. (2007). Convex optimization for deformable surface 3-d tracking. IEEE International Conference on Computer Vision.
  17. Salzmann, M., Moreno-Noguer, F., Lepetit, V., and Fua, P. (2008). Closed-form solution to non-rigid 3d surface registration. pages 581-594. European Conference on Computer Vision.
  18. Shen, S., Shi, W., and Liu, Y. (2010). Monocular 3-d tracking of inextensible deformable surfaces under l2- norm. IEEE Transactions on Image Processing 19, pages 512-521.
  19. Srivastava, S., Saxena, A., Theobalt, C., and Thrun, S. (2009). Rapid interactive 3d reconstruction from a single image. In VMV, pages 19-28.
  20. Torresani, L., Hertzmann, A., and Bregler, C. (2003). Learning non-rigid 3d shape from 2d motion. NIPS, pages 580-591.
  21. White, R. and Forsyth, D. (2006). Combining cues: Shape from shading and texture. CVPR.
  22. Xiao, J., x. Chai, J., and Kanade, T. (2004). A closedform solution to non-rigid shape and motion recovery. pages 573-587. ECCV.
  23. Yang, R., Davis, J., and Nister, D. (2007). Spatial-depth super resolution for range images. pages 1-8. Computer Vision and Pattern Recognition, CVPR 7807. IEEE Conference, Minneapolis, MN.
  24. Zhou, H., Li, X., and Sadka, A. (2012). Nonrigid structurefrom-motion from 2-d images using markov chain monte carlo. 14(1):168-177.
Download


Paper Citation


in Harvard Style

Hosseini S. and Araújo H. (2014). Structure from Motion - ToF-aided 3D Reconstruction of Isometric Surfaces . In Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-018-5, pages 544-552. DOI: 10.5220/0004787505440552


in Bibtex Style

@conference{icpram14,
author={S. Jafar Hosseini and Helder Araújo},
title={Structure from Motion - ToF-aided 3D Reconstruction of Isometric Surfaces},
booktitle={Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2014},
pages={544-552},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004787505440552},
isbn={978-989-758-018-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Structure from Motion - ToF-aided 3D Reconstruction of Isometric Surfaces
SN - 978-989-758-018-5
AU - Hosseini S.
AU - Araújo H.
PY - 2014
SP - 544
EP - 552
DO - 10.5220/0004787505440552