Luis Almeida, Paulo Menezes, Jorge Dias


Building 3D body models is an important task for virtual and augmented reality applications in telerehabilitation, education, 3DTV, entertainment and tele-presence. We propose a real-time full 3D reconstruction system that combines visual features and shape-based alignment using low cost depth sensor and video cameras targeting three-dimensional conferencing applications. With this approach we overcome the classic video based reconstruction problem in low-texture or repeated pattern regions. Alignment between successive frames is computed by jointly optimizing over both appearances and shape matching. Appearance-based alignment is done over 2D SURF features annotated with 3D position. Shape-based alignment is performed using the motion transformation estimation between consecutive annotated 3D point clouds through a linear method. A solution to avoid wrong annotated 3D matched points is proposed. 3D mesh model representation is used to lower the processed data and create a 3D representation that is independent of view-point.


  1. Akbarzadeh, A., Frahm, J.-M., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Merrell, P., Phelps, M., Sinha, S. N., Talton, B., Wang, L., Yang, Q., Stewénius, H., Yang, R., Welch, G., Towles, H., Nistér, D., and Pollefeys, M. (2006). Towards urban 3d reconstruction from video. In 3DPVT, pages 1-8. IEEE Computer Society.
  2. Aliakbarpour, H., Almeida, L., Menezes, P., and Dias, J. (2011). Multi-sensor 3d volumetric reconstruction using cuda. 3D Research, 2:1-14. 10.1007/3DRes.04(2011)6.
  3. Almeida, L., Menezes, P., Seneviratne, L., and Dias, J. (2011). Incremental 3d body reconstruction framework for robotic telepresence applications. In Robo 2011: The 2nd IASTED International Conference on Robotics, Pittsburgh, USA.
  4. Arun, K. S., Huang, T. S., and Blostein, S. D. (1987). Leastsquares fitting of two 3-d point sets. IEEE Trans. Pattern Anal. Mach. Intell., 9:698-700.
  5. Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., and MacIntyre, B. (2001). Recent advances in augmented reality. IEEE Comput. Graph. Appl., 21:34- 47.
  6. Bailenson, J., Patel, K., Nielsen, A., Bajscy, R., Jung, S.-H., and Kurillo, G. (2008). The Effect of Interactivity on Learning Physical Actions in Virtual Reality. Media Psychology, 11(3):354-376.
  7. Bay, H., Tuytelaars, T., and Gool, L. V. (2006). Surf: Speeded up robust features. In In ECCV, pages 404- 417.
  8. Besl, P. J. and McKay, N. D. (1992). A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell., 14:239-256.
  9. Challis, J. (1995). A procedure for determining rigid body transformation parameters. Journal of Biomechanics, 28(6):733-737.
  10. Eggert, D. W., Lorusso, A., and Fisher, R. B. (1997). Estimating 3d rigid body transformations: a comparison of four major algorithms. Machine Vision and Applications, 9:272-290.
  11. Fischler, M. A. and Bolles, R. C. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM, 24:381-395.
  12. Henry, P., Krainin, M., Herbst, E., Ren, X., and Fox, D. (2010). RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments. In RSS Workshop on Advanced Reasoning with Depth Cameras.
  13. Isgro, F., Trucco, E., Kauff, P., and Schreer, O. (2004). Three-dimensional image processing in the future of immersive media. Circuits and Systems for Video Technology, IEEE Transactions on, 14(3):288 - 303.
  14. Jung, S.-H. and Bajcsy, R. (2006). A framework for constructing real-time immersive environments for training physical activities. Journal of Multimedia, 1(7):9- 17.
  15. Konolige, K. and Agrawal, M. (2008). Frameslam: From bundle adjustment to real-time visual mapping. Robotics, IEEE Transactions on, 24(5):1066 -1077.
  16. Kurillo, G., Koritnik, T., Bajd, T., and Bajcsy, R. (2011). Real-time 3d avatars for tele-rehabilitation in virtual reality. Stud Health Technol Inform, 163:290-6.
  17. Kurillo, G., Vasudevan, R., Lobaton, E., and Bajcsy, R. (2008). A framework for collaborative real-time 3d teleimmersion in a geographically distributed environment. In Multimedia, 2008. ISM 2008. Tenth IEEE International Symposium on, pages 111 -118.
  18. Lange, B., Requejo, P., Flynn, S., Rizzo, A., Valero-Cuevas, F., Baker, L., and Winstein, C. (2010). The potential of virtual reality and gaming to assist successful aging with disability. Physical Medicine and Rehabilitation Clinics of North America, 21(2):339 - 356.
  19. Lanier, J. (2001). Virtually there. j-SCI-AMER, 284(4):66- 75.
  20. Lowe, D. G. (2004). Distinctive image features from scaleinvariant keypoints. Int. J. Comput. Vision, 60:91- 110.
  21. May, S., Droeschel, D., Holz, D., Fuchs, S., Malis, E., Nüchter, A., and Hertzberg, J. (2009). Threedimensional mapping with time-of-flight cameras. J. Field Robot., 26:934-965.
  22. Menezes, P., Lerasle, F., and Dias, J. (2011). Towards human motion capture from a camera mounted on a mobile robot. IVC, 29(6):382-393.
  23. Mirisola, L. G. B., Lobo, J., and Dias, J. (2007). 3d map registration using vision/laser and inertial sensing. In EMCR.
  24. Nahrstedt, K., Yang, Z., Wu, W., Arefin, M. A., and Rivas, R. (2011). Next generation session management for 3d teleimmersive interactive environments. Multimedia Tools Appl., 51(2):593-623.
  25. Petit, B., Lesage, J.-D., Franco, J.-S., Boyer, E., and Raffin, B. (2008). Grimage: 3d modeling for remote collaboration and telepresence. In ACM Symposium on Virtual Reality Software and Technology.
  26. Rizzo, A. A. and Kim, G. J. (2005). A swot analysis of the field of virtual rehabilitation and therapy. Presence, 14(2):119-146.
  27. Shi, J. and Tomasi, C. (1994). Good features to track. In Computer Vision and Pattern Recognition, 1994. Proceedings CVPR 7894., 1994 IEEE Computer Society Conference on, pages 593 -600.
  28. Steuer, J. (1992). Defining virtual reality: Dimensions determining telepresence. JOURNAL OF COMMUNICATION, 42:73-93.
  29. Viola, P. and Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, volume 1, pages I-511 - I-518 vol.1.

Paper Citation

in Harvard Style

Almeida L., Menezes P. and Dias J. (2012). ON-LINE 3D BODY MODELLING FOR AUGMENTED REALITY . In Proceedings of the International Conference on Computer Graphics Theory and Applications and International Conference on Information Visualization Theory and Applications - Volume 1: GRAPP, (VISIGRAPP 2012) ISBN 978-989-8565-02-0, pages 472-479. DOI: 10.5220/0003866304720479

in Bibtex Style

author={Luis Almeida and Paulo Menezes and Jorge Dias},
booktitle={Proceedings of the International Conference on Computer Graphics Theory and Applications and International Conference on Information Visualization Theory and Applications - Volume 1: GRAPP, (VISIGRAPP 2012)},

in EndNote Style

JO - Proceedings of the International Conference on Computer Graphics Theory and Applications and International Conference on Information Visualization Theory and Applications - Volume 1: GRAPP, (VISIGRAPP 2012)
SN - 978-989-8565-02-0
AU - Almeida L.
AU - Menezes P.
AU - Dias J.
PY - 2012
SP - 472
EP - 479
DO - 10.5220/0003866304720479