TIME-OF-FLIGHT BASED SCENE RECONSTRUCTION WITH A MESH PROCESSING TOOL FOR MODEL BASED CAMERA TRACKING

Svenja Kahn, Harald Wuest, Dieter W. Fellner

Abstract

The most challenging algorithmical task for markerless Augmented Reality applications is the robust estimation of the camera pose. With a given 3D model of a scene the camera pose can be estimated via model-based camera tracking without the need to manipulate the scene with fiducial markers. Up to now, the bottleneck of model-based camera tracking is the availability of such a 3D model. Recently time-of-flight cameras were developed which acquire depth images in real time. With a sensor fusion approach combining the color data of a 2D color camera and the 3D measurements of a time-of-flight camera we acquire a textured 3D model of a scene. We propose a semi-manual reconstruction step in which the alignment of several submeshes with a mesh processing tool is supervised by the user to ensure a correct alignment. The evaluation of our approach shows its applicability for reconstructing a 3D model which is suitable for model-based camera tracking even for objects which are difficult to measure reliably with a time-of-flight camera due to their demanding surface characteristics.

References

  1. Becker, M., Bleser, G., Pagani, A., Stricker, D., and Wuest, H. (2007). An architecture for prototyping and application development of visual tracking systems. In Capture, Transmission and Display of 3D Video (Proceedings of 3DTV-CON 07 [CD-ROM]).
  2. Besl, P. and McKay, N. (1992). A method for registration of 3-d shapes. In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 14(2), pages 239- 256.
  3. Bleser, G., Wuest, H., and Stricker, D. (2006). Online camera pose estimation in partially known and dynamic scenes. In ISMAR 2006: Proceedings of the Fifth IEEE and ACM International Symposium on Mixed and Augmented Reality, pages 56-65.
  4. Cignoni, P., Callieri, M., Corsini, M., Dellepiane, M., Ganovelli, F., and Ranzuglia, G. (2008). Meshlab: an open-source mesh processing tool. In Sixth Eurographics Italian Chapter Conference, pages 129-136.
  5. Durand, F. and Dorsey, J. (2002). Fast bilateral filtering for the display of high-dynamic-range images. In SIGGRAPH 7802: Proceedings of the 29th annual conference on Computer graphics and interactive techniques, pages 257-266.
  6. Huhle, B., Jenke, P., and Stra├čer, W. (2008). On-the-fly scene acquisition with a handy multi-sensor system. International Journal of Intelligent Systems Technologies and Applications (IJISTA), 5(3/4):255-263.
  7. Kolb, A., Barth, E., Koch, R., and Larsen, R. (2009). Timeof-Flight Sensors in Computer Graphics. In Proc. Eurographics (State-of-the-Art Report).
  8. Lepetit, V. and Fua, P. (2005). Monocular model-based 3d tracking of rigid objects: A survey. In Foundations and Trends in Computer Graphics and Vision, volume 1, pages 1-89.
  9. May, S., Droeschel, D., Holz, D., Wiesen, C., and Fuchs, S. (2008). 3d pose estimation and mapping with time-offlight cameras. In IEEE/RS International Conference on Intelligent Robots and Systems(IROS), Workshop on 3D-Mapping.
  10. Oggier, T., Lustenberger, F., and Blanc, N. (2006). Miniature 3d tof camera for real-time imaging. In Perception and Interactive Technologies, pages 212-216.
  11. Prusak, A., Melnychuk, O., Roth, H., Schiller, I., and Koch, R. (2008). Pose estimation and map building with a time-of-flight camera for robot navigation. International Journal of Intelligent Systems Technologies and Applications, 5(3/4):334-364.
  12. Schiller, I., Bartczak, B., Kellner, F., and Koch, R. (2008a). Increasing realism and supporting content planning for dynamic scenes in a mixed reality system incorporating a time-of-flight camera. In Proceedings of the European Conference on Visual Media Production, CVMP, volume 5.
  13. Schiller, I., Beder, C., and Koch, R. (2008b). Calibration of a pmd camera using a planar calibration object together with a multi-camera setup. In The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, volume XXI. ISPRS Congress, pages 297-302.
  14. Shi, J. and Tomasi, C. (1994). Good features to track. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR'94), pages 593-600.
Download


Paper Citation


in Harvard Style

Kahn S., Wuest H. and Fellner D. (2010). TIME-OF-FLIGHT BASED SCENE RECONSTRUCTION WITH A MESH PROCESSING TOOL FOR MODEL BASED CAMERA TRACKING . In Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2010) ISBN 978-989-674-028-3, pages 302-309. DOI: 10.5220/0002820903020309


in Bibtex Style

@conference{visapp10,
author={Svenja Kahn and Harald Wuest and Dieter W. Fellner},
title={TIME-OF-FLIGHT BASED SCENE RECONSTRUCTION WITH A MESH PROCESSING TOOL FOR MODEL BASED CAMERA TRACKING},
booktitle={Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2010)},
year={2010},
pages={302-309},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002820903020309},
isbn={978-989-674-028-3},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2010)
TI - TIME-OF-FLIGHT BASED SCENE RECONSTRUCTION WITH A MESH PROCESSING TOOL FOR MODEL BASED CAMERA TRACKING
SN - 978-989-674-028-3
AU - Kahn S.
AU - Wuest H.
AU - Fellner D.
PY - 2010
SP - 302
EP - 309
DO - 10.5220/0002820903020309