AR Visualization of Thermal 3D Model by Hand-held Cameras

Kazuki Matsumoto, Wataru Nakagawa, Hideo Saito, Maki Sugimoto, Takashi Shibata, Shoji Yachida

2015

Abstract

In this paper, we propose a system for AR visualization of thermal distribution on the environment. Our system is based on color 3D model and thermal 3D model of the target scene generated by KinectFusion using a thermal camera coupled with an RGB-D camera. In off-line phase, Viewpoint Generative Learning (VGL) is applied to the colored 3D model for collecting its stable keypoints descriptors. Those descriptors are utilized in camera pose initialization at the start of on-line phase. After that, our proposed camera tracking which combines frame-to-frame camera tracking with VGL based tacking is performed for accurate estimation of the camera pose. From estimated camera pose, the thermal 3D model is finally superimposed to current mobile camera view. As a result, we can observe the wide area thermal map from any viewpoint. Our system is applied for a temperature change visualization system with a thermal camera coupled with an RGB-D camera and it is also enables the smartphone to interactively display thermal distribution of a given scene.

References

  1. Arthur, D. and Vassilvitskii, S. (2007). k-means++: The advantages of careful seeding. In Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms, pages 1027-1035. Society for Industrial and Applied Mathematics.
  2. Borrmann, D., Nüchter, A., Dakulovic, M., Maurovic, I., Petrovic, I., Osmankovic, D., and Velagic, J. (2012). The project thermalmapper?thermal 3d mapping of indoor environments for saving energy. In Proceedings of the 10th International IFAC Symposium on Robot Control (SYROCO), volume 10, page 1.
  3. Demisse, G. G., Borrmann, D., and Nuchter, A. (2013). Interpreting thermal 3d models of indoor environments for energy efficiency. In Advanced Robotics (ICAR), 2013 16th International Conference on, pages 1-8. IEEE.
  4. Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. (2011). Kinectfusion: real-time 3d reconstruction and interaction using a moving depth camera. In Proceedings of the 24th annual ACM symposium on User interface software and technology, pages 559-568. ACM.
  5. Kandil, A., Hastak, M., and Dunston, P. S. (2014). Rapid 3d energy performance modeling of existing build-
  6. Lepetit, V., Moreno-Noguer, F., and Fua, P. (2009). Epnp: An accurate o (n) solution to the pnp problem. International journal of computer vision, 81(2):155-166.
  7. Nakagawa, W., Matsumoto, K., de Sorbier, F., Sugimoto, M., Saito, H., Senda, S., Shibata, T., and Iketani, A. (2014). Visualization of temperature change using rgb-d camera and thermal camera. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings. Springer.
  8. Saito, H., Honda, T., Nakayama, Y., and de Sorbier, F. (2014). Camera pose estimation for mixed and diminished reality in ftv. In 3DTV-Conference: The True Vision-Capture, Transmission and Display of 3D Video (3DTV-CON), 2014, pages 1-4. IEEE.
  9. Szabó, Z., Berg, S., Sjökvist, S., Gustafsson, T., Carleberg, P., Uppsäll, M., Wren, J., Ahn, H., and Smedby, O . (2013). Real-time intraoperative visualization of myocardial circulation using augmented reality temperature display. The international journal of cardiovascular imaging, 29(2):521-528.
  10. Thachasongtham, D., Yoshida, T., de Sorbier, F., and Saito, H. (2013). 3d object pose estimation using viewpoint generative learning. In Image Analysis, pages 512- 521. Springer.
  11. Vidas, S., Moghadam, P., and Bosse, M. (2013). 3d thermal mapping of building interiors using an rgb-d and thermal camera. In Robotics and Automation (ICRA), 2013 IEEE International Conference on, pages 2311- 2318. IEEE.
  12. Yanai, O. (2014). Thermal imaging as a smartphone application: exploring and implementing a new concept. In SPIE Defense+ Security, pages 90700M-90700M. International Society for Optics and Photonics.
  13. Yoshida, T., Saito, H., Shimizu, M., and Taguchi, A. (2013). Stable keypoint recognition using viewpoint generative learning. In VISAPP (2), pages 310-315.
  14. Zhang, Z. (2000). A flexible new technique for camera calibration. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 22(11):1330-1334.
Download


Paper Citation


in Harvard Style

Matsumoto K., Nakagawa W., Saito H., Sugimoto M., Shibata T. and Yachida S. (2015). AR Visualization of Thermal 3D Model by Hand-held Cameras . In Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2015) ISBN 978-989-758-091-8, pages 480-487. DOI: 10.5220/0005290904800487


in Bibtex Style

@conference{visapp15,
author={Kazuki Matsumoto and Wataru Nakagawa and Hideo Saito and Maki Sugimoto and Takashi Shibata and Shoji Yachida},
title={AR Visualization of Thermal 3D Model by Hand-held Cameras},
booktitle={Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2015)},
year={2015},
pages={480-487},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005290904800487},
isbn={978-989-758-091-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2015)
TI - AR Visualization of Thermal 3D Model by Hand-held Cameras
SN - 978-989-758-091-8
AU - Matsumoto K.
AU - Nakagawa W.
AU - Saito H.
AU - Sugimoto M.
AU - Shibata T.
AU - Yachida S.
PY - 2015
SP - 480
EP - 487
DO - 10.5220/0005290904800487