Multispectral Data Fusion for Robotic Reconnaissance and Mapping

Petra Kocmanova, Ludek Zalud, Frantisek Burian, Tomas Jilek

2014

Abstract

The aim of the paper is to describe the data-fusion from optical sensors for mobile robotics reconnaissance and mapping. Data are acquired by stereo pair of CCD cameras, stereo pair of thermal imagers, and TOF (time-of-flight) camera. The fusion is realized by means of spatial data from a TOF camera to ensure "natural" representation of a robot’s environment; thus, the thermal and CCD camera data are comprised in one stereo image presented to a binocular, head-mounted display. The data acquisition is performed using a sensor head, which is placed on an Orpheus-X3 robot; both the head and the robot were developed by our working group. After the geometrical calibration of each sensor, the positions of the sensors in 6DOFs are computed. The corresponding data from the CCD camera and the thermal imager are subsequently determined via homogeneous and perspective transformations. The result consists in an image containing aligned data from the CCD camera and the thermal imager for each eye. TOF camera calibration and its impact to the precision of fusion is described. Although the fusion is used for two different tasks – automatic environment mapping and visual telepresence, the utilised calibration and fusion algorithms are, in principle, the same.

References

  1. P. Kocmanova and L. Zalud, “Spatial Calibration of TOF Camera, Thermal Imager and CCD Camera”. In: Mendel 2013: 19th International Conference on Soft Computing. Brno: Brno University of Technology, Fakulty of Mechanical Engineering, 2013, s. 343-348. ISBN 978-80-214-4755-4.
  2. SR4000 Data Sheet, MESA Imaging AG. Rev. 5.1, 2011.
  3. T. Luhmann, J. Ohm, J. Piechel and T. Roelfs, “Geometric Calibration of Thermographic Cameras”. In: International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences 2010, vol. XXXVIII, pp. 411-416.
  4. X. Ju, J.-C. Nebel, J. P. Siebert, H. Gong, Y. Cai and J.-P. Chatard,” 3D thermography imaging standardization technique for inflammation diagnosis”. Proceedings of the SPIE. 2005, vol. 5640, pp. 266-273.
  5. S. Prakash, P. Y. Lee and T. Caelli,” 3D Mapping of Surface Temperature Using Thermal Stereo”. 2006 9th International Conference on Control, Automation, Robotics and Vision. IEEE, 2006, pp. 1-4.
  6. Z. Zhang,”Flexible camera calibration by viewing a plane from unknown orientations”. Proceedings of the Seventh IEEE International Conference on Computer Vision. IEEE, 1999, vol. 1, pp. 666-673.
  7. L. Zalud and P. Kocmanova,”Fusion of Thermal Imaging and CCD Camera-based Data for Stereovision Visual Telepresence”.In: SSRR 2013: 11th IEEE International Symposium on Safety Security and Rescue Robotics, pp.1-6.
  8. L. Zalud, F. Burian, L. Kopecny, P. Kocmanova (2013). Remote Robotic Exploration of Contaminated and Dangerous Areas, International Conference on Military Technologies, pp 525-532, Brno, Czech Republic, ISBN 978-80-7231-917-6.
  9. L. Zalud, (2005). ORPHEUS - Reconniaissance Teleoperated Robotic System. In: 16th IFAC World Congress. pp. 1-6, Prague, Czech Republic.
  10. D. Lichti, D.-Rouzaud: Surface-dependent 3D range camera self-calibration. In: Proceedings of SPIE Three-Dimensional Imaging Metrology, Vol. 7239, 2009, pp. 72390A-1-72390A-10.
Download


Paper Citation


in Harvard Style

Kocmanova P., Zalud L., Burian F. and Jilek T. (2014). Multispectral Data Fusion for Robotic Reconnaissance and Mapping . In Proceedings of the 11th International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO, ISBN 978-989-758-040-6, pages 459-466. DOI: 10.5220/0005060504590466


in Bibtex Style

@conference{icinco14,
author={Petra Kocmanova and Ludek Zalud and Frantisek Burian and Tomas Jilek},
title={Multispectral Data Fusion for Robotic Reconnaissance and Mapping},
booktitle={Proceedings of the 11th International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO,},
year={2014},
pages={459-466},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005060504590466},
isbn={978-989-758-040-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 11th International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO,
TI - Multispectral Data Fusion for Robotic Reconnaissance and Mapping
SN - 978-989-758-040-6
AU - Kocmanova P.
AU - Zalud L.
AU - Burian F.
AU - Jilek T.
PY - 2014
SP - 459
EP - 466
DO - 10.5220/0005060504590466