Evaluation of the Fusion of Visible and Thermal Image Data for People Detection with a Trained People Detector

Achim Königs, Dirk Schulz

Abstract

People detection surely is one of the hottest topics in Computer Vision. In this work we propose and evaluate the fusion of thermal images and images from the visible spectrum for the task of people detection. Our main goal is to reduce the false positive rate of the Implicit Shape Model (ISM) object detector, which is commonly used for people detection. We describe five possible methods to integrate the thermal data into the detection process at different processing steps. Those five methods are evaluated on several test sets we recorded. Their performance is compared to three baseline detection approaches. The test sets contain data from an indoor environment and from outdoor environments at days with different ambient temperatures. The data fusion methods decrease the false positive rate especially on the outdoor test sets.

References

  1. Bay, H., Ess, A., Tuytelaars, T., and Van Gool, L. (2008). Surf: Speeded up robust features. Computer Vision and Image Understanding (CVIU), Vol. 110:pp. 346- 359.
  2. Cielniak, G. and Duckett, T. (2004). People recognition by mobile robots. Journal of Intelligent and Fuzzy Systems, 15:21-27.
  3. Dalal, N. and Triggs, B. (2005). Histograms of oriented gradients for human detection. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR).
  4. Davis, J. W. and Sharma, V. (2004). Robust backgroundsubtraction for person detection in thermal imagery. In In IEEE Int. Wkshp. on Object Tracking and Classification Beyond the Visible Spectrum.
  5. Enzweiler, M. and Gavrila, D. (2009). Monocular pedestrian detection: Survey and experiments. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 31:2179-2195.
  6. Gero andnimo, D., Lo andpez, A., Sappa, A., and Graf, T. (2010). Survey of pedestrian detection for advanced driver assistance systems. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 32(7):1239 -1258.
  7. Goubet, E. (2006). Pedestrian tracking using thermal infrared imaging. Proceedings of SPIE.
  8. Han, J. and Bhanu, B. (2007). Fusion of color and infrared video for moving human detection. Pattern Recognition, 40(6):1771 - 1784.
  9. Ikemura, S. and Fujiyoshi, H. (2011). Real-time human detection using relational depth similarity features. In Kimmel, R., Klette, R., and Sugimoto, A., editors, Computer Vision ACCV 2010, volume 6495 of Lecture Notes in Computer Science, pages 25-38. Springer Berlin Heidelberg.
  10. Königs, A. and Schulz, D. (2012). Evaluation of thermal imaging for people detection in outdoor scenarios. In IEEE International Workshop on Safety, Security & Rescue Robotics (SSRR).
  11. Lampert, C. H., Blaschko, M. B., and Hofmann, T. (2008). Beyond sliding windows: Object localization by efficient subwindow search. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR).
  12. Lehmann, A., Leibe, B., and van Gool, L. (2009). Featurecentric efficient subwindow search. In IEEE Conf. on Computer Vision (ICCV).
  13. Lehmann, A., Leibe, B., and Van Gool, L. (2010). Fast PRISM: Branch and bound hough transform for object class detection. Int. Journal of Computer Vision (IJCV), pages 1-23.
  14. Leibe, B., Leonardis, A., and Schiele, B. (2008). Robust object detection with interleaved categorization and segmentation. Int. Journal of Computer Vision (IJCV), Vol. 77:pp. 259-289.
  15. Leibe, B., Seemann, E., and Schiele, B. (2005). Pedestrian detection in crowded scenes. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR).
  16. Li, J., Gong, W., Li, W., and Liu, X. (2010). Robust pedestrian detection in thermal infrared imagery using the wavelet transform. Infrared Physics and Technology, 53.
  17. Lietz, H., Thomanek, J., Fardi, B., and Wanielik, G. (2009). Improvement of the classifier performance of a pedestrian detection system by pixel-based data fusion. In Serra, R. and Cucchiara, R., editors, AI*IA 2009: Emergent Perspectives in Artificial Intelligence, volume 5883 of Lecture Notes in Computer Science, pages 122-130. Springer Berlin Heidelberg.
  18. Miezianko, R. and Pokrajac, D. (2008). People detection in low resolution infrared videos. In Computer Vision and Pattern Recognition Workshops, 2008. CVPRW 7808. IEEE Computer Society Conference on.
  19. San-Biagio, M., Crocco, M., Cristani, M., Martelli, S., and Murino, V. (2012). Low-level multimodal integration on riemannian manifolds for automatic pedestrian detection. In Information Fusion (FUSION), 2012 15th International Conference on, pages 2223 -2229.
  20. Schiele, B., Andriluka, M., Majer, N., Roth, S., and Wojek, C. (2009). Visual people detection: Different models, comparison and discussion. In ICRA Workshop People Detection and Tracking.
  21. Serrano-Cuerda, J., Lopez, M., and Fernandez-Caballero, A. (2011). Robust human detection and tracking in intelligent environments by information fusion of color and infrared video. In Int. Conf. on Intelligent Environments.
  22. Spinello, L. and Arras, K. (2012). Leveraging rgb-d data: Adaptive fusion and domain adaptation for object detection. In Robotics and Automation (ICRA), 2012 IEEE International Conference on, pages 4469 -4474.
  23. Suzuki, S. and Abe, K. (1985). Topological structural analysis of digitized binary images by border following. CVGIP.
  24. Zhang, L., Wu, B., and Nevatia, R. (2007). Pedestrian detection in infrared images based on local shape features. In Computer Vision and Pattern Recognition, 2007. CVPR 7807. IEEE Conference on.
Download


Paper Citation


in Harvard Style

Königs A. and Schulz D. (2013). Evaluation of the Fusion of Visible and Thermal Image Data for People Detection with a Trained People Detector . In Proceedings of the 10th International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO, ISBN 978-989-8565-71-6, pages 345-352. DOI: 10.5220/0004484203450352


in Bibtex Style

@conference{icinco13,
author={Achim Königs and Dirk Schulz},
title={Evaluation of the Fusion of Visible and Thermal Image Data for People Detection with a Trained People Detector},
booktitle={Proceedings of the 10th International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO,},
year={2013},
pages={345-352},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004484203450352},
isbn={978-989-8565-71-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 10th International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO,
TI - Evaluation of the Fusion of Visible and Thermal Image Data for People Detection with a Trained People Detector
SN - 978-989-8565-71-6
AU - Königs A.
AU - Schulz D.
PY - 2013
SP - 345
EP - 352
DO - 10.5220/0004484203450352