Visual Perception of Obstacles: Do Humans and Machines Focus on the Same Image Features?

Constantinos Kyriakides, Marios Thoma, Marios Thoma, Zenonas Theodosiou, Zenonas Theodosiou, Harris Partaourides, Loizos Michael, Loizos Michael, Andreas Lanitis, Andreas Lanitis

2024

Abstract

Contemporary cities are fractured by a growing number of barriers, such as on-going construction and infrastructure damages, which endanger pedestrian safety. Automated detection and recognition of such barriers from visual data has been of particular concern to the research community in recent years. Deep Learning (DL) algorithms are now the dominant approach in visual data analysis, achieving excellent results in a wide range of applications, including obstacle detection. However, explaining the underlying operations of DL models remains a key challenge in gaining significant understanding on how they arrive at their decisions. The use of heatmaps that highlight the focal points in input images that helped the models reach their predictions has emerged as a form of post-hoc explainability for such models. In an effort to gain insights into the learning process of DL models, we studied the similarities between heatmaps generated by a number of architectures trained to detect obstacles on sidewalks in images collected via smartphones, and eye-tracking heatmaps generated by humans as they detect the corresponding obstacles on the same data. Our findings indicate that the focus points of humans more closely align with those of a Vision Transformer architecture, as opposed to the other network architectures we examined in our experiments.

Paper Citation

in Harvard Style

Kyriakides C., Thoma M., Theodosiou Z., Partaourides H., Michael L. and Lanitis A. (2024). Visual Perception of Obstacles: Do Humans and Machines Focus on the Same Image Features?. In Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP; ISBN 978-989-758-679-8, SciTePress, pages 357-364. DOI: 10.5220/0012453500003660

in Bibtex Style

@conference{visapp24,
author={Constantinos Kyriakides and Marios Thoma and Zenonas Theodosiou and Harris Partaourides and Loizos Michael and Andreas Lanitis},
title={Visual Perception of Obstacles: Do Humans and Machines Focus on the Same Image Features?},
booktitle={Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP},
year={2024},
pages={357-364},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012453500003660},
isbn={978-989-758-679-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP
TI - Visual Perception of Obstacles: Do Humans and Machines Focus on the Same Image Features?
SN - 978-989-758-679-8
AU - Kyriakides C.
AU - Thoma M.
AU - Theodosiou Z.
AU - Partaourides H.
AU - Michael L.
AU - Lanitis A.
PY - 2024
SP - 357
EP - 364
DO - 10.5220/0012453500003660
PB - SciTePress