Authors:
Hiroto Murakami
1
;
Jialei Chen
1
;
Daisuke Deguchi
1
;
Takatsugu Hirayama
1
;
2
;
Yasutomo Kawanishi
1
;
3
and
Hiroshi Murase
1
Affiliations:
1
Graduate School of Informatics, Nagoya University, Nagoya, Japan
;
2
Faculty of Environmental Science, University of Human Environments, Okazaki, Japan
;
3
Multimodal Data Recognition Research Team, Guardian Robot Project, Riken, Kyoto, Japan
Keyword(s):
Pedestrian’s Gaze Object Detection, Object Detection, Gaze Estimation, Traffic Scene, Dataset.
Abstract:
In this paper, we present a new task of detecting an object that a target pedestrian is gazing at in a traffic scene called PEdestrian’s Gaze Object (PEGO). We argue that the detection of gaze object can provide important information for pedestrian’s behavior prediction and can contribute to the realization of automated vehicles. For this task, we construct a dataset of in-vehicle camera images with annotations of the objects that pedestrians are gazing at. Also, we propose a Transformer-based method called PEGO Transformer to solve the PEGO detection task. The PEGO Transformer directly performs gaze object detection with the utilization of whole-body features without a high-resolution head image and a gaze heatmap which the traditional methods rely on. Experimental results showed that the proposed method could estimate pedestrian’s gaze object accurately even if various objects exist in the scene.