End-to-End Gaze Grounding of a Person Pictured from Behind

Hayato Yumiya, Daisuke Deguchi, Yasutomo Kawanishi, Hiroshi Murase

2023

Abstract

In this study, we address a novel problem with end-to-end gaze grounding, which estimates the area of an object at which a person in an image is gazing, especially focusing on images of people seen from behind. Existing methods usually estimate facial information such as eye gaze and face orientation first, and then estimate the area at which the target person is gazing; they do not work when a person is pictured from behind. In this study, we focus on individual’s posture, which is a feature that can be obtained even from behind. Posture changes depending on where a person is looking, although this varies from person to person. In this study, we proposes an end-to-end model designed to estimate the area at which a person is gazing from their 3D posture. To minimize differences between individuals, we also introduce the Posture Embedding Encoder Module as a metric learning module. To evaluate the proposed method, we constructed an experimental environment in which a person gazed at a certain object on a shelf. We constructed a dataset consisting of pairs of 3D skeletons and gazes. In an evaluation on this dataset,HEREHEREHEREwe confirmed that the proposed method can estimate the area at which a person is gazing from behind.

Paper Citation

in Harvard Style

Yumiya H., Deguchi D., Kawanishi Y. and Murase H. (2023). End-to-End Gaze Grounding of a Person Pictured from Behind. In Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP; ISBN 978-989-758-634-7, SciTePress, pages 898-905. DOI: 10.5220/0011899900003417

in Bibtex Style

@conference{visapp23,
author={Hayato Yumiya and Daisuke Deguchi and Yasutomo Kawanishi and Hiroshi Murase},
title={End-to-End Gaze Grounding of a Person Pictured from Behind},
booktitle={Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP},
year={2023},
pages={898-905},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011899900003417},
isbn={978-989-758-634-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP
TI - End-to-End Gaze Grounding of a Person Pictured from Behind
SN - 978-989-758-634-7
AU - Yumiya H.
AU - Deguchi D.
AU - Kawanishi Y.
AU - Murase H.
PY - 2023
SP - 898
EP - 905
DO - 10.5220/0011899900003417
PB - SciTePress