Weakly-supervised Human-object Interaction Detection

Masaki Sugimoto, Ryosuke Furuta, Yukinobu Taniguchi

2021

Abstract

Human-Object Interaction detection is the image recognition task of detecting pairs (a person and an object) in an image and estimating the relationships between them, such as “holding” or “riding”. Existing methods based on supervised learning require a lot of effort to create training data because they need the supervision provided as Bounding Boxes (BBs) of people and objects and verb labels that represent the relationships. In this paper, we extend Proposal Cluster Learning (PCL), a weakly-supervised object detection method, for a new task called weakly-supervised human-object interaction detection, where only the verb labels are assigned to the entire images (i.e., no BBs are given) during the training. Experiments show that the proposed method can successfully learn to detect the BBs of people and objects and the verb labels between them without instance-level supervision.

Paper Citation

in Harvard Style

Sugimoto M., Furuta R. and Taniguchi Y. (2021). Weakly-supervised Human-object Interaction Detection. In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP; ISBN 978-989-758-488-6, SciTePress, pages 293-300. DOI: 10.5220/0010196802930300

in Bibtex Style

@conference{visapp21,
author={Masaki Sugimoto and Ryosuke Furuta and Yukinobu Taniguchi},
title={Weakly-supervised Human-object Interaction Detection},
booktitle={Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP},
year={2021},
pages={293-300},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010196802930300},
isbn={978-989-758-488-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP
TI - Weakly-supervised Human-object Interaction Detection
SN - 978-989-758-488-6
AU - Sugimoto M.
AU - Furuta R.
AU - Taniguchi Y.
PY - 2021
SP - 293
EP - 300
DO - 10.5220/0010196802930300
PB - SciTePress