Can Super Resolution Improve Human Pose Estimation in Low Resolution Scenarios?

Peter Hardy, Srinandan Dasmahapatra, Hansung Kim

2022

Abstract

The results obtained from state of the art human pose estimation (HPE) models degrade rapidly when evaluating people of a low resolution, but can super resolution (SR) be used to help mitigate this effect? By using various SR approaches we enhanced two low resolution datasets and evaluated the change in performance of both an object and keypoint detector as well as end-to-end HPE results. We remark the following observations. First we find that for people who were originally depicted at a low resolution (segmentation area in pixels), their keypoint detection performance would improve once SR was applied. Second, the keypoint detection performance gained is dependent on that persons pixel count in the original image prior to any application of SR; keypoint detection performance was improved when SR was applied to people with a small initial segmentation area, but degrades as this becomes larger. To address this we introduced a novel Mask-RCNN approach, utilising a segmentation area threshold to decide when to use SR during the keypoint detection step. This approach achieved the best results on our low resolution datasets for each HPE performance metrics.

Paper Citation

in Harvard Style

Hardy P., Dasmahapatra S. and Kim H. (2022). Can Super Resolution Improve Human Pose Estimation in Low Resolution Scenarios?. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP; ISBN 978-989-758-555-5, SciTePress, pages 494-501. DOI: 10.5220/0010863700003124

in Bibtex Style

@conference{visapp22,
author={Peter Hardy and Srinandan Dasmahapatra and Hansung Kim},
title={Can Super Resolution Improve Human Pose Estimation in Low Resolution Scenarios?},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP},
year={2022},
pages={494-501},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010863700003124},
isbn={978-989-758-555-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP
TI - Can Super Resolution Improve Human Pose Estimation in Low Resolution Scenarios?
SN - 978-989-758-555-5
AU - Hardy P.
AU - Dasmahapatra S.
AU - Kim H.
PY - 2022
SP - 494
EP - 501
DO - 10.5220/0010863700003124
PB - SciTePress