Attention-Based Shape and Gait Representations Learning for Video-Based Cloth-Changing Person Re-Identification

Vuong Nguyen, Samiha Mirza, Pranav Mantini, Shishir Shah

2024

Abstract

Current state-of-the-art Video-based Person Re-Identification (Re-ID) primarily relies on appearance features extracted by deep learning models. These methods are not applicable for long-term analysis in real-world scenarios where persons have changed clothes, making appearance information unreliable. In this work, we deal with the practical problem of Video-based Cloth-Changing Person Re-ID (VCCRe-ID) by proposing “Attention-based Shape and Gait Representations Learning” (ASGL) for VCCRe-ID. Our ASGL framework improves Re-ID performance under clothing variations by learning clothing-invariant gait cues using a Spatial-Temporal Graph Attention Network (ST-GAT). Given the 3D-skeleton-based spatial-temporal graph, our proposed ST-GAT comprises multi-head attention modules, which are able to enhance the robustness of gait embeddings under viewpoint changes and occlusions. The ST-GAT amplifies the important motion ranges and reduces the influence of noisy poses. Then, the multi-head learning module effectively reserves beneficial local temporal dynamics of movement. We also boost discriminative power of person representations by learning body shape cues using a GAT. Experiments on two large-scale VCCRe-ID datasets demonstrate that our proposed framework outperforms state-of-the-art methods by 12.2% in rank-1 accuracy and 7.0% in mAP.

Paper Citation

in Harvard Style

Nguyen V., Mirza S., Mantini P. and Shah S. (2024). Attention-Based Shape and Gait Representations Learning for Video-Based Cloth-Changing Person Re-Identification. In Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP; ISBN 978-989-758-679-8, SciTePress, pages 80-89. DOI: 10.5220/0012315900003660

in Bibtex Style

@conference{visapp24,
author={Vuong Nguyen and Samiha Mirza and Pranav Mantini and Shishir Shah},
title={Attention-Based Shape and Gait Representations Learning for Video-Based Cloth-Changing Person Re-Identification},
booktitle={Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP},
year={2024},
pages={80-89},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012315900003660},
isbn={978-989-758-679-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP
TI - Attention-Based Shape and Gait Representations Learning for Video-Based Cloth-Changing Person Re-Identification
SN - 978-989-758-679-8
AU - Nguyen V.
AU - Mirza S.
AU - Mantini P.
AU - Shah S.
PY - 2024
SP - 80
EP - 89
DO - 10.5220/0012315900003660
PB - SciTePress