Combining Metric Learning and Attention Heads for Accurate and Efficient Multilabel Image Classification

Kirill Prokofiev, Vladislav Sovrasov

2023

Abstract

Multi-label image classification allows predicting a set of labels from a given image. Unlike multiclass classification, where only one label per image is assigned, such a setup is applicable for a broader range of applications. In this work we revisit two popular approaches to multilabel classification: transformer-based heads and labels relations information graph processing branches. Although transformer-based heads are considered to achieve better results than graph-based branches, we argue that with the proper training strategy, graph-based methods can demonstrate just a small accuracy drop, while spending less computational resources on inference. In our training strategy, instead of Asymmetric Loss (ASL), which is the de-facto standard for multilabel classification, we introduce its metric learning modification. In each binary classification sub-problem it operates with L2 normalized feature vectors coming from a backbone and enforces angles between the normalized representations of positive and negative samples to be as large as possible. This results in providing a better discrimination ability, than binary cross entropy loss does on unnormalized features. With the proposed loss and training strategy, we obtain SOTA results among single modality methods on widespread multilabel classification benchmarks such as MS-COCO, PASCAL-VOC, NUS-Wide and Visual Genome 500. Source code of our method is available as a part of the OpenVINO™ Training Extensions∗ .

Paper Citation

in Harvard Style

Prokofiev K. and Sovrasov V. (2023). Combining Metric Learning and Attention Heads for Accurate and Efficient Multilabel Image Classification. In Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP; ISBN 978-989-758-634-7, SciTePress, pages 388-396. DOI: 10.5220/0011603700003417

in Bibtex Style

@conference{visapp23,
author={Kirill Prokofiev and Vladislav Sovrasov},
title={Combining Metric Learning and Attention Heads for Accurate and Efficient Multilabel Image Classification},
booktitle={Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP},
year={2023},
pages={388-396},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011603700003417},
isbn={978-989-758-634-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP
TI - Combining Metric Learning and Attention Heads for Accurate and Efficient Multilabel Image Classification
SN - 978-989-758-634-7
AU - Prokofiev K.
AU - Sovrasov V.
PY - 2023
SP - 388
EP - 396
DO - 10.5220/0011603700003417
PB - SciTePress