Multimodal Personality Recognition using Cross-attention Transformer and Behaviour Encoding

Tanay Agrawal; Dhruv Agarwal; Dhruv Agarwal; Michal Balazia; Michal Balazia; Neelabh Sinha; Neelabh Sinha; François Bremond; François Bremond

Research.Publish.Connect.

*Please fill out at least one Field. *Value must be an number!

Title:
ISBN:
Year:
Acronym:
Subject:

Advanced Search Proceedings Search

If you're looking for an exact phrase use quotation marks on text fields.

*Please fill out at least one Field.

Title:
Author:
Affiliation:
Subject:

Advanced Search Papers Search

If you're looking for an exact phrase use quotation marks on text fields.

*Please fill out at least one Field.

Name:
Affiliation:
Country:
Conference:
Subject:

Advanced Search Authors Search

If you're looking for an exact phrase use quotation marks on text fields.

*Please fill out at least one Field.

Name:
Country:
Subject:

Advanced Search Affiliations Search

If you're looking for an exact phrase use quotation marks on text fields.

Proceedings

Proceedings Search *Please fill out at least one Field. *Value must be an number!

Title:
ISBN:
Year:
Acronym:
Subject:

Advanced Search Proceedings Search

If you're looking for an exact phrase use quotation marks on text fields.

Papers

Papers Search *Please fill out at least one Field.

Title:
Author:
Affiliation:
Subject:

Advanced Search Papers Search

If you're looking for an exact phrase use quotation marks on text fields.

Authors

Authors Search *Please fill out at least one Field.

Name:
Affiliation:
Country:
Conference:
Subject:

Advanced Search Authors Search

If you're looking for an exact phrase use quotation marks on text fields.

Advanced Search

Paper

Multimodal Personality Recognition using Cross-attention Transformer and Behaviour Encoding

Topics: Categorization and Scene Understanding; Deep Learning for Visual Understanding ; Event and Human Activity Recognition

In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 5 VISAPP: VISAPP, 501-508, 2022

Authors: Tanay Agrawal ¹ ; Dhruv Agarwal ^{2

;

1} ; Michal Balazia ^{1

;

3} ; Neelabh Sinha ^{4

;

1} and François Bremond ^{1

;

3}

Affiliations: ¹ INRIA Sophia Antipolis - Méditerranée, France ; ² Indian Institute of Information Technology, Allahabad, India ; ³ Université Côte d’Azur, France ; ⁴ Birla Institute of Technology and Science, Pilani, India

Keyword(s): Multimodal Transformer, Multimodal Data, Feature Engineering, Personality Recognition.

Abstract: Personality computing and affective computing have gained recent interest in many research areas. The datasets for the task generally have multiple modalities like video, audio, language and bio-signals. In this paper, we propose a flexible model for the task which exploits all available data. The task involves complex relations and to avoid using a large model for video processing specifically, we propose the use of behaviour encoding which boosts performance with minimal change to the model. Cross-attention using transformers has become popular in recent times and is utilised for fusion of different modalities. Since long term relations may exist, breaking the input into chunks is not desirable, thus the proposed model processes the entire input together. Our experiments show the importance of each of the above contributions.

CC BY-NC-ND 4.0

Guest: Register as new SciTePress user now for free.

SciTePress user: please login.

My Papers

You are not signed in, therefore limits apply to your IP address 13.59.196.41

In the current month:

Recent papers: 100 available of 100 total

2⁺ years older papers: 200 available of 200 total

Paper citation in several formats:

Agrawal, T., Agarwal, D., Balazia, M., Sinha, N. and Bremond, F. (2022). Multimodal Personality Recognition using Cross-attention Transformer and Behaviour Encoding. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP; ISBN 978-989-758-555-5; ISSN 2184-4321, SciTePress, pages 501-508. DOI: 10.5220/0010841400003124

@conference{visapp22,
author={Tanay Agrawal and Dhruv Agarwal and Michal Balazia and Neelabh Sinha and Fran\c{c}ois Bremond},
title={Multimodal Personality Recognition using Cross-attention Transformer and Behaviour Encoding},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP},
year={2022},
pages={501-508},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010841400003124},
isbn={978-989-758-555-5},
issn={2184-4321},
}

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP
TI - Multimodal Personality Recognition using Cross-attention Transformer and Behaviour Encoding
SN - 978-989-758-555-5
IS - 2184-4321
AU - Agrawal, T.
AU - Agarwal, D.
AU - Balazia, M.
AU - Sinha, N.
AU - Bremond, F.
PY - 2022
SP - 501
EP - 508
DO - 10.5220/0010841400003124
PB - SciTePress