loading
Papers Papers/2022 Papers Papers/2022

Research.Publish.Connect.

Paper

Paper Unlock

Authors: Mathieu Pagé Fortin and Brahim Chaib-Draa

Affiliation: Department of Computer Science and Software Engineering, Laval University, Québec and Canada

Keyword(s): Multimodal, Sentiment Analysis, Emotion Recognition, Multitask Learning, Text and Image Modalities.

Abstract: Multimodal sentiment analysis has recently received an increasing interest. However, most methods have considered that text and image modalities are always available at test time. This assumption is often violated in real environments (e.g. social media) since users do not always publish a text with an image. In this paper we propose a method based on a multitask framework to combine multimodal information when it is available, while being able to handle the cases where a modality is missing. Our model contains one classifier for analyzing the text, another for analyzing the image, and another performing the prediction by fusing both modalities. In addition to offer a solution to the problem of a missing modality, our experiments show that this multitask framework improves generalization by acting as a regularization mechanism. We also demonstrate that the model can handle a missing modality at training time, thus being able to be trained with image-only and text-only examples.

CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 54.196.27.171

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Fortin, M. and Chaib-Draa, B. (2019). Multimodal Sentiment Analysis: A Multitask Learning Approach. In Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods - ICPRAM; ISBN 978-989-758-351-3; ISSN 2184-4313, SciTePress, pages 368-376. DOI: 10.5220/0007313503680376

@conference{icpram19,
author={Mathieu Pagé Fortin. and Brahim Chaib{-}Draa.},
title={Multimodal Sentiment Analysis: A Multitask Learning Approach},
booktitle={Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods - ICPRAM},
year={2019},
pages={368-376},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0007313503680376},
isbn={978-989-758-351-3},
issn={2184-4313},
}

TY - CONF

JO - Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods - ICPRAM
TI - Multimodal Sentiment Analysis: A Multitask Learning Approach
SN - 978-989-758-351-3
IS - 2184-4313
AU - Fortin, M.
AU - Chaib-Draa, B.
PY - 2019
SP - 368
EP - 376
DO - 10.5220/0007313503680376
PB - SciTePress