loading
Documents

Research.Publish.Connect.

Paper

Authors: Mathieu Fortin and Brahim Chaib-Draa

Affiliation: Department of Computer Science and Software Engineering, Laval University, Québec, Canada

ISBN: 978-989-758-351-3

Keyword(s): Multimodal, Sentiment Analysis, Emotion Recognition, Multitask Learning, Text and Image Modalities.

Abstract: Multimodal sentiment analysis has recently received an increasing interest. However, most methods have considered that text and image modalities are always available at test time. This assumption is often violated in real environments (e.g. social media) since users do not always publish a text with an image. In this paper we propose a method based on a multitask framework to combine multimodal information when it is available, while being able to handle the cases where a modality is missing. Our model contains one classifier for analyzing the text, another for analyzing the image, and another performing the prediction by fusing both modalities. In addition to offer a solution to the problem of a missing modality, our experiments show that this multitask framework improves generalization by acting as a regularization mechanism. We also demonstrate that the model can handle a missing modality at training time, thus being able to be trained with image-only and text-only examples.

PDF ImageFull Text

Download
Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 54.196.116.3

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Fortin, M. and Chaib-Draa, B. (2019). Multimodal Sentiment Analysis: A Multitask Learning Approach.In Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-351-3, pages 368-376. DOI: 10.5220/0007313503680376

@conference{icpram19,
author={Mathieu Pagé Fortin. and Brahim Chaib{-}Draa.},
title={Multimodal Sentiment Analysis: A Multitask Learning Approach},
booktitle={Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2019},
pages={368-376},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0007313503680376},
isbn={978-989-758-351-3},
}

TY - CONF

JO - Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Multimodal Sentiment Analysis: A Multitask Learning Approach
SN - 978-989-758-351-3
AU - Fortin, M.
AU - Chaib-Draa, B.
PY - 2019
SP - 368
EP - 376
DO - 10.5220/0007313503680376

Login or register to post comments.

Comments on this Paper: Be the first to review this paper.