Happy or Sad, Smiling or Drawing: Multimodal Search and Visualisation of Movies Based on Emotions Along Time

Francisco Caldeira, João Lourenço, Teresa Chambel

2023

Abstract

Movies are a powerful vehicle for culture and education and one of the most important and impactful forms of entertainment, largely due to the significant emotional impact they have on the viewers, in our lives. Technology has been playing an important role, by making a huge amount of movies more accessible in pervasive services and devices, and helping in emotion recognition and classification. As such, it is becoming more pertinent the ability to search, visualize and access movies based on their emotional impact, although emotions are seldom taken into account in these systems. In this paper, we characterize the challenges and approaches in this scenario, then present and evaluate interactive means to visualize and search movies based on their dominant and actual emotional impact along the movie, with different models and modalities. In particular through emotional highlights in words, colors, emojis and trajectories, by drawing emotional blueprints or through users’ emotional states, with the ability to get us into a movie in serendipitous moments.

Paper Citation

in Harvard Style

Caldeira F., Lourenço J. and Chambel T. (2023). Happy or Sad, Smiling or Drawing: Multimodal Search and Visualisation of Movies Based on Emotions Along Time. In Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 2: HUCAPP; ISBN 978-989-758-634-7, SciTePress, pages 85-97. DOI: 10.5220/0011896400003417

in Bibtex Style

@conference{hucapp23,
author={Francisco Caldeira and João Lourenço and Teresa Chambel},
title={Happy or Sad, Smiling or Drawing: Multimodal Search and Visualisation of Movies Based on Emotions Along Time},
booktitle={Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 2: HUCAPP},
year={2023},
pages={85-97},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011896400003417},
isbn={978-989-758-634-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 2: HUCAPP
TI - Happy or Sad, Smiling or Drawing: Multimodal Search and Visualisation of Movies Based on Emotions Along Time
SN - 978-989-758-634-7
AU - Caldeira F.
AU - Lourenço J.
AU - Chambel T.
PY - 2023
SP - 85
EP - 97
DO - 10.5220/0011896400003417
PB - SciTePress