loading
Documents

Research.Publish.Connect.

Paper

A Case Study on using Crowdsourcing for Ambiguous Tasks

Topics: Applications: Image Processing and Artificial Vision, Pattern Recognition, Decision Making, Industrial and Real World applications, Financial Applications, Neural Prostheses and Medical Applications, Neural based Data Mining and Complex Information Proces; Deep Learning; Higher Level Artificial Neural Network Based Intelligent Systems; Learning Paradigms and Algorithms

Authors: Ankush Chatterjee 1 ; Umang Gupta 2 and Puneet Agrawal 2

Affiliations: 1 Indian Institute of Technology, Kharagpur and India ; 2 Microsoft and India

ISBN: 978-989-758-327-8

Keyword(s): Crowdsourcing, Deep Learning, Label Aggregation Techniques.

Abstract: In our day to day life, we come across situations which are interpreted differently by different human beings. A given sentence may be offensive to some humans but not to others. Similarly, a sentence can convey different emotions to different human beings. For instance, “Why you never text me!”, can either be interpreted as a sad or an angry utterance. Lack of facial expressions and voice modulations make detecting emotions in textual sentences a hard problem. Some textual sentences are inherently ambiguous and their true emotion label is difficult to determine. In this paper, we study how to use crowdsourcing for an ambiguous task of determining emotion labels of textual sentences. Crowdsourcing has become one of the most popular medium for obtaining large scale labeled data for supervised learning tasks. However, for our task, due to the intrinsic ambiguity, human annotators differ in opinions about the underlying emotion of certain sentences. In our work, we harness the multiple p erspectives of annotators for ambiguous sentences to improve the performance of an emotion detection model. In particular, we compare our technique against the popularly used technique of majority vote to determine the label of a given sentence. Our results indicate that considering diverse perspective of annotators is helpful for the ambiguous task of emotion detection. (More)

PDF ImageFull Text

Download
CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 35.173.57.202

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Chatterjee, A.; Gupta, U. and Agrawal, P. (2018). A Case Study on using Crowdsourcing for Ambiguous Tasks.In Proceedings of the 10th International Joint Conference on Computational Intelligence - Volume 1: IJCCI, ISBN 978-989-758-327-8, pages 242-247. DOI: 10.5220/0006955002420247

@conference{ijcci18,
author={Ankush Chatterjee. and Umang Gupta. and Puneet Agrawal.},
title={A Case Study on using Crowdsourcing for Ambiguous Tasks},
booktitle={Proceedings of the 10th International Joint Conference on Computational Intelligence - Volume 1: IJCCI,},
year={2018},
pages={242-247},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006955002420247},
isbn={978-989-758-327-8},
}

TY - CONF

JO - Proceedings of the 10th International Joint Conference on Computational Intelligence - Volume 1: IJCCI,
TI - A Case Study on using Crowdsourcing for Ambiguous Tasks
SN - 978-989-758-327-8
AU - Chatterjee, A.
AU - Gupta, U.
AU - Agrawal, P.
PY - 2018
SP - 242
EP - 247
DO - 10.5220/0006955002420247

Login or register to post comments.

Comments on this Paper: Be the first to review this paper.