Visual Question Answering Analysis: Datasets, Methods, and Image Featurization Techniques

Vijay Kumari, Abhimanyu Sethi, Yashvardhan Sharma, Lavika Goel

2023

Abstract

Holistic scene understanding is a long-standing objective of core tenets of Artificial Intelligence (AI). Multimodal tasks that aim to synergize capabilities spanning multiple domains, such as visual-linguistic capabilities, into intelligent systems are thus a desideratum for the next step in AI. Visual Question Answering (VQA) systems that integrate Computer Vision and Natural Language Processing tasks into the task of answering natural language questions about an image represent one such domain. There is a need to explore Deep Learning techniques that can help to improve such systems beyond the language biases of real-world priors that presently hinder them from serving as a veritable touchstone for holistic scene understanding. Furthermore, the effectiveness of Transformer architecture for the image featurization pipeline of VQA systems remains untested. Hence, an exhaustive study on the performance of various model architectures with varied training conditions on VQA datasets like VizWiz and VQA v2 is imperative to further this area of research. This study explores architectures that utilize image and question co-attention for the task of VQA and several CNN architectures, including ResNet, VGG, EfficientNet, and DenseNet. Vision Transformer architecture is also explored for image featurization, and a myriad of loss functions such as cross-entropy, focal loss, and UniLoss are employed for training the models. Finally, the trained model is deployed using Flask, and a GUI for the same has been implemented that lets users input an image and accompanying questions about the image to generate an answer in response.

Paper Citation

in Harvard Style

Kumari V., Sethi A., Sharma Y. and Goel L. (2023). Visual Question Answering Analysis: Datasets, Methods, and Image Featurization Techniques. In Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-626-2, pages 281-288. DOI: 10.5220/0011655900003411

in Bibtex Style

@conference{icpram23,
author={Vijay Kumari and Abhimanyu Sethi and Yashvardhan Sharma and Lavika Goel},
title={Visual Question Answering Analysis: Datasets, Methods, and Image Featurization Techniques},
booktitle={Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2023},
pages={281-288},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011655900003411},
isbn={978-989-758-626-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Visual Question Answering Analysis: Datasets, Methods, and Image Featurization Techniques
SN - 978-989-758-626-2
AU - Kumari V.
AU - Sethi A.
AU - Sharma Y.
AU - Goel L.
PY - 2023
SP - 281
EP - 288
DO - 10.5220/0011655900003411