Video-based Behavior Understanding of Children for Objective Diagnosis of Autism

Abid Ali, Abid Ali, Farhood Negin, Susanne Thümmler, Susanne Thümmler, Francois Bremond, Francois Bremond

2022

Abstract

One of the major diagnostic criteria for Autism Spectrum Disorder (ASD) is the recognition of stereotyped behaviors. However, it primarily relies on parental interviews and clinical observations, which result in a prolonged diagnosis cycle preventing ASD children from timely treatment. To help clinicians speed up the diagnosis process, we propose a computer-vision-based solution. First, we collected and annotated a novel dataset for action recognition tasks in videos of children with ASD in an uncontrolled environment. Second, we propose a multi-modality fusion network based on 3D CNNs. In the first stage of our method, we pre- process the RGB videos to get the ROI (child) using Yolov5 and DeepSORT algorithms. For optical flow extraction, we use the RAFT algorithm. In the second stage, we perform extensive experiments on different deep learning frameworks to propose a baseline. In the last stage, a multi-modality-based late fusion network is proposed to classify and evaluate performance of ASD children. The results revealed that the multi-modality fusion network achieves the best accuracy as compared to other methods. The baseline results also demonstrate the potential of an action-recognition-based system to assist clinicians in a reliable, accurate, and timely diagnosis of ASD disorder.

Paper Citation

in Harvard Style

Ali A., Negin F., Thümmler S. and Bremond F. (2022). Video-based Behavior Understanding of Children for Objective Diagnosis of Autism. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP; ISBN 978-989-758-555-5, SciTePress, pages 475-484. DOI: 10.5220/0010839200003124

in Bibtex Style

@conference{visapp22,
author={Abid Ali and Farhood Negin and Susanne Thümmler and Francois Bremond},
title={Video-based Behavior Understanding of Children for Objective Diagnosis of Autism},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP},
year={2022},
pages={475-484},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010839200003124},
isbn={978-989-758-555-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP
TI - Video-based Behavior Understanding of Children for Objective Diagnosis of Autism
SN - 978-989-758-555-5
AU - Ali A.
AU - Negin F.
AU - Thümmler S.
AU - Bremond F.
PY - 2022
SP - 475
EP - 484
DO - 10.5220/0010839200003124
PB - SciTePress