Exploring the Dependencies between Behavioral and Neuro-physiological Time-series Extracted from Conversations between Humans and Artificial Agents

Hmamouche Youssef, Hmamouche Youssef, Ochs Magalie, Prévot Laurent, Prévot Laurent, Chaminade Thierry

2020

Abstract

Whole-brain neuroimaging using functional Magnetic Resonance Imaging (fMRI) provides valuable data to localize brain activity in space and time. Here, we use a unique corpus including fMRI and behavior recorded when participants discussed with a human or a conversational robot. Temporal dynamic is crucial when studying conversation, yet identifying relationship between the participants’ behavior and their brain activity is technically challenging given the time resolution of fMRI. We propose here an approach developed to extract neurophysiological and behavioral time-series from the corpus and analyse their causal relationships. Preprocessing entails the construction of discrete neurophysiological time-series from functionally well defined brain areas, as well as behavioral time-series describing higher-order behaviors extracted from synchronized raw audio, video and eyetracking recordings. The second step consists in applying machine learning models to predict brain activity on the basis of various aspects of behavior given knowledge about the functional role of the areas under scrutiny. Results demonstrate the specificity of the behaviors allowing the predictions of the activity in functional brain areas.

Paper Citation

in Harvard Style

Youssef H., Magalie O., Laurent P. and Thierry C. (2020). Exploring the Dependencies between Behavioral and Neuro-physiological Time-series Extracted from Conversations between Humans and Artificial Agents. In Proceedings of the 9th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-397-1, pages 353-360. DOI: 10.5220/0008989503530360

in Bibtex Style

@conference{icpram20,
author={Hmamouche Youssef and Ochs Magalie and Prévot Laurent and Chaminade Thierry},
title={Exploring the Dependencies between Behavioral and Neuro-physiological Time-series Extracted from Conversations between Humans and Artificial Agents},
booktitle={Proceedings of the 9th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2020},
pages={353-360},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0008989503530360},
isbn={978-989-758-397-1},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 9th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Exploring the Dependencies between Behavioral and Neuro-physiological Time-series Extracted from Conversations between Humans and Artificial Agents
SN - 978-989-758-397-1
AU - Youssef H.
AU - Magalie O.
AU - Laurent P.
AU - Thierry C.
PY - 2020
SP - 353
EP - 360
DO - 10.5220/0008989503530360