Automated Medical Reporting: From Multimodal Inputs to Medical Reports through Knowledge Graphs

Lientje Maas, Adriaan Kisjes, Iman Hashemi, Floris Heijmans, Fabiano Dalpiaz, Sandra Van Dulmen, Sjaak Brinkkemper

2021

Abstract

Care providers generally experience a high workload mainly due to the large amount of time required for adequate documentation. This paper presents our visionary idea of real-time automated medical reporting through the integration of speech and action recognition technology with knowledge-based summarization of the interaction between care provider and patient. We introduce the Patient Medical Graph as a formal representation of the dialogue and actions during a medical consultation. This knowledge graph represents human anatomical entities, symptoms, medical observations, diagnoses and treatment plans. The formal representation enables automated preparation of a consultation report by means of sentence plans to generate natural language. The architecture and functionality of the Care2Report prototype illustrate our vision of automated reporting of human communication and activities using knowledge graphs and NLP tools.

Paper Citation

in Harvard Style

Maas L., Kisjes A., Hashemi I., Heijmans F., Dalpiaz F., Van Dulmen S. and Brinkkemper S. (2021). Automated Medical Reporting: From Multimodal Inputs to Medical Reports through Knowledge Graphs. In Proceedings of the 14th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2021) - Volume 5: HEALTHINF; ISBN 978-989-758-490-9, SciTePress, pages 509-514. DOI: 10.5220/0010261605090514

in Bibtex Style

@conference{healthinf21,
author={Lientje Maas and Adriaan Kisjes and Iman Hashemi and Floris Heijmans and Fabiano Dalpiaz and Sandra Van Dulmen and Sjaak Brinkkemper},
title={Automated Medical Reporting: From Multimodal Inputs to Medical Reports through Knowledge Graphs},
booktitle={Proceedings of the 14th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2021) - Volume 5: HEALTHINF},
year={2021},
pages={509-514},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010261605090514},
isbn={978-989-758-490-9},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2021) - Volume 5: HEALTHINF
TI - Automated Medical Reporting: From Multimodal Inputs to Medical Reports through Knowledge Graphs
SN - 978-989-758-490-9
AU - Maas L.
AU - Kisjes A.
AU - Hashemi I.
AU - Heijmans F.
AU - Dalpiaz F.
AU - Van Dulmen S.
AU - Brinkkemper S.
PY - 2021
SP - 509
EP - 514
DO - 10.5220/0010261605090514
PB - SciTePress