Multimodal Interaction Techniques in Scientific Data Visualization - An Analytical Survey

Jannik Fiedler, Stefan Rilling, Manfred Bogen, Jens Herder

2015

Abstract

The interpretation process of complex data sets makes the integration of effective interaction techniques crucial. Recent work in the field of human–computer interaction has shown that there is strong evidence that multimodal user interaction, i.e. the integration of various input modalities and interaction techniques into one comprehensive user interface, can improve human performance when interacting with complex data sets. However, it is still unclear which factors make these user interfaces superior to unimodal user interfaces. The contribution of this work is an analytical comparison of a multimodal and a unimodal user interface for a scientific visualization application. We show that multimodal user interaction with simultaneously integrated speech and gesture input improves user performance regarding efficiency and ease of use.

References

  1. Arabzadeh, E., Clifford, C. W., and Harris, J. A. (2008). Vision Merges With Touch in a Purely Tactile Discrimination. Psychological Science, 19(7).
  2. Bolt, R. (1980). Put-That-There. In Proceedings of the 7th annual conference on Computer graphics and interactive techniques, pages 262-270.
  3. Bowman, D., Kruijff, E., Laviola, J., and Poupyrev, I. (2001). An Introduction to 3-D User Interface Design. Presence, 10(1).
  4. Bryson, S. (1996). Virtual Reality in Scientific Visualization. Commununications of ACM, 39(5):62-71.
  5. Cohen, P., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Smith, I., Chen, L., and Clow, J. (1997). QuickSet: Multimodal Interaction for Distributed Applications. In Proceedings of the fifth ACM international conference on Multimedia, pages 31-40.
  6. Haynes, W. (2013). Student's t-Test. In Encyclopedia of Systems Biology, pages 2023-2025. Springer New York.
  7. Jégo, J., Paljic, A., and Fuchs, P. (2013). User-Defined Gestural Interaction: a Study on Gesture Memorization. In IEEE Symposium on 3D User Interfaces (3DUI), pages 7-10.
  8. Nielsen, J. (2000). Why You Only Need to Test with 5 Users. http://www.nngroup.com/articles/why-youonly-need-to-test-with-5-users/. [Online; accessed last on 16-12-2014].
  9. Nigay, L. and Coutaz, J. (1993). A Design Space For Multimodal Systems: Concurrent Processing and Data Fusion. In Conference on Human Factors in Computing Systems, pages 172-178.
  10. Oviatt, S. (1999). Ten Myths of Multimodal Interaction. Commun. ACM, 42(11):74-81.
  11. Oviatt, S., Lunsford, R., and Coulston, R. (2005). Individual differences in multimodal integration patterns: What are they and why do they exist? In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 241-249. ACM.
  12. Turk, M. (2013). Multimodal Interaction: A review. Pattern Recognition, 36:189-195.
  13. van Dam, A. (1997). Post-WIMP User Interfaces. Communications of the ACM, 40(2).
  14. Xiao, B., Girand, C., and Oviatt, S. L. (2002). Multimodal integration patterns in children. In 8th International Conference on Spoken Language Processing, Korea.
  15. Xiao, B., Lunsford, R., Coulston, R., Wesson, M., and Oviatt, S. (2003). Modeling multimodal integration patterns and performance in seniors: Toward adaptive processing of individual differences. In Proceedings of the 5th international conference on Multimodal interfaces, pages 265-272. ACM.
Download


Paper Citation


in Harvard Style

Fiedler J., Rilling S., Bogen M. and Herder J. (2015). Multimodal Interaction Techniques in Scientific Data Visualization - An Analytical Survey . In Proceedings of the 10th International Conference on Computer Graphics Theory and Applications - Volume 1: GRAPP, (VISIGRAPP 2015) ISBN 978-989-758-087-1, pages 431-437. DOI: 10.5220/0005296404310437


in Bibtex Style

@conference{grapp15,
author={Jannik Fiedler and Stefan Rilling and Manfred Bogen and Jens Herder},
title={Multimodal Interaction Techniques in Scientific Data Visualization - An Analytical Survey},
booktitle={Proceedings of the 10th International Conference on Computer Graphics Theory and Applications - Volume 1: GRAPP, (VISIGRAPP 2015)},
year={2015},
pages={431-437},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005296404310437},
isbn={978-989-758-087-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 10th International Conference on Computer Graphics Theory and Applications - Volume 1: GRAPP, (VISIGRAPP 2015)
TI - Multimodal Interaction Techniques in Scientific Data Visualization - An Analytical Survey
SN - 978-989-758-087-1
AU - Fiedler J.
AU - Rilling S.
AU - Bogen M.
AU - Herder J.
PY - 2015
SP - 431
EP - 437
DO - 10.5220/0005296404310437