Better Choice? Combining Speech and Touch in Multimodal Interaction for Elderly Persons

Cui Jian, Hui Shi, Nadine Sasse, Carsten Rachuy, Frank Schafmeister, Holger Schmidt, Nicole von Steinbüchel

Abstract

This paper presents our work on developing, implementing and evaluating a multimodal interactive guidance system that features spoken language and touch-screen input for elderly persons. The development foundation of the system comprises two systematically designed and empirically improved aspects: a set of development guidelines for elderly-friendly multimodal interaction according to common ageing-related decline of important human abilities, and a hybrid dialogue modelling approach with a formal method triggering and agent-based management for the elderly-centered multimodal interaction. To evaluate the minutely developed and implemented system, an experimental study was conducted with thirty-three elderly persons and empirical data were analyzed by applying an adapted version of a general evaluation framework, which provided overall positive analysis results and validated our effort to develop an effective, efficient and elderly friendly multimodal interaction.

References

  1. Alston, P. W., 2000, Illocutionary acts and sentence meaning. Cornell University Press.
  2. Birdi, K., Pennington, J., Zapf, D., 1997. Aging and errors in computer based work: an observational field study. In Journal of Occupational and Organizational Psychology. pp. 35-74.
  3. Broadfoot, P., Roscoe, B., 2000. Tutorial on FDR and Its Applications. In K. Havelund, J. Penix and W. Visser (eds.), SPIN model checking and software verification. Springer-Verlag, London, UK, Volume 1885, pp. 322.
  4. Charness, N., Bosman, E., 1990. Human Factors and Design. In J.E. Birren and K.W. Schaie, (eds.), Handbook of the Psychology of Aging. Academic Press, Volume 3, pp. 446-463.
  5. Craik, F., Jennings, J., 1992. Human memory. In F. Craik and T.A. Salthouse, (eds.), The Handbook of Aging and Cognition. Erlbaum, pp. 51-110.
  6. Folstein, M., Folstein, S., Mchugh, P., 1975. “mini-mental state”, a practical method for grading the cognitive state of patients for clinician. In Journal of Psychiatric Research. Volume 12, 3, pp. 189-198.
  7. Fozard, J. L., 1990. Vision and hearing in aging. In J. Birren, R. Sloane and G. D. Cohen (eds), Handbook of Metal Health and Aging. Academic Press, Volume 3, pp. 18-21.
  8. Goetze, S., Fischer, S., Moritz, N., Appell, J. E., Wallhoff, F., 2012. Multimodal Human-Machine Interaction for Service Robots in Home-Care Environments. In Proceedings of the 1st International Conference on Speech and Multimodal Interaction in Assistive Environments. The Association for Computer Linguistics, pp. 1-7.
  9. Hall, A., Chapman, R., 2002. Correctness by construction: Developing a commercial secure system. In IEEE Software. Vol. 19, 1, pp. 18-25.
  10. Hawthorn, D., 2000. Possible implications of ageing for interface designer. In Interacting with Computers. pp. 507-528.
  11. Jaimes, A., Sebe N., 2007. Multimodal human-computer interaction: A survey. In Computational Vision and Image Understanding. Elsevier Science Inc., New York, USA, pp. 116-134.
  12. Jian, C., Scharfmeister, F., Rachuy, C., Sasse, N., Shi, H., Schmidt, H., Steinbüchel-Rheinwll, N. v., 2011. Towards Effective, Efficient and Elderly-friendly Multimodal Interaction. In PETRA 2011: Proceedings of the 4th International Conference on PErvasive Technologies Related to Assistive Environments. ACM, New York, USA.
  13. Jian, C., Scharfmeister, F., Rachuy, C., Sasse, N., Shi, H., Schmidt, H., Steinbüchel-Rheinwll, N. v., 2012. Evaluating a Spoken Language Interface of a Multimodal Interactive Guidance System for Elderly Persons. In HealthInf 2012: Proceedings of the International Conference on Health Informatics. SciTepress, Vilamoura, Algarve, Portugal.
  14. Kline, D. W., Scialfa, C. T., 1996. Sensory and Perceptual Functioning: basic research and human factors implications. In A. D. Fisk and W. A. Rogers. (eds.), Handbook of Human Factors and the Older Adult, Academic Press.
  15. Kotary, L., Hoyer, W. J., 1995. Age and the ability to inhibit distractor information in visual selective attention. In Experimental Aging Research. Volume 21, Issue 2.
  16. Krajewski, J., Wieland, R., Batliner, A., 2008. An acoustic framework for detecting fatique in speech based human computer interaction. In Proceedings of the 11th International Conference on Computers Help People with Special Needs. Springer-Verlag Berlin, Heidelberg, pp. 54-61.
  17. Lutz, W., Sanderson, W., Scherbov, S., 2008. The coming acceleration of global population ageing. In Nature. pp. 716-719.
  18. Mackay, D., Abrams, L., 1996. Language, memory and aging. In J. E. Birren and K. W.Schaie (eds), Handbook of the psychology of Aging. Academic Press, Volume 4, pp. 251-265.
  19. Mandel, C., Lüth, T., Laue, T., Röfer, T., Gräser, A., Krieg-Brückner, B., 2009. Navigating a Smart Wheelchair with a Brain-Computer Interface Interpreting Steady-State Visual Evoked Potentials. In Proceedings of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE Xplore, St. Louis, Missouri, United States, pp. 1118- 1125.
  20. McDowd, J. M., Craik, F. 1988. Effects of aging and task difficulty on divided attention performance. In Journal of Experimental Psychology: Human Perception and Performance 14. pp. 267-280.
  21. Moeller, S., Goedde, F., Wolters, M., 2008. Corpus analysis of spoken smart-home interactions with older users. In N. Calzolari, K. Choukri, B. Maegaard, J. Mariani, J. Odjik, S. Piperidis, and D. Tapias, (eds.), Proceedings of the Sixth International Conference on Language Resources Association. ELRA.
  22. Rodríguez, M. D., García-Vázquez, J. P., Andrade, Á. G., 2011. Design dimensions of ambient information systems to facilitate the development of AAL environments. In Proceedings of the 4th International Conference on PErvasive Technologies Related to Assistive Environments. ACM Press, New York, United States, pp. 4:1-4:7.
  23. Roscoe, A. W., 1997. In The Theory and Practice of Concurrency, Prentice Hall.
  24. Salthouse, T. A., 1994. The aging of working memory. In Neuropsychology 8, pp. 535-543.
  25. Schieber, F., 1992. Aging and the senses. In J. E. Birren, R. B. Sloane, and G. D. Cohen, (eds.) Handbook of Mental Health and Aging, Academic Press, Volume 2.
  26. Shi, H., Bateman, J., 2005. Developing human-robot dialogue management formally. In Proceedings of Symposium on Dialogue Modelling and Generation. Amsterdam, Netherlands.
  27. Shi, H., Jian, C., Rachuy, C. 2011. Evaluation of a Unified Dialogue Model for Human-Computer Interaction. In International Journal of Computational Linguistics and Applications. Bahri Publications, Volume 2.
  28. Traum, D., Larsson, S., 2003. The information state approach to dialogue management. In J.v. Kuppevelt and R. Smith (eds.), Current and New Directions in Discourse and Dialogue. Kluwer, pp. 325-354.
  29. Walkder, N., Philbin, D. A., Fisk, A. D., 1997. Agerelated differences in movement control: adjust submovement structure to optimize performance. In Journal of Gerontology: Psychological Sciences 52B, pp. 40-52.
  30. Walker, M. A., Litman, D. J., Kamm, C. A., Kamm, A. A., Abella, A., 1997. Paradise: a framework for evaluating spoken dialogue agents. In Proceedings of the eighth conference on European chapter of Association for computational Linguistics, NJ, USA, pp. 271-280.
Download


Paper Citation


in Harvard Style

Jian C., Shi H., Sasse N., Rachuy C., Schafmeister F., Schmidt H. and von Steinbüchel N. (2013). Better Choice? Combining Speech and Touch in Multimodal Interaction for Elderly Persons . In Proceedings of the International Conference on Health Informatics - Volume 1: HEALTHINF, (BIOSTEC 2013) ISBN 978-989-8565-37-2, pages 94-103. DOI: 10.5220/0004244800940103


in Bibtex Style

@conference{healthinf13,
author={Cui Jian and Hui Shi and Nadine Sasse and Carsten Rachuy and Frank Schafmeister and Holger Schmidt and Nicole von Steinbüchel},
title={Better Choice? Combining Speech and Touch in Multimodal Interaction for Elderly Persons },
booktitle={Proceedings of the International Conference on Health Informatics - Volume 1: HEALTHINF, (BIOSTEC 2013)},
year={2013},
pages={94-103},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004244800940103},
isbn={978-989-8565-37-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Health Informatics - Volume 1: HEALTHINF, (BIOSTEC 2013)
TI - Better Choice? Combining Speech and Touch in Multimodal Interaction for Elderly Persons
SN - 978-989-8565-37-2
AU - Jian C.
AU - Shi H.
AU - Sasse N.
AU - Rachuy C.
AU - Schafmeister F.
AU - Schmidt H.
AU - von Steinbüchel N.
PY - 2013
SP - 94
EP - 103
DO - 10.5220/0004244800940103