Cui Jian, Frank Schafmeister, Carsten Rachuy, Nadine Sasse, Hui Shi, Holger Schmidt, Nicole von Steinbüchel


This paper presents a multimodal interactive guidance system for elderly persons for the use in navigating in hospital environments. We used a unified modelling method combining the conventional recursive transition network based approach and agent-based dialogue theory to support the development of the central dialogue management component. Then we studied and specified a list of guidelines addressing the needs of designing and implementing multimodal interface for elderly persons. As an important step towards developing an effective, efficient and elderly-friendly multimodal interaction, the spoken language interface of the current system was evaluated by an elaborated experiment with sixteen elderly persons. The results of the experimental study are overall positive and provide evidence for our proposed guidelines, approaches and frameworks on interactive system development while advising further improvements.


  1. Alston, P. W., 2000, Illocutionary acts and sentence meaning. Cornell University Press.
  2. Birdi, K., Pennington, J., Zapf, D., 1997. Aging and errors in computer based work: an observational field study. In Journal of Occupational and Organizational Psychology. pp. 35-74.
  3. Bolt, R. A., 1980. Put-That-There: Voice and Gesture at the Graphics Interface. In Proceedings of the 7th International Conference on Computer Graphics and Interactive Techniques. Seattle, USA, pp. 262-270.
  4. Broadfoot, P., Roscoe, B., 2000. Tutorial on FDR and Its Applications. In K. Havelund, J. Penix and W. Visser (eds.), SPIN model checking and software verification. Springer-Verlag, London, UK, Volume 1885, pp. 322.
  5. Charness, N., Bosman, E., 1990. Human Factors and Design. In J.E. Birren and K.W. Schaie, (eds.), Handbook of the Psychology of Aging. Academic Press, Volume 3, pp. 446-463.
  6. Craik, F., Jennings, J., 1992. Human memory. In F. Craik and T. A. Salthouse, (eds.), The Handbook of Aging and Cognition. Erlbaum, pp. 51-110.
  7. Folstein, M., Folstein, S., Mchugh, P., 1975. “mini-mental state”, a practical method for grading the cognitive state of patients for clinician. In Journal of Psychiatric Research. Volume 12, 3, pp. 189-198.
  8. Fozard, J. L., 1990. Vision and hearing in aging. In J. Birren, R. Sloane and G.D. Cohen (eds), Handbook of Metal Health and Aging. Academic Press, Volume 3, pp. 18-21.
  9. Hawthorn, D., 2000. Possible implications of ageing for interface designer. In Interacting with Computers. pp. 507-528.
  10. Hall, A., Chapman, R., 2002. Correctness by construction: Developing a commercial secure system. In IEEE Software. Vol. 19, 1, pp. 18-25.
  11. Holzinger, A., Mukasa, K.S., Nischelwitzer, A. K., 2008. Introduction to the special thematic session: Humancomputer interaction and usability for elderly. In Proceedings of the 11th International Conference on Computers Help People with Special Needs. Springer Verlag, Berlin, Germany, pp. 18-21.
  12. Hoyer, W. J., Rybash, J. M., 1992. Age and visual field differences in computing visual spatial relations. In Psychology and Aging 7. pp. 339-342.
  13. Jaimes, A., Sebe N., 2007. Multimodal human-computer interaction: A survey. In Computational Vision and Image Understanding. Elsevier Science Inc., New York, USA, pp. 116-134.
  14. Jian, C., Scharfmeister, F., Rachuy, C., Sasse, N., Shi, H., Schmidt, H., Steinbüchel-Rheinwll, N. v., 2011. Towards Effective, Efficient and Elderly-friendly Multimodal Interaction. In PETRA 2011: Proceedings of the 4th International Conference on PErvasive Technologies Related to Assistive Environments. ACM, New York, USA.
  15. Kline, D. W., Scialfa, C.T., 1996. Sensory and Perceptual Functioning: basic research and human factors implications. In A.D. Fisk and W.A. Rogers. (eds.), Handbook of Human Factors and the Older Adult, Academic Press.
  16. Kotary, L., Hoyer, W. J., 1995. Age and the ability to inhibit distractor information in visual selective attention. In Experimental Aging Research. Volume 21, Issue 2.
  17. Mackay, D., Abrams, L., 1996. Language, memory and aging. In J.E. Birren and K.W.Schaie (eds), Handbook of the psychology of Aging. Academic Press, Volume 4, pp. 251-265.
  18. McDowd, J. M., Craik, F. 1988. Effects of aging and task difficulty on divided attention performance. In Journal of Experimental Psychology: Human Perception and Performance 14. pp. 267-280.
  19. Moeller, S., Goedde, F., Wolters, M., 2008. Corpus analysis of spoken smart-home interactions with older users. In N. Calzolari, K.Choukri, B. Maegaard, J. Mariani, J. Odjik, S. Piperidis, and D. Tapias, (eds.), Proceedings of the Sixth International Conference on Language Resources Association. ELRA.
  20. Morris, J. M., 1994. User interface design for older adults. In Interacting with Computers. Vol. 6, 4, pp. 373-393.
  21. Oviatt, S. T., 1999. Ten myths of multimodal interaction. In Communications of the ACM. ACM New York, USA, Vol. 42, No. 11, pp. 74-81.
  22. Roscoe, A.W., 1997. The Theory and Practice of Concurrency, Prentice Hall.
  23. Ross, J. R., Bateman, J., Shi, H., 2005. Using Generalized Dialogue Models to Constrain Information State Based Dialogue Systems. In Symposium on Dialogue Modelling and Generation.
  24. Salthouse, T. A., 1994. The aging of working memory. In Neuropsychology 8, pp. 535-543.
  25. Schieber, F., 1992. Aging and the senses. In J.E. Birren, R.B. Sloane, and G.D. Cohen, (eds.) Handbook of Mental Health and Aging, Academic Press, Volume 2.
  26. Shaie, K.W., 1996. Intellectual development in adulthood. In J.E. Birren and K.W. Shaie, (eds.), Handbook of the psychology of aging. Academic Press, Volume 4.
  27. Shi, H., Bateman, J., 2005. Developing human-robot dialogue management formally. In Proceedings of Symposium on Dialogue Modelling and Generation. Amsterdam, Netherlands.
  28. Sitter, S., Stein, A., 1992. Modelling the illocutionary aspects of information-seeking dialogues. In Journal of Information Processing and Management. Elsevier, Volume 28, issue 2, pp. 165-180.
  29. Traum, D., Larsson, S., 2003. The information state approach to dialogue management. In J.v. Kuppevelt and R. Smith (eds.), Current and New Directions in Discourse and Dialogue. Kluwer, pp. 325-354.
  30. Walkder, N., Philbin, D. A., Fisk, A.D., 1997. Age-related differences in movement control: adjust submovement structure to optimize performance. In Journal of Gerontology: Psychological Sciences 52B, pp. 40-52.
  31. Walker, M. A., Litman, D. J., Kamm, C. A., Kamm, A. A., Abella, A., 1997. Paradise: a framework for evaluating spoken dialogue agents. In Proceedings of the eighth conference on European chapter of Association for computational Linguistics, NJ, USA, pp. 271-280.

Paper Citation

in Harvard Style

Jian C., Schafmeister F., Rachuy C., Sasse N., Shi H., Schmidt H. and von Steinbüchel N. (2012). EVALUATING A SPOKEN LANGUAGE INTERFACE OF A MULTIMODAL INTERACTIVE GUIDANCE SYSTEM FOR ELDERLY PERSONS . In Proceedings of the International Conference on Health Informatics - Volume 1: HEALTHINF, (BIOSTEC 2012) ISBN 978-989-8425-88-1, pages 87-96. DOI: 10.5220/0003783800870096

in Bibtex Style

author={Cui Jian and Frank Schafmeister and Carsten Rachuy and Nadine Sasse and Hui Shi and Holger Schmidt and Nicole von Steinbüchel},
booktitle={Proceedings of the International Conference on Health Informatics - Volume 1: HEALTHINF, (BIOSTEC 2012)},

in EndNote Style

JO - Proceedings of the International Conference on Health Informatics - Volume 1: HEALTHINF, (BIOSTEC 2012)
SN - 978-989-8425-88-1
AU - Jian C.
AU - Schafmeister F.
AU - Rachuy C.
AU - Sasse N.
AU - Shi H.
AU - Schmidt H.
AU - von Steinbüchel N.
PY - 2012
SP - 87
EP - 96
DO - 10.5220/0003783800870096