DEVELOPMENT OF VOICE-BASED MULTIMODAL USER INTERFACES

Claudia Pinto P. Sena, Celso A. S. Santos

Abstract

In the last decades, the interface evolution made the visual interfaces popular as standard and the keyboard and mouse as input device most used to the human-computer interaction. The integration of voice as an input style to visual-only interfaces could overcome many of the limitations and problems of current human-computer interaction. One of the major issues that remain is how to integrate voice input into a graphical interface application. In this paper, we introduce a development method of multimodal interfaces combining voice and visual input/output. In order to evaluate the proposed approach, a video application multimodal interface was implemented and analysed.

References

  1. Carneiro, M. 2003. Interfaces Assistidas para Deficientes Visuais utilizando Dispositivos Reativos e Transformadas de Distância. Rio de Janeiro. 162p. Phd Thesis, Pontifícia Universidade Católica do Rio de Janeiro, Brazil.
  2. Carvalho, J.O.F. 1994. Referenciais para Projetistas e Usuários de Interfaces de Computadores Destinadas aos Deficientes Visuais. MSc. Thesis, Univ. of Campinas, Brazil.
  3. Damper, R. I., 1993. Speech as an interface medium: how can it best be used?, in Baber, C. and Noyes, J. M., Eds. Interactive Speech Technology: Human Factors Issues in the Application of Speech Input/Output to Computers, pages pp. 59-71. Taylor and Francis. UK.
  4. DE Souza, C.S.; Leite, J.C.; Prates, R.O.; Barbosa, S.D.J., 1999. Projeto de Interfaces de Usuário: perspectivas cognitivas e semióticas. Jornada de Atualização em Informática, Brazilian Symposium of Computing, Rio de Janeiro, Brazil.
  5. Gavaldà, M., 2000. La Investigación em Tecnologías de La Lengua. Quark Ciencia, Medicina, Comunicación y Cultura. N. 19, Jul-Dec. 2000, p. 20-25.
  6. Grasso, M. A., 1996. Speech Input in Multimodal Environments: A proposal to Study the Effects of Reference Visibility, Reference Number, and Task Integration. Technical Report TR CS-96-09, University of Maryland, Baltimore Campus.
  7. Hix, D.; Hartson, H. R., 1993. Developing User Interfaces: Ensuring Usability Through Product & Process. Caps 1, 2 e 3. John Wiley & Sons, Inc.
  8. IBM, 2003. Multimodal Application Design Issues. (URL: ftp://ftp.software.ibm.com/software/pervasive/info/mu ltimodal/multimodal_apps_design_issus.pdf, access on 03/07/2005)
  9. Maybury, M., 2001. Coordination and Fusion in Multimodal Interaction. (URL:http://www.mitre.org/work/tech_papers/tech_pa pers_01/maybury_coordination/maybury_coordination .pdf).
  10. Mountford, S. J.; Gaver, W. W., 1990. Talking and listening to computers. In Laurel, B. (Ed.), The art of human-computer interface design, 319-334. Reading, MA: Addison-Wesley.
  11. Nunes, L. C.; Akabane, G. K., 2004. A Convergência Digital e seus Impactos nas Novas Formas de Interação Humana. In Anais XI SIMPEP - Bauru, SP, Brasil.
  12. Oviatt, S., 2002. Multimodal Interfaces. Handbook of Human-Computer Interaction, Lawrence Erlbaum: New Jersey.
  13. Preece, J. et al., 1994. Human-computer interaction. Great Britain: Addison-Wesley Publishing Company, Inc.
  14. Raskin, J., 2000. The Humane Interface: New Directions for Designing Interactive Systems. ACM Press.
  15. Santos, C.A.S.; Rehem Neto, A. N; Tavares, Tatiana Aires. 2004. Um Ambiente para Anotação em Vídeos Digitais com Aplicação em Telemedicina. In: Webmedia & LA Web 2004, Ribeirão Preto, Brazil.
  16. Shneiderman, B., 1998. Designing the User Interface: Strategies for Effective Human-Computer-Interaction. 3rd Ed. Addison-Wesley.
  17. Siqueira, E. G., 2001. Estratégias e padrões para a modelagem da interface humano-computador de sistemas baseados na arquitetura softboard. São José dos Campos: INPE.
  18. SUN MICROSYSTEMS., 1998. Java Speech API Programmers Guide. October, 26, 1998. 156 p. (URL: http://java.sun.com/products/java-media/ speech/forDevelopers/jsapi- guide.pdf)
Download


Paper Citation


in Harvard Style

Pinto P. Sena C. and A. S. Santos C. (2006). DEVELOPMENT OF VOICE-BASED MULTIMODAL USER INTERFACES . In Proceedings of the International Conference on Signal Processing and Multimedia Applications - Volume 1: SIGMAP, (ICETE 2006) ISBN 978-972-8865-64-1, pages 310-316. DOI: 10.5220/0001573103100316


in Bibtex Style

@conference{sigmap06,
author={Claudia Pinto P. Sena and Celso A. S. Santos},
title={DEVELOPMENT OF VOICE-BASED MULTIMODAL USER INTERFACES},
booktitle={Proceedings of the International Conference on Signal Processing and Multimedia Applications - Volume 1: SIGMAP, (ICETE 2006)},
year={2006},
pages={310-316},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0001573103100316},
isbn={978-972-8865-64-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Signal Processing and Multimedia Applications - Volume 1: SIGMAP, (ICETE 2006)
TI - DEVELOPMENT OF VOICE-BASED MULTIMODAL USER INTERFACES
SN - 978-972-8865-64-1
AU - Pinto P. Sena C.
AU - A. S. Santos C.
PY - 2006
SP - 310
EP - 316
DO - 10.5220/0001573103100316